I had similar concerns about M4 before recently getting a GB2, as I live in central Florida and it’s almost always very humid here. Like the OP, I generally opt for stainless steels. I’ve been carrying it very frequently since I’ve had it (probably two months now, which I concede is not an exorbitant amount of time), but I have seen absolutely no issues so far. I absolutely love the knife, and am not worried one bit about it crumbling into dust. I’ve also noticed no issues on the bare edge of my Mini Adamas in Cruwear, but the rest of the blade is coated.
I don’t spend much time around saltwater these days, so I have no input regarding marine exposure, but I do sweat plenty, and that has been inconsequential so far.
I did service a buddy’s M4 Bailout recently, which had apparent rust starting on the uncoated edge and on the exposed lasered logo on the blade, but he is notoriously neglectful of his knives, which is exactly why he sends them to me to sharpen and tune up. Some oil and elbow grease cleaned it up fine. Knowing him, he probably dressed a deer in a swamp somewhere and threw it back in his bag coated in who knows what.
I am very impressed with the edge M4 takes, and will not rule it out if something else that catches my fancy in it comes up just because it’s not stainless.