This is definitely true. For instance, .460 S&W from an XVR feels better than .454 Casull from a Redhawk.
I think .40S&W can be great for carry in the right gun. That's the main factor. In something like a USP, PX4 Storm, or one of better metal-framed pistols; it can be very comfortable to shoot. The smallest gun where .40S&W is comfortable is probably the PX4 Compact. Of course, the PX4 was specifically built for .40S&W and its rotating barrel system helps to mitigate recoil. (That's also probably why it is such a remarkably soft shooter in 9mm.)
That's the other thing. Lots of guns chambered in .40S&W over the years were originally built around 9mm and many were already polymer guns trying to be light for carry. I tried both a Shield and a PPS in .40S&W. Neither felt good at all.