I wear jeans about 8 months out of the year, and shorts the other 4. In jeans, I can get away with a knife over 5 oz. in weight, but 5 oz. is where I draw the line in shorts -- and even then, I prefer to go lighter if possible. My experience with everything that winds up in my pockets is that even an ounce of difference is a lot.
When I'm wearing shorts in the summer, and running or playing with my kid, most knives north of 4 oz. feel like a brick flopping around on the edge of my pocket. Every time I get excited about a heavier knife, and carry one for a little while, my pocket reminds me why I prefer that weight limit.
That said, GB940Rookie makes a point I absolutely agree with:
For example: I used to own a G10 Manix 2, and that knife and my current EDC -- an Emerson CQC-7 -- are both right around 5 oz. Despite being the same weight, the Manix is an unmistakable presence in my pocket; it never really disappears. The CQC-7 is much narrower than the Manix, and carries better for me (it even does pretty well in shorts).
Lately, I've been experimenting to see how light I can go and still love a knife. My Manix 2 LW (2.8 oz.) is simply delightful, and the 4.4 oz. G10 Military is an insane amount of knife for that weight. Neither is a prybar, no, but nor do I feel like I'm sacrificing strength or solidity on the altar of weight.