I live in a large city and work (well, worked) at a software company. I carry a fixed blade every day in plain view and have yet to get a comment or visible reaction from anyone. There seem to be a few simple factors that make this possible...
1) Horizontal Sheath: Even though my sheath is kydex, so it's quite obvious what's in it, the sideways profile just doesn't register with people the way a more traditional vertical sheath does. As a bonus, it also gets in the way far less.
2) Blending: The matte black sheath and dark ironwood grip don't stand out against my black belt and black or navy pants. Except for the full tang along the sides of the grip there is no shiny metal to attract attention.
3) Natural Materials: Leather would probably be an even lower-profile choice for a sheath, but I just like kydex too well.

Ironwood and other natural grip materials seem to register as general decoration and don't scream
knife. The more decorative an item is the less threatening it seems to be.
I would think that carry location should be a factor, but I carry mine just to the weak side of my belt buckle, where it should be fairly conspicuous. Having it right up front doesn't seem to bother anyone.
Here's what my sheathed knife looks like:

It is 6.7" overall with a 3.3" blade and weighs 3.4 oz. If you click on the link you can see full specs and pictures of the unsheathed blade.
--Bob Q