I tend to carry a SAK for those sheeple situations. If I have to use a larger blade, I try to be discreet. However, when people say 'THAT'S A WEAPON!' (even with a SAK) I'll try some humor to lighten the atmosphere or, depending on how I feel, sarcasm.
sheeple: "THAT'S A WEAPON"
Me: Weapons are illegal in this facility according to section 2, paragaph 6 of the OSHA manual, fifth edition. I keep the weapons in my car.
sheeple: "THAT'S A WEAPON"
Me: (looking confused) That's not a weapon (pulls out Manix, Skirmish, etc and holds it high) THIS is a weapon.
sheeple: "THAT'S A WEAPON"
Me: You're right. I'm sorry to carry it. I'll just carry these 4 inch blade scissors around. I couldn't possible hurt anyone with these. Thank you so much for setting me on the straight and narrow.
sheeple: "THAT'S A WEAPON"
Me: (glares in their eyes with a hateful look and says) Yep. They walk away after a few seconds of staring.
Strangely enough, few people, especially at work, even ask me about my knives.