Interesting. I came here to research and ask about laws in other countries because I've traveled extensively and for years have always had something with me, at least my little (old 1st generation) Delica. Seems that may not have always been kosher, depending on where I was.
Anyway, back on topic... Take this for what it is... I lived in NYC for many years (W.Village), still have my apartment and travel to NYC several times a month for work. When I go to work I'm usually wearing a suit. The most casual I'll ever be is if I'm going in on a weekend or know I'll be working late I may wear slacks, button down shirt and a blazer/sport coat... also maybe on Friday's in the summer --or on occasion I'll take my tie off and travel home in a suit with no tie.
Up until I lost it a few days ago, I carried/carry a knife with me every day, everywhere for the last 25+ years. Inside the subway stations the NYCPD will often have a table setup where they do random bag checks/inspections. It is somewhat an unwritten rule that during commuting hours if you are not late for work people will often volunteer to be searched so they don't stop and ask someone who's already running late. I had and still have zero issues carrying --but there's a caveat to that. #1 I'm usually wearing a suit or dressed professionally, #2, if stopped I can say I live there (have a NYC address on my current driver's license) which honestly makes a big difference... If you are a tourist why the heck would you need a "utility tool/knife" in NYC... #3 the knife I carried was small, old and well used (meaning could easily pass as a utility tool), #4 it was never displayed, always either in my pants pocket, clipped inside my briefcase or clipped/in my inside breast pocket of my suit jacket.
Other caveats (for my situation) included me usually carrying more than one knife, my Spyderco and usually a SAK with many tools (like flat & Philips screwdrivers) or a Gerber/Leatherman (which were always stored in my briefcase or elsewhere but not on my person). And, I work in the finance space but do a lot of IT stuff. I'm forever buying nerdy computer things like hard drives or server parts --and I install those parts as well as do my own shipping & receiving. As such I keep a small set of screwdrivers and some other stuff (needle nose pliers, tweezers, etc.) my briefcase/bag (Timbuk2 bag). I've been harassed on several occasions (and have a couple NYC PBA cards in my wallet) but never once had to use/pull the cards, gotten a ticket, had anything confiscated, etc. Couple other caveats are that I sailed in the city at least one night a week and always try to if I'm back there on the right days. I also keep a Vespa in the city and always had/have tools/crap in the glovebox, etc. Between all of what I had going on, every time I was ever asked (unlawfully harassed) I always had/have a good and valid reason for having a pocket knife in my pocket.
I think it all comes down to using your head and asking yourself why you are carrying a knife in the first place. If you are a tourist you probably don't have much of a reason to be carrying a knife. Maybe, but probably not. Most NYC residents go out of their way to mind their own business, stay under the radar and avoid drama as much as possible. If you have a flashy Rambo knife clipped to your belt or hanging out of your pocket it'll draw unwanted attention. Most that live in NYC wouldn't do that in the first place. If you get stopped or harassed then perhaps ask yourself why you were stopped in the first place. If you volunteer to a bag check the answer is easy: its a tool, just as legal as carrying a screwdriver or box cutter... and why would I have ever volunteered to an inspection/check if I knew it was a problem...
TL;DR: Probably the best rule of thumb is "if you have to ask you probably shouldn't do it".
