Okay, legal issues aside, here is what will happen.
If you are walking down the street with a knife on your belt, and the cop sees it, he will very likely stop you and ask you about it, unless it is fairly obvious you have it for work (like you are dressed up as an electrician toting tools and such). If you don't have a qualifying answer, such as going to work and this is what I do and I use a knife like this for my job, or he doesn't believe what you tell him, then he will probably tell you it illegal to have it and will seek to confiscate it from you. If you resist, you will likely get hauled into the precinct, where depending on your demeanor at the time, you could still lose your knife or you could get cited or even charged with a felony. It may not be on the up and up, but now look at the hassle you are in. If it looks like it could be used to inflict injury, then you are going to be in a tough battle convincing the judge otherwise. Yeah, you can appeal his decision, to what avail?
Prudence would say that you ought not carry a knife, or any other potential weapon, out where the cops can see it. I can and have walked around Manhattan with a loaded semi auto that no one but me knew about. I don't do it all the time. I darned sure don't advertise it, and I wouldn't a knife either. You are just asking for trouble that ain't worth it. If you want to pack something for self defense, then do what you must, but be discreet.
Laws are great in a courtroom, but the practicality of streets preclude due process up to the point that you are in custody and the offending item is removed. It does no good for yourself, and usually a lot of harm, to argue with a NYPD that you think your rights are being violated. You may eventually get your day in court, but they will get a whole lot of other days you would otherwise have enjoyed being somewhere else.
The point is, legal or not, why carry it out in the open if you don't have to? They don't want to see it, but I'll be damned if I am going to put my safety in their hands. I may go to jail if I have to use it, but at least I have a chance of surviving. If I am going to pack a knife for work and I can't keep it put away, then I will make sure I look the part and my line is believable. If it is for self defense, then the only person that will need to know I have it is me and the guy I have to use it on. Most of the time the cops are only there to clean up the mess after the fact, but if you are stupid about it or act the fool, then they will do their best to act pre-emptively.