I'm not a lawyer, but my reading of the Illinois code is that being "armed with a dangerous weapon" is not an offense unto itsself, but is just a component of certain other offenses. In other words, it's ok to be "armed" with a 3"+ knife, if you are doing nothing else wrong.
For instance, if you have a 3"+ knife and intend to use it unlawfully against another, you would likely be in violation of the following statute:
Illinois Criminal Code 720 ILCS 5/24-1. Unlawful Use of
Weapons. (a) A person commits the offense of unlawful use
of weapons when he knowingly... (2) Carries or possesses
with intent to use the same unlawfully against another, a
dagger, dirk, billy, dangerous knife, razor, stiletto,
broken bottle or other piece of glass... or any other
dangerous or deadly weapon of like character...
Of course, even a smaller knife would be judged to be a dangerous weapon if used in a manner dangerous to another's well being.
As another example, assaulting someone with your fists or robbing someone are crimes. Having a "dangerous weapon" on you at the time might escalate the charge to one of "armed violence" which likely has stiffer penalties.
Also, remember that Chicago has greater restrictions than state law. You can't conceal anything longer than 2.5" there.