Welcome to blade forums!
You have gotten some good advice thus far. I am glad that you were able to visit a local shop and realize just how big a 5.5" folder feels. I think your decision to focus your search around the 4" mark is the right move in the long run.
The remainder of this post will be dedicated to categorizing some of the selection criteria you may have not yet considered. I will make some suggestions along the way. We're off...
Weight:
Your initial post mentioned that you wanted a lightweight blade to carry on a daily basis. With a 4" blade you are going to be toting a bit of weight around. In my opinion, you want to look at knives that are under 5 ounces. Anything over this mark begins to get tiresome for EDC. The closer to four ounces you can get the more comfortable it will be and the less likely you will be to leave it on the dresser.
Handle material/texture:
Some handles are highly abrasive and some are much smoother. You are probably going to want something in between so that the knife slides in and out of clothing easily but still offers decent traction.
My favorite handle material is probably G10. It generally offers good traction and chemical resistance for minimal weight. FRN (fiber reinforced nylon) and micarta are also good options.
Lock type:
The most common lock types are liner lock, back lock, axis lock, compression lock, and frame lock. Each have different attributes and actuation methods. Try to experiment with each of them next time you are at a dealer. If you are left handed, you will want to steer clear of liner and frame locks.
Steel/grind:
This is a complex subject. I would recommend staying with a high grade steel like CPM-S30V, 154CM, 440C, AUS-8, or VG-10. There are many others, but these are all safe bets. My favorite is CPM-S30V as it has amazing edge retention and takes a fine edge.
The grind is a very personal choice. I prefer flat grinds but also like a properly executed hollow grind. A full flat grind would be my suggestion here. I think you may be happier with a plain edge as a sharp plain edge can do anything a serrated edge can do IMO. Plain edge will also be easier to sell/trade if you ever need to do so.
Care/maintenance:
Folding knives have moving parts and corrosion/wear sensitive materials. Many people do not think about this when purchasing and carrying a blade. At a minimum you will want to get some lubricating oil for the pivot/edge and some sort of basic sharpening system. You may also want to consider some tool(s) for disassembly, pivot adjustment, clip relocation, etc.
Spyderco and Benchmade blades both come with lifetime sharpening services. If you pay post to and from the factory, the company will sharpen, lube, and clean your blade for you. This may be a good option if you only have one knife and don't want to bother with sharpening and cleaning your blade. I of course recommend doing it all yourself but I am a knife enthusiast.
Clip/carry method:
Examine your chioces in detail and see if they offer a movable clip. The clip can often be relocated for tip up/down carry and left/right handedness.
Model suggestions:
Spyderco military- 4" blade, 4.25 ounces, CPM-S30V, Full flat grind, grippy G10 handles, liner lock, obligate right hand tip down carry, $125. Available in many blade/handle varieties including bright orange
Spyderco police G10- 4.4" blade, 5 ounces, Vg-10, Full flat grind, grippy G10 handles, back lock, four way adjustable clip, $125. Available in black only.
Benchmade 710- 4" blade, 4.5 ounces, D2 semi-stainless steel, grippy G10 handles, axis lock, left/right adjustable clip, $115. Available in black only. Stay away from D2 steel if you live near the ocean.
Benchmade 930- 3.6" blade, 3.00 ounces, S30V steel, smooth G10 handles, axis lock, left/right adjustable clip, $125. Available in beautiful black/blue milled handle only.
Benchmade 940- 3.4" blade, 2.9 ounces, S30V steel, anodized aluminum handles (nice chalky grip), axis lock, left/right adjustable clip, $125. Available in dark green only. A simply amazing folding knife.
Good luck with your choice,
John