I think the main thing to look out for is that you can find a wider range of quality in that price point than nearly anywhere else. There are $4 knives that are worth $4, and $60 knives that are worth $4. the main thing is to look at what you want to use it for.
Things to think about, reputation of the brand, scale of production, finish and final edge, handle material, steel. Take for example an opinel folder. they are a bit rough, need a bit of work to get them really nice, but every dollar not in the handle and finishing is in the steel and treatment. They are produced in vast numbers, and have a very good reputation. Then take your average middle of the road "tactical" folder. maybe its got G10 scales, those are not exactly cheap to produce. Its got a lock, thats another piece to design and manufacture, maybe its got lots of little jimpy bits, and a clip. By the time the handle is done whats left for the blade? cheap steel, a coating to cover-up the finish of the steel, and some hardening to some degree. But if what you want is a nice handle, with a mean looking blade, then buy what you like.
Best of both worlds, blade and handle, Victorinox pioneer. can't go far wrong from there.