rockgolfer, I think it's worth discussing on a discussion forum. That's why we're here. For me it's not an issue but for you it is. Neither of us is right or wrong.
For me, it's in the same category as sharpening the blade. But sharpening the blade to my liking would take longer. On Swiss army knives, I'll often square up the screwdrivers since they the edges are polished off. I'm used to tinkering with knives so that they work best for me.
If the spring is so loosy goosy that it doesn't hold no matter what then that would be a bigger problem for me. That's why I asked about the spring.
This type of implement is a full time job for companies like Victorinox/Wenger. Some of their designs actually depress and lock in place to secure the driver. And they use rivets instead of peened pins. This Trestle Pines knife and most from other traditional knife makers are very old school.