I'll let some others answer about reamers, but will point out that sharp and properly ground bits are the most important part of a good hole. I have never reamed a hole in a wood handle. I use brad point bits for many holes in wood that need precision diameters. The rest of the time I use jobber length bits. If a bit is not drilling right, I replace it. I have a good drill bit sharpener, but rarely use it on a small size bit. Buy them in 10-packs and toss them in a "USED" box for non-precise holes and/or use them to align scales on the second and third holes.
Much of understanding why a hole gets oval is understanding how the bit cuts in wood. The cutting edges go through the grain in various directions. In each revolution they will cut: down the grain - across the grain - up the grain - across the grain. The cut will have more resistance on the cross-grain cuts and try to move the bit left or right as the edge bites it. If the bit is very sharp and running at the right speed, the left from one flute cancels te right from the other flute .... in theory. In reality, one direction will always have more resistance than the other. Since the drill bit is locked in a rigid drill press, the wood tries to move left and right. This can make the hole wider in the long-grain directions than the cross-grain direction, creating a non-round hole. The following things will help control that inherent problem.
Besides good quality sharp bits, technique is important.
Make sure the drill press table is trammed to be 90° to the bit. If not, you get an oval hole.
Short bits flex less than long ones. If you are drilling holes that are less than an inch deep, use jobber length bits.
Clamp the handle or scale flat against a backer block of hard wood or micarta. Obviously, the board needs to be flat and smooth. Both should be clamped firmly in a drilling jig or flat to the drill press table. A good drill press table vise is very helpful. All of these measures will help prevents tear-out on the exit from the back and also assures the grain doesn't try to move the wood about as the drill enters and cuts.
Pilot holes are wise if you are using twist bits, especially for larger holes.
Feed the bit at the right speed and pressure. Let it do its work, don't try and punch it through the wood. Small bits in most wood should run fast. Be aware that certain woods will clog the bits and need the bit run in-out-in-out constantly. Oily and dense woods like ebony, cocobolo, desert ironwood, ringed gidgee, lignum vitae, and other similar woods must be drilled carefully and slower, or they can literally blow the block of wood apart (especially on drilling deep holes for hidden tangs).
Leave enough wood thickness in your handle block or scales to sand down after drilling the holes. This removes the frayed fibers at the surface. The hole will be much cleaner a millimeter below the entry/exit point.
Pin stock diameter varies and rarely is exactly the stated size, so measure it before drilling the holes. It is good to stick with one supplier of pin stock and get a batch of it at a time to assure all pins will fit the same. A cheap HF plastic micrometer is very handy in the shop.
Taper the end of the pins so they don't catch as they go through and exit the handle when being installed. A perfect pin/hole fit up should be a snug fit that just requires a bit of a twist and push to pass through. If you have to use a hammer, it is too tight. If it slides through easily, it is too loose.
