For mine, I drill the holes the exact size of my pins. Drill a hole, pin that one, then the next, etc. I do each side with temporary pins (that is lightly sand and then polish so they slide out a little easier) and shape my handles to final shape. Then I drive out the temp pins, finish the blade and tang. I don't peen radically with the final pins, just a light tap before smoothing them to size. That is done as part of my final glue up and finishing.
There is nothing wrong with the bigger holes, the epoxy will take care of the space. I just like my handles to basically be affixed to the point epoxy would probably not be necessary. There is nothing wrong with finishing your handles with the knife. For some projects I finish then this way. I etch a lot my blades though, and have had the FeCl color light spacers faintly in the past. So now I finish the blade with scales separate. It also allows me to cerakote or blue the knife without the chemicals getting on the scales. It also allows me to pin just the scales together to ensure they are exact copies of each other... Works well for bookmatching spalted scales and such also. Always finish the leading edge of the scale to final finish before its on the knife, then you don't have an issue of working against the blade.
I also try for either a blade heavy or dead neutral balance. I also like as few pins as possible and may only expose one or two, and use stubby thread rod for the others. For that I undercut the drilled holes in the scales slightly to create a ledge of sorts for the epoxy. With this method, I have driven a stake with the handle of my knife before and had no issues... With modern epoxies, only a couple of pins are necessary. They basically retard lateral movement saving the epoxy bond... For a small project I have gone with no pins at all... just texture and a couple if small holes to hold the scales. It was a personal project and I'm waiting to see how it holds up. I use G Flex BTW.
I also go to at least 600-800 grit prior to HT. That plus atmosphere control equal very little scale. Reducing scale saves time and belts...
Cheers... good luck.