All it takes is patience and persistence! It helps if the wood is overly dry (a morning or afternoon in the oven at 120-150 F, or tucked overnight behind the wood stove, if not sure). Once it's partially started in to the eye and tapped back out that leaves you with a perfect imprint on the wood to replicate using file/rasp/chisel/sharp knife, on the remainder of the hang. By rights it should slide on tight enough not to wobble and yet loose enough to be able to move without serious effort. What you've described on your unfinished hang doesn't allow space for the wedge to tap in but at least you have now created a template.
The more often you do re-hafts the smoother and faster you'll get.
I cannot claim to be an 'axe master', by the way. Very few folks qualify. That category would have to go to one of those axemen of yore who could masterfully re-haft using only a pocket knife (or an axe head) during a coffee break.
Had Square_Peg's response been entered before I starting typing this response I wouldn't have bothered to submit an opinion. His advice is excellent!
