I am inherently distrustful of relying solely on epoxy to hold handles in place. Had a lot of bad experiences with the Devcon stuff. However, lots of guys say some of the higher grade epoxies are worlds above what you usually find in your local hardware store or walmart. A good portion of airplane wings are held together with nothing but epoxy, after all. Search the shop talk section for Lock Tite- they also make some super adhesives that supposedly bond stronger than the handle material itself.
That said, I would still personally try to come up with some solution to mechanically fix the handles in place even if you don't like pins. Maybe some hidden pins, or dovetailed bolsters fore & aft, or pins hidden under a euscutcheon, or a bezel setting, or something.