Why are you using a belt drive setup for the buffer? Why not put the buff wheel direct to shaft? I use a 1/3 hp 1725 rpm motor for buffing and find it just about perfect. I'm not trying to cut with the buffer, but only to polish, and mostly for wood. That doesn't take any power at all, you're just "touching" the buff wheel to the wood when polishing. The wood should already be at 1,000+ grit, better at 2,000+ grit.
Even for putting a mirror finish to a blade, it should be sanded to 1,000+ grit before using buffer. Again, it doesn't take much power at all there. I like a loose buffer wheel when doing blades with the lower power motor. The loose buff wheel will collapse before putting that much pressure to "grab" the blade from hand. Still have to be careful, and always buff on lower part of wheel, and don't allow wheel to catch at top of blade edge.
Later