Dan,
Horsepower doesn't mean as much as having a machine that is more rigid. If it's geared right and the belt set up is right, a small motor will push a 1/4" end mill fast enough, but if the column isn't stiff, it vibrates. That's what causes the problem and breaks end mills. Learning feeds and speeds is a must, regardless.
I'm not bashing anyone that uses the mill/drills. Guys that know feeds and speeds, and proper set up get good results with them. Just my opinion, but since I've owned and used them from the small ones to the large ones, it's a valid opinion.