It would burn your fingers long before burning the temper. Go to town, and remember to clear the swarf and grit frequently. I use one of the abrasive belt cleaning sticks (cheap, and super handy for extending the life of belts that are just loaded with gummy material) when the sandpaper loads up more than a quick brushing can clear.