I keep a tube (3 or 4" PVC schedule 40 with a sealed cap on one end and a clean-out plug on the other) of "pickling solution" on hand for scale removal when necessary or desirable. Just buy Sodium Bi-sulfate from the Pool Supply section of the hardware store, and mix with distilled water.. Doesn't take a ton, a cup or two maybe.. It's faster than vinegar IMHO and more effective.
That being said, I almost *never* bother with mill-scale removal anymore.
As far as forge scale goes, when making damascus for example, I'm with Don. Nothing beats a quality angle grinder.. Although I've been in the shops of other's who have dinky little angle grinders with big-box store grinding wheels, and wanted to kill myself. Contractor grade (very different than the standard) DeWalt is the minimum (much higher amperage than the smaller ones), but a large 7" or so Metabo is ideal. I either use the big cone stones or Radnor wheels from the welding store. They break down rather than glaze, and keep cutting.