It provides far less surface area for decarb, and makes post-HT clean up much simpler. Deep scratches are as much as five times harder to remove on hardened steel than annealed steel. On steels like CPM-S35VN, sanding post-HT is a real bear. My friend takes it to a mirror polish before HT. I can't recall the exact numbers, but consider that the Rockwell scale is logarithmic, not linear. The difference between 45 and 60 is not a 33% increase. It is hundreds of times as hard.
As to the surface area, I would have to do some math to be exact, but a quick mental image says that a 120 grit surface has at least twice the surface area of a 400 grit surface. This pushes decarb deeper into the steel, necessitating more surface grinding after HT.