How fast metal is removed is dependent on many things:
1 - HARDNESS of the grit, as compared to the hardness of the metal being ground upon.
2 - SHAPE of the abrasive grit doing the grinding - more angular, 'sharper' shapes will cut more aggressively than blockier or rounder shapes.
3 - SIZE of the grit doing the work - in simplest terms, a larger grain of hard, sharp grit will cut a deeper and wider trough into the metal being cut.
4 - TOUGHNESS of the grit doing the work - this refers to it's resistance to breaking or fracturing as it's working.
5 - WEAR RESISTANCE of the grit, which refers mainly to it's ability to maintain it's shape without dulling by abrasion.
6 - FRIABILITY of the grit, which is the tendency of individual grit grains to break down or fracture into smaller grains.
7 - LUBRICATION of the grit, which helps prevent clogging or accumulation of metal swarf as the work is being done. Also helps with cooling of the grit in powered grinding applications generating a lot of heat. With regard to heat, some abrasives are more stable and harder/longer-working at high temperatures than others will be.
8 - How the abrasive is bound or held together - the 'binder' for the grit can be designed to hold it firmly, as with a plated diamond hone, or to release worn grit more easily from the surface and expose fresh, sharp grit to keep the stone working aggressively (think of water stones in particular, and oil stones secondarily). With a plated diamond hone, the toughness and wear-resistance of the plating itself is a big factor in determining how well it holds onto the embedded grit. And the bond itself, between the grit and the plating, can vary in strength as well, which influences how long the plated hone will last and retain the grit.
In terms of hardness, the abrasive only needs to be somewhat harder than the material being cut. Extra hardness just allows the work to be done with a lighter touch to cut to a given depth (assuming other factors are equal, like size, shape, etc.). Anyone who's ever accidentally left some deep, ugly scratches on a blade with a featherlight, errant brush against a diamond hone will immediately understand this concept.
Friability is often designed deliberately to advantage with some abrasives (like aluminum oxide especially, and with silicon carbide to a narrower degree), allowing the grains to break down into smaller, but still sharp & jagged, cutting edges. Silicon carbide is inherently more brittle or friable in general - but some engineering tweaks can also make it a little less so, as desired, according to usage.
Between friability and the binders used, these are both factors influencing how a stone can be kept working without 'glazing' (dulling/polishing of the surface) or otherwise being worn out.