Depends on the strop and the compound used, for me. For strops with very aggressive compounds, which load up fast and very darkly, I'll usually just use something like a microfiber towel or paper towel to wipe away any black buildup that'll come off (and that's usually quite a lot), and then apply some new compound. I have at times moistened the paper towel or rag with some WD-40, which aids in removing more of the dirty swarf and old compound from the leather. After letting the leather dry for a day or two, I'll then apply fresh compound.
Some strops don't load up very much anyway, like those used with green compound. With those, I'll still periodically give it a few wipes with the paper towel or whatever, but that's generally not as necessary. I've also found green compound lasts a long while on a leather strop, so I don't need to reapply it very often at all. If I actually notice a loss of performance in the strop, then I'll apply some more compound.
If you want to completely remove compound from a leather strop, sanding it down to clean leather is the easiest way to do that. I'll usually use WD-40 or some isopropyl alcohol to clean the leather a bit before sanding it, to minimize rubbing or grinding dirty swarf and old compound into the leather while sanding it. After the leather has been sanded down to a clean & velvety 'nap' on the surface, then apply new compound of your choice.
David