It would take the same sequence of grits that wood does. I don't know how much material you have to remove, but take it slow, with fresh belts/abrasives, always, or you'll burn the stuff badly.
I would probably go 80, 220, 400, 600, the 400/600should be by hand, with a backing for the paper, as you can control it better, and no chance of burning with these finer grits.
Then buff with a loose buff loaded with white compound. The darker compounds like green etc., will leave color in the ivory.
Just use a light touch when you do it, and you'll be fine.
Edited to add; If you want to leave that rough colored bark like surface, then it will require a completely different approach.
Something along the lines of wiping super glue on the outside, and buffing.
I could better say if I knew what type of handle you were planning.
If it's a full tang, you can flatten the inside with 40-60 grit, and leave it.