OP, I'm a fixed blade maker that almost exclusively uses natural handle materials. I've made many thousands of knives that are used very hard as working cowboys/ranchers etc are my market. I also back my knives and will fix them if ya break em. The number one handle material that I've seen needed fixing is smooth camel bone, (4 or 5 out of several hundred). Enough that I'm working with a stabilized camel bone now and so far so good. Second has been jigged bone but thats been two or three out of probably a couple thousand. One was dropped on a tile floor and another had kind of a cool story. Belonged to a charter fishing boat Capt down in FL. Had a line break, the fishing rod snapped back and smacked the handle of his sheathed knife and knocked a big chip off of it. This guy doesn't care, I fixed it for him and have made him probably 20 of the same knife always with the yellow jigged bone handle.
Heck even made him the only machete I've ever made but he went with yellow G10 on that.
My most popular single handle material is sheep/rams horn. I have never repaired one that I can recall. Thats out of several thousands of knives. Sheep horn is very grippy and its not just the texture that gives you this. Smooth sheep horn will get more grippy when wet too, its just a characteristic of sheep horn. From a knife making standpoint it does have a tendency of wanting to go back to being curved. The darker Himalayan Rams horn has less of this tendency. Merino sheep horn:
Himalayan Rams horn:
Buffalo horn, I've repaired several with the scales wanting to lift. Fact I've pretty much quit using it for some years now. I use to use it so much I was importing it from India myself. Nowadays I'll only use it with Loveless bolts or Corby bolts. Tough stuff though I don't ever recall one breaking. I just finished re handling one of my ancient history knives, I mean old, and pretty crude, looked like Freddy Flintstone made it. It had buffalo horn scales that were separating from the tang. Replaced them with sheep horn for the guy.
Woods. I've used lots of different woods and I don't recall ever repairing one except one ironwood one that had cracked. Pure durability I like cocobolo, a rosewood.
I've made thousands of knives with Sambar and now probably the last decade, strictly American Elk. I have never had an antler handled knife break that I know of. Sambar was the preferred antler for a long time because it had a single vein hole down the center of the antler whereas our elk or your red deer have a lot of pith. Doesn't matter really though because the exterior is the same as far as hardness and durability and I cut the pith off while making scales. Done correctly there is no difference between Sambar, Red Deer or Elk. Lots of folks say that Sambar has better color but its been dyed. They use Potassium Permanganate as a disinfectant and a bye product of this treatment is the color. I do the same with my elk. Meanwhile part of the problem with getting the really "good" sambar is that this particular species has been interbreeding with other species and the sheds are not so solid anymore. They are having alot more pith in them too. Pith on an Elk shed:
But I cut four scales off of that:
After the PP treatment:
Sambar. I can't tell the difference:
And there is none practically. Anyhoo thats my experience regarding durability in natural materials. I certainly wouldn't discount stabilized woods either. Sometimes the best of both worlds and way more soul than micarta or G10: