My 1 ton arbor press from Harbor Freight was way less than $100, like closer to $30.
You can put your stamp in place, the weight of the ram without any force holds the stamp upright. You can then check it for location and make sure your stamp is not leaning forward/backward or to either side. Then just press it in.
You can smack the top of the ram with a hammer to leave a really deep mark but you can also press it in plenty deep by hand. I usually press as hard as I can with my hand then give the ram a light rap with a ball peen to make sure it's a good imprint.
the whole process only takes about 30 seconds. I personally have seen no reason whatsoever to purchase an expensive stamp holder or even modify an arbor press to hold the stamp. If your stamp is straight up and down, it can't move anywhere but straight down.
The other plus side to the press is: you get to buy a tool that can be used for other things and is not limited to just holding a stamp.
This is just the method that works for me. Your method may vary.