I used to cut and collect for a living, cutting stone is really a whole hobby to learn on it's own. But you can do it!
You can cut softer stones by hand; serpentine, marble, soapstone and alabaster are nice to work with hand tools. None of those stones are very tough but can still be great to learn on. I'd trust serpentine or marble on a knife not meant for hard use. (it will scratch though) Harder stones really need a dedicated water cooled diamond or silicon carbide setup. You can make machines on a budget but once again, it is a fair bit of work.
A silicon carbide wheel and a rubber expando wheel/belts is a great starter machine that you can make. Make sure to always use a GFCI when mixing water and electricity.
Drilling is tough but it can be done with diamond tile bits or dedicated diamond bits that can be found cheaply. Drill in very short spurts making sure to keep the hole cleaned as you go.
Checkout some rock collecting and cutting forums for more info or pm me. Check out machines like the "Diamond Genie" to see a real pro machine and also check out handmade machines, if you like making your own gear lapidary is one hobby you can save a lot on.