Truly budget yet machine powered and useable for other tasks:
1x30"/1x42"/4x36" grinders with disc attachment (underpowered, but good enough if you have patience, useable around the house, and not expensive- most of mine were done on the 4x36- it can be done)
Good belts for the machine!
Drill press and good drill bits
Good files- learn how to properly use them if you don't already know- I was never taught, especially draw filing.
Good wet dry sandpaper/vice grips/high tension hacksaw with numeral blades.
Go to a cabinetry or headstone business and see if they have guaged granite block pieces- I got a couple hand sized that way, and a larger slab and did not have to pay machinist prices (also got a good granite plate on sale after, functional knives can be made without one)- free and useful for flattening stock/handles, making sheaths.....)
Supplies:
Steel- most specialized ingredient so buy clean known steel
Pins- 1/8"&1/4" brass rod from dept stores
Glue- I use G2 epoxy because that was what a knifemaker first recommended to me, is local from Lee Valley, and I've had better luck with it than other systems like Acraglas!
Handle material- local hardwoods- cabinetry shop offcuts, exotic hardwood dealers, cabinetry shop off cuts for Corian/other synthetics.... This can be very budget, but just don't go for the $70 burl blocks on your first piece!
Leather/kydex- heat gun and tin snips will make a workable kydex sheath, leather "sinew"/glue/sealant will make workable leather sheath- can always expand into rivets/gum traganth/leather stamps after.
There is nothing wrong with going inexpensive and still useful tools while deciding if the hobby is for you. A drill press/small grinder is handy regardless, and small models can be found cheap second hand online. I Started this way (profiling blades with dremel cutoff discs before I learned hacksaw and drilled holes!)and after 20 or so knives I've upgraded some things and plan to on others, but I'm approaching this as a hobby not as a career (at least until retirement!), and I work at a glacial pace pay of the time. And this way if you hate knifemaking you aren't out a lot of money for specialized tools that you can only unload at a discount to a "lucky" new maker......