There's all sorts of ways to start out with DE's. Some things I've learned. The razors are really small compared to what you might be used to but much heavier. See the picture at the bottom of the post, that's my superspeed on the left next to a normal Q-tip, it's a typical size for a DE razor. It'll feel a bit odd at first but you get used to it and then wonder why on earth a mach 3 is so damn big. "canned goo" is okay, if you're having trouble lathering a soap or cream, or you get a de razor and don't have the other stuff, they work just fine together. Cheap soaps are good too. VDH Deluxe is the first soap I recommend you get. It's one of the easiest to lather soaps and responds quickly to face or bowl lathering. You can get it at walmart for just over 1.50, and it'll last you for a couple months. I don't recommend Williams, maybe it can be lathered, all I get is thin airy foam, super slick but there's no cushion and something in it burns my face.
So first off, my short answer for a first timer, based off my experience and what I'd want to start with if I knew 4 years ago what I do today. Gilette Superspeed off ebay, VDH deluxe soap(walmart), Wilkinson DE blades (walmart) cheap salsa bowl (walmart), Muhle black fibre synthetic brush(direct from muhle online). Total cost of everything but the brush is under 30 dollars, maybe under 15 if you can nab a sweet deal on that superspeed. The brush is 39-55 dollars, but it's a sweet brush and maybe the last you'll ever need, I don't have one, but I'll be fixing that soon. Alternatively get an Omega boar brush for 11 bucks, I'd skip the VDH brush due to shedding and general quality issues.
The salsa bowl is your lathering bowl, I like mine, it's cheap, the perfect size, and doesn't break my toes or leave shards of ceramic all over the bathroom if it gets dropped. Get the brush nice and wet, then one light pump/shake over the sink to get excess water out. Swirl it on the soap until the soap looks like it's penetrated up to the handle and filled the bristles, then swirl it around in the bowl until to fully build the lather. When you lather, you should end up with something as dense as canned shaving cream, but generally better all around. If it's airy, or dissapears quickly, then it's not right, usually the answer is swirling on the soap more. Only add extra water if the lather is hard and pasty. Once this setup is mastered, you can find a favorite soap, blade, razor combo to stick with. I recommend trying (it'll probably be hard) to use up whatever new product you're trying before buying another one, otherwise you'll end up with a huge collection like some guys, which kinda offsets the money saving aspect of the DE razor.
Now for the long answer.
Razors... well all sorts of options.
I recommend finding an NDC (1948-50, they weren't date coded like all the others) Gillette Superspeed TTO(twist to open). This is same razor I have and I use it daily. It's very mild and you'd have to work pretty hard to cut yourself. I get more nicks with cartridge razors. You can find really nice ones on ebay or badger and blade's FS forums for less than 20 dollars shipped. It's totally safe to use an old razor like this, the seller will probably have done all the cleaning needed before you get it, a quick scrub with some scrubbing bubbles, barkeeper's friend, and a quick dip in alcohol is all that's needed if they didn't. Some people think that DE razor shave quality and comfort are perfectly balanced with this razor and hasn't improved much if at all in the last 60 years.
Most of the merkur stuff is good, though I hear they're having some plating issues recently. For people with coarse beards, the Slant (merkur's slant bar design, models 37c and 39c) gets recommended alot. If it weren't for the weber razor this would be my 2nd razor. As it stands it'll be my 3rd, maybe later this year.
Some people have grip issues with the smooth handled futur if they get some soap on their hands, I've also heard it's a very agressive razor compared to superspeeds and merkur's 3 piece razors even on lower settings. I recommend holding off on a Future as a 2nd razor.
Weber DLC and ARC bulldog razors. These available with DLC and some other kind of Chrome based PVD coating . The head and handles are solid stainless steel, and they've generally gotten rave reviews aside from the "utilitarian" appearance. Made in the USA, 70 bucks, I don't think you can find a better deal, so I'll be getting one as my 2nd razor as soon as they're back in stock.
Feather Popular. Made in japan, can be bought new under 20 dollars, uses more plastic than most razors but the quality is high, and it's another mild shaver/easy to learn with. TTO (twist to open) design.
Soaps: VDH deluxe is right now my favorite, my experience is rather limited though. I've used Mike's natural soaps and really like the scents, cusion, and moisturizing, but my razorglide isn't as good as it is with VDH. I think the potential is there, it just doesn't have the wide lathering tolerance VDH does. Everyone's skin is different which can affect glide, and many others think Mike's is some of the best stuff you can get right now.
Cella is on my short list of "must try soon" soaps, it's an Italian made soap with history, not terribly expensive, sposed to be easy to lather, and has a huge fanbase.
Brushes: I have two, the VDH boar from their deluxe kit, and a Parker Synthetic. The VDH boar sheds, alot, you'll have hairs on your face with most shaves. The parker is an upgrade, I like the way it lathers but the bristles have a bit too much spring to them. The Muhle black fiber I mentioned earlier is much different, and I look forward to getting one soon.
The eternal Boar vs Badger question. After years of reading, it seems you can't just say "badger is better". They are for sure different, and there's more variety with badger brushes. This is a YMMV thing.
Blades: The walmart Wilkinson swords are made in germany, and by most reports just "okay" blades. They'll get you started though, and blades are another one of those YMMV things. Tons of places online offer blade sampler packs.
Astras are super popular and super cheap, 10 bucks/100 blades when they go on sale, sometimes even less. They're made in Russia, and seem to be the "all around" blade of choice in regards to actual performance.
The only US made blades that I know of are personna Lab, Med Prep, and barber blades. The barbers are hard to find, more expensive, and from most reports just not that good. The labs and med preps are pretty much the same blade with different markings and the meds are cleaner. The lab/med blades are well regarded by most.
From left to right, Qtip, Gilette SS, Bic disposable, Sensor XL handle, Mach 3 handle, Fusion Power handle.