All good advice.
Proper insulation
Proper refractory coating(s)
Proper burner and regulator.
All that matters on a forge is those three things. Saving in these places by substitution is foolish. The items needed for a forge build are not expensive, and picking the right materials will make a forge that will last for years. $ 100 will build a first class forge start to finish. Much less with some sourcing skills. As crank has discovered, it is easy to be near that in expense and still have a very low quality forge.
For local sourcing on refractory wool and coatings check for:
Local Pottery shops and suppliers
Local boiler supply and repair places
Plumbing and Heating repair shops
Nearby knifemakers
That said, it is hard to beat HTT&R for a one-stop-shop source.
Talking about Old Mexico - Years back I saw a South American ( Mexican?) forge build link. It was a good size forge sitting outside a rather primitive knife shop shack. The base was a pile of rubble with sand mounded on it. Over that was a long chicken wire tube about 16" round. This was lined with several layers of fiber wool salvaged from a closed factory. This was covered in a thick layer of home brew refractory made from ash, clay, and some sort of fireplace cement. The builder said the mix was used by the locals to line bread ovens. A floor of firebrick ( salvaged from the factory) was placed on the bottom. The back and front were made with piled up and mortared together firebricks. The front resemble a classic bread oven. The final semi-circular chamber was about 8" wide, 5" high, and 24" long. The outside was covered in 6" of adobe, with more chicken wire for strength. The propane burner was 3" pipe and a big squirrel cage blower. The forge worked very well according to the maker. Don't know if this was the one Eric saw, but it sure showed ingenuity.