To me, a $50 or $X deposit represents the buyer's labor, from a job he probably hates or at least would rather be somewhere else, that could have gone into his kid, his gas tank, his electric bill. What am I going to do with it? I don't need it to make his knife. I don't need it to pay my bills, keep my "doors open" or "things moving." I'm not going to charge him opportunity cost of his labor, when if anything I would just stash it away so I could give it back to him at his request for whatever reason.
Taking money from someone in (relatively)immediate exchange for goods and services I can do. Expectations are either fulfilled or not met with the item in hand. If they're not, a refund is a mouse click away. I can't give back 3 months of whatever that deposit could have done for the buyer.
I guess for me it amounts to this: When a buyer and I come to an agreement about price for a knife, I know exactly what he's trading me. $500 is $500. Doesn't matter if it comes from Bill, or Jim, or Pete. It's $500. He doesn't know what he's getting from me. He has an idea, maybe a good idea, maybe he's even said "Make me one identical to this one you've already made." But he still doesn't know until he's got it in his hand, because I'm human, and I make mistakes, or misinterpret expectations, and "identical" hand made items rarely are. So I believe that the burden is on me to provide what was asked for, for the price stated, and not on the buyer who's trading me objectively exactly what I asked for in trade. It's not that I'm not confident in my ability to supply what I agreed to supply. It's that I'm not confident that the expectations agreed to were communicated perfectly and when I provide exactly what I thought we agreed to, it's not exactly what the buyer thought we agreed to.