Best I know, the "Heinlein" quote originally comes from Gauss. I can't say exactly when or where he said or wrote that though.
(That was on the door of my neighbor in the dorms, a few years back. He and I were both math majors. On my own door, I chose to put a poster of Einstein saying "Do not worry about your difficulties in mathematics; I assure you mine are greater.")