Quite the contrary; collectors tend to go for pretty stuff (and heads much more so than hafts), and much like a fancy-figured gunstock that is prone to break at the grip in the field (think elephant gun or some other real 'kicker'), grain orientation of a handle is no biggie on a wall hanger or show piece. But someone that constantly uses an axe (or a sledge hammer, shovel or rake for that matter) could care less about what the wood looks like as long as the grain is oriented as best possible and shows minimal run-out.
I blindly used whatever axes were on hand (Lands & Forests Dept, for instance, had 3-4 dozen of them for daily issue out of the tool crib) for years and only through inspecting the cause of breaks and learning to re-haft came to be more appreciative of what to look for when selecting axes and handles. Axe handles rarely break under normal circumstances but it's overstrikes and prying motions that really overstress them.