The dirty secret about most fine watches, and this is a tough one to come to terms with when you're a watch guy, is that almost none of them are worth the money charged for them.
And yet that shouldn't stop you from buying one if you love it, and it makes you happy.
Put it this way; is a Strider, a CRK, a Busse worth the money charged for one? In other words, did it cost enough to make it, that with a reasonable profit for maker and dealer plus cost of materials the knife equals exactly what you pay for one at the checkout line? No, and it doesn't really matter, because the guys that own them get a product experience, and for that matter a product that makes them happier than even a seemingly identical one from someone else. That's not even to say that they're not all nice knives, they absolutely are.
In the same way, a Panerai, an Rolex, or an Omega (for example) are all nice watches, and none of them are worth the money charged for them. People should still buy them if it makes them happy, and gives them a product experience they wouldn't be as satisfied with had they bought another watch.
Panerai's, even though they use relatively inexpensive movements (I've heard they're starting to build their own, but that's almost irrelevant in the grand scheme) are really nice watches. They will almost certainly last you your lifetime, and then some, with proper maintenence. Frankly a "better built" watch may actually need more maintenence to keep ticking. Ever known a guy with a Patek, or a hand-made German watch made by some magic elf in the Black Forest? Those watches are like Ferraris, they are bred to do things that other ostensibly similar things are not bred to do. As a result, they need CONSTANT maintenance, they leak in the rain, they get horrible gas mileage, the parts are horrendously expensive, and good luck using one as your commuter car (or EDC watch) without having a couple backups. You know what? They're still awesome. Totally, horrendously, unbelievably overpriced and overhyped, but show me a guy who says he genuinely doesn't want one, even if it were free, and I'll show you a new kind of buffoon.
What you need to do is keep looking for watches that speak to you (unless the Panerai already sings loudly) and ask a few questions, read a few responses, and then ask your watch dealer or maker of choice if the one you've chosen seems to have good reliability, easy and quick service (preferably reasonably priced, including parts) etc. You will get people that say that any watch, I don't care the brand, price or model is either the best thing since sliced bread, or the worst thing since bread mold. Somewhere in the middle is the answer, just like it is with all things. Your local watch dealer who actually services watches will be a great person to ask, unless he's a crook. If nothing else, call up Alex Watchworks in Portland, OR and talk to either the watch makers or the owner. They will tell you if your choice of watch seems well built, if parts are easily available without shipping the watch half-way across the globe for untold amounts of money, and if in the grand scheme of things the watch is "worth" the money. Frankly Watchworks will probably tell you to buy a Bell & Ross, but even though those are awesome watches, they don't sing to me, I have no idea what they do for you.
Once you make a decision, be open to new ideas until you buy the watch. After that, pay no attention to anybody else's opinion, and don't try to pretend that yours matters more than others'. It's just a watch, and one that you liked when you paid the bill. If you spend all your time second-guessing your decision, you are almost certain to come to the conclusion that you made the wrong decision, which is untrue. If the watch sang to you, and seemed like a halfway decent choice, there is no more important proof that you, personally, at that particular time, made the right decision. You can always buy another watch, but in the meantime, you'll have a cool one on your wrist. There are some things in life that you can second guess; your choice of bread, your choice of wine, maybe even your choice of wife. Watches, in the absolute grand scheme of things, do not matter one tiny little bit; they're just possessions, and there will always be a chance to fix any tiny little mistakes that you MIGHT have made some time in the future.
Buy and enjoy.