This is pretty much how I used to use my older model. I ended up buying a high tech one and really do like it. As every other person said, the GPS is no substitute for a map and compass but it can be an excellent augment to one. On my new one, I find I can get away with using a GPS with its included topo map 99.9% of the time and pretty much use my physical map for 1) backup on failure of my GPS or 2) to get a bigger picture compared to the tiny screen of the GPS to help plan routes and such.
Look, maybe it is lazy, but having a 1:24,000 detailed topo map pre-loaded on your GPS which shows you not only where you are but lets you scale up and down the map is damn nice, even if you just really use it to help locate yourself on your hard copy map. Mine also comes with a 3-d profile view which is a bit gadgety, but it really does help you 'read' topo data by switching it back and forth and looking at your terrain. Another cool thing you can do is make a custom map. For example, lets say I have a trail map from the park. I can scan that it and then use Google earth to geo-position it and export to my GPS. I can then flip back and forth between the custom map and the topo map. Its nice. Not essential, but its nice. People have already mentioned it, but I do find it really useful to mark not only location of my car and camp, but also interesting areas I come across. I also find it pretty cool to download my GPS tracks and look at them after the trip was done, especially useful for computing distance hiked etc.
Finally, I have noticed some real improvements in GPS technology today compared to past units I've owned. The electronic compass of older units was crap. Not only did the electronic compass seem to suck up battery life very quickly, the darn things were never that stable and the pointer needle keeps jumping all over the place. I'd always have to increase my pace so that the GPS - triangulation compass would kick in to give any accurate heading. Anyhow, I never trusted the e-compass on my GPS' in the past. This new one is rock stable and gives really good accuracy relative to my magnetic one. My new unit also doesn't need to be level to give a good reading which is kind of cool. Its easier to get a heading at a glance with than a regular compass. Battery life is also improved. I get 8 h steady use on a set of double A's. Not bad and by switching to rechargables and keeping a set of 12 I have all the juice I need on a week long trip that also serves my flashlight.Signal reception on the newer units are also both faster and better. My new one actually tracks satellites in buildings now whereas my older ones wouldn't. My old ones would sometimes get confused in heavy fog and high canopy whereas the new one doesn't.
The downside is that these new fangled units can be expensive. Mine was quite costly, but I enjoy the unit and it lets me enjoy the outdoors. Not needed, true, but as somebody already mentioned, why not take every advantage you can have especially when doing more challenging back-country navigation?