I started out with a very simple Garmin GPS 12 (probably don't even make them any more).
I eventually upgraded to a more fancy Garmin (the 76CS). Even have all the topo software for it and stuff.
Playing with all the features is cool, but I have come full circle to using the GPS in the most simplest of terms. I don't use "trackback" because I rarely walk back the way I came in. I also found the maps not much use, because continually zooming in and out on a small screen takes too much time. I just use topos of the area anyway.
I basically mark the point I want to return to. Whether you want to put it away while still on, or do the turn-off, turn-on method, I do agree with putting it away. There is no need to be walking with your head down, looking at a small screen. Personally, I use the turn-off, turn-on method. Once the satellites are acquired in the area, it only takes a couple seconds to pick them up again. But, that is beside the point. Here is how I use it.
If you just whip the thing out, there are 2 ways to use it without further movement required.
1) My preferred method - Just initiate a "GOTO" to you starting point. On my map screen, I have it set up to report "bearing." It will only report a number when the "GOTO" is initiated. No walking needed, no history. Just fire it up, or pull it out of the pouch, read the bearing, and put it away. I then navigate with my compass. You can always pull the GPS out again to double check your navigation.
2) Works well too, but not as accurate. Most GPS units have the "map" page defaulted to "track up." Meaning that whichever direction you are moving is up. I don't like to have to move to figure direction. So, I set the GPS to "north up." The GPS does not require movement to know which direction is north. So, you pull it out of its case (or turn it on...whatever method you are using) and face north. Since the map is set to north up, you can visually see what direction you need to go.
Don't think that you have to go spend a ton of money to get usefulness out of a GPS.
B