Good advice has been given.
I can't comment on color, since that is really a matter of personal taste. Non-color displays can still provide a lot of information at a much lower price and lower battery drain.
As I see it, when buying your first GPS you have a couple of key decisions to make:
1. Will you use it while driving a moving vehicle?
If yes, then you'll likely want a GPS with maps, sufficient map storage space (either internal or on removable memory cards), and you'll need a display that is large enough to easily view while driving (a real problem for me). You'll also need some kind of mounting system so your nice GPS doesn't go splat during a panic stop.
Another option is to go with a laptop computer and a GPS/mapping software like those sold by Delorme. This was my preferred solution for use in a vehicle. Again though, you'll need some way of securing the laptop in your vehicle.
2. Do you need to view your location, or the location of waypoints, relative to nearby roads and/or towns?
If the answer is yes, then you'll need a GPS with maps and the storage necessary to use them.
3. Do you just want to be told where you are (via coordinates), or be given a bearing/distance from where you are to a waypoint?
If the answer is yes, then any basic GPS will likely do what you need. These are the very basic functions of most GPS's - they tell you where you are and the direction/distance to a waypoint (either where you were or a point whose coordinates were somehow entered into the GPS).
4. Are you managing a large number of waypoints?
If the answer is yes, I would strongly suggest you make sure your GPS will interface with a computer, and that you consider purchasing software like ExpertGPS which lets you create waypoints and routes on a map, it saves the waypoints and routes, and then lets to move them from your PC to your GPS and vise versa.
5. Do you need a GPS that will automatically calculate a route of travel if you tell it where you want to go?
If yes, some GPS's will do that, but since roads change, I tend to think a PC with good software will do that better than a handheld GPS.
Keep in mind that the some GPS's do come with electronic compasses. These electronic compasses are usually accurate to about 2-3 degrees and draw a fair amount of battery power. Most magnetic compasses are also accurate to 2 degrees or so, but don't rely on batteries. Of course you can do the same thing by getting a bearing from the GPS and using a compass to identify the direction. If you are using your GPS to do geocaching, where you are trying to "point" to a nearby target, then I could see an electronic compass being useful. If out hiking longer distances, then I can't see it being all that useful. In that case you can use a magnetic compass for hours without "drawing down" any batteries. If you choose to buy a GPS with an electronic compass, do make sure that the compass feature can be turned off in order to save battery power.
Most will tell you that you should ALWAYS carry a magnetic compass of some kind whether or not you carry a GPS, just to be safe.
Make sure you learn about UTM coordinates and how to use UTM grids to convert a UTM coordinate from a GPS to a location on a map, or to convert a location on a map to a UTM coordinate for entry into a GPS. A great site for this is
http://www.maptools.com .
Also, you'll need to learn a bit about Datums and make sure that your GPS is set to the same Datum as your map, otherwise there will be some extra error in your coordinates.
