I'm not much of a set drummer, but I have a couple of comments.
First of all, a lot of this depends on your price range and what your willing to spend. You mention being new to drumming - I would suggest getting somewhat of a starter kit, if your going to buy new. If you decide down the road that you don't like it or aren't interested in it, your wallet won't be too far drained. I have heard a few stories about guys starting out buying 1000 dollar kits (or more

) and wind up selling for a loss after figuring out it wasn't for them. Also, I would avoid an electronic kit. They tend to be expensive and lack the feel of an acoustic set - Plus, you learn alot about drums by owning and taking care of an acoustic kit - tuning, maintenance, etc. Electric kits have some plus's though - Good for volume control and apartment like settings.
That said, I have seen a lot of nice kits - If you find something you like in a major brand, you should be fine. I had a Pearl Export series that was a great kit for me - I don't know if they are still producing them though. I have played on a couple different kits - By far my favorite is Mapex. I would really suggest going through craigslist and looking for a kit that someone is selling, rather than buying new. With any luck you should be able to find something decent at a price range of a new "budget" kit. Lower end kits tend to have cheaper wood (basswood and mahogany vs. maple), hardware, cymbals - They still work just fine but you will lack in sound and durability that "nicer" kits tend to have. Most of the time, you pay for what you get.
The cool thing is that if you really get into it, you can always upgrade - Either buy purchasing a brand new kit or by adding on cymbals, a new snare, etc. Sabian and Zildjian both produce great cymbals.
BeardTamer mentioned a lot of good things - customization, cymbals, heads - Don't worry too much about buying the nicest kit out there (unless you have the extra cash

). Buy something and use it! Alot of the "know how" will come over time. Once you get into it, you'll get a feel for what you like and what you don't.
If you find something you like and have questions, feel free to send me an email or PM. I'd be glad to help you out, give advice, etc. Like I said, I haven't played on a set in a while but I'm regularly involved in marching percussion. I also have a buddy/co-worker who knows way more than I do. If you have questions, I can get a hold of him and get you answers I may not have.
Good luck and let us know what you decide to get!