I've used lots of bivys. I find that a slightly fuzzy lining makes the bivy work much better than just smooth Gore-tex on the inside. For example, the OR bivys with the smooth Gore-tex don't work nearly as well as the Bibler bivys with Todd-tex (which has a slightly fuzzy lining).
So, my first recommendation is to get one with a fuzzy lining. I think the ones from Integral Designs also have the fuzzy lining.
Secondly, the ones with the "clamshell"-type opening at the end are a lot easier to get into and out of than the type with a short zipper along the side from shoulder to hip. They're more waterproof, too.
Thirdly, don't bother with the ones that have poles at the head. They make it heavier, bulkier, slower to set up, but not generally that much better. But you do want there to be something to keep the fabric off of your face. I like the type with a replaceable, cheap, pliable wire running through them around the head area. It's also nice when they have a tiny loop sewn in at the head area, so that you can string a cord through it, and tie the cord to a branch, to hold the fabric off your face.
Fourthly, make sure it has a mosquito netting. It's nice to be able to have it partially unzipped, but still have a barrier from insects stinging your face, and various critters climbing in.
Fifthly, I like a bit of extra room at the top of the bivy, so that I can stash some essential gear in the bag and keep it dry.
The Bibler makes the best bivy I've seen (but it's pricey), though there are doubtless some great ones from other brands, which I'm not familiar with.
--Mike