I reckon Will is right about letting steam escape. But yes, it's true, the paper doesn't burn because the water transfers and absorbs the heat so well. If the cup/milk jug/whatever is only half full, it will burn down to the level of the water. By the same token, the icewater in your drinking glass doesn't start to get warmer until the ice melts. It's packed away somewhere right now, but in my (few years old) Guinness Book, it lists regular water as the strangest substance on the planet for its various interesting properties.
You guys are creepin' me out, once again, me and a guy at work were talking about this just last night!