I've been beatin my head against this wall, too. I like those Swedish surplus wool pants, but I haven't found them in my size yet. (I suspect all the XL's and tall sizes get used up and don't make it the surplus stores?)
I have a couple pairs of fleece-lined, thinsulated jeans from Fleet Farm which I love in town. If someone made a pair like that but with lightweight wool instead of cotton denim on the outside? I'd be all over it.
In warm weather I get by just fine with jeans or bdu's, I keep a spare pair of shorts in my pack in case I get soaked, or know I'm going to get soaked. Only problem is, you can only count on warm weather here for about 60 days... spring and fall are a real gamble.