dipbait, it really depends on a lot of factors as to whether or not you get a good crop. If it froze at all during the flowering in the spring, crops will be small or non-existent. Also, the leaf roller moth, peduncle borer and larva of the zebra swallowtail have ruined my crops in the past. Usually the leaf roller is more of a problem during dry years. I predict a pretty good crop this year as wet as it has been. You really have to keep checking the trees consistently starting about the last week in September and into October because it seems they can all fall overnight sometimes. Once they fall, they don't last very long. What doesn't quickly rot gets eaten by critters.
Also, I have to differ here with Richard. Color is not the best indicator of taste. I go by smell, feel, and finally taste to make sure. Actually, most times, the best tasting ones I've had have been splotchy and greenish, yellowish brown. If it is green but feels soft or the skin splits and gives to moderate pressure, and it smells fruity, it is probably good to go. Usually, if it is still on the tree and you don't see any others laying around, they aren't ripe. If it is green, others are on the ground, and the greenish ones fall when you lightly shake the tree, they are usually ripe or will be with laying on your counter a couple of days. Just bust one open and test it out to make sure. You'll know if it isn't ready.
Small trees can produce more than larger sometimes, sometimes not. You just never know.
One thing you have to be aware of also is how a patch grows. Generally, if you find a large grove, there are a few "mother trees" and the smaller ones are actually "clones" that sprout off the roots of the mothers. This can be a problem because trees with the same genes generally can't produce fruit. They need to be pollinated by a tree with different genes. Now, if you have the Sunflower variety of Paw Paw like we have here in KS, there is some evidence that it is the only exception to the rule and can pollinate itself. There is a good chance you have this variety since you are right next door. However, I have found groves that are seemingly of the same genes and with nary a paw paw to be found.
Let us know how you do this fall. I'll do the same! I love the Paw Paw. Planted a couple of my own a few years ago out back. Generally, a tree needs to be about 5-6 years old I think to produce fruit.