Someone with far more knowledge than me can explain the finer points of the differences, but it won't really mean anything if you're not sharpening a lot. To really understand the difference, you have to use them all and get a sense of how they "feel" different when sharpening, affect the bevel and edge refinement differently, work at different speeds, etc. It's like wine.
Someone can explain the finer points to you, but it won't mean anything unless you start drinking and getting a sense of them yourself. If someone tells you a wine is "oakier" or "brighter" than another wine, that won't mean anything until you've tasted a lot of wines and gotten a sense of it. Generally, the main difference that will be useful to know is that they have different abrasive materials that cut differently. Diamonds are obviously the hardest and cut very aggressively. Aluminum Oxide (India stones, for example) isn't as aggressive. Ceramic is in the middle and is fairly hard and aggressive. The harder and more wear-resistant the steel, the more you'll benefit by using a harder, more aggressive medium like diamonds. But keep in mind that all of these different materials are harder than steel and will sharpen a knife.
However, that won't really mean anything to you unless you sharpen a lot of knives with a lot of different stones and "feel" the difference yourself. Different stones work differently, too, because of their design. Even two water stones from two different companies rated at the same approximate grit will "feel" different and affect the bevel differently because they are manufactured differently, using different bonding materials/techniques, a different source of abrasives, etc.
But to address your issue directly, "I'm really only looking for differences in the actual stone," in a nutshell, they are made of different stuff that cuts the steel differently. They are also used differently. Diamonds can be used dry or with water. India and Arkansas stones are usually used with honing oil or cleaning solution (Windex, Simple Green, etc.). Ceramic can be used dry or with water. Water stones are used with water. The price points of different types of stones vary considerably, too. DMT bench stones are expensive by most peoples' standards. Natural Japanese and Belgian stones can be astronomically expensive and are usually only purchased by razor honers (or serious hobbyists). India stones can be incredibly cheap. You can pay anything you want for water stones.
EDIT: One important point to keep in mind: all sharpening stones will sharpen. In fact, you can use a cinder block and a red masonry brick from your garden to sharpen a knife. The difference is mainly in aggressiveness, refinement, and "feel." Some prefer one kind, some prefer others.
