Angle guides won't help you hold a consistent angle for the entire pass. They will only help you set the same angle at the beginning of each pass. It's up to you to maintain the angle after that, and it sounds you're not doing it very well.
Here's a video that I found recently that might help you:
It uses an alternative method to angle guides. It's more of a trick or a hack. You just mark a spot on your fingers, put your finger against the stone, and hold the back of the blade against that spot. This way you can easily hold the same angle for the entire pass.
Another forum user on here also did something interesting where they clipped something under the blade so that it would hold the same angle for them. I forget who it was or what exactly they were using as their angle clip. But anyway, it's an alternative approach that works by attaching the angle guide to the blade instead of anchoring the angle guide to the stone. I don't think I would ever do it that way, but it evidently worked well enough for him.
