Try this:
Sand the magnet surface with 120 grit paper, and clean with acetone.
Make the end of the dowel slightly concave, so it will have a pool of epoxy between the wood and the magnet.
If you really want security, use a dowel slightly larger than the magnet, and drill the end to make a recess that holds the magnet.
Use slow cure epoxy.
I just about guarantee that this will solve your problem.
For an alternative to epoxy ( and easier to remove) use double sided sticky foam tape to mount the magnets. I use this to put rare earth magnets anywhere I want one to hold things up/open/closed.
Go to a good hardware store or the German equivalent of Home Depot and look at the double sided foam tapes. Some are construction grade, and will hold as well as nails.
A second alternative is Liquid Nails Construction Adhesive. It comes in a small squeeze tube as well as the big cylinders. This will work as well or better than epoxy on your project. You will find it at the hardware store near the tape, most likely.
