If the guard and handle are the same height you can chamfer both. Don’t over do it, all you need to do is just break the sharp edges. If your handle is bigger, only do the handle material. Two or three layers of masking tape on your guard will not only protect the surface but also give you a good indication of how much bigger the handle needs to be. I would not recommend chamfering after glue up. Do a dry fit, glue when you are happy.