I tend to prefer a 4.5lb for moving metal. I use a 4.5 rounding hammer for about 90% of my forging. I use a 4.5 cross pein for the rest of it. The only forging I do with a lighter hammer is when I am working on stock 1/8 thick or less, as even light blows to the edge with the 4lb hammers tend to cause the edge to roll. When forging thin stock, finalizing bevels, and planishing, I use a Peddinghaus 800gm German Pattern. I essentially never touch my 2.5lb hammers. I have a number of other hammers that I also almost never use.
I find that the smaller hammers are exhausting and hard on joints when you try to move metal with them. And even then, they just don't move it well. A properly radiused, heavy rounding hammer forges quickly, and I have found to be nearly as precise as a smaller hammer in almost all forging situations. I find this doubly true when you are really trying to move metal, because I have a tendency to swing a lighter hammer instead of just dropping it and letting the gravity do the work. I find that swinging a hammer significantly reduces precision. When I am 'dropping' the hammer instead of swinging it, I have essentially removed all but two variables: the height from which I drop it will control the impact energy, and the position and angle of the blow will be set essentially by the hammer's orientation as I guide it down.
