Your backswing is a product of your starting position. If you need more speed, you start the
ball higher. With a free armswing, the
ball will have more speed going in the swing, and it will end up higher, possibly above shoulder height. If you need less speed, start the
ball lower, again, keep a free swing, and the
ball will end up lower.
The key is that, no matter if your swing is high or low, you should not be pulling the
ball up, and you should not be stopping the
ball short and pulling it down.
Gravity is key.