The pumps are probably pretty large just so it doesn’t take that long. It’s more of a large volume of water problem than a push it forcefully problem if that makes any sense. There’s something called hydraulic head or pump head that determines how high a pump can pump. It’s sort of a trade off between height it can pump and flow rate. The pumps used are more than likely centrifugal pumps rather than positive displacement.
As an aside on pumps, on the ship I work on, we have two fire pumps that are run off 11kV motors, they have to pump a large volume of water potentially pretty high for a deluge/sprinkler system. Our normal ballast pumps are 480v and when they come on the lights flicker but we don’t do much ballasting of the same type as the heavy lifts, it’s more for trim and heel.
There are also considerations that a partially filled tank can lead to instability (free surface effect) and they have a definite sequence that they must pump out the tanks to prevent excess stress on both the vessels (on ships the bending is called hogging and sagging). Some ships if they are loaded/ballasted in a certain configuration can actually potentially break in half.