Tape recorders use a capstan and pinch roller to draw the tape past the heads:
The tape is pinched between these two parts and the precise rotational speed of the capstan (the shiny metal pin) ensures that the tape always travels at a constant speed. The reel that is taking up the tape after the capstan is driven via a clutch, so that the drive to it can run faster than is necessary to take up all the tape slack even when the reel is empty and at its smallest diameter. The clutch is always slipping and maintains the correct torque to wind the tape around the reel as it plays without slack forming.
When the pause button is pressed, two things happen. The pinch roller is lifted from the capstan instantly removing drive from the tape, and drive to the reel is disengaged and braked. Why lift the pinch roller? Two reasons. One is so that the tape can be stopped instantly, the capstan usually has a large flywheel directly connected to it to reduce speed flutter, and this cannot be stopped quickly enough. The other is to prevent damage to the pinch roller and tape. Leaving pinch pressure applied without rotation can allow a small depression to form in the rubber and slightly kink the tape.
You’ve probably already figured out the answer. If the pinch roller lifts before the reel is stopped, there will be a position in the mechanism’s action where drive is maintained to the reel but the capstan can no longer govern the tape speed. At this position the tape will run fast as the clutch no longer has any resistance to make it slip. Manufacturers ensure that the actions occur in this order, if the reel stopped before the pinch roller lifted, tape would continue to travel and bunch up as the reel is no longer taking up the slack.
With mechanical pause buttons, it’s possible to find that half way point where the reel is still driving and the pinch roller is just lifted, the tape runs fast and you hear chipmunks.