The close button has always been there, you double clicked the top left menu button. That worked all the way until Microsoft started redoing window decorations in desktop mode with Windows 8.1, and even for a short period after.
This was also copied into other X window control styles. Even today, a Motif replicates the Windows 1.0-3.11 top-left menu+close button.
Only in win32 applications. When UWP and its successors arrived, the OS stopped providing that functionality. Some applications may still support it, but the automatic equivalence of double clicking the application icon to the close button was removed, because the application is mostly tasked with drawing these UI controls now.
Well, the standard window title bar still does. But with so many apps implementing their own borders, it's a bit of a crapshoot if it (or the window menu itself) will work with many apps. Even Microsoft apps sometimes forget, like Teams (of course...).
And that answer is precisely why (1) Windows 95 was such a revelation to the market and (2) nerds like us remain oblivious to that[1] even three decades on.
Yeah, yeah, I know CUA allows for a window close. No one knew. I worked IT at the time (as did lots of us here in our youth I'm sure) and was constantly teaching and re-teaching this trick to the poor people trapped with their CUA environments.
But suddenly with Windows 95 you could see how it worked.
[1] Even if we knew in our bones, c.f. this very discussion about the popularity of a cloned hack on Linux, that it was the Right Thing.
FVWM users with virtual desktops disagree. Windows 95 was a step back compared to the FVWM configurability. Deskbars? Why when you can have 3x3 desktops by default, and people even had a 16 (4x4) pane based environments?
You didn't switch between tasks, you switched between full opened desktops with Windows inside, one or two, the rest was somewhere else.
This was also copied into other X window control styles. Even today, a Motif replicates the Windows 1.0-3.11 top-left menu+close button.