The problem with actually owning hardware is that you need a lot of it, and need to be prepared to manage things like upgrading firmware. You need to keep on top of the advisories for your network card, the power unit, the enterprise management card, etc. etc. If something goes wrong someone might need to drive in and plug in a keyboard.
Eventually we admitted to ourselves we didn't want those problems.
You have to deal with a lot more stuff. You have to order/pay for a server (capex), mount it somewhere, wire up lights-out-mgmt and recovery and do a few more tasks that the provider has already done.
Then, say if the motherboard gives up, you have to do quite a bit of work to get it replaced, you might be down for hours or maybe days.
For a single server I don't think it makes sense. For 8 servers, maybe. Depends on the opportunity cost.
Have you done this yourself? If you haven't I think you'd discover server hardware is actually shockingly reliable. You could go years without needing to physically touch anything on a single machine. I find that people who are used to cloud assume stuff is breaking all the time. That's true at scale, but when you have a handful of machines you can go a very long time between failures.
Yes, having done this for decades, it happens often enough that you need to plan for it. You need to have redundancy, spare parts, and staffing or you are basically gambling. All of this has to be tested, too, or you might find that your failover mechanism has dependencies you didn’t plan for or unexpected failure modes (I’ve twice experienced data center hard outages due to the power distribution system failing oddly when switching between mains and UPS power, or UPS and generator).
Using something like AWS can make it easy to assume that servers don’t fail often but that’s because the major players have all of that behind the scenes, heavily tested, and will migrate VMs when prefail indicators trigger but before stuff is done.
If you have failover redundancy of services across your systems of some kind to mitigate then great. With proper setup no worries. I guess it depends how much you want to take on vs hand off.
“Your own server in a colo” means going to the colo to swap RAM or an SSD when something goes wrong. You rent a server and the benefit is the rentor has spare parts on hand and staff to swap parts out.
My experience with Equinix varies wildly by facility and somewhat by which tech gets the ticket. Chicago has been good, Seattle tolerable, Dublin is dumber than rocks. We flew somebody up there rather than take chances for a big project.
Asking the obvious question: why not your own server in a colo?