I use to believe that too until someone rightfully reminded me of all of the ugly solutions I’ve used over the years to deploy to VMs. Then I became a big believer in always containerizing.
This is the insight. You'll a worse job even for something as certificate management with letsencrypt. Let alone all the other stuff. If your workload doesn't need that, and you are fine with some downtime, then and only then don't do it.