Couldn't agree more. It seems Gitlab is severely underestimating the cost of having similar infrastructure on bare metal as their AWS infra. I would probably start to re-architect their software and replace Ceph with something better, probably with S3. Much easier to scale up and operate. Also their Ruby stack has a lot of optimisation potential. Starting to optimise on the other end (hardware, networking, etc.) is a much harder job, starting with staffing questions. AWS, Google and MSFT has the best datacenter engineers you can find and there is a huge amount of effort went into engineering their datacenters. Not to mention your leverage you have being a small startup vs a cloud vendor when talking to HW vendors. Anyways, in few years we gonna know if they managed to do this successfully.