Agreed. I worked on a team that built out a solution like Ably described and we would run into lots of weird issues with ec2 lifecycle (often init script / configuration management issues) and deploys would take a long time (had to drain ec2 instances and bring new instances up) and there's just a lot more to manage (autoscaling groups, instance profiles, AMIs, configuration management, process management, log exfiltration, SSH keys, infra as code, etc). If you really don't want to use Kubernetes, you can get a lot of the benefits by using ECS/Fargate, but I really don't know why you wouldn't just go all-in on EKS at that point.
I would be tempted to skip the Docker part entirely and use raw EC2 instances plus Packer or whatever tool of choice to.deploy the code. You still get autoscaling that way, and additional isolation of processes too. In addition, Amazon handle all the packing of VMs into hosts. With the Docker container route, you're still doing some of that work yourself to minimise waste.
> A small custom boot service on each instance that is part of our boot image looks at the instance configuration, pulls the right container images, and starts the containers.
> There are lightweight monitoring services on each instance that will respawn a required container if it dies, and self-terminate the instance if it is running a version of any software that is no longer the preferred version for that cluster.
They've built a poor-mans Kubernetes that they won't be able to hire talent for, scales slower and costs more.
I didn't think they were using containerd; this phrase made me think that:
> Functionally, we still do the same, as Docker images are just a bunch of tarballs bundled with a metadata JSON blob, but curl and tar have been replaced by docker pull.
Yeah the wording is confusing. But they are definitely using “docker run” rather than say “docker pull” then somehow extracting the image and executing the contents. That would be totally bonkers.
Jokes aside, it sounds like they should just use ECS instead.