> Packing servers has the minor advantage of using spare resources on existing machines instead of additional machines for small-footprint services. It also has the major disadvantage of running heterogeneous services on the same machine, competing for resources. ...
Have a look at your CPU/MEM resource distributions, specifically the tails. That 'spare' resource is often 25-50% of resource used for the last 5% of usage. Cost optimization on the cloud is a matter of raising utilization. Have a look at your pods' use covariance and you can find populations to stochastically 'take turns' on that extra CPU/RAM.
> One possible approach is to attempt to preserve the “one VM, one service” model while using Kubernetes. The Kubernetes minions don’t have to be identical, they can be virtual machines of different sizes, and Kubernetes scheduling constraints can be used to run exactly one logical service on each minion. This raises the question, though: if you are running fixed sets of containers on specific groups of EC2 instances, why do you have a Kubernetes layer in there instead of just doing that?
The real reason is your AWS bill. Remember that splitting up a large .metal into smaller VMs means that you're paying the CPU/RAM bill for a kernel + basic services multiple times for the same motherboard. Static allocation is inefficient when exposed to load variance. Allocating small VMs to reduce the sizes of your static allocations costs a lot more overhead than tuning your pod requests and scheduling prefs.
Think of it like trucks for transporting packages. Yes you can pay AWS to rent you just the right truck, in the right number for each package you want to carry. Or you can just rent big-rigs and carry many, many packages. You'll have to figure out how to pack them in the trailer, and to make sure they survive the vibration of the trip, but you will almost certainly save money.
it's absolutely insane, they claim several thousands (!) of VMs and each autoscaling group having their own load balancer. Just the cost worth saving when they have only 10-20% wasted resource and an ingress controller instead of multiple hundred ELBs pays for several years of development (+ headcount to maintain).
Great point. By packing heterogeneous workloads on the same underlying VM, you can amortize the spare capacity against non-correlated workloads versus the one workload per VM they are pushing.
> Packing servers has the minor advantage of using spare resources on existing machines instead of additional machines for small-footprint services. It also has the major disadvantage of running heterogeneous services on the same machine, competing for resources. ...
Have a look at your CPU/MEM resource distributions, specifically the tails. That 'spare' resource is often 25-50% of resource used for the last 5% of usage. Cost optimization on the cloud is a matter of raising utilization. Have a look at your pods' use covariance and you can find populations to stochastically 'take turns' on that extra CPU/RAM.
> One possible approach is to attempt to preserve the “one VM, one service” model while using Kubernetes. The Kubernetes minions don’t have to be identical, they can be virtual machines of different sizes, and Kubernetes scheduling constraints can be used to run exactly one logical service on each minion. This raises the question, though: if you are running fixed sets of containers on specific groups of EC2 instances, why do you have a Kubernetes layer in there instead of just doing that?
The real reason is your AWS bill. Remember that splitting up a large .metal into smaller VMs means that you're paying the CPU/RAM bill for a kernel + basic services multiple times for the same motherboard. Static allocation is inefficient when exposed to load variance. Allocating small VMs to reduce the sizes of your static allocations costs a lot more overhead than tuning your pod requests and scheduling prefs.
Think of it like trucks for transporting packages. Yes you can pay AWS to rent you just the right truck, in the right number for each package you want to carry. Or you can just rent big-rigs and carry many, many packages. You'll have to figure out how to pack them in the trailer, and to make sure they survive the vibration of the trip, but you will almost certainly save money.
EDIT: Formatting