This should be titled "Kubernetes API is so simple you can explore it with curl". Which is true. I really like the API consistency, clear resource schema, use of HTTP verbs and conventions, official SDKs and even the kubectl CLI.
However, behind this simple and intuitive API there is still a massively complex and mostly unintuitive system which cannot be reduced to "just make some curl calls and you are good to go". While it is easy to do things in Kubernetes, figuring out what to do and why to do it is anything but.
I've been working in distributed systems for 20 years and k8s is the best yet. The docs are quite good: https://kubernetes.io/docs/home/ The community is growing and despite the Google connection, it's not dominated or controlled by them. Distributed systems are a huge problem space to learn and IMHO I think too many people get frustrated trying to learn all of that, on top of all the k8s tooling and conventions. There's no real shortcut other than getting in and building experience. K8s is by far the easiest system yet to do that--spin up a cluster with kind right now (https://kind.sigs.k8s.io/), it will take 5 minutes, and start playing with the learning tutorials: https://kubernetes.io/docs/tutorials/kubernetes-basics/
I think you’re both right. It is definitely the easiest and most intuitive distributed system, and it’s also difficult to understand how all the pieces fit together. Just getting a typical web app deployed requires a container for the app, a pod definition, a DB container, a stateful set, a persistent volume for DB and possibly app storage, a persistent volume claim, a service, an ingress, something like cert manager to to handle creating a cert and attaching it to the ingress, etc. It all makes sense when you’ve done it a few times but it can seem quite overbearing even to someone who’s a fan of docker-compose.
I'm new to distributed systems and k8s was a long steep learning curve to even get something running. Unfortunately, most of what i've learnt is by running into "how can I do that" followed by intense googling, reading blogs, docs and stackoverflow. Its after i've spent hours getting it to work, that i now can start looking into why and how it works which are usually rabbit holes on their own.
I wish i could start with the concepts of distributed systems, play around in something meant for learning and then map that knowledge to the kubernetes world.
But nothing i could find was even remotely helpful. Most content around this originates in research or academia making it harder to apply and learn on real problems -- the way I'm used to learning.
I’ve just started down this path and I agree. K3os makes setting up a cluster in a VM dead simple and from there it’s really easy to play around with. Especially using snapshots for the VM in case you break your cluster and need to revert.
> However, behind this simple and intuitive API there is still a massively complex and mostly unintuitive system
And that's a good thing! You can't expect such a complex use case to be resolved by something simple.
Having such an easy, consistent and coherent API to abstract away all that complexity is truly a godsend.
K8S on single node cluster, or hundreds of nodes, it just work the same. Moving from one company to another, you don't have to relearn everything. What works on your local kind cluster will most likely work on GKE.
What other system do that? And would that system be "simple"?
The API absolutely does not "abstract away all that complexity" though. Which is fine. Any massive distributed system like Kubernetes is either complex or useless.
My problem is that in today's "devops" world developers are more and more just expected to be familiar with Kubernetes as another tool in their belt. Speaking from experience at my own workspace, that is a recipe for disaster.
As long as the learning curve (which could be weeks or even months) is acknowledged and factored in, Kubernetes can be a very powerful weapon to wield.
At my previous job, the developers were only expected to provide a Docker image with a list of requirements to run it (volumes, environment, configuration). Then it was my job to write the Helm chart and CI/CD pipeline. It was also my job to teach the developers to be autonomous on that aspect.
I never got a job where developers were expected to know how to deal with (pre)production servers. Except maybe in startups where everyone is expected to do everything because you never got enough people with the right skill.
I even got a developer in my team who didn't even know that Kubernetes existed while the company was migrating from on-premise to Kubernetes (and probably still does not).
Kubectl get, edit, delete and describe work roughly the same way on every object type. If that's a service, pod, deployment, or cronjob, once you know the verbs you can interact with all of them.
Every object has a kind, metadata, and spec.
If that doesn't lead to some intuition on how to use it...
The complexity is in what is a service, pod, job, etc. not in the CRUD model. It's like describing driving a car as "the accelerator, brake, and clutch are all pedals; you press them to activate their functionality; pretty intuitive"
Sure it's easy to make HTTP requests to an HTTP REST-like API. That's not the complexity. Look at the data structures, how many fields a simple thing like a Pod have. It's actually insanely complex, and if you are using Kubernetes, you have to understand most of them.
Look at what Kubernetes can do... it's beyond even an operating system, it's like if systemd controlled a whole datacenter. It handles everything too--networking, storage, processes, jobs, etc. Of course that's going to be a complex thing with a large surface area. The fact you can just curl into the API server with a consistent and discoverable (all OpenAPI) API is pretty darn amazing.
There's a full open API schema served by the API server. Tools can read it and build specific interfaces or validate requests before even sending them. And in general k8s is really just a big database of objects. Once you learn the core objects like pod, deployment, service, etc. it becomes very natural to interact with anything else--just find what object it is and interact with its attributes.
I guess this means that you're able to learn how the API works by using the API to download metadata.
In fact, one can generate a complete API client by pointing a script to a running kubernetes instance, which will generate all code for interacting with exactly that version. And the resulting client is huuuge.
Single node isn't going to buy you much over a basic docker-compose.yml setup, but IMHO you now have a home 'lab' to experiment and hone your skills. Helm charts, despite their flaws, when done well can be really simple and easy to install and maintain application stacks too. Just a couple CLI commands to get a selfhosted app setup with all of its dependencies like a database, ingress proxy, etc.
I find that Kubernetes is more complex than docker-compose, but still has advantages for production deployments. Things like rolling upgrades, config management, versatile services with label selectors (and readiness probes). I have been bitten multiple times by docker-compose reading a variable from my current shell environment instead of an env-file.
I'm no fan of Helm though, templating structured files just feels wrong. It clashes a lot with the built-in "valueFrom:" stuff built into Kubernetes; why couldn't Helm use similar "helmVar:" and keep being YAML?
> Single node proxmox VM/LXC mix homeserver. Is learning K8S worthwhile?
Maybe, although if you're fully happy with your current setup then probably not unless you just want to explore and learn new things.
I use kubernetes "at home" (ok, well it's an OVH server, but I use it for personal/home stuff), and I like it. I happen to run a wide variety of apps for myself and friends, already used docker, and I was somewhat reaching the limits of docker compose.
I really like the consistent ecosystem, and that k8s cleanly deals with problems like TLS termination with LE[1] and automatically updating my DNS[2]. I like that I can just write down what I want in a YAML file and don't have to mess around with docker commands or package managers or what else.
It's not for everyone, but I wouldn't say there's no value in it, especially if you're already using containers or a similar tool.
On a homeserver I think it's a wash, just pick the thing you like best. You can of course run kube inside proxmox if you want to experiment.
I think of k8s as being two distinct things. One, it's a way to run and manage software.
Two, it's a way to consume software. There are lots of packages out there for k8s (helm charts, pre-made manifests) and so on. If you want to run eg HA gitlab, the "supported" way to do that is on k8s. There are quite a few complex software stacks now where the simplest practical way to set them up is run them in k8s.
Most of the time these big beasty things are not what you want to run for home use.
Unless you can use the knowledge for work or just want something you can spend endless hours tweaking/tinkering on, not at all, not even a little bit. It will be less robust and an order of magnitude more difficult to troubleshoot and keep updated.
I work on k8s all day long. I welcome some actual responses contrary to my opinion. If he isn’t doing it for knowledge, k8s on a single node is a significant downgrade from proxmox for a home lab.
He currently has containers, vms, robust storage, and an extremely easy to use GUI out of the box with proxmox. He gets none of that from k8s without significantly more effort.
Once again, people are quick to read the title and jump into the comments to write their opinion. The article itself addresses that it clickbaited a little and it actually means the API.
That’s a legit thing. The local node_modules directory is a copy of the dependency so if you want to test a fix you can edit the files directly to try it out.
Articles with title like this is disingenuous to my taste. And frankly, it's plainly wrong. The most natural and easy tasks are always exceedingly complex. If something really appears easy to interact with, it must be proportionally more complex.
However, behind this simple and intuitive API there is still a massively complex and mostly unintuitive system which cannot be reduced to "just make some curl calls and you are good to go". While it is easy to do things in Kubernetes, figuring out what to do and why to do it is anything but.