> as a small developer

Do you need kubernetes?

I know the hype cycle is mad for copying big tech, but if stackoverflow can operate on a couple of IIS instances I’d argue that you almost never need kubernetes.

People jump on bandwagons.

Ten years ago you started with PHP + memcache + MySQL running on just a physical box running linux. No Docker or Kubernetes. No virtual machine for what it is worth. Then you split it in multiple PHP frontends with a load balancer or MySQL master and slaves as the traffic demanded it.

I think a lot of systems these days are over-engineered and over-paid from day one. It may cost you 10-20 times what you would if you keep the "old day" approach.

That said Docker and Kubernetes are nice. They give you a lot of flexibility. But the most important part, I think, has been that they consolidated the shift from "pet" to "cattle" servers that simplifies a lot the sysOp and sysAdm work.

Disclaimer : I get paid to setup k8s clusters for the bandwagon folks

From my perspective k8s is really good for a company who has a team of experienced SRE who can manage k8s well and the company has a marketing team driving new development that needs to get to market fast

It is not cheaper than simple autoscale groups, it is not easier than rebuilding packages with jenkins ( rpm spec files or debian src rebuilds )

It is however the current popular framework so if you want to ride the wave learn it. Also learn how to migrate away from it as that will be a future role as several early adopters are pulling back out of it.

The larger issue is the simple fact automating cloud infrastructure is not trivial so abstract layers lets more people implement things without understanding the lower levels giving folks the appearance of having a complete framework.

It works great until it doesn't. Then it is interesting watching people try to figure out how to fix it.

Learn the lower levels of Linux and k8s will come to you organically

In your experience, what is the strongest point of evidence leading to the successful utilization of k8s?

> the company has a marketing team driving new development that needs to get to market fast

At what point in the growth curve? Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping?

What development process or cycle has overheated with friction that requires k8s? Where is that friction which k8s relieves?

:)

Successful utilization of k8s, interesting metric, what is your criteria ?

>At what point in the growth curve?

I have seen properly staffed startups (in silicon valley) leverage the platform to pivot direction fast but overall I would say you need a large enterprise to support it. Ironically the large enterprise which could benefit the most wants to lay ITIL on top of k8s, call it agile and the methodologies conflict creating more issues than they had.

> Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping?

Yes to both

>What development process or cycle has overheated with friction that requires k8s?

Upgrading k8s :)

>Where is that friction which k8s relieves?

What k8s relieves is once you get a pipeline getting product to production fast, the approval process from legacy ITIL groups being the problem ( I have yet to see an enterprise company use k8s properly though I have been to presentations where some claim to have done this )

If someone attempts to use ITIL and k8s you are going to have problems. Training people to NOT do this is the #1 issue in my experience.

All my mentions of issues or friction in my previous comment was about pre-k8s team's experiences... I try and clarify a few key cases in the following:

> Successful utilization of k8s, interesting metric, what is your criteria ?

Sorry, what pain motivated using k8s, and using k8s relieved that pain.

> I have seen properly staffed startups (in silicon valley)

What is 'proper' for staffing?

>> Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping? > Yes to both

Is it the appropriate use of a Reliability engineer's skills to develop or change arbitrary features? Or are you saying they are at least Engineers so they should be able to pitch in everywhere... CSS accessibility features or k8s config.

>> What development process or cycle has overheated with friction that requires k8s? > Upgrading k8s :)

This tautology keeps me away from k8s. Before k8s what pain in the process required using k8s to solve?

> a pipeline getting product to production fast

Does k8s make it fast? Does k8s make it possible? I think I'm deploying "fast", without k8s... But because you keep mentioning ITIL I suppose it's more about the infrastructure changes? Or are you talking about ITIL tension because k8s requires a more liberal policy than ITIL allows?

There is so much assumed context when talking about k8s, the only comprehensible part of these discussions is k8s is a rabbit hole to end all rabbit holes.

What I've learned from the comments above is k8s consulting business is the best thing since OOP. Just tell the marketing team you have a magic button that being installed will give them ability to change the course of business every 5 minutes by handwaving and a PowerPoint slide.

>Sorry, what pain motivated using k8s, and using k8s relieved that pain.

CTO / CIO / $SOME_C reads a blog and decides they want k8s is how it usually is introduced

> What is 'proper' for staffing?

Experienced C Developers who can do operations ( like google level SRE )

Summary of k8s : if you use the proper methodology to implement k8s and you are in a cloud framework you likely dont need k8s :)

What shines about k8s is it leverages the current container fad to ship code faster because devs like it. The current container fad pushes the burden of supporting buggy code to operations teams. IF the operations teams have issues they had better be highly skilled to figure them out

I have yet to see a company let requirements drive the choice to use k8s.

I don’t think kubernetes is nice because I’ve seen more than one small team throw a lot of resources at it and fail.

I’m not attacking docker, however, I can see why you would want containers. We still haven’t found an efficient usage for them at my place, but we never need to spin up more than one instance of our software. I think docker can sometimes be a way to cheat unsafe software around operations, but that’s more of an anti-pattern than an issue with docker.

Then again, I’m probably old and grumpy, but that sometimes has the advantage of not adopting techs before they are easy to use.

could you elaborate more about failure of kubernetes? I'm considering using docker/kubernetes in our production so to hear warning stories would be extremely helpful.

For me containerization was always about deterministic environments and ease of deployment instead of performance and clustering. But even with these advantages I am currently not using any solution for that.

For cloud services this is probably a good idea, even for users to a degree if the provider doesn't already give you a fitting box.

But otherwise it is not a must have in my opinion. Maybe that is a mistake and the apps I develop today are not going to work in 10 years. Well, worst case: I have to be paid again.

> For me containerization was always about deterministic environments and ease of deployment instead of performance and clustering. But even with these advantages I am currently not using any solution for that.

You can get 99% of the way using a stable distribution and a configuration management system (ansible, chef and the like). It's much much simpler than running an orchestration service. I feel most people don't need containers and orchestration, just config management running redundant system designs.

To be honest I only very recently got to know ansible and related techs so I maybe missing an opportunity to learn something. Even so I think you are forgetting the DEV part. With ansible and chef you can make a deployment to the real infra. With containers you can have infra locally in your DEV environment and have clean slates. The similitude of the DEV environment and the production are crucial for devops. There is nothing more annoying for developers than having something work locally and then needing some weird quirk for the production/ci. A lot of political infighting and hate for devops. I saw this being a tech lead for build system in a fortune 500 company. Ah they have redhat based distro. Ultra stable! Problem is nothing from outside the company works out of the box, leading to blessed machines. A disaster that lead to so much unofficial workarounds that it is not funny. Lol the kernel is so old it cannot run docker:) Ubuntu is better but ultra stable machines will tend to massive customizations that are very hard to keep when you finally want to upgrade. It was very common to reach End of Life of LTS distros, and then have the server upgrade being a nightmare due to the long evolution that happened in the mean time.

> With containers you can have infra locally in your DEV environment and have clean slates.

True, but you can do that with plain system containers such as with lxd, rather than having that bundled with the huge paradigm shift that Docker comes with.

My experience with lxd is very limited. Actually I worked with liblxc which is the underlying paradigm, and i kind of disagree with you. The paradigm of lxd is much more foreign to me than docker. I am pretty familiar with my application and the distro of the container in a user perspective. I am definitely very insecure about cgroups and kernel namespaces. In the end my application is connected with my business/work orders. Kernel minutiae is not and the technical skill requirements is much higher. That will put a higher price tag on my team's human resources.

> The paradigm of lxd is much more foreign to me than docker.

The paradigm of lxd is pretty much exactly the same as the paradigm of a regular distribution installed on bare metal or inside a VM. If you can operate a regularly installed distribution, then you can operate inside a lxd container. The commands to create and destroy lxd containers are trivial ("lxc launch ubuntu:bionic" for example).

> Kernel minutiae is not and the technical skill requirements is much higher.

I'm not sure why you think you need to know kernel minutiae, cgroups or kernel namespaces. Operating lxd needs none of that.

> I am pretty familiar with my application and the distro of the container in a user perspective.

That's all you need.

So, speaking as a dev, I feel like Docker's killer app is that it makes the config management a lot easier.

Dockerfiles give you a fairly easy and consistent way to express, "The runtime environment needs to have Python 3.5 and these packages," in a format that doesn't introduce too many concepts over and above the basic command line junk you'd use to manage your environment without Docker. If your stack requires multiple services, docker-compose gives you another fairly easy way to describe what all goes into that. And then it gives you a _super_ easy interface for starting and stopping all those services, keeping track of what you have running, all of that.

(And it's all fairly disposable, which is nice, since, as devs, we tend to break things. TBH, if Docker has done nothing else for me, it's that it's turned nuking PostgreSQL to get back to a clean install a 10 second process instead of a 30 minute one.)

It's not really that simple, and I've spent my fair share of time screaming at Docker for being flaky and having confusing under-documented configuration. (And I don't think I'd use it at all if I were working on a platform that weren't so annoyingly susceptible to systemic dependency hell. But worse is better, so the unix philosophy won, so here we are.) But eventually you get over that hump, and it starts feeling fairly easy to understand.

I don't know Chef, but I've seen Ansible used in production, and it just doesn't seem nearly so attractive. It could just be how it's being used, but it felt like there was this infinite regress of complexity where everything was tied to something else and you have to have been the person who built it to understand it, kind of like the bad old days when people were trying to put too much smarts into the database itself so they'd just become this rat's nest of triggers and whatnot. I'm sure it's not that bad. . . but my initial impression was that Docker is great for scratching a developer's itches, but slightly sucks for ops, but maybe is still worthwhile there if you're dealing with microservices or elastic scaling or something like that and you can use Kubernetes to smooth over some of the flakier bits. Ansible is much more for ops, and does a great job there, but I don't see it scratching many dev itches at all.

> So, speaking as a dev, I feel like Docker's killer app is that it makes the config management a lot easier.

It starts with a Dockerfile, which is a limited shell script, and it does not get any better beyond that. Shell scripts are simple, I'll give you that, but please don't sell them as some magic bullet. Dockerfiles are no configuration management system.

I've seen my fair share of hairy chef, puppet and ansible in the wild. Don't read into config management from those. I've also seen beautiful ansible installs, which deploy from dev setups all the way up to full infrastructure setup and deployment with blue-green deployment.

> Dockerfiles are no configuration management system.

Y'know, we might violently agree. That is a much more concise statement than my rambling attempt to explain why I think Docker is so much more palatable for development workflows.

You're right, it is no magic bullet. And I misspoke when I said "configuration management"; I forgot that that's a term of art in operations. By "management" I really just meant "stick it all in one or two files so I can get my checklist down to one step, and manage shared packages in a way that's at least a little bit less kludgey than simply abusing environment variables." So I find that it save some yak shaving, and for that I can deal with it under certain circumstances.

I actually hate using it for deployment or production config management, because IMO it seems to do a crap job at it. And it does a crap job at it precisely because of the features (or lack of features) that make it convenient for development. Even using it to manage our integration tests' runtime dependencies is kind of a hot mess. But I'm willing to concede that, together with Kubernetes, it might be nice for cloud-native elastic scaling microservice-y stuff, insofar as it seems to be popular for that. I don't actually know firsthand; I'm allergic to complexity, so try to avoid building things that way.

> For me containerization was always about deterministic environments and ease of deployment instead of performance and clusterin

This! I come from the embedded world with a bit of webui and having yocto for the embedded reproducibility and docker for the infrastructure, I am so happy. No more uncertainty when moving to another machine or upgrading my DEV machine. Nope Everything running right everywhere. Some scriptology and I had a full boot up from tftp for kernel and an ephemeral nfs from an ext4 master, and I could reliably make full system component tests all the way to web browser experience validation.I even had an autopilot simulator spawning ephemerally in a docker. Restarting the containers gave me a clean env again. Pure piece of mind and productivity. The initial investment was big though.

Am also an embedded dev. What toolchains do you use and how do you set them up using docker? Can you provide some sort of guide, because you already said. First investment is pretty high.

I use embedded Linux, not rtos chips. The tool chain is generated by yocto. Yocto is an embedded Linux from scratch distro creator. It creates images and environment for cross development. My connection with the docker was not so toolchain related, but system component test related. I modified the testing harnesses of yocto to start docker containers that serve a kernel zImage(over tftp) and a copy of the pristine ext4 image over nfs. With some program scriptology I made in python I was able to have serial boot log expects as well as boot time monitoring. I made a write up about it but it seems not many people dabble in such topics as SCTs for embedded devices[1]. I must say that embedded developing is a kind of a lonely experience,compared with for example the docker and web developer communities.

Regarding the investment it is an embedded industry problem. There is very little re-use. My experience is that we are an industry of wheel reinvention, where all of us are deep experts so we roll everything on our own. Maybe against myself I speak as I indeed developed this harnesses even though other solutions exist. My quip with what exists is that Intel absolutely owns, or used to own yocto and embedded tooling open source projects. This lead to very crappy code that was Intel specific being half merged into upstream. Mind you their featues are really not working,and were accepted because Intel is a gold yocto project sponsor. When you try to remove the broken code to a more sane one, you cannot because then the actual good rules of making small changes will bite you back and your changes will not be accepted. So you need to wait for Intel to cave in to remove their broken features. I digress sorry :). Even so Yocto project is a great step towards embedded projects productivity and knowledge re-use. https://www.reddit.com/r/embeddedlinux/comments/bk8a8k/yocto...

I'm currently using k8s in a staging/uat environment (running on a tiny 2 node cluster). So far, I've found it to be super useful in CI, as you can use container orchestration to emulate the larger production IaaS stack with much lower cost and time overhead than, say, a real terraform deployment. And if your team is already drinking the "cloud-native" kool-aid and wanting to use AWS lambda or a similar service, I'd argue it would be a much better investment to deploy those types of workloads as kubernetes jobs or with a framework like OpenFaaS[1] on kubernetes, giving the team much greater flexibility and avoiding vendor lock in.

https://www.openfaas.com/

Yeah, for my side projects I just use gitlab CI + docker compose.

Builds use the dind images on gitlab's runners to build an image and push to their container registry.

For deployments I have a host with a personal CI runner instance on Linode's smallest instance type which can access a user on the "production" host when SSHing over a private network, and has the docker-compose command allowed in the sudoers file. Then it can run docker-compose up to deploy. The key for this is passed to the job via gitlab's secrets UI so someone getting read access to either of my hosts wouldn't be able to do anything.

While people will rightly point out that this does mean the CI builder effectively has root on the "prod" host, for a side project it's enough for me. I might investigate podman/buildah some weekend when I have time as apparently that allows for rootless container launches.

I've used Docker Compose and Swarm for small-scale stuff too - it's really easy to work with, doesn't use much CPU for management (something k8s os/was notorious for - not sute if that's still valid?), the docs are pretty good, and there are a gazillion YAML templates on GitHub to use as a reference.

Compose and Swarm can take you pretty far, but TBH, it felt like Docker gave up on them years ago, even before k8s "won" the container orchestration war. A real shame :(

I don't need Kubernetes, no. But what do I use? What's there to automatically deploy my software when I push in a simple and reliable way? All the other stuff I can do myself, but I want something to deploy stuff to servers.

"git checkout X && git pull" ?

If I wanted to bring production down I'd just rm -rf / the database machines instead of using half-measures like that.

I'm just saying you don't need fancy CI-de-jour tech for a home project. If you're on the Atlassian stack Bamboo does pretty solid deployment/test/rollback. With a proper staging and QA environment even a large corp doesn't need much more than git.

I've never used Bamboo, so maybe that's the solution, but I disagree with your comment otherwise. No one should ever need to SSH to the servers to deploy things, and I would prefer to keep the hand-rolling to a minimum. Is there an OSS solution that will take care of that? Maybe Nomad would work.

>> I as a small developer and small server owner...

You'll never ever need either Docker or Kubernetes or even the latest and greatest javascript frameworks.

I started running a server before I knew anything and that server is still purring along happily.

But if you're ever targeting an enterprise, either as a freelancer or as an employee, those words are invaluable in your resume.

It of course helps if you actually know about those technologies!

And when you do get to learn them you'll wonder why everyone is coming full circle!

> You'll never ever need either Docker or Kubernetes or even the latest and greatest javascript frameworks.

I agree with "no need for k8s and the latest JS frameworks", but strongly disagree on not needing Docker. It is extremely useful for setting up separate development instances for your projects - no matter if you're doing PHP development with N different versions of PHP (as some sites may still be stuck at 5.6 while others are already requiring 7.2 due to Composer dependencies) or, worse, nodejs and Java where each project will have its own requirements for node, Tomcat and whatnot.

I personally set up one mega-container for each project which runs all the services required - mysql/pgsql for the database, apache as frontend / mod-php, if needed Tomcat - and can simply shut them down when I'm done working on a project instead of having the databases and servers all consuming memory and resources all the time.

>It is extremely useful for setting up separate development instances for your projects this. and not only the ability to have different software and versions for each project but also the ability to more or less match the production environment in your dev container for each project.

The point is to deploy immutable images that were clean rebuilt from scratch and tested prior to deployment, rather than upgrading an environment which becomes risky after some time. Another added value is automatic actions based on container watchers, traefik for example will self-configure on the fly when you just spawn a container with the hostname in a label. If deployment becomes that easy then why not leverage gitlab dynamic environment feature and deploy on $branchname.ci.example.com so that you can have say a product owner to review the development prior to merging to master (which would deploy to staging if you have an aggressive CD strategy - which I do)

When you break fast and move things, Docker is invaluable. I have more than one product and something like Docker makes it tractable for a single person to support.

Still, I run it all on one server using Docker so there is a medium :-)

How often are you spinning up new servers? In one to two hours, I can configure a production ready Debian server from base install with firewall rules, correct network interfaces, cron jobs, all dependencies and tooling, monitoring, Postgres, and my application server (either C epoll passed or Spring Boot) sitting behind an nginx proxy. I would consider myself neither a Sysadmin nor particularly fast at configuring Linux.

One to two hours per project multiplied by potentially one or two QA/UAT environments, and this is a tiny blip in the total time spent developing a solution. I think it would take me a few years before I paid back the debt of learning docker/kubernetes for it to actually start saving me time as a freelancer (or if multitenancy suddenly becomes out of the question).

At my job I regularly spin up ~1000 servers to test some workloads or do some data processing. Would be a pain to do that manually, and spinning up 1000 servers isn't significantly more expensive than 100 or 10 but it is significantly faster.

I actually run all my docker services on one 16-core server.

Great thing about Docker is a commit is all that's needed to reproduce everything since I also use terraform to deploy the infrastructure.

Docker and Kubernetes are on completely different layers. You can't do those comparisons.

Kubernetes is like an operating system, Docker is like a format for the executable files.

There is no problem with servers. We're running Kubernetes in production on a single Linux machine, essentially using Kubernetes as an alternative OS.

Kubernetes seems to be intended for large, complex environments but yours seems to not meet that category. What have you gained by using it?

(Not OP) I'm running a single node kubernetes cluster because it provides much needed isolation between services as well as, and that's the biggest part, a aingle and simple way to have everything in one place.

I can duplicate a service I'm running 1:1 with a new version for testing in 2 minutes, I can tear it down in 2 seconds. I can roll back changes in one command, I can wipe the server and reinstall everything from scratch in 25 minutes.

The environment is completely reproducible, and I can with a single command see every config that applies to a service.

All the usual deployment trouble is gone, no more weird setups and config situations. All the weirdness is nicely encapsulated.

This. Debugging k8s issues can be tricky, but you do get the above-mentioned benefits and in my opinion can make small teams very productive. It's also a great skill to have.

What did you use to create the single node "cluster"? I was using k8s for a similar purpose but had to create 3 nodes at the time (>2 years ago).

Sounds too good to be true. What challenges have you faced by using Kubernetes?

Like https://www.youtube.com/watch?v=b2F-DItXtZs but for devops.

Precisely. I'm reading the article and not grasping the technical point.

Mainly because the point is non-technical; while I'm not sure that Google is exactly a saintly organization, things like the Summer of Code and giving K8s away have procured some goodwill for them.

The whole article is over the head of Joe User (me) who is competent to use Docker but nowhere near deep enough to contribute code to it.

> Why a cluster?

For availability. If you have promised 99.9% or more availability to your customers you probably need some sort of redundancy or hot failover. Kubernetes is a good option to get that.

If you don't need 99.9%, simpler is better.

> Too cheap

You can get a single node Kubernetes cluster from Digital Ocean for $10 per month.

That's silly because a single node cluster doesn't have redundancy which is the main reason for Kubernetes, but it lets you get started. A minimum redundant cluster would have two nodes and a $15 load balancer, so $35 per month at DO.

Just use Docker.

Let others run to Kubernetes, it doesn't sound like the right tool for you. That's ok!

Why bother with containers at all at that scale then?

We keep on using plain old VMs, while watching everyone rush into containers fashion.

I like the tooling better and suspect that many other developers feel the same. Docker is more like managing and configuring software libraries and dependencies. You just declare what type of environment you want and it's there. If you change the version number of a dependency the old image is discarded and a fresh one is created. The Dockerfile is managed with the source code.

VirtualBox feels like installing a regular computer. It takes a long time and is a lot of manual work. If you want to change something you login on the existing VM until you reach a point where you no longer remember all the changes you've made over the years. The machine is unclean.

I realize that there are solutions out there for automating VM deployments but Docker did a good job of catering, and perhaps marketing, to developers.

> VirtualBox feels like installing a regular computer. It takes a long time and is a lot of manual work. If you want to change something you login on the existing VM until you reach a point where you no longer remember all the changes you've made over the years. The machine is unclean.

Unfair comparison. People running on VMs usually rely on configuration management to do the install+config part. Think of stuff like Ansible as a Dockerfile for VMs/bare-metal.

Config management isn't a magic bullet. At the core every config management platform is manual work in a for loop. Lot's of layers in the case of Salt/Puppet to make it a bit more ergonomic but you're ultimately still on the hook for all the server maintenance. No config management is all-encompassing and so without extreme diligence you will lose the state of your servers over time.

I use Ansible all day every day -- it's not comparable container tooling in the slightest.

Containers aren't a magic bullet either. Surely the core of container systems is also "manual work in a for loop". You're right that the default 'ephemerality' of containers is (or can be) a plus but config management tools don't require one to treat one's servers as 'pets'.

> No config management is all-encompassing and so without extreme diligence you will lose the state of your servers over time.

I'm not sure exactly what you mean by "all-encompassing". Surely all of the popular config management systems have 'escape hatches' to let you do anything you could do from a shell, which seems pretty "all-encompassing" to me. But if you mean the config management system don't completely enforce a specific 'state' for managed servers then of course you're correct. Containers don't do that either, beyond periodically replacing running containers with new ones created from a base image (and even that's almost certainly not perfect either).

Unless your container hosts are entirely managed by someone else (and probably even not then, in the fullness of time), you're always going to "lose the state of your servers over time". There's (almost always?) some state somewhere that has to be explicitly managed and thus requires, generally, "extreme diligence".

I do agree that containerization and config management are very different but, like everything, it's 'just' another set of tradeoffs to be made, hopefully depending on one's actual or expected needs and wants.

> I use Ansible all day every day -- it's not comparable container tooling in the slightest.

Only if you are not actually cleanly managing containers. With containers you just pile the shit into them, close the doors and say "Oh look, we have a clean surface!" which is certainly fine in the dev.

If it is fine in production, then it does not matter if the shit is in containers, VMs, dedicated servers, etc. Container may as well be curl http://mylservice/containername.img followed by dd if=containername.img of=/dev/sdb ; set-boot-flag-sdb ; reboot

Docker manages dependencies and libraries as well as whoever created the docker images manages those dependencies and libraries.

Most of developers that are suddenly becoming more productive with docker do equivalent of a full install of an distribution that they run in production, call it a container and get the claps from management because "We managed to do it faster". Never mind that the surface for brokenness is now even higher than the surface for brokenness on a VM that the container is going to be ran on had the VM got the full install of of the distribution.

> VirtualBox feels like installing a regular computer. It takes a long time and is a lot of manual work. If you want to change something you login on the existing VM until you reach a point where you no longer remember all the changes you've made over the years. The machine is unclean.

That's because in this approach no one bothered to do equivalent of what one does when creating a docker image - write chef/puppet/salt/ansible baseline configuration so the new VM is nothing other than "base-VM + special config for this function".

Might be, on large organisations we just make out an order and get our VM.

I have more experience with VMware and Hyper-V, as type 1 hypervisors, VirtualBox always felt a bit underpowered.

Containers are just groups of processes, essentially a chroot that isn't limited to the filesystem; I find them much simpler than full VMs, which are overkill for a single server.

It's not like containers are new technology, even on Linux; we were using OpenVZ a decade ago. Now they're just integrated into the mainline.

Sure, after all I used HP-UX vaults back in the day, however in the context of Java and .NET application servers they hardly bring anything new.

And we can use native containers if we actually need them, so I see such tooling more as yet another consulting wave.

Deployment and transfer of environments between people.

Using docker images means that we can take the thing that developer A built and that "runs on their machine" (without necessarily having that developer on hand) and have developer B easily launch it on their machine without having any dependency or isolation or reproducibility issues, and after making some trivial changes put it in a new, fresh production server without having to ask the original dev how it was/should be configured.

We do that with VM images, WAR/EAR, RPM, NuGET, MSI packages.

You don't need a whole cluster to get started, you can use Minikube locally for example : https://minikube.sigs.k8s.io/docs/

If uptime is not a concern and only one node is intended, docker-compose is a far more efficient solution to any k8s solution.

It might be more efficient but it's definitely not a solution to get up to speed with k8s

I'm sorry, I somehow missed that the top-level commenter was interested in the one-node k8s to get started in it.

Have they solved the high CPU usage problem while it is running (even when no containers are inside)? I don't like my laptop hot, so never could make a switch.

They haven't. As an example of where this lack of efficiency comes from, recently I've fixed an issue during which I've discovered that the Etcd component of k8s uses periodic (10 sec) liveness checking, where for every check (launching of the etcdctl client), the `runc` binary (from container.d) is executed 3 (three) times. You can imagine this probably just scratches the surface.

Ugh! Was the fix committed upstream?

Yes, this one was a systemd detection in runc, which was causing big log flooding and runtime overhead due to runc being executed so frequently. I initially cached the checks, but later the devs removed them altogether. Still, this is not solving the inefficiencies in k8s. https://github.com/kubernetes/kubernetes/issues/76531

Yes, it's a real issue. Kubernetes burns too much RAM and CPU even with no workload.

Docker on macOS also suffers from that issue though.

It will use more on MacOS and Windows than on Linux, as it needs to use a sidecar Linux "Moby" VM to run containers. In practice on Windows, this hasn't been an issue for me (64GB RAM in my dev laptop).

This will actually finally change soon'ish with Windows at least, I think as part of WSL2. Not sure if anything is going to change on MacOS.

Not for me, except for the RAM blocking - I wish that Hyperkit could dynamically allocate and release RAM for the guest machine instead of blocking 2GB (in my case) by default all the time.

I haven't noticed that.

--vm-driver=none helps a lot with the CPU and especially RAM usage, using k3s instead of minikube helps more.

We've had the same problems - we didn't need all the overhead of kubernetes. Have you seen BeeKube? https://beekube.cloud It is a managed container platform, similar to a paas but with containers

Docker is more than enough for most of the setups TBH, prometheus and what will work just as well. I would never recommend K8 locally, if you can even avoid docker it's a win.

It's just a matter of preferences, where you want to invest your time to learn etc... Kubernetes being the fully fledged "state-of-the-art"

I would just recommend the work of Stefan Prodan who's repository and blog are full of open and well thought devops work, even if you are just interested in K8 or only Docker

You can run a single-node cluster. Not the best option and overkill if you don't have the need for redundancy, but it might save you some time deploying software. Honestly at that point you might be better running your software using docker-compose or something like that.

For development, microk8s is actually very nice: https://microk8s.io

I feel until you have atleast a clusterful of services, you shouldn't add that much complexity to your applications. It's much easier to manage a single tower for as long as possible, and slowly migrate to a new one when it seems ripe

I was looking at https://k3s.io/ as a better configuration management solution than just pure-docker. Because I like kubernetes, I used it, I know it.

Dabble with LXC on Linux