You just described the actual draw of CoreOS. CoreOS is just Linux. Instead of going the Xen route you just described, it can capitalize on being just Linux by supporting everything Linux already supports. At this point, you can view CoreOS, which is just Linux, as the "hypervisor." Containers don't have to replicate key subsystems in every paravirtualized instance, as containers share all the same subsystems like e.g. the TCP/IP stack. You're also not throwing away any distribution experience. Want CentOS? Create a CentOS container with Docker. Want Ubuntu? Create an Ubuntu container with Docker. These are incredibly powerful ideas I'm just now being exposed to.
Try telling your SAN vendor that you're running "just Linux" and can you have a driver and support package for that please, and let us know how you get on.
When you're done there, now try plugging your nice new machine into the corporate DC. Oh it seems it needs to support VLAN tagging. No problem, better just write some new code for that.
Time to migrate a bunch of performance sensitive services. Uh oh, no support for FusionIO PCI SSDs! Better write some more code.
Time to migrate your remote sites, only policy dictates certain services must be physically encrypted. No problem, better just cutpaste Debian's cryptsetup scripts and be done with it.
Oops, turns out we deployed 1000 machines with a duff BIOS setting. No problem, I'm sure the server vendor has a support package for CoreOS..
We could come up with examples until we've basically reinvented a modern Redhat/Debian initramdisk and boot environment.
There are certainly environments where enterprisey vendor testing and certification makes sense; mostly small or heterogeneous environments. But at large scale it's cheaper to do stuff yourself.
Also, CoreOS has more of a read-only stateless philosophy, so I think it'd still be interesting even if it ends up having to duplicate a lot of effort from mainstream distros.
That would have been a problem with every new distribution, including RHEL and Debian when they were brand new. In reality, your SAN should be able to accommodate a generic Linux kernel with any userland you can containerize.
Yes, and ideally Microsoft Windows should be open source and IBM OS/360 would have a build supporting Firefox running on my Nexus One, but that's not how the world operates..
Your sarcasm is tedious. If a SAN cannot support a generic Linux kernel, then it's the fault of the vendor for not allocating enough dev hours to the Linux kernel, and their product should not be chosen for a Linux environment.
I'm not sure how else to respond to "let's reinvent a complete Linux distribution because we've written a 20 line auto-updater".
This would have made sense to me a decade ago, but back then I just wouldn't have understood the amount of work involved. You simply cannot produce a fully generalized container OS without producing a fully generalized OS.
That aside, I actually like the central idea of a better approach to managing/updating a large group of host machines. I just don't think it warrants yet another bikeshedded support/security/certifiability nightmare (and lets face it, this is bikeshedding, there's absolutely no value in the core claim of "it's just Linux", but it gives unfettered license to reinvent the same old wheels all over yet again).
Just wanted to add something to this thread, which has not been mentioned yet. Sure, most (if not all) SANs are supported vie generic linux kernel drivers for HBAs, something like a QLogic card, or an exported iSCSI LUN. The issue is that you spent tons of money on the SAN and a truck load of money on hardware and software to hook up to that SAN. So you want to have a vender supported OS to go along with these items. If you have performance issues or stability issues, the first things a vender will ask is, what OS are you running, so they can direct you to the correct support group. The support group will want to debug the issue, so you have to generally run something like RHEL or Windows to get support. This might only be a mega corp issue though, since most startups don't have Oracle/Sybase, fiber channel SANs, etc. To sum this up, in the mega corp world, if you are not running a vender supported OS, you likely cannot get support for your device.
