I don't understand why people are so fork-a-phobic, and anti-patching these days. Distributions (like Red Hat and SUSE, but also Debian, Ubuntu, Fedora, openSUSE, etc) have been doing this for decades. Forking a project is something that is unique to the free software community, and we're doing ourselves a disservice by not taking advantage of this freedom. Forking a fast-paced project like k8s is fairly ambitious, as you've said, but that doesn't make it a bad idea from the outset.
> If RedHat doesn't have the clout to get their improvements in upstream, why should I presume they have the clout to avoid their additions won't be broken by other changes?
That's not how free software development or maintenance work. Believe it or not, the engineers at Red Hat (or SUSE, Canonical, etc) are actually pretty clever. An upstream not accepting a change can be for any number of reasons unrelated to the technical aspects of the patch itself. It could be a conflict with their roadmap or scope, it could break something else they're working on that is of higher priority, it could require more discussion on whether the use-cases can be solved by existing features, it could require further research into whether the proposed feature is the best way of solving the problem, etc. I've seen all of those reasons (and more) for some of my changes not being merged upstream (and I also maintain some upstream projects, so I've used those reasons before too). Not to mention that usually "no" in an upstream review means "not yet, I'm still thinking about it".
If an upstream rejects a patch, but a customer needs the patch in order for them to be able to effectively use the project, then Red Hat (or SUSE, Canonical, etc) are entirely within their rights to add that patch to the packages they ship. And that's the correct thing to do. Upstreams generally are not good at release engineering, so in order for hotfixes a distribution would have to patch the project anyway. What makes a feature patch any different? Not to mention that Red Hat (or SUSE, Canonical, etc) also provides documentation on how to migrate to the upstream feature (if the upstream feature ends up being different).
Kernel development has worked this way for more than 25 years, with distributions carrying patches that eventually get pushed upstream asynchronously (usually with some improvements through discussions that make them more generic for all kernel users). While stable kernels have made the massive patchsets much less of a burden to maintain, this model still is in practice today.
I'm against forks like OpenShift because as an upstream maintainer on a major open source project, distribution patches caused us nothing but headaches. Their well meaning patches almost always caused problems. Users routinely ended up at our doorstep with the issues they caused. We then either told the user to go bug their distribution, spent time digging into the issue, or got lucky and the distribution maintainers actually paid attention to our lists. The latter one was actually pretty rare.
You presume I don't have experience with open source projects and don't understand why things might not be accepted, which isn't really true. I used clout as short hand for doing the work to actually get changes into upstream. I'll admit I probably chose a poor word there.
I really can't fathom why Routes wasn't just adopted into Kubernetes directly instead of a complete rewrite of Ingress being added. I don't know all the details, but when I went digging what I found was RedHat folks explaining what they did and Google engineers writing Ingress.
Please also understand, I totally agree that RedHat and other packagers are fully within their rights to patch things. I think they really shouldn't. They cause at least as many problems as they solve in my experience. But it's also my right to say that I don't want to use their patched stuff.
I don't see the situation with the Linux Kernel as a success story. I see it as a failure. It's downright impossible to tell someone if something is going to work with their kernel because the version numbers are utterly meaningless since the distributions patch all sorts of things in and out. I have been running Linus' mainline kernels for the last several years and I've been broken exactly once and even then only very slightly.
I tend to think that if distributions avoided patching unless absolutely necessary and worked with upstream to get things included first we'd all be a lot better off. Those reasons why the patches weren't accepted quicker would get dealt with before things were in the hands of users.
But frankly the distributions incentive is to create value for themselves. Not help the project along. Helping the project is utterly secondary to any value creation they are doing for themselves. In fact you give an excellent example. You say that upstream is terrible at release engineering. So rather than just applying a patch to a distribution, it's really beyond me why distros don't work with the upstreams to improve their release processes if that's the problem with staying with a pure upstream.
That's not to say that distros don't create any value for the overall community. It's just my opinion that they don't create as much value as I think they should. These companies are taking in massive amounts of money off the open source projects. Sure, in some cases they have maintainers/contributors to upstream projects on their staff. But those are usually the cases where what I'm talking about isn't what is happening.
Now all of that sounds like I don't think distros should ever patch. Which is probably an exageration of my position. I think there are times when it's needed. Security fixes, unresponsive upstreams, etc...
But adding completely distinct functionality. I don't want to touch that with a 10 foot pole.
Edit: Forgot to say, your comment about the migration to upstream feature bit. I flat out asked RedHat how they planned to get people to migrate to Ingress. Their answer was they didn't have a plan.
Is in 3.6, and other improvements will continue to be added. The one downside is that you have to grant the router proxies access to secrets, which means if someone compromises your edge ingress controller they can root your cluster unless you are very careful about only giving the routers access to exactly the secrets they need. That's partially why Routes contain their own secrets, so that you can't accidentally expose yourself to a cluster root.
This sort of stuff is the details the OpenShift team spends most of its time on. Kube will eventually get most of this. But most people are running single tenant Kube clusters and so in Kube we spend more time focusing on making that work just right. It's pretty difficult to build a fully multitenant Kube setup without making choices that we're just not ready to do in Kube yet.
> If RedHat doesn't have the clout to get their improvements in upstream, why should I presume they have the clout to avoid their additions won't be broken by other changes?
That's not how free software development or maintenance work. Believe it or not, the engineers at Red Hat (or SUSE, Canonical, etc) are actually pretty clever. An upstream not accepting a change can be for any number of reasons unrelated to the technical aspects of the patch itself. It could be a conflict with their roadmap or scope, it could break something else they're working on that is of higher priority, it could require more discussion on whether the use-cases can be solved by existing features, it could require further research into whether the proposed feature is the best way of solving the problem, etc. I've seen all of those reasons (and more) for some of my changes not being merged upstream (and I also maintain some upstream projects, so I've used those reasons before too). Not to mention that usually "no" in an upstream review means "not yet, I'm still thinking about it".
If an upstream rejects a patch, but a customer needs the patch in order for them to be able to effectively use the project, then Red Hat (or SUSE, Canonical, etc) are entirely within their rights to add that patch to the packages they ship. And that's the correct thing to do. Upstreams generally are not good at release engineering, so in order for hotfixes a distribution would have to patch the project anyway. What makes a feature patch any different? Not to mention that Red Hat (or SUSE, Canonical, etc) also provides documentation on how to migrate to the upstream feature (if the upstream feature ends up being different).
Kernel development has worked this way for more than 25 years, with distributions carrying patches that eventually get pushed upstream asynchronously (usually with some improvements through discussions that make them more generic for all kernel users). While stable kernels have made the massive patchsets much less of a burden to maintain, this model still is in practice today.
[I work for SUSE.]