The reality is that the WHATWG (a) only writes descriptive standards, describing what already exists, usually with pseudocode and prosa instead of ABNF or EBNF (see the URL standard replacement), and (b) only describes something once it’s actually been implemented on larger scale.
On the topic of what standards are supposed to do – prescriptively shape and replace what exists – the WHATWG isn’t useful. WHATWG "standards" are the equivalent of Microsoft Office Open XML, a standards body just taking an existing implementation, defining whatever it does as standard, and doing it so incomplete that the result is useless.
Yes, WHATWG and W3C are doing the best they can do in the current climate (where Google can roll out QUIC and SPDY before even any standard is defined across websites accounting for 6% of global traffic, 65%+ of web browsers, and 85%+ of mobile phones), but this is just misleading. It helps no one to pretend to do standardization work when you don’t actually have any power to decide anything – neither WHATWG nor W3C can actually force, or even ask, Google to change SPDY or QUIC. They’re papertigers.
I work for Chrome, as an editor of the HTML Standard. So let me give you my perspective on this.
In Chrome we ensure that all features we ship to the web go through a public standards process. This allows them to be developed by a collaborative community, including other browser vendors and web developers who would use them. It ensures that if we happen to ship a feature sooner than other vendors, there's a specification and a shared test suite (https://github.com/w3c/web-platform-tests) that allow others to quickly follow. Note that a specification is better than requiring them to read the Chromium source, because specifications are at a higher level that doesn't depend on individual browser architecture details.
In the WHATWG we don't only write descriptive standards. But we do ensure that whatever standards we write, are ones browsers are willing to implement. And we ensure that standards accurately describe how browsers operate, even for legacy features, because that is all part of the mission of allowing browsers to compete on an even playing field and build themselves from scratch without having to go through the kind of costly reverse-engineering that Firefox 1.0 did to catch up to IE6. In practice we've found that algorithmic specs are better for this than BNFs, as it's harder to specify error-handling behavior for BNFs while still staying compatible with the web (i.e. while still producing a standard browsers are willing to ship).
And yes, we're not interested in just creating a standard out of thin air, with no vendor collaboration, calling it "standard", and then hoping some magical power would force browsers to implement. It is indeed much more collaborative than that.
But the fact that we require standards to be developed in tandem with implementations doesn't mean that implementations (such as Chrome) just go ahead and do whatever they want, and we at the WHATWG transcribe it into the spec at some lower level of detail. Instead, the public, collaborative standards process helps to extract out all testable and observable aspects of the feature into a codebase-agnostic description others can use, and provides a forum for them to comment on ideas before any final shipping decisions are made. And, per our working mode (https://whatwg.org/working-mode#changes), changes and additions do require multi-implementer support before they're ready to graduate to a WHATWG Living Standard; proposals not yet at that point are said to be in incubation, and are often developed elsewhere (see https://whatwg.org/working-mode#new-proposals) such as the W3C's WICG.
> In Chrome we ensure that all features we ship to the web go through a public standards process.
Ehm, basically every major feature Chrome has shipped has been shipped before the standard was even discussed. SPDY shipped long before HTTP/2 was even finalized, and QUIC is doing the same. NaCl shipped in the same way, without any standardization, and to this date, earth.google.com depends on it.
In general, your problem is that you only consider browser developers. In the past, the WHATWG has decided to redefine the URL standard, then shame cURL for not following the standard, without ever involving anyone from the curl project in the discussion. The URL discussion affects everything from Android’s IPC system to curl, from industrial machinery to the web. The WHATWG explicitly declared that the URL spec is designed to completely, and exhaustively, obsolete and deprecate any existing URL or URI spec.
Yet, the only people ever contacted about this, and who were given the ability to take part in the discussion, were representatives from the three large browser vendors.
Those are fair counterexamples. Perhaps I should be distinguishing between the Blink team and the rest of the Chrome team. I realize that distinction isn't very important to an outsider, but at least realize that that there's a large portion of the Chrome team that cares very much about the web evolving through an open standards process.
The URL Standard was designed in the open with input from many different constituencies. The cURL author has chosen not to participate, for reasons of his own, but e.g. Node.js, PHP, Google's GURL (used by Android IPC, I believe), and others are quite involved.
The URL Standard didn’t even consider contacting any industrial vendor that relies on them - e.g. SIEMENS. There’s entire industries out there that use these standards, and rely on them to be stable.
The only participants were all either browsers, affiliated with browsers, or a handful of web serving projects.
Other projects that rely on URLs include everything from KDE to Gnome, Microsoft’s OS to the systems used in your car.
Changing a URL standard and only involving web vendors is basically like changing the A4 paper standard and only talking to the Microsoft Office team, the Google Docs team, and HP’s printer team – while entirely ignoring paper manufacturers, envelope manufacturers, the mail companies around the world that will have to ship the envelopes, fax manufacturers that have to build faxes able to fax the new format, newspapers and magazines that have to replace their paper, newspaper shelf manufacturers that build newspaper shelves for newspaper stores, etc.
Most of the time, it’s easy to only think of the web as browsers and servers, but some of the specs the WHATWG touches go through entire industries, sometimes there are millions of companies that have to be notified months or years beforehand to replace their software, update it, potentially even do a recall, and standardize. Not everything moves as fast as the web.
And this entirely disregards the people that are trying to parse the web with HTML parsing, which everyone loves to ignore. And so many other groups of people and companies.
https://github.com/whatwg/url/issues/118 is the curl issue, I believe. As I love curl more than any browser I personally side with it. It has support for ldap urls; curl wins. :)
The actual HTML specification which browsers follow is maintained by WHATWG at:
https://html.spec.whatwg.org/multipage/