I'm using npm / browserify etc in anger for the first time today. This is a horrible issue to have run into and it's left a pretty sour taste.
The fact that it's possible for someone to unpublish 17 lines of js and break the install of major bits of infrastructure for everybody is pretty insane. It seems like at a minimum the dependency tree should be traversed to see what the flow on effect will be. Should it even be possible to revoke an old version that's depended on by other libs?
These few lines of js cost me a couple of hours tonight! :-/
I don't want to sound like a old grumpy man but here goes anyhow. I was looking into using node.js, react, etc after many years of writing web apps using Python and Quixote (obscure web framework like Flask). The whole Javascript technology stack looks pretty insane of me. Getting a working React environment requires a huge number of packages to be pulled down by npm. Browserify requires a bunch more. Recursive dependency resolution is nice and all but isn't this going to create a massive technological debt that needs to be maintained? Semver is not a magic bullet.
Also, seems like a security disaster waiting to happen (I don't use CDN because I like to make sure I review code before putting it on my important websites). Linux distributions like Debian have put in huge effort into making a packaging system that is secure and doesn't lead to dependency hell. You can argue how successful they have been but, and maybe I'm ignorant, I don't see the same effort and level of maturity in the Javascript/npm ecosystem.
As an alternative to React, I'm looking at Mithril. It's relatively small and has no dependencies. I'm quite a bit more comfortable building on that foundation as opposed to a giant house of cards.
> Recursive dependency resolution is nice and all but isn't this going to create a massive technological debt that needs to be maintained?
Spot on. Imagine deploying an application, in 2018, that pulls down 1000 libraries, 300 of which are 6 years old versions and contain vulnerabilities (or just bugs) involving data on transit.
Who is going to do all the work to backport fixes in every affected version of each library?
If things continue on the current path, I would be surprised if a typical 2018 node/NPM-based application only pulled in 1000 libraries (transitively). I just checked an application under development here:
$ ls node_modules | wc -l
517
Then I checked the current Angular 2 repository:
$ ls node_modules | wc -l
804
(Kudos to NPM 3 for finally flattening node_modules; in addition to reducing duplication and making life less miserable on Windows, it is now much more obvious how the transitive dependencies explode.)
Given the ongoing trend toward each dependency having more dependencies of its own, 1000 doesn't sound like much of a stretch. How many of those 1000+ will be up to date and lacking in critical security or functionality bugs? It sure won't be 100%.
It makes me look longingly at languages which ship with a reasonable standard library.
What dependencies are required? I would have thought that for basic, no-frills use, you could just load react.js and use it with no other dependencies. Are you referring to the tooling setup required for JSX? I've never used react, just curious.
react_demo$ du -sh node_modules/
25M node_modules/
react_demo$ ls node_modules/|wc -l
79
react_demo$ cat $(find node_modules/ -type f ) | wc -l
287767
I think I installed react, react-dom, browserify (globally), babel. The exact details are not really important, being inexperienced with react I probably installed stuff that is not strictly necessary. However, the ease of pulling in a huge amount of dependent code makes me concerned. For a long lived stable application you will have to maintain those dependencies (at a minimum, have 3rd party maintainers who you can trust to maintain it, not break your app, not inject security issues into your app, etc).
Have had the same experience tonight - browserify etc 12 modules, 75MB, circa 12,000 files. I guess it's just the npm way but it's definitely a bit shocking for those of us who are unfamiliar with it.
But are those strictly react dependencies? Seems more like a demo that was doing a lot more than just using react. Eg, browserify is mentioned, but that's a totally separate project from react, completely orthogonal. So inexperience seems like a big factor here..
It's the ecosystem that's being complained about here it seems, rather than react specifically.
Javascript has an almost non-existent standard lib. Obviously Python ships with most of the kitchen sink in core. You're not grumpy, but you're misguided.
A lot of people like to think that software fragmentation comes without any drawback.
Having many tiny libraries and/or multiple versions hurts in the long term.
"I don't want to sound like a old grumpy man". Hmm you failed :). On a serious note you might have noticed that there are 2 major camps of node users those who came from web frontend devs & ones adapting node at large scale enterprises/projects
(paypal, walmart, netflix, joyent) so as you might guess there are solutions to address situations like this (e.g. run your own registry etc).
> Recursive dependency resolution is nice and all but isn't this going to create a massive technological debt that needs to be maintained?
Keeping each package small and self-contained reduces the technical debt. If you depend on a monolithic library like Django, an update can be months of work with impact all over your code. If you depend on the same amount of code, but split into 100 smaller libraries, everything becomes much easier to maintain.
> Semver is not a magic bullet
It's pretty close. Having an ecosystem where everyone follows it religiously because all the tooling is built around it is really nice.
> Also, seems like a security disaster waiting to happen (I don't use CDN because I like to make sure I review code before putting it on my important websites).
It doesn't have to be. Reviewing all the code you depend on is a fool's errand IMO, but if you want to do it then NPM makes it very easy.
> Linux distributions like Debian have put in huge effort into making a packaging system that is secure and doesn't lead to dependency hell. You can argue how successful they have been but, and maybe I'm ignorant, I don't see the same effort and level of maturity in the Javascript/npm ecosystem.
The node ecosystem has been very good at avoiding dependency hell. It's well designed. They could stand to learn from Debian on the security front though.
Should one not be able to unpublish? What if there's a bug in a released version, and it's going to take a while to fix? Or what if you simply no longer wish to be associated with the organization anymore?
I don't think it's relevant - if there are other libraries depending on it, the code probably needs to stay. Maybe there's a bug but it's been worked around. The library has been up long enough for others to depend on it and publish their own code using it.
You could have a "soft" unpublish so the module is still available for any libraries that depend on that specific version but it won't be picked up in the "greater than" version case.
In terms of the owner wanting to have it taken down, there should probably be a licence flag that hands control over to the host so at the very least an audit can be carried out before removal.
EDIT some discussion rather than downvotes would be nice. In this case we have a library with a totally permissive license, used by screeds of other libraries, with 2.4M downloads a month. The pragmatist in me says that there should be some mechanism by which pushing a button shouldn't screw up everyone's day. Let's discuss potential solutions.
Rust/Cargo solves this with a `yank` option. Cargo will no longer automatically switch to that version, but if you already use it, nothing changes (though there might be a warning or something)
> if you simply no longer wish to be associated
> with the organization anymore?
Given that it's open source, (specifically WTFPL in this case), that's not something you can actually. Or rather, it _is_ okay for npm to republish stuff: that's part of the license.
> These few lines of js cost me a couple of hours tonight! :-/
Every dependency has some costs and some risks stemming from the need to trust in another person. If you're only using 17 lines of code, those costs dwarf the cost of code maintenance.
Though, there are probably ways to improve the package system such that less trust is necessary. Or even just ways to use the existing system better.
The fact that it's possible for someone to unpublish 17 lines of js and break the install of major bits of infrastructure for everybody is pretty insane. It seems like at a minimum the dependency tree should be traversed to see what the flow on effect will be. Should it even be possible to revoke an old version that's depended on by other libs?
These few lines of js cost me a couple of hours tonight! :-/