Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm not sure what's anachronistic about efficient use of disk space, but in any case that's nothing that can't be solved via hardlink deduplication.


> I'm not sure what's anachronistic about efficient use of disk space

Consider this: A frequently used module in one of my trees is "glob". It's repeated four times in two different versions. It takes 207K each time, including subdependencies. The wasted space is 414K... more than a whole floppy! ;-)

Or to put it another way, I've wasted 0.00017% of my rather small 250GB SSD.

Optimizing that is anachronistic.

> that's nothing that can't be solved via hardlink deduplication

Yep, and `npm dedupe` [1] does something similar. This does have the potential to become massively complex, though. You have to re-dupe when one dependency upgrades but another doesn't, and you also need to deal with the fact that two modules' dependencies may be sharing memory space that a module author was expecting to have to herself. (Modules are cached, so changes to "module global" variables are shared, but modules loaded from different locations are cached independently.)

[1] https://www.npmjs.org/doc/cli/npm-dedupe.html


> Optimizing that is anachronistic.

On the other hand, my Java .m2 directory counts 2469 jar files for a total of 2.4G. My considerably smaller .cabal is still 253MB. I won't complain if hardlinking prevents package repo sizes from going out of hands.

> Yep, and `npm dedupe` [1] does something similar.

Not quite. From what I understand, it attempts to manipulate the package hierarchy. On the other hand, hard link deduplication doesn't need to do any such thing, it simply needs to hardlink files with the same checksum. No intelligence required. Some time ago, I used rsync to do this kind of thing when deploying large binary dependencies, and it is very practical (IMHO, hard links get too little love).


Fair enough.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: