> Those 2 libraries are the closest thing to the modern internet equivalent of the "Library of Alexandria". I hope they don't manage to burn in down.
This just reminds of how in the late nineties and early 2000s we used to have torrent sites for music enthusiasts with an archive of all the different versions, editions, bootlegs, all painstakingly added and catalogued, representing tens if not hundreds of man-hours worth of work by people who simply love music and wanted to share that love with other enthusiasts, creating some of the richest catalogues of music and music metadata the world has ever seen. And then it all was destroyed.
And this was true for many other topics of interest too.
We have burned many digital libraries in the last few decades, and we will probably continue to do so for a while longer.
More like hundreds of man-years, when you factor in all the editions, bootlegs etc that are now forever lost to time.
This should be a golden age for information preservation. Instead it's a dark age. I personally know several eye-wateringly good bands whose catalogs are not available anywhere, at all.
Given that those were torrent sites, they weren’t hosting the content. The individuals seeding the content still likely have copies of it. They just have no easy, obvious place to share it where they won’t get in trouble for doing so... for now. But if such a technology shows up later, the (already ripped and preserved) content might just show back up along with it.
This is a good point; it's amazing what's still out there. Bitrot does set in over time, though, and the less distributed a work is the worse its odds of living through each new tech transition. I've known people who still have cassette recordings of concerts that are publicly considered "lost". Hopefully they digitize and back up, but not all of them will.
Even the brief bloom of the torrent and sharing scene was incredible, though. Scarce works that circulate manually are in near-constant jeopardy, because every new copy takes new effort. But a whole lot of unique content, from jam-band tapes to obscure translated books, just showed up on torrent sites and got shared widely. Even if it's inaccessible at the moment, that stretch may have spawned enough copies to endure across the next few transitions.
> Bitrot does set in over time, though, and the less distributed a work is the worse its odds of living through each new tech transition.
Eh, I feel like we've almost fully gotten away from bit-rot. Bit-rot was a problem because individuals don't bother to go to the effort of constant IT administration, and don't have the economies of scale required to afford good data redundancy.
But individuals don't have to back things up "personally" any more. With one command, I can send three copies of any file to three object-storage providers. Each one (according to the Dynamo architecture, which they mostly all implement) is holding 17 copies of the data on 17 shards, and replaces copies from the good ones whenever they go bad. Each provider's copies only exist in one region, but the region is different for each provider selected. As individual object-storage providers die, I can find new ones and sync my data over from the surviving ones. As long as "cloud [redundant] object storage" as a concept exists, I won't have to do much at all to ensure continued integrity of my data. All the details—including porting my data to new physical substrates when old formats die—are being handled under the abstraction, and converted into (tiny!) monthly fees that are lower than what even a zero-redundancy tape library would cost me.
And, of course, the data I'm sending is encrypted, and not even with a symmetric key, but rather a PKI key where the decryption half of the key is held in cold storage (i.e. in hardcopy base64 in a safety deposit box at a bank; and a few other places.) If I'm uploading rips of something copyrighted, nobody will ever know that but me. I'll be able to get the data back out when the time comes to dust it off and share it.
In the HD movie torrent community – people sharing full Blu-ray images – a person’s collection can run into the many terabytes. My own collection amounts to 8TB and is only content I have watched personally and am interested in, while torrent communities greatly benefit from obsessive hoarders who download and seed content beyond their personal interests. There is no economical solution to back up that much data on the cloud, and bitrot does set in.
If you're not spending at least as much money (+ labor opportunity-cost) on your own data-storage solution as it costs to back the data up to the cloud, your data isn't safe.
The /r/datahoarder subreddit is full of people like this. I'm happy they exist, if nothing else so at least there may be a digital record of our civilization if it's destroyed.
"Data hoarding" along these lines is, IMO, an admirable use of time and money, and I wish that I had more of both so I could participate. One day I hope to.
Is there any technical reason we can't make a distributed tracker/community like what.cd, given that torrents are already distributed and ZeroNet et al exist? All we'd need is the forums and ratio bookkeeping.
Sure. Mooi were formed in 1999, split in 2006, made a bunch of great music in between, and all that remains is this one song on YouTube: https://www.youtube.com/watch?v=gF5cP8S0zOk
The masters still exist, I'm told, but that's no good to anyone - they won't be re-released because it would require too much intellectual property wrangling and nobody cares enough to bother, and sooner or later that hard drive will die and that will be that.
Did they ever release their music on CDs? If so, surely some of those CDs are still out there somewhere and could be ripped, if only they could be tracked down. I guess that just hasn't happened yet though.
"Formed in 1999, Mooi are duo Rachel Lloyd and Laura Dickinson. The pair write heartfelt and intimate songs, with both members playing guitars and singing."
What.cd hosted, among other things, three unpublished Salinger stories. They were swiftly taken down by the site hosts over demands from his estate, but narrowly made it onto BitTorrent and other sites, and are still available today.
Meanwhile, a lot of unique live-concert recordings (often made with the band's permission, unlike the Salinger tapes) vanished with the demise of What.CD. Presumably some people still have them, if bit rot hasn't caught up, but they simply ceased to be available.
I'm not sure these ever landed on torrent sites, but the first album from Godspeed You! Black Emperor is considered lost, and Eminem's debut EP was unavailable for more than a decade. Both are (early-)digital-era works, which raises interesting questions about the fragility of even modern content. The archive-less disappearance of quite a few online short stories, flash games, and Soundcloud tracks makes me suspect that in a few decades, quite a few more "first works" of notable creators will be unavailable.
This is the negative of private invite-only trackers like what.cd. If they end up closing a lot of data gets dies with them. In contrast to that piratebay and nyaa.pantsu regularly upload their databases and there are even offline database readers for them.
Years ago, I was on a university private tracker. It was used more to share hard-to-find resources than for any actual piracy, much less piracy of anything new or policed. (I'm sure people stole plenty of 50-year-old textbook PDFs.) But when the hoster shut it down abruptly, everything simply vanished, including all kinds of hand-scanned and annotated stuff which may be literally irreplaceable.
Meanwhile, something like Kickass Torrents can be shut down and abandoned by the owners, but the content all reappears almost immediately. The private-tracker model has lots of advantages for community building, but as far as robustness it undermines the entire concept of P2P sharing.
I still have a 56kbps MP3 copy of Eminem’s first EP sent to me, pre-Napster, over AIM. I didn’t even realize it wasn’t unavailable until this post. And my son loves Eminem.
It looks like some copies hit the open internet around 2009, and a few of the songs has been remastered as promotional gimmicks. But yeah, I remember looking a bunch of years ago and finding that the title and one other song (Open Mic, maybe?) were the only things in circulation.
It's sort of funny to see that even something released post-internet can float in and out of availability like that. I generally class recent content as "lost" or "available", but obviously there's still a third category.
As a non-music example (and please somebody prove me wrong), I cannot find the 1954 movie "So This is Paris", starring Tony Curtis. No copies available anywhere on amazon, ebay, or searching through internet sites. There was a short clip of one of the songs on Youtube a few years ago, but it was taken down for copyright infringement. I simply cannot find it anywhere.
what.cd. I only used it sparingly, but apparently they had a lot of rare releases that existed nowhere else. Vanished overnight with the feds taking it down.
Actually it was a reverse proxy that was taken down, and they didn’t even target what.cd, it was collateral damage. The admins got scared nevertheless and pulled the plug.
I remember a torrent site for old games.
You had tremendous community efforts documenting and cataloging games. Gathering every thing from the different releases, patches, music, art, etc.
It was a gaming historian's wet dream. They were even beginning to record the releases from different warez scene groups.
But everything went away because, form what I gathered, someone in EA noticed you could download FIFA 96 for the Mega Drive/Genesis.
Unfortunately, I don't remember the name of the site. RIP
It was Underground Gamer. The quality of the collection was astonishing, many games were impossible to get or purchase anywhere else due to abandonment. And everything just disappeared overnight.
People still have all this on their drives. I'm sure the community can be revived someday.
You're trying to make a snarky comment, but this is exactly what patents are supposed to do...
Reveal how inventions and large companies work, as opposed to the old cloak-and-dagger method of 'secret formulas' and such. Then allowing anybody to see the method and licence it from the company, or be liable for a lawsuit.
What we used to have is if somebody somehow stole the secret to an invention it became fair game. Nowadays though companies try to game the system by having intentionally labrinthian patchworks of patents that do their best to not reveal anything about how the invention works while still being able to be used to sue people.
> I'm sure that "library of Alexandria" would be burned quite quickly.
Rightly so-- and nothing of value would be lost. SV companies don't have much in the way of "secret sauce", they just have $$$ to burn (from VC weenies buying equity stakes) for hosting costs and for developing increasingly-crappy web frontends.
Well, they weren't torrent sites in the late 90s and probably not the early 2000s either. The Bittorrent protocol wasn't released until July 2001 and it took a few years to get popular.
Back then, they were just servers you downloaded stuff from.
Napster started in 1999. Search was centralised, but downloads were p2p.
Edit: it shut down in 2001, and everyone I knew at the time was using it. vanderZwan might have misused the word torrent, but the rest of his post is spot on.
This just reminds of how in the late nineties and early 2000s we used to have torrent sites for music enthusiasts with an archive of all the different versions, editions, bootlegs, all painstakingly added and catalogued, representing tens if not hundreds of man-hours worth of work by people who simply love music and wanted to share that love with other enthusiasts, creating some of the richest catalogues of music and music metadata the world has ever seen. And then it all was destroyed.
And this was true for many other topics of interest too.
We have burned many digital libraries in the last few decades, and we will probably continue to do so for a while longer.