Can someone explain why the old books could really be relevant. I get the pre-nuclear steel analogy, but why is this relevant given how much more modern texts exist. A few years ago, millions of yahoo groups were erased but now a few thousand books are what is needed to run a successful AI company? I mean, it can barely be about the information in those books (that would be very often outdated), but just for a little more text (with ever less marginal gain), what is the benefit?
It sounds like it is about the information in the books. The titles they're looking for are all nonfiction. Not everything is on the internet, and just because it's a few years old doesn't mean it's outdated.
Speaking from experience, the information density of published books is a lot higher than most internet text. It's very high quality training data.
The goal here is to have all human knowledge in a single file, which is pretty neat IMO.
The internet basically never delivered on the promise of replacing textbooks or even education as a whole. Wikipedia sucks on many topics, has insane internal politics, and is a tertiary source by design (redigesting blogs and books), whereas textbooks are generally secondary.
Booksellers noticed a huge uptick in non-fiction purchases, that is not the same thing as them not targeting fiction at all.
Edit: Actually, I have real evidence, the Bartz in "Bartz v Anthropic" is Andrea Bartz, a novelist, and the complaint specifically lists four of her novels as infringed works.
I see. So it is much less about 150 year old fiction books, but more about 1980s science literature that was only ever printed five times. That makes a lot more sense than what the public debate seems to be about.
> but now a few thousand books are what is needed to run a successful AI company?
They're scanning millions of books.
It's the diversity of text that helps. One of the lessons we've learned is that more training data leads to better models. Even old books have different mixes of word sequences that will improve the model. The returns are diminishing, but when you have the pipeline set up to ingest it you might as well keep adding to the dataset.
They are the memories of the productive part of society. You can leaf through them and get the feel of what it was like. You don’t need most of your memories, personality, or core ideals to be productive to the State.
The information density is a lot less for millions of yahoo groups. They are far more likely to cover the same topics and not have new information in them.
Books are more likely to be about a specific topic or story or time or setting and be more information dense
> why the old books could really be relevant
> it can barely be about the information in those books (that would be very often outdated),
LOL, not being rude: have you ever read a book outside of what they forced you to read in school? Most old books are not O'reilly's manuals for Visual Studio 2014, they don't go out of date.
They are interesting to human beings for the same reason they are interesting to the labs. If it was just about quantity of text then the labs could generate text with the prev. gen model and use that alone to scale to the next model, there is something of immeasurable value contained in books (hint: it starts with an i and rhymes with bin formation).
Cognition is encoded in language, they weren't brain-damaged yet. There's better (real, not token-exchange) thinking, which LLMs can copy and reproduce in novel arrangements.
@fsflover: and my mobile phone provider. And my insurance. And my eletricity provider. And my housing associations customer portal. Getting doctors appointments. Mortgage. Union. National retirement savings account. Some of these may have some alternative left, but far too many you will be left out.
Well, I can buy a train ticket without it, so I could still leave.
Do you mean to say that all these services require strong device integrity to function in Sweden?? In Australia, no local app that I'm aware of (banking, finance, government, medical) requires any form of device integrity - otherwise I couldn't use them. The only exception I've encountered is Google Wallet.
I wonder why there would be such a difference in policy between countries, not only in government but across the private sector? This doesn't make any sense to me. If anything, I'd expect Sweden to have a more sensible, left wing attitude than Oz.
The comment that I was replying to was suggesting (to me) to live in sweden without mobile bankid (centralized 'banking' 2FA app on smartphone). I have here listed services that require said mobile bankid. Mobile bankid works for me and others on graphene os, but would likely not work on an arbitrary linux smartphone.
And Europe would be in even bigger black-out trouble right now as half of those 20 GW of nuclear power would not be running due to missing cooling water, just like is the case in the other European countries.
The german green part certainly isn't perfect, but they had a plan. A plan that included far more renewable energies than currently build. The ones that orchestrated the ultimate end of nuclear energy while shutting down build up of renewables are the conservatives. Blaming someone for a half executed plan is childish. Especially when it is from a party that supposedly had economic competence yet killed tens of thousands of jobs.
If look up the original plan from 1980s, you will see that the German Green idea was to replace nuclear power and oil with renewable and coal and natural gas. Climate change was not very important, the most important was to remove nuclear power. (The founding idea of German Greens, as was for many anti-nuclear groups, was that by removing nuclear power they will remove the danger of nuclear weapons.)
Energie-Wende: Wachstum und Wohlstand ohne Erdöl und Uran [Energy transition: growth and prosperity without petroleum and uranium]
I assume, ultimately western nations will have to adopt what already exists for agricultural goods for production as well. I am afraid it will end as a per-worker subsidy analog to per-area-of-land for lack of a better metric but in the end, that is what will be necessary if one wants to keep any industrial capabilities.
It’s equal parts funny and frustrating to see how deep into the ”a few hypothetical percent of profits for corporations matters more than customer rights and less e-waste” some are.
reply