There's this implied trust we all have in the AI companies that the models are either not sufficiently powerful to form a working takeover plan or that they're sufficiently aligned to not try. And maybe they genuinely try but my experience is that in the real world, nothing is certain. If it's not impossible, it will happen given enough time.
If the safety margin for preventing takeover is "we're 99.99999999 percent sure per 1M tokens", how long before it happens? I made up these numbers but any guess what they are really?
Because we're giving the models so much unsupervised compute...
> If it's not impossible, it will happen given enough time.
I hope you might be somewhat relieved to consider that this is not so in an absolute sense. There are plenty of technological might-have-beens that didn't happen, and still haven't, and probably will never—due to various economic and social dynamics.
The counterfactual—all that's possible happens—ie almost tautological.
We should try and look at these mechanisms from an economic standpoint, and ask "do they really have the information-processing density to take significant long-term independent action?"
Of course, "significant" is my weasel word.
> we're giving the models so much unsupervised compute...
Didn't you read the article? It's wasted! It's kipple!
> A happier future is one with humans in control over their own tools and their own livelihoods.
People should own the product of their work and owning companies should be illegal.
Every good product starts as somebody's weekend project or and experiment with a buddy in the garage. Then they start getting users and making money. And then they sell it and the new owner ruins is.
Molecules form cells, cells form humans, humans form organizations. Slavery is illegal, yet it's legal to own a group of humans, replace parts against the group's will and order the group to do something against their will. Owning companies is just an abstraction built to replace slavery with enough indirection that people don't object.
> Our values are irrelevant to the wider population.
This is the saddest part. Things are getting worse for everyone and most people just don't care. They are either ignorant or accept it as inevitable.
I see 2 issues:
1) People don't have real power. There's too many steps between an issue you care about and a solution which requires changing laws. I don't see a solution other than people voting on laws directly and possibly votes weighted by how much they actually know about the stuff. How to implement it at reasonable cost is a very difficult question.
2) You can't make people care. People only start caring once they personally get hurt. Theoretical downsides don't interest most of the population. Freedom of speech is something they've learned at school about and they know they are supposed to cheer for it but when a platform requires spelling fuck as f*ck, that's OK with them - it might be the canary in the coal mine before more sophisticated censorship (analysis of sentiment/meaning, shadow bans) is rolled out but that's a theoretical concern, if they are able to comprehend it at all. And even when they get hurt, they often don't learn from it. I've seen plenty of people lose accounts on various platforms but all they do is switch to another proprietary platform, without looking for real alternatives.
One good metric of quality of life (which includes various freedoms) is how many people emigrate or immigrate.
Anybody who defends authoritarians has to explain why so many people want to leave and why the regime wants to keep them in. (With some exceptions such as China which weaponizes emigrants by threatening their families.)
If that's the case the theocratic monarchy in UAE takes the cake, I think, although maybe there are similar amounts elsewhere.
Pretty much all the highest % immigration countries are monarchy that I can think of, since in those country another tax payer is an easy win and immigrants that cause problem can be instantly booted so there is very little downside to taking anybody with $1 or a job who cares to come.
Top Countries by Percentage of Immigrants (approximate recent figures):
Qatar: Around 77% (or 76.7%).
United Arab Emirates (UAE): Around 74-88% (some sources show higher figures for earlier years).
Kuwait: Around 69-73%.
Bahrain: Around 55%.
Singapore not far behind (~40% from memory), a one party state but with voting, sometimes described as essentially an elected recallable monarchy. Also note most of those countries have relatively low emigration rates of native citizens.
I think "immigrants" is the wrong statistic here, since it includes workers with no path to citizenship. (In some cases, they can't leave because their employer stole their passport.)
It confuses "this is a good place to resettle" with "here I can arbitrage higher wages in order to send money back home."
I'm not sure it is wrong. I'd have no path to UAE citizenship, nor do I particularly want one, I'd likely have lower wages. And I'd still like to live there more than most places.
> > it includes workers with no path to citizenship
> You're still an immigrant even if you can't become a citizen.
Yes, like I said. My point is that these two scenarios are very different:
1. "I am +1 to the immigrant count because this is a great place for me and my family to live and I wish for us to move here permanently."
2. "I am +1 to the immigrant count temporarily because the wages here are so much more than I could earn at home, and I'm remitting that money back to my family who live somewhere else. As much as this is an opportunity for me, we could never move here because same wages would have my family homeless and starving, making this a terrible place to move permanently."
Both people are happily adding to the "immigrants inside" count, but they are very different judgements about the country.
That's a good point. Perhaps a better statistic would be people who want to emigrate or immigrate. We're introducing a bias by measuring only those who actually do.
For me, this is the maxim that governs speaking with someone defending a totalitarian regime.
If the person has no issue that people have to be kept by force INSIDE for the country to function, then we have a fundamental disagreement on what is good and what is bad and any further discussion is a waste of time.
1) Not sneering at it but everything has a cost. If authoritarians get the impression that all their past offenses will be forgiven if they hold everyone hostage and negotiate well, then there's no risk for them. And it's disrespectful to the victims.
There should be things you don't come back from.
For example, if you imprison people for political reasons, the time they spent in prison should be added up, multiplied by a punitive constant (2-3) and given to the offenders. And if that is a just punishment (I believe it it), then not doing that to them is unjust. Simple as that.
2) We should be looking for ways how to have both a peaceful transition and just punishment for the offenders.
The people most responsible got away for free by skillful negotiation (immunity in exchange for data).
Instead, the proposition should have been a) you give us the data and graciously accept your death penalty b) we repeat the experiments on you, nonlethal first. That's harsh and will make many people today recoil (because they've been indoctrinated into a 1-step moral system which seems to correlate with stability but injustice), but it's fair and just. They think those experiments were OK to perform on innocent people, so they are very much OK to perform on them (guilty people) by their own logic.
Yes, here in Poland 36 years later people still seriously argue the country would be much better if we hanged the communists off lampposts (like it was done in few other places).
There ws a great cost to a "peaceful transition". The entire judiciary was basically full of extremely corrupt people, half of the political class. Even today when the old judges are almost all gone the horrible culture they had still corrupts many younger ones (although today it is more towards incompetence and indifference rather than corruption).
Would it be better to have half a million (or possibly entire million if you count inevitable victims on the other side) die to avoid it? We are still paying the price.
There is an argument that had we sorted the communist problem successfully back then we wouldn't have politicians later that let themselves be corrupted by Putin into funding his army. And perhaps there would never be an invasion of Ukraine.
Or if we done away with the peaceful transition, the communists in other neighbouring countries would attempt to hold on to power with everything they got. Who knows.
"Should we have put 500,000 people to death?" sounds like pub conversation, to be frank. There are plenty of options between 'no repercussions for the old regime' and Rwanda.
Hm. I am not sure if a lynchmob and more blood would have helped the transition. The main important thing to the people was, that the wall was down and Stasi (secret police) out of power.
There has been prison time and the careers of anyone important connected to the Stasi ended.
You need "a little bit" of politician/judge/enforcer lynching to keep the government in line the same way they make a big show of "a little bit" of kicking in people's doors at 4am to keep the peasants in line.
I didn't say a lynchmob, why do people always assume a bad implementation?
Obviously, if you intend to abduct ("imprison") or kill ("execute") somebody as punishment, then you should have very high certainty they deserve that punishment. One of the methods of achieving that is giving them a chance to defend themselves ("court process").
I don't see any difference between individuals and monopolies on violence ("states") doing this, as long as they both have sufficient levels of certainty.
Because the optimum is a public process which proves their guilt beyond reasonable doubt so that every good person supports their punishment and has the confidence (certainty of guilt) to support it publicly.
But if the choice is between no punishment and somebody gunning them down in the street or droning them, i prefer the latter.
Court processes are useful when guilt is uncertain at first look and you want to increase certainty. But dictators and their close supporters, the certainty is often sufficient by nature of many their actions being public. Sometimes they literally go on TV and declare they're going to a foreign country to kill their people and take their land. At that point, it only becomes a matter of making sure you have the right person.
And don't forget the victims. Many authoritarian regimes don't kill opposition outright (for various reasons) but imprison them instead. Such a victim knows many of the people (cops, judges, informants, etc.) responsible for / guilty of falsely imprisoning them. After a regime change, the victims go free and have often more knowledge of the offenses than can be proven to a court by the simply virtue of being there and therefore have more than enough confidence to deliver a just punishment.
>I don't see any difference between individuals and monopolies on violence ("states") doing this, as long as they both have sufficient levels of certainty.
This peasant is faulty. He's not indoctrinated enough. Someone nab him and send him for reeducation. /s
It's a hard one. I can tell you something which doesn't work because the Americans have tried it twice so far. It won't work to say "Well, that was naughty, please don't do it again".
That silliness is how you get Jim Crow, it's how you got Trump 2.0
In a civilized country I can believe jail time would be good enough, but the US still uses capital punishment, so seems to me that if you want to be taken seriously some of those responsible have to be executed
In practice I remain doubtful that such an orderly transfer is likely. If there's chaos, for even a few days, that's how you get France's "Wild Purge" in the period when German withdrawal and Allied liberation are happening one town at a time. The accused are punished, sometimes even executed, without anything resembling due process.
> The accused are punished, sometimes even executed, without anything resembling due process.
I also don't like this but I wonder, if this is because the choice is between a) full punishment with less certainty of guilt now b) lenient or no punishment with high certainty of built later.
The ideal would be to hold those people until they can be tried and punished in an orderly fashion. And in principle all you need for this is enough food to keep them alive, though in such situations, even that might be a luxury.
> After all, demand has a hard ceiling at 24 hours per day.
I agree consumption is capped. I constantly struggle with whether to watch a given video or read a given article. I have an ever increasing to-watch and to-read list and unfortunately human life is too short to learn all the things I would want to.
That being said, on the production side, it's a complex interplay between quality, quantity and discoverability. If it takes 10x the effort to increase quality 2x, then it might economical to produce 10x the number of videos with 1x the quality. I say might because those videos will be shared less, rated worse and will therefore have lower discoverability. But by how much?
And you can't judge quality until after you've consumed the "content"[0]. So if the goal is to serve as many ads as possible, it's more economical to just make more "content". That's why I much prefer individual "creators" who clearly do what they do because they enjoy it.
OPs idea is about having a new GPL like license with a "may not be used for LLM training" clause.
That the LLM itself is not allowed to produce copyrighted work (e.g. just copies of works or too structurally similar) without using a license for that work is something that is probably currently law. They are working around this via content filters. They probably also have checks during/after training that it does not reproduce work that is too similar.
There are law suits about this pending if I remember correctly e.g. with the New York Times.
The issue is that everyone is focusing on verbatim (or "too similar") reproduction.
LLMs themselves are compressed models of the training data. The trick is the compression is highly lossy by being able to detect higher-order patterns instead of fucusing on the first-order input tokens (or bytes). If you look at how, for example, any of the Lempel-Ziv algorithms work, they also contain patterns from the input and they also predict the next token (usually byte in their case), except they do it with 100% probability because they are lossless.
So copyright should absolutely apply to the models themselves and if trained on AGPL code, the models have to follow the AGPL license and I have the right to see their "source" by just being their user.
And if you decompress a file from a copyrighted archive, the file is obviously copyrighted. Even if you decompress only a part. What LLMs do is another trick - by being lossy, they decompress probabilistically based on all the training inputs - without seeing the internals, nobody can prove how much their particular work contributed to the particular output.
But it is all mechanical transformation of input data, just like synonym replacement, just more sophisticated, and the same rules regarding plagiarism and copyright infringement should apply.
---
Back to what you said - the LLM companies use fancy language like "artificial intelligence" to distract from this so they can they use more fancy language to claim copyright does not apply. And in that case, no license would help because any such license fundamentally depends on copyright law, which as they claim does not apply.
That's the issue with LLMs - if they get their way, there's no way to opt out. If there was, AGPL would already be sufficient.
I agree with your view. One just has to go into courts and somehow get the judges to agree as well.
An open question would be if there is some degree of "loss" where copyright no longer applies. There is probably case law about this in different jurisdictions w.r.t. image previews or something.
I don't think copyright should be binary or should work the way it does not. It's just the only tool we have now.
There should be a system which protects all work (intellectual and physical) and makes sure the people doing it get rewarded according to the amount of work and skill level. This is a radical idea and not fully compatible with capitalism as implemented today. I have a lot on my to-read list and I don't think I am the first to come up with this but I haven't found anyone else describing it, yet.
And maybe it's broken by some degenerate case and goes tits up like communism always did. But AFAICT, it's a third option somewhere in between, taking the good parts of each.
For now, I just wanna find ways to stop people already much richer than me from profiting from my work without any kind of compensation for me. I want inequality to stop worsening but OTOH, in the past, large social change usually happened when things got so bad people rejected the status quo and went to the streets, whether with empty hands or not. And that feels like where we're headed and I don't know whether I should be exited or worried.
With LLMs, if you did the first in the past, then no matter what license you chose, your work is now in the second category, except you don't get a dime.
> Not just to the people I agree with, but to anyone who needs to use a computer.
Why not say "... but to the people I disagree with"?
Would you be OK knowing your code is used to cause more harm than good? Would you still continue working on a hypothetical OSS which had no users, other than, say, a totalitarian government in the middle east which executes homosexuals? Would you be OK with your software being a critical directly involved piece of code for example tracking, de-anonymizing and profiling them?
As for me that's a risk I'm willing to accept in return for the freedom of the code.
I'm not going to deliberately write code that's LIKELY to do more harm than good, but crippling the potential positive impact just because of some largely hypothetical risk? That feels almost selfish, what would I really be trying to avoid, personally running into a feel-bad outcome?
I think it would be most interesting to find ways to restrict bad usage without crippling the positive impact.
Douglas Crockford[0] tried this with JSON. Now, strictly speaking, this does not satisfy the definition of Open Source (it merely is open source, lowercase). But after 10 years of working on Open Source, I came to the conclusion that Open Source is not the absolute social good we delude ourselves into thinking.
Sure, it's usually better than closed source because the freedoms mean people tend to have more control and it's harder for anyone (including large corporations) to restrict those freedoms. But I think it's a local optimum and we should start looking into better alternatives.
Android, for example, is nominally Open Source but in reality the source is only published by google periodically[1], making any true cooperation between the paid devs and the community difficult. And good luck getting this to actually run on a physical device without giving up things like Google Play or banking apps or your warranty.
There's always ways to fuck people over and there always will be but we should look into further ways to limit and reduce them.
> Open Source is not the absolute social good we delude ourselves into thinking.
Historically the term "Open Source" was specifically developed to divorce the movement from the "social good" ideas that were promoted by Free Software.
That's where I stand. I don't do Open Source to make the world better. I do Open Source because I believe that makes my software better.
I'm not an activist. I'm an engineer. Nothing wrong with activism, all the power to the people doing it, but the licensing I chose for my code doesn't take it into account.
I agree with the GP. While I wouldn’t be happy about such uses, I see the use as detached from the software as-is, given (assuming) that it isn’t purpose-built for the bad uses. If the software is only being used for nefarious purposes, then clearly you have built the wrong thing, not applied the wrong license. The totalitarian government wouldn’t care about your license anyway.
The one thing I do care about is attribution — though maybe actually not in the nefarious cases.
> The totalitarian government wouldn’t care about your license anyway.
I see this a lot and while being technically correct, I think it ignores the costs for them.
In practice such a government doesn't need to have laws and courts either but usually does because the appearance of justice.
Breaking international laws such as copyright also has costs for them. Nobody will probably care about one small project but large scale violations could (or at least should) lead to sanctions.
Similarly, if they want to offer their product in other countries, now they run the risk of having to pay fines.
Finally, see my sibling comment but a lot of people act like Open Source is an absolute good just because it's Open Source. By being explicit about our views about right and wrong, we draw attention to this delusion.
It’s fine to use whatever license you think is right. That includes the choice of using a permissive license. Restrictions are generally an impediment for adoption, due to their legal risk, even for morally immaculate users. I think that not placing usage restrictions on open source is just as natural as not placing usage restrictions on published research papers.
Tragedy of the commons. If all software had (compatible) clauses about permitted usage, then the choice would be to rewrite it inhouse or accept the restrictions. When there are alternatives (copyleft or permissive) which are not significantly worse, those will get used instead, even if taken in isolation, the restricted software was a bigger social good.
During the gold rush, it is said, the only people who made money were the ones selling the pickaxes. A"I" companies are ~selling~ renting the pickaxes of today.
(I didn't come up with this quote but I can't find the source now. If anything good comes out of LLMs, it's making me appreciate other people's more and trying to give credit where it's due.)
To be honest, I haven't looked at any statistics but I imagine a tiny few of those looking for gold found any and got rich, the most either didn't find anything, died of illness or exposure or got robbed. I just like the quote as a comparison. Updated the original comment to reflect I haven't checked if it's correct.
There's this implied trust we all have in the AI companies that the models are either not sufficiently powerful to form a working takeover plan or that they're sufficiently aligned to not try. And maybe they genuinely try but my experience is that in the real world, nothing is certain. If it's not impossible, it will happen given enough time.
If the safety margin for preventing takeover is "we're 99.99999999 percent sure per 1M tokens", how long before it happens? I made up these numbers but any guess what they are really?
Because we're giving the models so much unsupervised compute...
reply