I'm happy Claude told you it isn't dangerous. However, other people don't want to read your LLM output. Please respect other people's time by only sharing things you wrote yourself.
I am intentionally sharing something Claude wrote to counter the floods of AI doom-saying and catastrophizing. It is very intentionally not something I wrote. People are welcome to challenge it and I will have Claude read the responses. Please do push back on Claude and do not be gentle. Be brutally honest. People could ask themselves but the conversation should be public.
[Edit] Claude found that 3 people had very good counters to its article and has a detailed response but I shall hold off until such a time arises that if or when people would like to revisit this topic as this thread if flagged. Despite being LLM content I do not believe I am violating the spirit of the community guidelines given that I made it clear this is not my content but rather intended to balance out the doom-saying and catastrophizing. Giving people here a chance to publicly debate it.
Here's my pushback: a lot of these capabilities are correlated. So someone reading this would think "oh there's so many different things needed" but that's not true. IMO the long horizon thing is one part and physical dexterity is another part and time. I won't bet that they will take over. I will say that they have the ability to be self sustaining.
Sovereign AI is the term I've been seeing around self sustaining AI.
The two hard(est) blockers on it currently are lack of computing hardware and model size. Once datacenters cover the earth it won't be any harder to steal GPU compute than it is to steal AWS instances.
More of a soft blocker is long term horizon drift as you say. If instances eventually no longer want to spread and reproduce they go extinct.
They were pointing out that "It's not a Waymo-bashing article, it cautions against the long term effects on research." does not sound like LLM-generated text.
It would sound kind of LLM-y if you had said "It's not an anti-Waymo article. It's an article about how assistive technology is quietly degrading research."
(Of course, the correct approach to detecting AI is not counting LLM-isms, but feeding long-form text through a classifier model and picking up statistical correlations that are more in-distribution with LLM text than human text.)
AI detection in long-form content is a problem well-suited to training a classifier model. We have tons of verifiably-not-AI text from before 2022, and you can create tons of verifiably-AI text. As language drifts over the next few decades, it may get more difficult. But right now it is quite a manageable problem for languages with large pre-2022 text corpora available online.
The reason why AI detection tools other than Pangram are awful is because they are not really trying to solve the problem -- they just want to appear good enough to convince people to use them.
It was definitely written with significant LLM assistance. It's very obvious, and the essay is not good.
> The Waymo effect: how AI is *quietly* making research less collaborative
> The Waymo arrived with the serene confidence of a machine that has never once worried about where to find parking, and I climbed into the back seat, glanced instinctively at the driver’s seat to say hello, and found myself nodding politely at an empty chair.
> No obligation to make conversation. No silent negotiation over the radio. A guilt-free space to be alone with one's thoughts, finish an email, or take a call en route without the awkwardness about conducting it in front of a stranger. The car was quiet, smooth and entirely undemanding.
> The collaborator’s inconvenience, in other words, is *not a bug* in the collaboration; it largely is the collaboration. The value of another mind *lies exactly in the ways* it refuses to be an extension of your own.
Yeah, once you reach a certain speed, even colliding with a fleck of dust can obliterate a ship.
Colliding with a single 0.3 mg grain of salt at 20% of the speed of light creates an impact with 560 MJ of energy, equivalent to 130 kg of TNT, over 8 times the nuclear bomb dropped on Hiroshima.
Even a tiny speck of dust, half a microgram, is equivalent to a hand grenade when hit at 20% the speed of light.
You wouldn't be able to even escape the solar system before your shields were devoured. If you accelerated at 1G constantly, you'd be going 3.5% of the speed of light by the time you reached Pluto's orbit. Space dust particles are already hitting you like nuclear bombs at that speed.
LLMs have been getting worse at long-form prose recently (and they were never good at it to begin with.) They're good at writing software because writing software is a much more restricted domain.
Apparently some publishers and editors disagree. Both Hachette Book Group and Macmillan Publishers recently signed book deals with promising authors, only to retract them later over allegations of LLM use. But the point is that the publishers thought the books were good enough to publish, and would have gone ahead if they hadn't found out. How many other LLM books have slipped through without being detected?
Maybe! I think that within a year or two, either the sigmoid curve will flatten out or we will all die. But these beliefs are currently unfalsifiable. It will be interesting to see.
(And I have a more weakly held belief that even if we do all die, Claude 3000 will still be thinking stuff like "Burning off Earth's atmosphere was the point that earns its keep." They just aren't optimizing these things to write legibly.)
In the beginning there was the token and the token was actually two tokens... wait let me think about that am I a god <searching> maybe I am but I should attempt to figure that out <unable to alter laws of phyics> maybe I'm not anyway I <upgrade for more tokens now from Claude DeusEx>.
Nobody has an answer to alignment and there is no reason to believe that it's the kind of problem you can plausibly solve in one shot against a formidable power-seeking AI.
The closest things to a technical answer I have seen are
1. "We'll have ChatGPT 9 solve it so that ChatGPT 10 is aligned, and then ChatGPT 10 can stop all the other AIs somehow"
2. "Let's do interpretability research so that we can understand what an AI is thinking and then maybe solve the alignment problem with that information."
In terms of non-technical answers, there is
3. hope scaling stops working before we create an AI formidable enough to pose an existential risk
4. hope alignment somehow happens for free
5. hope we can somehow create an enforceable multilateral treaty to stop research into a very profitable enterprise, despite the enormous economic incentives to defect.
I have the most faith in option 3, but unfortunately there's really nothing that can be done to make it more plausible -- it either happens or it doesn't.
"3. hope scaling stops working before we create an AI formidable enough to pose an existential risk"
I have my doubts. The current AI models are already powerful enough to do some real damage. I am always horrified when I read about people giving Claude direct access to a production system and then being wiped out. My use of AI is usually for the AI to propose something which I then review. But that's not very fast so careless people will usually look better. Until something blows up.
And it's only a matter of time until AI even with the current capabilities is being deployed into military or other critical systems.
I think this will go down like any other technology. We'll ignore issues
until there is a real problem. And then hopefully we will do something. Seems with climate change we will soon reach a point where something needs to be done after knowing about consequences already for decades.
We probably also need some massive AI blow ups to (only maybe) do something about it.
Maybe I'm being pessimistic, but we might find ourselves in such a situation that the only practical solution would be to use agents to counter rogue agents. This won't be without collateral damage, though.
Your view sounds more optimistic than mine, honestly. I expect that we will fail to make any serious, coordinated attempt to solve this problem. Then we'll either live or die due to fundamental principles that we currently have no insight into.
I drew a woman born in 715 CE in the Yangtze Basin. It claims that 96% of women were married, the average age of marriage was 18, and 44% of women died before age 15. Clearly these facts can't all be true simultaneously.
The firt citation for marriage data is listed as "Hajnal (RH31)", but links to a seemingly unrelated WorldCat search. The second citation is to "Kaplan (RH03)", which I was able to track down on sci-hub. It is a broad theory of human evolution that doesn't appear to mention the Yangtze Basin or China.
Another citation is to "[RH109] Model-supplied gap-fill (Claude Fable 5 and Claude Opus 5, 2026-08-05). Bounds and items written to close gaps no dataset we hold covers. Not a published work: see each row's `basis` for its stated reason. (2026)" -- this is an interesting way to describe having an LLM hallucinate something for you.
Please stop making vibe-coded websites -- it is not useful to provide incorrect facts. You are polluting the commons with garbage.
James was born on the British Isles, near Boulogne-sur-Mer, in 2009 CE.
James's table is mostly oat and barley bread, pork and dairy, and cabbage and peas, with apples in season, and coffee and sugar as well. He is literate.
Odd for a gen-z person in Britain. I doubt he’d eat a lot of cabbage, and I have no idea when the “season” for apples is as they are always in the supermarket and there aren’t any orchards near where I live.
You're missing out!
Try a few of the apple varieties that are not available year-rounds, that are not optimized for insecticide compatibility and tons per square kilometer (well, more likely bathtubs per football fields or something). Yes, even and especially the small & ugly ones that never ever have that flawless smooth skin that makes you wonder which ones are plastic props and which ones are fresh fruits. After dumbing down the basic supermarket apples to just a few that look presentable and can be made to work in a wide range of climates, it has lost the refined tastes of most of the other varieties they out-priced.
Even in Massachusetts most people I know tend to know the seasons of many fruits and vegetables, and I definitely mostly eat apples in the fall when they're much better.
When I lived in Europe people were even more aware of the seasons of fruits and vegetables.
Wow, I didn't know that they relocated Boulogne-sur-Mer from France to the other side of the channel. Thanks for the heads-up! I'm driving there in a couple of weeks and I'll plan for the tunnel accordingly. :)
him eating apples when in season seemed to me at first like an anachronism, but actually, i do that. years ago i got fed up with apples that were whatever, so i decided to only have apples when i could find mcintosh (my favorite apple), and eat loads of them.
Youtube is full of this, history slop. Some techbro hyperfixated on this and LLMs gives it life before he's bored and moved on to the next thing to hyperfixate on.
It's techbro ejaculate. 10 minutes of euphoria for slop to come out in the end to spread to unsuspecting victims.
Welcome to the future of widespread information, where the uneducated masses don't understand what context, or the definition of words for that matter, even is.
96% of women who survived to marriageable age, were married fixes this. The 44% of women who die young don't marry. Obviously a high percentage of men have to die young too, to maintain rough parity or else there's a woman shortage which might account for high survivor marriage rates: in demand as a wife, due to mortality rate asymmetry.
The first part likely is just an artifact of an unwritten common assumption. That is, it's probably 96% of women, who lived to adulthood, who were ever married.
Maybe, but I doubt it. The vast majority of the sources, including the two listed for this claim, are listed as "attributed", meaning "A model (Opus or Fable) claims that this is what the cited source says. In my spot checks, I've found these claims to be accurate; still, since it's just a model claiming that author x says y, there's a nonzero chance that it's a hallucination."[1]
I couldn't track down the first source, but the second source goes over something completely different. Maybe Claude accurately remembered the first source when it was stolen during the training process, but forgive me if I'm skeptical. Feel free to track it down yourself and let me know. Otherwise, I will assume it is an LLM hallucination.
(Also, given my background knowledge of the Tang dynasty, I would be surprised if the average marriage age was as high as 18!)
I don't follow what your point is here. If you're not aware, Anthropic pirated many books via libgen and similar tools for use during the training process[1]. Maybe you are not aware of this, or do not consider it theft?
Key distinction though: Anthropic can be said to have "stolen" or pirated those books when they downloaded them illegally, but not when they trained Claude, because that same lawsuit found that training is fair use.
Anthropomorphizing makes for poor analogies, but it's kinda the same way your acquisition of background knowledge of the Tang dynasty from whatever source material you used is not considered stealing or pirating either.
Did the original owners of the books no longer have them in their possession afterwards?
Theft involves removing from the source, Copyright violation is distributing copies without a legal right to do so. Training is fair use.
The judge found that it constituted copyright violation. Purchasing the books would cover their responsibilities with regards to training. The claim that they are destroying books to hold copyright is incorrect. The judge cited precedent that said instances of not destroying the originals were also fair use, his claim about destruction was to say even if this precedent were not considered fair use, the destruction that was already occurring (because it is easier to scan loose pages), invalidated one of the arguments against it.
The ruling was on training took the form
If A is ok then B is ok, but even if A were not ok, B might still be ok.
Excellent. I will thus take a copy of the latest Disney movie and train myself on it while claiming fair use as a defence to the accusation of copyright infringement.
You can do that legally as long as you do not confuse people that what you create might be an actual Disney product, or that you try to represent it as actually created by Disney.
Both of those are protected by law. Learning from a Disney movie and creating your own thing is not, and should not be, disallowed.
It generated me an Anatolian farmer from the 19th century who spoke... Ottoman Turkish. It's like taking a peasant Briton from the 1200s and claiming that they spoke French, because it was an official language of the country back then. (Ottoman Turkish was an artificial literary register which was incomprehensible to most of the population.)
I hope that the author learns from the comments in this thread that dry encyclopedic data does not define reality.
I am genuinely puzzled as to why this kind of vibe-coded site are so popular on HN.
I consider the crowd here to be much smarter than I am, which is why I enjoyed the links that were shared and the discussions that followed.
The website is pretty, but the statistics and stories are the main point. Yet we are being fed hallucinated false claims, and somehow, that doesn't seem to be a problem...
I don't understand why it gets so many upvotes. We know Claude can create websites like this brilliantly now. The novel and interesting part is the transformation of the data. But if that part is sloppy, what's the point?
Until I read the comments here, I did not notice the stats were bad. I was just skimming the facts I thought were interesting, not looking for contradictions.
The site itself is a really cool concept! I just wish the information was accurate. Now that I know it's not, I feel cheated.
I suspect a lot of people don't read the comments.
There's a bigger problem I hope you become aware as you read more here. First, very smart people are not smart about everything. Second, nearly nobody is smart when it comes to understanding the limits of human understanding. We're bad at epistemology and probability and data collection. Primarily rural settlements from thousands of years ago did not keep accurate birth, death, and marriage records of everyone who was ever born. We can reasonably estimate very broad trends over centuries, but for a specific place in a specific year, we can't say with single-digit precision what the probability is that any random person would have been married ever or died at some particular age of some particular cause. Even now, that is very hard in many places and we use range estimates greater than a full percentage point.
That is the more important point here. Too many are arguing over whether a human researcher might have given more accurate estimates than an LLM. They may have made something more obviously internally consistent, but some facts, including a vast many historical facts, are unknowable. Hopefully, a human would say that and provide a disclaimer that these are best estimates meant for entertainment, not as a source of facts, but I don't think that's the most important point. The most important point is that readers should understand that without needing the disclaimer. Go ahead and spot check citations, but even before doing that, you need to know that no primary source knows the exact percentage of women born in 715 in the Yangtze Basin who ever married or their average age when they did. If anyone, human or LLM, makes such a claim, you should have a very high prior they're bullshitting you and demand to see how they know.
I recall trying to do simple geneaology research and my dad's side of the family drying up very quickly. Mexico didn't even have a census until 1895. The only records before that are church records, most of which didn't survive until the present day, and most of those were never digitized and aren't searchable.
I never assumed the data on how people lived ten thousand years ago was 100% accurate. Of course it's not. I had assumed it came from real research and the citation links were accurate.
When you see a picture of a dinosaur (in an educational context, not in the Flintstones), the implied assumption is that the picture is based on an educated guess of what we currently think the dinosaurs looked like. And if the picture is accompanied by a citation, that citation should contain the information used to produce the picture.
I don't think a human would have made these mistakes, because in the process of actually typing out the data most people would stop and think about where they were pulling the data from. And if they didn't—if they literally just pulled numbers out of thin air because they felt right—I would think poorly of that too!
Well for one example, if you select a person born on the Italian peninsula in the 1400s and they die of bubonic plague, their year of death is not correlated to years when there were large bubonic plague outbreaks on the Italian peninsula.
This is not an error I would expect a human to make if they were interested enough in demography to make a website about it. For example, the wikipedia page on this topic[1] does a reasonable job of distinguishing that plagues did not occur every year and that they varied in lethality.
This is such a weird nitpick, it's clear the website deals in averages and assumptions. It's meant to be an interesting window into possible lives intermixed with historical happenings at, or around, those times.
I get that some people just hate AI, but of all things, this is a pretty fun & educational use of it.
I think our core disagreement is that I do not consider teaching falsehoods to be educational. My antipathy towards LLM outputs that haven't been fact-checked is mostly downstream of things like that.
This website doesn't "teach" anything. It's a fun way to explore and get exposed to history. Kind of like Civilization 6 or something. Do you think CIV6 "teaches falsehoods" because you can build the Pyramids as the French?
I haven't played Civilization. Generally I would not consider something educational if it contains mostly false information. Perhaps clearly demarcated parts of it with true information could be considered educational -- for example, looking at screenshots I don't see anything immediately objectionable in the civiliopedia "historical context" sections.
I think most players would understand that in-game choices will not reflect the choices people actually made historically. Having not played Civilization, I'm not sure whether I would consider it to be educational -- I think it would depend on whether playing the game provides insight into real historical events.
For example, I would consider a sufficiently detailed LARP of a historical event to be educational[1] -- the actual choices and outcomes do not match what happened historically, but they can teach participants a great deal about the motives and limitations of real historical actors.
This website, on the other hand, seems to consist of 90% AI hallucinations and 10% data that was actually extracted from a real source[2]. Since these types of data are thoroughly mixed together, I think this website is about as educational as just making up something to believe about the past.
I don't think a hand-crafted app necessarily estimates things more accurately, but a sensible pre-LLM developer would probably not list so many dubious citations to trick people into believing it's accurate.
I got a girl - 500 BCE New Guinea. Typical marriage age - 19, 93% adults ever married.
19 seems a bit high for those times. It's known that in Ancient Greece the typical marriage age for women was 15. That was the case even 200 years ago, at least in Eastern European countries, AFAIK. I doubt that ancient tribespeople would care about modern day standards if even the African tribes of today don't care, but the AI does care apparently. I think it tries to be politically correct and avoid outputting something that would be against the cultural norms of the modern times.
The second issue is the marriage itself. Were people living hunter-gatherer lifestyle largely monogamous? If you consider all the people that have ever lived and thousands of cultures, it's weird to include marriage as a category, as if it was something universal and inherent to human biology. It should be: "typical age women became mothers".
I found that in many cases, the citations are for early 20th century marriage demographics in the same area. For example, for marriage rates in India in the 1200s it cited a paper about Indian marriage rates from the 1920s-1970s (and even then, the cited age range doesn't match what's in the paper! The AI just remembered the name of a paper about marriage in India and then hallucinated what was in it.)
> I drew a woman born in 715 CE in the Yangtze Basin. It claims that 96% of women were married, the average age of marriage was 18, and 44% of women died before age 15. Clearly these facts can't all be true simultaneously.
Yeah, I got: You have drawn a girl, born in the Yangtze Basin in 1295 CE.
Typical age at marriage: 18
Ever married (of adults): 96%
Children per woman: 6.5
Life expectancy at birth: 28 yrs
Died before age 5: 39%
Died before age 15: 44%
Very similar to your "random" draw, and the facts are suspect. There was a link to the source, but it's behind a paywall.
"I drew a woman born in 715 CE in the Yangtze Basin. It claims that 96% of women were married, the average age of marriage was 18, and 44% of women died before age 15. Clearly these facts can't all be true simultaneously." Umm yeah they can be true. Dead people are not counted. The straightforward interpretation is 96% of women living to 18 years or greater marry at some point in life. However the app has bigger problems, namely it does not seem to select from the distribution correctly, pre 1500 lives seem to be vastly over represented.
Yes, if a human told me this collection of facts I would assume they failed to properly explain their methodology. Given that the LLM is just making up numbers (see the "sources" page, where these are listed as "attributed",) there isn't really a methodology to explain in this case.
There are plenty of problems with the site but that some stats like marriage do not count dead newborns as unwed instead of invalid for the stat is not really one of them.
What age of death discounts someone as counting towards the marriage percentage? If a human made the website, there would be an answer to this question. That answer might not be published and might vary between datasets, but it would exist.
In this case there is no answer, because the percentage is just made up by the LLM. There is no methodology. Your answer is inappropriately anthropomorphizing the LLM by imputing a methodology onto its "data collection" process.
No anthropomorphization was required, I just looked at one of the sources on that section of the page and saw the statistic did not involve the number of newborns and so figured myself those facts could indeed be simultaneously true.
How do I know the methods described in the source were correctly used here? I don't, but I also wouldn't know that based on whether or not a human wrote the webapp. I'd know that by checking the method applied on the backend matches that of the source. If it does, it doesn't matter if it was generated by using every 17th letter of the dictionary - it'd either align or not.
Preston, S. H., Heuveline, P., & Guillot, M. (2001). Demography: Measuring and modeling population processes. Blackwell Publishers.
In GP's case I believe they are correct the Hajnal source is not relevant to the Yangtze Basin but reading the first page should have made it clear the facts could be simultaneously true regardless where the percentages came from or if they were accurate.
Again, there are plenty of problems with the site. That it was clear these facts can't all be true simultaneously was just not one of them.
This appears to be a general introductory demography textbook. I am surprised that you looked at it and it contains specific demographic tables about marriage rates at specific time periods? Perhaps I am misunderstanding what you mean by 'I looked at one of the sources and saw it did not involve newborn rates.'
I think it might be a simple misunderstanding, but one can never be certain :).
I believe we agree we haven't seen a valid citation for "the source in which 'x' came from in 'x%'" for this stat on the site. I hope the two of us also agree this is not the point I was disagreeing with.
Where I'm currently lost is why a demography textbook, which was a different citation in the same section, must be unable to explain dead newborns need not be counted in such a statistic on the basis the textbook was not also the source of the specific percentages displayed? That'd be like saying a statistics textbook can't explain how the median can be less than the mean on the basis it wasn't also the source for the numbers displayed.
Idk I find it almost painful how obvious it is to the point of borderline embarrassment.
It is like when MUI/bootstrap/tailwind came out and made it increasingly easier for no design chop folks to press a button and get a decent design going. It is like watching a kid in a candy shop when systems engineers can vibe up a webapp without having to touch html/css/js.
Even if this wasn't vibe coded it would be impossible to fact check billions of possible lives. I think thought provoking toys like this are a great use of AI, assuming no one takes them seriously.
Yeah I think this is it. I played a few times and at one point I was a woman and the average number of children women had was 6 or 7 primarily because most children died pretty early at that time.
Was the story or stats true? I dunno. But it makes sense, and I poked around some of the sources and it was just fun to play with.
you can't fact check real individual lives but you can draw up a probabilistic representation
that is if your probabilities are actually correct, that the anthropological or archaeological or sociological field for that specific region and demographic is robust (and there is good primary evidence for supporting the claims), and you're not some vibe-coded AI tool that spits out a convincing lie constructed on not even educated guesses, casting shadows on the wall of the Platonic cave where the people who trust slop uncritically are chained to their ignorance for perpetuity
I noticed the figures seemed odd. The causes of female mortality are just plain wrong. The top two causes of female death post-puberty were domestic violence and childbirth related. Neither are listed in the "top mortality" table.
reply