Worst case is warming oceans create a hypoxic environment where anaerobic bacteria thrive en masse and generate H2S over country-scale areas undersea. The chemocline breaches the surface and the ocean and atmosphere become poisonous to most complex life and agriculture, also stripping the ozone layer in the process, irradiating the surface. This is one mechanism posited for the end-permian mass extinction that eradicated most ocean and surface life, including the trilobites.
Not sure how long we'd survive such a scenario, even sheltering underground. But surely it couldn't happen to us.
(However, it now seems like the AI might get us first.)
Runaway warming leading to a hot "supergreenhouse" climate was theorized. Alterbatively a snowpiercer scenario creating a snowball earth.
Or, once billions of people die from climate change, planetary wars will start on the last hospitable areas, leading to the end of humanity.
In contrast, AI ending "humanity" is limited by our own material power, would likely be stoppable, and would not easily reach areas isolated from technology
Why would it not easily reach areas isolated from technology? If a nefarious AI wins a war, it would be one of the easiest thing for it to locate any survivors anywhere on the planet.
I think you (and I, and everyone who wants AI development to pause while we catch up with the implications at least) is looking where the ball is going.
I have the general (non specific to anyone in this thread) impression that people who think it can't end humanity, are looking where the ball is today.
Current AI obviously can't "locate any survivors anywhere on the planet".
Where the ball is going… well, much as I don't believe Musk's timelines for anything, he is trying to sell his Optimus robots as a "robot army", about them running factories, about factories on the moon etc.
I suspect the moon-factory "idea" was someone asking Grok, given how the numbers don't really work for building compute as well as power, but the scale of that is enough to change Earth's equilibrium temperature by… I forget, but IIRC it's many tens of Kelvin rather than single-K from global warming if this was all put in LEO for some reason.
There's a lot to unpack with this, and I am not an expert, but: crop failure across industrial monocrops leads to food shortages, wild fires destroy significant areas which support pollinator ecologies, fresh water become contaminated by rising sea water and blighted by drought, and perhaps most significantly, as resources become strained, humans become desperate and armed conflict proliferates. This is just top of mind.
Bad, sure; I wish they were not present to threaten us, but they are not close to "all".
Humans are extremely resilient omnivores. We are able to eat a huge range of things from algae to zebra, and get food (farmed and hunted) from salt water as well as fresh.
Perhaps I have a bit more imagination—or a bit more humility towards our humanity—such that I believe this is not "either/or". In the vast search space of potential future outcomes, of course even I can see humanity survive nature as I've seen nature survive humanity. However, I can also so easily see probable branches where we are our own undoing. Apropos of TFA, if one is to believe LLMs can carve a threatening path to humanity end, how could one not see the same for our much older extant threats?
> Apropos of TFA, if one is to believe LLMs can carve a threatening path to humanity end, how could one not see the same for our much older extant threats?
Which "much older" extant threats?
Global warming and nukes are industrial era threats. Pre-industrial threats were the four horsemen of the apocalypse: war, famine, pestilence, and death.
War and death didn't go away, but are not (and never were) extinction threats. Famine and pestilence, well, there's a reason people point the finger of blame at the Chinese government for the Great Leap Forward, and why we had lockdowns for Covid while we tested that the vaccines were safe (we went from "genome sequenced" to "first human test" in 66 days; the rest of the wait being mix of "yes but how sure are we it's safe?" tests and manufacturing at scale).
If everywhere except North Sentinel Island was wiped out, or everywhere but Hawaii, or everywhere but Greenland, humanity would be back to the level of the 1750s within a few millennia at most; and other than the North Sentinel Island example, likely even faster as books with many advanced solutions to historical problems would survive and be comprehensible.
And the problem with LLMs (and other AI) is, we're giving them control of stuff, and that stuff includes robots. The *current* versions mainly concern me for economic and cybersecurity reasons, but the tech moves fast, and the companies working on them are reckless.
My core expectation is these models will cause 1e3-1e7 deaths in a single event before people collectively actually take the risk seriously. The lower end of that is "industrial accident", the upper range includes "convinces people to go to war", and these are examples of ways current LLMs can already plausibly do wrong, given who uses them, why they use them, and how careful they are about their usage.
However, a common failure mode for people is "that was an unfortunate incident, but we studied the problem and we fixed it, it can't happen again" right before a different error they hadn't though of explodes in their faces. This is how we can get to 8e9 deaths: people keep pushing the models to do more, and keep lying to themselves that "this time we fixed all the problems, everything will work, it will be great".
Global warming causing war between nuclear armed opponents. Migration flows and competition over ever shrinking resources.
It's silly to treat these things as separate categories as if only one will happen at a time. These issues are happening, and are going to happen all at once. That is the fundamental challenge. AI plays into that because it can make wars and nuclear exchanges so much more effective.
If an AI tells the President the USA can survive China's nuclear barrage mostly unschathed, perhaps he'll press the button...
Shouldn't we be trying to do something even better than that, where the superintelligent AIs help every human to live a meaningful and thriving life? Perhaps substantially maintaining human autonomy over the trajectory of the future, even while doing things like keeping the peace, preventing people from becoming victims of crime or random disasters or disease.
Completely unrealistic. We could already be working toward such goals without super intelligent AI. I don’t see any reason to assume that ASI created through an evolutionary process would have a different outcome.
If ASI ended up being align-able and not an evolutionary process, why would we assume folks who already are currently in power would change their behavior?
Because I think that a world that is increasingly isolated by technology and divided by ideology is craving authentic personal interaction with real humans. And I think that’s something AI will not be able to duplicate because it requires real human beings.
reply