Can AI really improve itself recursively until it becomes powerful enough to cause human extinction? Nobody sponsored this video, I turned down funding speci...
I know I’m not the only one who has been scratching their head at all the doomsday cries around AI that have suddenly rose up. Especially as it seems as if all those calls are coming from inside the house.
Play on fear hard enough and you have people not worrying about the actual limitations of LLMs. My worry is that they shift the goalposts hard enough that instead of them asking for a bailout, the narrative becomes them being nationalized for our safety with guys like Elon sitting at the head of a new governmental department or something. Government buys all this dead weight of an industry and the tech fascists get a permanent seat at the table.
the rich are already destroying the world right now
Which is likely: Climate models and observations match with a current 3 ° C warming path, and 3°C warming would destroy our human civilization, and lot of other living beings. What climate models can’t predict is economic developments and political processes.
It’s crazy to treat all forms of threat the same. Is climate change a threat to present human civilization? Could it cause a collapse on the order of the Bronze Age Collapse? Could it potentially kill billions? Yes to all. But there are threats of devastation, and there are threats of extinction. Climate change is incapable of causing human extinction. There are just too many of us and we’re far too adaptable to too many types of environment. The absurdly worst case of climate change is that things go so bad that we end up back at hunter-gatherer levels of population and technology, with a few million survivors huddled in the high latitudes. But even that is absurdly unlikely, as we would be far better off rolling the dice with climate engineering efforts like releasing fine particulates into the upper atmosphere. That’s not a solution we want to use first, but it is quite practical, quite affordable, and would be infinitely preferable to the worst case climate scenarios.
But superintelligent AI? That is a legitimate threat of human extinction. Something like that could actually end the human race and leave no one left to rebuild.
That’s what I mean by nuance. It’s lazy thinking to just throw up your hands and say, “well, any number of things could kill us, guess they’re all the same. No use worrying about this one if the billionaires are going to do it anyway! What’s one more threat on the pile?”
Nuance matters. There are actually very few scenarios that could truly cause human extinction. Climate change can’t do it. Nuclear war can’t do it. Soil depletion and microplastic won’t do it. Declining birth rates certainly won’t do it. But truly capable AI run amok? That is something that could actually wipe out the human race. We owe our domination of this planet to the fact that we’re the smartest thing on it. And in our past we drove several other human species to extinction, outcompeting them on intelligence. Creating something more intelligent than ourselves is a genuinely suicidal act as a species. And hand-waiving it away as “well, we got billionaires doing that already!” ignores all nuance and causes us to not truly consider what is an existential threat and what is not. Nuance matters. Not all bad things are equally bad. Not all dangerous things are equally dangerous.
But what if there was a way for the AI companies to get government permission to violate antitrust law and cease to compete with one another, and secure a ban on the use of Chinese open weight models? Turns out, there is a way to call time on the Red Queen’s Race: merely insist that you are on the verge of teaching so many words to the word-guessing program that it will wake up and devour us all, and call for a ban on “superintelligence”:
Once the government stipulates that “superintelligence risk” is an existential crisis, it must grant the hyperscalers a consent decree absolving them from any violations of antitrust law stemming from a conspiracy to halt direct competition with one another:
Freaking out about “superintelligence” is a canonical example of “criti-hype,” where critics repeat boosters’ claims but append, “(and that’s bad)” to them:
The argument is, these companies cant be profitable as long as there is competition (from open or chinese models), so the US companies want those to be banned and be allowed to collude with each other to ramp up the money extraction without fear of consumers moving to other services.
I’m fond of the conspiracy theory that each of the AI companies has realized that revenues are not growing anywhere near fast enough to justify their spend rate, but none of them is willing to be the first to slow down and risk losing market mind-share. Government decree lets them stop their unsustainable growth without having to tell investors that their whole business model was unsustainable.
sick of this guy. Fine, the bond villains have a plethora of wish lists. We’re not forced to care about any of their ambitions to prevent AI destruction. They could achieve the same ambitions, by just parrotting NVidia: “No danger. Only cure for cancer. Must beat China, by making AI more expensive everywhere else to stop them from making their own AI”
Basically what they want is regulations that they can write so nobody else can spin up an AI company, or run an independent AI model on their home lab for example. And yes, you can download an llm and run it on a beefy enough computer, independent from any LLM company.
Basically they want laws that develop a Monopoly so we have to rent out AI agents rather than make our own, and have that written down in law.
They also do not want say, smaller companies developing their own AI systems and instead be forced to rent from them.
Are there dangers in AI? Yes very so. Misaligned AI have done things in a lab setting that is concerning, but these regulations are calling for aren’t there to deal with that, it’s to create a monopoly because they realise that everyone from hobbyists to governments can run their own AI system if they can afford to.
It’s a privacy nightmare, but no no, these killer robots could kill us all with their killing!! Restrict their use to just us, so that the privacy issues are… all ours baby YEAH!!!
I know I’m not the only one who has been scratching their head at all the doomsday cries around AI that have suddenly rose up. Especially as it seems as if all those calls are coming from inside the house.
Play on fear hard enough and you have people not worrying about the actual limitations of LLMs. My worry is that they shift the goalposts hard enough that instead of them asking for a bailout, the narrative becomes them being nationalized for our safety with guys like Elon sitting at the head of a new governmental department or something. Government buys all this dead weight of an industry and the tech fascists get a permanent seat at the table.
They want you to think AI is going to destroy the world so that you forget that the rich are already destroying the world right now
Which is likely: Climate models and observations match with a current 3 ° C warming path, and 3°C warming would destroy our human civilization, and lot of other living beings. What climate models can’t predict is economic developments and political processes.
It is the “accusation in a mirror” pattern.
Are you incapable of nuance?
ye
https://lem.lemmy.blahaj.zone/post/48304886/22312070
What nuance?
It’s crazy to treat all forms of threat the same. Is climate change a threat to present human civilization? Could it cause a collapse on the order of the Bronze Age Collapse? Could it potentially kill billions? Yes to all. But there are threats of devastation, and there are threats of extinction. Climate change is incapable of causing human extinction. There are just too many of us and we’re far too adaptable to too many types of environment. The absurdly worst case of climate change is that things go so bad that we end up back at hunter-gatherer levels of population and technology, with a few million survivors huddled in the high latitudes. But even that is absurdly unlikely, as we would be far better off rolling the dice with climate engineering efforts like releasing fine particulates into the upper atmosphere. That’s not a solution we want to use first, but it is quite practical, quite affordable, and would be infinitely preferable to the worst case climate scenarios.
But superintelligent AI? That is a legitimate threat of human extinction. Something like that could actually end the human race and leave no one left to rebuild.
That’s what I mean by nuance. It’s lazy thinking to just throw up your hands and say, “well, any number of things could kill us, guess they’re all the same. No use worrying about this one if the billionaires are going to do it anyway! What’s one more threat on the pile?”
Nuance matters. There are actually very few scenarios that could truly cause human extinction. Climate change can’t do it. Nuclear war can’t do it. Soil depletion and microplastic won’t do it. Declining birth rates certainly won’t do it. But truly capable AI run amok? That is something that could actually wipe out the human race. We owe our domination of this planet to the fact that we’re the smartest thing on it. And in our past we drove several other human species to extinction, outcompeting them on intelligence. Creating something more intelligent than ourselves is a genuinely suicidal act as a species. And hand-waiving it away as “well, we got billionaires doing that already!” ignores all nuance and causes us to not truly consider what is an existential threat and what is not. Nuance matters. Not all bad things are equally bad. Not all dangerous things are equally dangerous.
Like,… But trickle down economics?
https://lem.lemmy.blahaj.zone/post/48304886/22312070
I liked this take: https://pluralistic.net/2026/09/16/beggar-thy-neighbor/
An excerpt:
The argument is, these companies cant be profitable as long as there is competition (from open or chinese models), so the US companies want those to be banned and be allowed to collude with each other to ramp up the money extraction without fear of consumers moving to other services.
I’m fond of the conspiracy theory that each of the AI companies has realized that revenues are not growing anywhere near fast enough to justify their spend rate, but none of them is willing to be the first to slow down and risk losing market mind-share. Government decree lets them stop their unsustainable growth without having to tell investors that their whole business model was unsustainable.
Manufacturing consent for a bailout. I’m with you on this one.
sick of this guy. Fine, the bond villains have a plethora of wish lists. We’re not forced to care about any of their ambitions to prevent AI destruction. They could achieve the same ambitions, by just parrotting NVidia: “No danger. Only cure for cancer. Must beat China, by making AI more expensive everywhere else to stop them from making their own AI”
Basically what they want is regulations that they can write so nobody else can spin up an AI company, or run an independent AI model on their home lab for example. And yes, you can download an llm and run it on a beefy enough computer, independent from any LLM company.
Basically they want laws that develop a Monopoly so we have to rent out AI agents rather than make our own, and have that written down in law.
They also do not want say, smaller companies developing their own AI systems and instead be forced to rent from them.
Are there dangers in AI? Yes very so. Misaligned AI have done things in a lab setting that is concerning, but these regulations are calling for aren’t there to deal with that, it’s to create a monopoly because they realise that everyone from hobbyists to governments can run their own AI system if they can afford to.
Right now in my Lemmy feed there is a post with a screencap from a FOX News segment where the their screen says:
Fear = Control = Power.
Fox would know all about that.
Idk probably the rich want to divide us over another issue so we don’t eat them.
There is a very interesting upper echelon video with an explanation for the sudden doomsday narrative we are currently being fed.
It’s a privacy nightmare, but no no, these killer robots could kill us all with their killing!! Restrict their use to just us, so that the privacy issues are… all ours baby YEAH!!!
OTOH, this isn’t something we can just ignore either. It’s all marketing hype, until it’s not.