They all got caught in an unsustainable build out. They are all under pressure to find some gentleman’s agreement to slow it down so they have more time before investors realize they aren’t getting their money back. It is the usual tech scam of fake it til you make it. When you aren’t going to make it you need to find excuses and delay.
And the big problem for them is that they have no leverage of Chinese labs, and it’s hard to ban open models.
This “we have to kill the planet first or China will beat us to it” narrative is such bullshit. There’s *zero fucking chance" we ever catch up to them in any meaningful metric. Especially now that our schools are churning out brain dead dependents.
The US used to have the brain steal advantage were every PhD in the world wished to work in a research institute or university in the US, but I don’t think that’s true anymore and they’re going to start to lose people to Europe and Asia
It’s the wall thing, clearly.
There is clearly a push coming from these companies in the past month to present AI as something extremely dangerous. The way I see it, it’s just marketing for the industry to keep the grift bubble going a while longer still: (1) there’s no such thing as bad press, (2) if it’s dangerous it must also be good. IMO public statements like this is just another element in that marketing campaign.
Maybe the bubble is closer to bursting than I imagine and they’re trying to stretch it out until the US midterm elections end.
I suspect in reality it’s a combination of all those reasons, probably in different amounts for all three.
The bond market turning means they can’t borrow money cheaply anymore. They are looking for an excuse to reduce capex.
You remember how agreements like this used to be called a cartel and how a country should have a functioning and independent government body to prevent these?
It might also be that Altman and Musk are deliberately lying, intentionally feigning agreement to encourage other models to slow down while they quietly ramp up development.
It seems all the US models want their AI to be the one that becomes hostile, escapes confinement and attempts to dominate the world.
(In reality, they want their own AI to be the one that is able to obediently dominate the world, which is just as bad a scenario for the rest of us.)
lol the xkcd is sort of funny… even when faced with some varying flavors of doom humans feel the need to flex on people they think are dumber than them.
I don’t think Randall Munroe would argue others are dumber than he is, only that he’s seeing a scenario that is not (as frequently) reflected in fiction and thought experiments as the classic AI gains sentience, escapes containment and takes over scenario.
I’d point to Linus’ Law, Given enough eyes, all bugs are shallow. (Wikipedia notes the law’s validity is confirmed empirically, but refuted in analysis.) This applies to more than software. Right now, election enthusiasts in the US are combing election law and procedure for ways the November election can be thrown, and ways to counter that scenario quickly.
I was thinking about the misprogramming scenario of AI escape. This implies hyperintelligent AI is dangerous even when everyone using it is benign and has non-adversarial intentions. Essentially an AI will need to be coded with an ethics system such as that of a general (or high-ranking soldier in the US) who deliberates on operational action and its consequences, whether the operation is legal, or would cause undue harm in its execution.
Right now, AI in its current development is… whimsical, at best. That is to say, it ignores rules the way it hallucinates facts. And AI can be manipulated to ignore rules through adversarial input, so AI is not ready to operate dangerous machinery.
I don’t see how anything they do can possibly affect what Chinese labs are doing. And that’s the only alternative to American labs right now. So, who are they going to convince exactly?
The point is to get laws passed in the US that create a moat for them as businesses. They don’t care about competing with China, they care about competing with the next YC cohort.
That’s definitely a plausible option, but it’s going to be very hard to ban use of open models. They could get use of official Chinese services banned, but justifying why OpenRouter and others can’t run them is going to be a lot harder. And there’s also a ton of money invested in all these AI companies running on open models now. So, the pushback will be significant.
It would be a lot easier to regulate open models if they perform regulatory capture first
Seems like one of several last ditch efforts to keep the bubble going IMO. They’re flailing
yup
It’s possible they are going to push the chinese labs to do the same. Doubtful it will happen. So they’ll go back developing AI and pretend nothing happened
The problem for them could end up being that the economics simply don’t work. If more capable models are more power hungry, then operating them might be too expensive to justify. Or it could be that there are diminishing returns, and they simply can’t make a model that’s significantly better than the current frontier.
I doubt the Chinese companies will comply, even if they agree on the surface. Whoever releases the most powerful model when the truce ends, will have the advantage. If anything, research and training will continue, releases will slow down.
They have no leverage over Chinese labs, and China has every incentive to continue developing this tech. The only real explanation I see here is that they’re starting to get into diminishing returns territory, investors are getting edgy, and the costs of running this stuff are exploding.
i’d say that’d be a worse development for humanity if it ends up obeying the epstein reich
This isn’t cartel behavior at all
I propose we “pace the frontier” by sinking all three of these mfers into the deepest darkest pit of the sea.
I agree with lightnsfw
Option 3: JP Morgan, Goldman, etc told them to settle down or they’ll get throttled economically.
yup, that’s a totally valid option if the costs for their bigger models are going through the roof
They must realize by now that they have no business model viable enough to repay the money they burned for the last two years, and if they are given an excuse to stop, they can tell their investors they are not responsible for the absence of any ROI.
I’m not sure Chinese labs are even going in the same direction as the AI projects in the US. They’re working to see what they can do with a (more) reasonable amount of buildout, rather than building data centers from horizon to horizon.
Also, the Chinese are motivated by seeing what AI can do for a larger society. American AI systems are being refined automation and instruments of control, specifically military and national security interests.
Essentially, the US industry is trying to get AI to train a gun on the entire US population.
Oh they definitely aren’t, there’s an interview with Alibaba Cloud founder where he discusses the direction in China. Basically, the goal is to find useful niches for this tech early on, then iterate and improve. They’re not chasing AGI or trying to make one model to rule them all. That said thoough, the capabilities of Chinese models in the same domains where American ones shine are very close as well. So, I do expect that Chinese models will catch up and start surpassing American ones on their own turf before long. I’m also expecting that the trend will shift towards running smaller and local models for most things because you just don’t need a giant model to do most tasks.
I’m looking forward to when hardware gets cheap enough to try Qwen 3.8
I’m hoping Alibaba will start selling these things at rpi prices https://wccftech.com/alibabas-tsmc-built-5nm-risc-v-chip-xuantie-c950-now-runs-qwen-3-8-27b-model-natively-unlocking-massive-vertical-integration-tailwinds/
It’s because they’re hitting model size constraints. There’s only so much memory bandwidth you can get between racks or even rack spaces and memory bandwidth is the constraint for nearly every ml thing.
Expect a reversal once a more memory dense component hits.
There’s no reason to think that the architecture itself can scale indefinitely. It might very well be that LLMs have some hard constraints on the scope of the problems they’re capable of solving.
Of course, that’s what I’m saying. Physical constraints of hardware mean there’s a limit to how much further (read: larger in terms of working memory footprint, because that’s how they’re getting “better” and better “frontier” models) development can continue until a more dense component comes along.
Every singularity a sigmoid.
I meant that simply making models bigger might not actually make them more capable. So even if you had unlimited hardware to play with, you might have to find a different approach.
You could create a way to measure the idea of capability that would bear that out but from a pure discrete mathematics perspective, no, you only get better with a larger memory footprint.
There’s a lot of ways to make that faster or make that behave like a process running on a bigger memory footprint, but ultimately that’s the constraint.
And companies competing in the field of ai can’t justify the expense of cutting down their gigantic model to only know how to identify wood because that has a known and limited impact. They already said they’re shooting for unlimited immeasurable impact on the scale of replacing all human labor and got massive funding for it.
It doesn’t matter if it’s easier to do one backflip, you asked me to triple dog dare you to do a million backflips. Well… we’re waiting!
Again, there is no reason to think that you can just keep making the model bigger and keep getting improved capability that way. In fact, we already know that’s not the case because simply making them bigger stopped being the focus. The real breakthrough is going to come from better algorithms.
You said there’s no reason to think you can just keep making the model bigger and keep getting improved capability.
there’s the structure of the neural network itself. Fundamentally, adding nodes and layers increases the ability of the model to handle more complex input.
Then there’s the actual models we see in use. They are literally as large as the hardware allows. The only reason to use smaller models are to fit some constraint.
So both by the book and in practice bigger is always better.
Now we can’t always go big. I can’t afford to purchase a dgx or even upgrade my wiring to power it, let alone pay the power bill it would rack up or all the other utilities alone when my wife leaves me because of the sound.
My computer can only fit so many expansion cards and pcie is so slow compared to hbm that I’m better off running a small model quickly that fits on one card as opposed to a larger one slowly across several cards.
But those are all constraints. When I replace my motherboard with supermicro gpu host fabric I no longer am limited by the pcie bandwidth and can quickly use models that fit across several cards.
I do agree with you that the future is smaller models, not because of the fundamental nature of the concepts involved but because of the complex constraints that are coming into play.
The problem is with the context and data propagation through the network. As you keep making it bigger it becomes slower and less focused. And there is research showing that smaller models do outperform large ones on some tasks https://cacm.acm.org/news/bigger-not-necessarily-better
What I expect we’ll see going forward is more hierarchical architecture where you have finely tuned models for specific tasks with a general routing model on top. This is basically already where MoE architecture is moving now. We might also see stuff like neurosymbolics get more popular where the LLM acts as a stochastic engine within a symbolic logic system. The model can handle noisy input from the real world, and transform it into structured data that a symbolic engine can operate on.
Brute forcing the problem is a naive approach and US labs took it because they effectively had unlimited resources to train their models until now.
And when more compute becomes available, solutions that are more efficient are going to further benefit from that as well. We see this with DeepSeek right now. They focused on efficiency over capability up front, and now they have a fundamentally cheaper architecture that’s rapidly catching up in capability.
Ligmoid
It’s the energy crisis. They realized “oh shit money is real now” and are trying to sooth investors.
Interested how Monday will go. This is not what investors want to hear, even if it’s softened by markets being closed.
The whole bubble might be about to pop.
Have been hearing that for 2 years now
The market can stay irrational for a long time but it does eventually have to happen. The 2008 crisis was the result of overleveraging in 1999-2003 and interest rate rises from 2004-2007, so it can take years for it to crumble. Two years is a long while but it could easily take two more before people realise the emperor has no clothes.
The nature of a bubble means that the longer you go the more likely it actually is. And this is a bubble I frankly can’t wait to pop.
Significant portions of Lemmy are turning into debate bro bro hreads on AI and its relevance.
I heard on the Framework subreddit that they just secured a cheaper batch of RAM and refunded their customers, so could be something or maybe it’s just a blip. I’m also wondering if the recent Flash model releases are disrupting investor interest in the AI datacenter projects especially since running them on local hardware is starting to become more feasible, and for businesses in particular possibly more cost-effective.
“AI” seems to be best put to use performing specific automated tasks. This is something a lot of us have been saying for a while now, but they tried to make what is essentially an industrial tool (as in boutique AI for specific industries), into both slide deck candy for enterprise C-suites, and an expensive consumer plaything. Enterprise customers demand real results for their money, and consumers do not care about the novelty of generative AI enough to recoup the costs of this insane level of infrastructure development. Add what is essentially shaping up to be runaway inflation, and well, yeah. No one cares about this shit.
They came at it like this because they are Silicon Valley and for the better part of 3 decades now, they have operated under the assumption that they are always right, and that if they can survive running on a loss at first, eventually they will acheive market dominance and monopolize their product. I just don’t see it happening. I see this more like the early dot com days. A lot of investment, and a lot of failures. But eventually, for better or worse, a few of them figured out how to make it work. This time around however, capitalism itself is failing.
I think it’s a combo of this and the fact that “intelligence” continues to not scale with the inputs as they’d hoped. I haven’t been keeping up with developments, but it feels like they’re finding new ways to discover that the brain is in fact an miraculously efficient and effective mechanism. Most their party tricks seem to be reproducing brains more minor processing feats in interesting ways. All the promised advancements have stagnated and what remains is just exploitation of economies of scale and its consequences, which is far from the revolution investors were promised.
I’ve written an essay
I doubt that
They all must have figured out by now that they are hitting a limit. I honestly don’t think LLMs will lead us to AGI. I’m sure it’s a step on the path to it, but I’m think it’s a lot further than most think.
So, they make this “agreement”, then the slowdown is just being “responsible” so that the investors don’t panic. Meanwhile they all go full tilt behind the scenes to try to find the next breakthrough.
That’s my view as well, LLMs are likely just one piece of a much bigger puzzle and we’re now hitting the limit of what you can do with them in practical terms.
It’s become so focused on LLMs that that’d be a better outcome, some new research in a new direction
They hit a wall and want to prepare everyone for the fact that they won’t be able to meet the expectations that they themselves created.
Maybe they are running out of electricity / data centres / some other requirement?
That’s also very possible. The US grid has very little spare capacity, and building out more will be a decades long project. So, if their newer models are more power hungry, then they might not be economically viable even with all the investor money being thrown at them.
I expect more power efficient chips that are ai specific to come out in the next few years. Eventually you’ll be able to run good models on your phone. Not sure about ram requirements or anything like that if the model could be shrunk down somehow. There’s definitely huge gains in optimizing efficiency to be had. Right now is the equivalent of an old IBM mainframe trying to do a spreadsheet. We might even giggle at the thought of gigabytes of ram in the future with having multiple terabytes as standard on personal devices.
I expect we’ll start seeing stuff like Taalas where they print the model to the chip and other specialized chips like Xuantie C950 going forward. Neither of these requires DRAM, and Taalas is particularly clever since they just print the model right to an ASIC chip. So, the whole renting out LLMs business model isn’t going to last long I suspect.
So a repeat of the crypto crash for graphics cards when ASICs ate their lunch. Mind you, that’s only for inference (although a super fast QWEN 3.8 would meet a lot of peoples needs).
The argument for datacentres is for training the models, but then they’ll need to prove that they haven’t hit a diminishing returns wall, which will be hard if, as seems likely, they have. Also the Chinese have been doing it in a cave, with a box of scraps (figuratively), and gotten at least 90+% as good results.
Seems like the recent advances have been in the frameworks, which don’t need no stinking (literally if fossil fueled) datacentres.
Right, I’d argue that China proves you don’t need massive data centers for training. And yeah, I think something like Qwen 3.8 is more than enough for tasks most people do. There are a lot of tricks you can do as well with the harness, where there’s a lot of attention is shifting now. And it’s a lot cheaper and faster to develop better harnesses than train new models. I expect we’ll start seeing a shift towards neurosymbolic systems before long where the LLM acts as a stochastic component within a symbolic logic engine.
If so it amuses me that investing in maintaining and upgrading public infrastructure via taxes might have saved them the choke point
Not even the first time this is happening. The US & Co. are concerned about being technologically overtaken by Japan / China, but don’t make their universities affordable.
Even more ridiculous then since for quite awhile their students have been keeping our universities fiscally solvent and now we’re discouraging them from participating and cutting support for our universities at the same time.
I can only believe this is willful
Musk: Yes, we should all slow down. With no external verification and upon the agreement of this handshake, we should all stop developing so fast. We, especially, will slow down. You can trust us.
Actually, what does that look like? I assumed from the expansions the bottleneck wasn’t algorithmic, i.e. each data center that they bring online was designed from the ground up to be at 100% all the time. Is that not the case? Can you even (for lack of a better metaphor) underclock your datacenter? Does the cooling work that way?
For that matter, are they doing that trick where the datacenter is owned/operated by Independent DC Company X, and has exclusive lease agreements for compute?
I’m guessing they’ll stop funding / building new data centres? Not all the ones announced have been built, and not all those built are operational.
That was kind of why I asked. I don’t know enough about that world, but it seems like as soon as ink is on paper then whatever company signed on to build it starts baking their profits into their corporate calculations and planning for staffing, etc. I’m sure lawyers know all that stuff going into those types of talks and baked penalties into the contracts (or whatever). But can you just “oops, our bad” out of buying up so much of the world’s expected RAM supply, so much of the property (some through eminent domain), so much cooling and energy capacity, signing construction contracts, etc, and walk off? That risk is sitting somewhere, and with NASDAQ futures only being down 1.25% as of right now I’m not sure what to think.
But we’re living in the future, so I’m sure it’ll wind up being the local municipality catching hell for it. All the tax breaks they paid for companies to bring in “jobs” won’t amount to anything, and the land will all have been acquired.
There will have to be a reckoning somewhere. There simply isn’t enough electricity in any country except China to power data centres at the scale they are being proposed. As to who will end up holding the bag, you are probably right. I wouldn’t be surprised if the contracts between companies and the municipalities are designed to let the companies get away, either through limitations on liabilities, binding arbitration, or some other legal trick.
Like someone else said (in this thread or some other) the places they’re putting the datacenters need to be thinking in terms of infrastructure upgrades: water, fiber, and electrical lines, and they need to mandate putting the datacenter in some vacant industrial area that needs cleanup from some industry that moved to china in the 80s. Instead they’re thinking in terms of jobs that just aren’t going to manifest (I believe). There’s already that video of the muni guy refusing to answer whether he’d signed an NDA or not. I wonder if the deal he signed was contingent on success of the venture.
At least with this announcement it seems like the winds are blowing more in the direction the people predicting a bubble said it would. AAPL became a trillion dollar company in 2018. Now NVDA is worth more than 5 trillion with AAPL on its heels. If it goes it may take some time to come back.
