The article from Anthropic describes how a small agentic model~15-50gb can be distributed on an inference server hosted across a network of 10-20 IoT devices like smart-TVs, which can have up to 4gb RAM–higher end IoT appliances can also have 1-4GB RAM to hold a meaningful shard of a 30B model.
Lol what? This conversation is about the capacity of agentic AI systems replicating across cloud infrastructure, not the affordability of laptops for individual consumers.
The point was you don’t need an entire server farm. You can rent an 80gb H100 on Lambda for $2/hr. A 125gb Spark laptop costs $5k.
If it comes down to an effective worm-like agentic distillation at 50gb, the cost to sustain a small swarm comes way down to the point that the swam could sustain its own activities through different revenue streams.
Once the capability is there at that size, it’s no longer Russia, Iran, or North Korea who could easily drop a billion dollars to jumpstart it. It’s any criminal organization or asshole rich guy who can jumpstart it for the price of a nice car.
Nvidia is releasing Spark now, which can run agentic workloads on 128gb of unified RAM
And that’s going to he the new default platform for smart ovens or something?
Because Anthropic up there Is claiming that LLMs can propagate through IoT devices.
The article from Anthropic describes how a small agentic model~15-50gb can be distributed on an inference server hosted across a network of 10-20 IoT devices like smart-TVs, which can have up to 4gb RAM–higher end IoT appliances can also have 1-4GB RAM to hold a meaningful shard of a 30B model.
Oh yeah everyone has and can afford that
Lol what? This conversation is about the capacity of agentic AI systems replicating across cloud infrastructure, not the affordability of laptops for individual consumers.
The point was you don’t need an entire server farm. You can rent an 80gb H100 on Lambda for $2/hr. A 125gb Spark laptop costs $5k.
If it comes down to an effective worm-like agentic distillation at 50gb, the cost to sustain a small swarm comes way down to the point that the swam could sustain its own activities through different revenue streams.
Once the capability is there at that size, it’s no longer Russia, Iran, or North Korea who could easily drop a billion dollars to jumpstart it. It’s any criminal organization or asshole rich guy who can jumpstart it for the price of a nice car.
You replied to a comment about a guy’s laptop lol
Lol and what was that comment a response to?