- cross-posted to:
- linux@programming.dev
- cross-posted to:
- linux@programming.dev
cross-posted from: https://piefed.social/c/linux@programming.dev/p/2509642/linus-torvalds-posts-his-first-ai-assisted-patch-to-the-linux-kernel-synthetic-test-clo
cross-posted from: https://piefed.social/c/linux@programming.dev/p/2509642/linus-torvalds-posts-his-first-ai-assisted-patch-to-the-linux-kernel-synthetic-test-clo
Our short term memory is our context window. Some people have larger capacity for information, others very little. Same thing, different terminology.
I use Ollama daily. The context limit of 10k is indeed very limiting and noticeable compared to the extended lengths of cloud models. I’ve had to recruit a model specifically trained on condensing salient information to perform context compression frequently. I also use the limited local context to establish effective strategies for maximizing token efficiency in the agent.md.
It’s no different from how many tasks a person can simultaneously perform, or disparate words they can hold concurrently. Eventually thoughts are forgotten. What exactly is determined by the structure itself - the context-pruning strategy inherent. We don’t know what our own are, but I believe we have an equivalent.
All I’m saying, in the end, is that anything that can occur in a physical entity can be emulated in a digital entity. Are we capable of doing that flawlessly yet? No, but it is looking inevitable to me. Neural networks are ubiquitous, and our ability to simulate them will only improve.
Sure, but I still think calling it “memories” is a reach and a bit disingenuous. It’s not remembering anything. It’s re-reading it will the every run, making it appears as if it has remembered. Calling it “reminders” or something else I think would me more appropriate.
The “memories” terminology comes from the big AI companies and they are doing everything in their power to make AI seem smarter than it actually is. A lot of reporters in media has a hard time understanding this as well and will report on AI as having it’s “own thoughts” and “making own decisions”.
ChatGPT breaches confinement because their engineers are stupid. Not because of some self determination to fuck with us or our infrastructure. OpenAI are responsible and let it happen. Same with Anthropic.
In the end an AI is a dumb computer program that only know and does what we tell it to do.