Zos_Kia

I’d love to take the credit but i actually stole it from that link that made the rounds on Hacker News

Dude, we work for the same company and I could have typed that in, and maybe I did. I wanted your experience with it, that’s why I asked you.

To me it’s like sending the “let me google that for you” link to answer a question. It’s just bad form. I don’t want your whole reasoning trace man, i just want to know what you understand of it and maybe you’ll catch some detail i’m missing or whatever. It’s simple, i won’t read LLM output, my colleagues know it and i get shit for it but no i am not digesting this material for you. Give me a 3 bullet-point version in your own words, the point is not just in the data exchange it’s also to make sure you are aware of the answer and we have a common truth.

Or failing that, just give me the fucking prompt and at least i’ll know if you understand the question.

I see, that’s interesting. I do a lot of transcoding but offline so i don’t have usage for such a cache. I’ve tested various storage solutions but on my setup, transcoding is always CPU-bound, even on old ass HDDs the bottleneck is never I/O.

Yeah i specifically don’t do any AI workloads on my server, that would be stupid slow with my old ass hardware. But my buddy (who’s a bit impulsive with money) bought two Spark GX10s and we’re likely to get some fun out of them :)

VMs mostly

oh yeah i see how that can be hungry

What are you hosting on Minecraft that isn’t using >=4 gigs?

Just a vanilla server i play on with my son, it’s got 2G and i haven’t noticed anything out of the ordinary. Chunk gen is slow-ish but i suppose that’s CPU-bound.

BTW i exagerated in my initial comment, i looked at the machine and it’s sitting just under 8G of used RAM.

Also ZFS

Jesus christ 😅 no idea if you’re jesting

Serious question, what does RAM help with in the context of self hosting? I recently bought 32G for my server, and it’s DDR3 ecc so it’s so cheap I could have afforded 64 but I just kept wondering what will I use it for? I rarely go north of 6G usage and that’s with half a dozen services, a Minecraft server etc… I just don’t know what kind of services are RAM hungry.

Oh man believe me I’m all for it. I totally understand having an approach of engineering that is not bankable or tailored for Californian degen culture.

I’m not saying there’s anything wrong with your stance. Just saying it will become an aesthetic niche just like there’s some people who still track music on magnetic tape when it would be exponentially faster to use cubase.

I don’t have your specific axe to grind against AI but my personal angle is to only use old hardware and make software that runs on it.

Not everything has to be superlative, and self imposed constraints are great for quality of life.

Exactly. And a commit is a commit. Unless it’s 10Kloc in one go you can just read what’s in it and decide for yourself.

At my previous job we used to jokingly (?) tell our engineering manager “no commits, no opinions” well I think it’s kinda like that.

And that’s great for you but I still think you’ll be in a minority. Which is not necessarily bad of course.

Open Source devs mostly come from the industry and the penetration of agentic coding in the industry has been massive over the last six months. I don’t think I’ve ever seen anything of this scale.

I think disclosure is good and should be tackled as soon as possible because being transparent in your communication is just good practice in general.

However I feel like this will soon be rendered useless as all projects will move to agentic (or otherwise ai-assisted) coding.

Maybe there’ll be a movement of hand coded FOSS but realistically they’ll have a hard time. Resources are already tight for most projects, and rejecting productivity in favor of aesthetics is a rich guy’s strategy.

This whole debacle is showing that people fundamentally misunderstand how code works. They are trying to declare code good or code bad because of some silly heuristics like ai/not-ai, as if it wasn’t literal lines of text which you can read before you form an opinion and make a fool of yourself.

Yes i saw that benchmark and was honestly not surprised with the results. It seems that Anthropic really focused on those issues above and beyond what was done in other labs.

Honestly Claude is not that sycophantic. It often tells me I’m flat out wrong, and it generally challenges a lot of my decisions on projects. One thing I’ve also noticed on 4.6 is how often it will tell me “I don’t have the answer in my training data” and offer to do a web search rather than hallucinating an answer.

I think a lot of it is still done by hand, and there is also synthetic data distilled from larger models of course.

I mean, making fuel is one thing

Actually i was more thinking of crude metallurgy and materials processing. You could quite easily get aluminium from lunar regolith, and also tons of silicates. This allows you to produce shielding, radiators and the structural elements of solar panels without having to kaboom-boom the tons of raw material from the Earth. And it’s not particularly high-tech stuff either, just some furnaces and basic extruding would go a long way. If you just have to ship the delicate electronics from Earth you’re already saving a lot.

Interestingly NASA had an idea of a plan that sounds at least technically possible, but it’s a multi-decade operation and doesn’t look anything like what the current startups are pitching. Of course you can have your data centers in space, why the fuck not, but a data center sits on top of a lot of boring old infrastructure which nobody’s excited to talk about.

It’s going to be prohibitive if you have to pay the gravity tax every time you want to move 1 ton of metal, so realistically this kind of high-tech project cannot even begin without having substantially industrialized the moon. Nothing fancy but you’ll need at least some mining and refining, and solid trans-lunar logistics routes. Probably some housing for a bit of personnel too. At that point the space data center would be dwarfed by the size of its own support system.

The best way to learn to write is to write and have someone critique you. That someone can be an AI it doesn’t change anything about the process, as long as the initial input is your own best effort and the final result is your own edit based on the feedback you received.

That’s an excellent point! On that topic I recently listened to an interview of the founder of EleutherAI, who focuses on training small language models. She said they were able to train a 1B parameters reasoning model with 50K Wikipedia articles and carefully curated RL traces. The thing could run in your smartphone and is at parity with much larger models trained on trillions of tokens.

She also scoffed at Common Crawl and said it contained mostly cookies and porn. She had a kind of attitude like “no wonder the big labs need to slurp trillions of tokens when the tokens are such low quality”. Very interesting approach, if you understand french I can only recommend the interview.

OK that’s a fair observation. Honestly my naive guess would be that they simply do not optimize mainline gpt models for the kind of use case you generally have on Api (tool use, multi-step actions, etc…). They need it to be a perky every day assistant not necessarily a reliable worker. Already on gpt-4 i found it extremely mediocre compared to the Claude models of the same time.

I think that’s a more likely explanation than model collapse which is a really drastic phenomenon. A collapsed model will not just fail tasks at a higher rate, it will spit garbled text and go completely off the rails, which would be way more noticeable. It would also be weird that Claude models keep getting better and better while they’re probably fed roughly the same diet of synthetic data.

The switch you mention (from 4th gen to 5th gen GPT) is when they introduced the model router, which created a lot of friction. Basically this will try to answer your question with as cheap a model as possible, so most of the time you won’t be using flagship 5.2 but a 5.2-mini or 5.2-tiny which are seriously dumber. This is done to save money of course, and the only way to guarantee pure 5.2 usage is to go through the API where you pay for every token.

There’s also a ton of affect and personal bias. Humans are notoriously bad at evaluating others intelligence, and this is especially true of chatbots which try to mimic specific personalities that may or may not mesh well with your own. For example, OpenAI’s signature “salesman & bootlicker” personality is grating to me and i consistently think it’s stupider than it is. I’ve even done a bit of double blind evaluation on various cognitive tasks to confirm my impression but the data really didn’t agree with me. It’s smart, roughly as smart as other models of its generation, but it’s just fucking insufferable. It’s like i see Sam Altman’s shit eating grin each time i read a word from ChatGPT, that’s why i stopped using it. That’s a property of me, the human, not GPT, the machine.