• joey_builds@eviltoast.orgBanned
    link
    fedilink
    English
    arrow-up
    91
    arrow-down
    3
    ·
    3 days ago

    What gets me is that this is roughly the best-case scenario for “capable hands” — Linus knows this codebase better than anyone alive — and a race condition plus a 9% read regression still slipped through on the first attempt. Reviews and testing caught it, which is the system working, but it moves the bottleneck from writing code to reviewing it. And review doesn’t scale the way generation does; skimming a patch is quick, but actually proving an LLM-assisted one is correct takes real effort. I say that as someone who uses AI tooling daily — reading code you didn’t write is genuinely more tiring than writing it, and that tax compounds the more you lean on generation. The maintainer-burnout angle might end up being the real story here, more than the AI one.

    • raspberriesareyummy@lemmy.world
      link
      fedilink
      English
      arrow-up
      31
      arrow-down
      10
      ·
      3 days ago

      reading code you didn’t write is genuinely more tiring than writing it

      What I keep saying. Good take except that you don’t use “AI” tooling, you use copyright evasion slop tooling.

      • AppleTea@lemmy.zip
        link
        fedilink
        English
        arrow-up
        5
        arrow-down
        2
        ·
        2 days ago

        I thought code couldn’t be copywritten? It’s instructions, isn’t it?

        • 1995ToyotaCorolla@lemmy.world
          link
          fedilink
          English
          arrow-up
          4
          ·
          2 days ago

          I think it’s still murky as far as AI generated code goes. (IIRC You can’t, but not sure enough to say so authoritatively)

          But code historically has been very copyright-able. Oracle’s business model was mostly buying companies and then suing people for infringing on their newly-acquired IP.

    • Orygin@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      19
      arrow-down
      9
      ·
      3 days ago

      Linus knows this codebase better than anyone alive

      Does he? Is it even possible to know the entirety of the Linux kernel within one head?

      • partofthevoice@lemmy.zip
        link
        fedilink
        English
        arrow-up
        22
        arrow-down
        4
        ·
        3 days ago

        Verifying 10 pounds of incoherent architectural choices and over engineered submodules is certainly harder than reviewing tee bits of coherent human work.

          • partofthevoice@lemmy.zip
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            1
            ·
            2 days ago

            LLMs make dumb shit by default. Even Linus had a race condition caught in review when he used AI to bulldoze his way through a feature.

            Agentic loops have major flaws which humans don’t, a tradeoff you seem to keep ignoring while you suggest that developers and agents are comparable in their output quality. The only comparison is, at surface value, they both produce code. Deeper than that, there are plenty of tradeoffs.

            Yes tradeoffs in code quality, project quality, maintainability, technical debt, … you need to be a good programmer to get a good project out of an agent — and even that isn’t enough at times.

            • At the architectural level, there’s definitely a difference yes. But at the smaller scoped implementation level the latest models are pretty good compared to humans. Not perfect by any means, sure, but it’s not all drivel anymore. It was 2 years ago, but today’s definitely different.

              Race conditions are generally harder to catch, even for human devs. And you’re saying it as if Linus wouldn’t have introduced it had he coded everything by hand, which I don’t think is remotely guaranteed (like he never makes mistakes). And the review system clearly works just fine still.

              Good developers can accelerate their work with agents, and you’re right that you can generate tons of LLM-powered slop with it too. But that’s why dismissing anything an agent wrote immediately is also wrong. The review determines if it’s good enough, whether a human or an agent wrote it.

              • partofthevoice@lemmy.zip
                link
                fedilink
                English
                arrow-up
                2
                ·
                22 hours ago

                I’m not really discussing smaller scoped patches, though. An LLM can change a single line in a file, sure, but the prerequisite there is that the file adequately contains the necessary context to do so. With small patches, you’re advantaged in that way.

                Larger patches suffer from a distinct kind of problem where the LLM can heavily over optimize submodules within the code, without proper consideration for the larger premise. This is a problem I’ve seen referred to as “decoherence.” In large projects, it can sort of take over the codebase after enough LLM engagement.

                Decoherence makes your codebase unmaintainable. The cost of every change increases logarithmically and the situation gets worse. The heavily optimized submodules are often incompatible semantically, so the agent also optimizes channels between them. This is an explosion in unwarranted complexity. This isn’t a human problem.

                Good developers can accelerate their work with agents. However, back to the topic on hand, reviewing agentic code changes is necessarily harder — because it’s prone to more issues that run far deeper. You have to review the code almost adversarially, with more cognitive load being used to unravel the gargantuan mess of barely sensible changes when viewed from a birds eye.

                • BJW@lemmus.org
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  7 hours ago

                  You can also have multiple models review the result, seek consensus, and then review the results of that report. I find that’s the best way to find legitimate issues, prior manual review. Huge time saver because the multitude of models can converge while you make yourself a cup of coffee, and you can find things that need to be fixed before you ask someone to spend their precious time in reviewing a pull request.

                  • partofthevoice@lemmy.zip
                    link
                    fedilink
                    English
                    arrow-up
                    1
                    ·
                    7 hours ago

                    You don’t throw more oil on oil fires.

                    Dont get me wrong, you can’t use AI to speed things up. But if you’re facing the problems I’m describing, then you need a targeted and explicit recovery effort — not just more bots.

                    Remember part of the problem is the increased cost of every subsequent change.

                    Humans will have this little voice in the back of their head, always screaming at them about how the architecture of an app should be adjusted to better account for a feature. We’re constantly reorganizing our apps, or backlogging features because we know that it should come with a reorganization.

                    By contrast, an agent just does as it’s told. When you start using agents in a manner that outpaces your capacity to review and understand the breadth of its work, you’re necessarily ignorant of what changes should and shouldn’t necessarily be occurring. It creates the conditions for the problem which I’m referring to. Also, AI is really good at moving fast — so it created the preconditions of the conditions for the problem which I am referring to. It’s turtles all the way down.

        • Railcar8095@lemmy.world
          link
          fedilink
          English
          arrow-up
          9
          arrow-down
          10
          ·
          3 days ago

          And verifying 10 pounds of incoherent architectural choices and over engineered submodules of human work is certainly harder than reviewing tee bits of coherent agent work.

          You are? I can also make false dichotomy fallacies!

          • partofthevoice@lemmy.zip
            link
            fedilink
            English
            arrow-up
            12
            arrow-down
            1
            ·
            edit-2
            3 days ago

            The difference is that AI is prone to this issue, even a single agentic process. Humans only really run into this issue with large, disorganized teams.

            Humans burn out.

            https://maevyn.codes/decoherence

            edit: hold up. Presenting as though what I said works in reverse,

            verifying 10 pounds of incoherent architectural choices and over engineered submodules of human work is certainly harder than reviewing tee bits of coherent agent work.

            as though agentic and human contribution is equatable… wouldn’t that be false equivalency? Tisk tisk…

            Human work is not swappable with agentic work without tradeoffs, which is kind of why I was making a point at all.