• dan@upvote.au
    link
    fedilink
    arrow-up
    3
    ·
    3 hours ago

    Clean-room slopimentations

    A lot of the recent AI slop software aren’t actually using proper clean room techniques though.

    With clean room reverse engineering, you need two completely separate parts:

    1. Someone that reverse engineers the original program and writes a spec specifying how it behaves
    2. Somebody else, who has never inspected the original program in any way, writes a new program based entirely on the spec.

    For the recent AI slop reimplementations, some of them are essentially one person using two different AI models and agents for these two roles. Some of the potential problems are that:

    • If the person was ever exposed to data from the original app, or has ever inspected or reverse engineered the original software in any way, even just by looking at debugging output or temporary artifacts from the reverse engineering agent, then it’s not a clean room any more.
    • There’s no guarantee that the implementation agent won’t inspect the proprietary software, violating the clean room contract. Telling an agent not to do something won’t actually stop it from trying.
    • There’s no guarantee that the implementation agent didn’t have leaked code from the original program in its training data.
      • Related to this, there’s no guarantee that the model didn’t ingest data from original developer because they used an agent that trained on their data.