128
StoryScope: Investigating idiosyncrasies in AI fiction
arxiv.orgAs AI-generated fiction becomes increasingly prevalent, questions of authorship and originality are becoming central to how written work is evaluated. While most existing work in this space focuses on identifying surface-level signatures of AI writing, we ask instead whether AI-generated stories can be distinguished from human ones without relying on stylistic signals, focusing on discourse-level narrative choices such as character agency and chronological discontinuity. We propose StoryScope, a pipeline that automatically induces a fine-grained, interpretable feature space of discourse-level narrative features across 10 dimensions. We apply StoryScope to a parallel corpus of 10,272 writing prompts, each written by a human author and five LLMs, yielding 61,608 stories, each ~5,000 words, and 304 extracted features per story. Narrative features alone achieve 93.2% macro-F1 for human vs. AI detection and 68.4% macro-F1 for six-way authorship attribution, retaining over 97% of the performance of models that include stylistic cues. A compact set of 30 core narrative features captures much of this signal: AI stories over-explain themes and favor tidy, single-track plots while human stories frame protagonist' choices as more morally ambiguous and have increased temporal complexity. Per-model fingerprint features enable six-way attribution: for example, Claude produces notably flat event escalation, GPT over-indexes on dream sequences, and Gemini defaults to external character description. We find that AI-generated stories cluster in a shared region of narrative space, while human-authored stories exhibit greater diversity. More broadly, these results suggest that differences in underlying narrative construction, not just writing style, can be used to separate human-written original works from AI-generated fiction.
Abstract page for arXiv paper 2604.03136: StoryScope: Investigating idiosyncrasies in AI fiction



This isn’t what I’d call “reliable”.
I’m also not impressed with their methodology, which is heavily based on Gemini.
That’s too many steps removed from anything you’d encounter in the wild. They’re not even testing against human-prompted output.
And then they use Gemini again to analyze all the stories. Relying on proprietary cloud models for the core of your analysis is like building on sand.
Removed by mod
only 7% of human work got flagged for ai it is reliable though
Percentages are somewhat misleading here. That’s around 2 in 25, enough to falsely accuse a student or 2 in a fairly standard classroom for example.
I remember somebody mentioning a quote along the lines of “A 95% success rate sounds good, until you become the 1 in 20 that it kills”
Removed by mod
Getting your degree or being kicked out and banned from your college and still requires to pay back the loans. IS life or death.
It literally can determine the quality of life of a person for the next 30-40 years. If not forever. This is NOT something to take lightly.
Removed by mod
Holy the ignorant self-centered anecdote. Too long, no relevance, do not recommend. I‘d probably give it a failing grade.
Removed by mod
Traits of the ignorant, as one would expect.
Okay so what I’m taking away from this is that you were lucky to find gainful employment without proof of a degree and didn’t have any debt to pay off? Because the context here is people having all the debt and none of the credentials required for jobs where they make enough to pay off that debt. If your card is declined, how you feel about it doesn’t matter.
Removed by mod
Still more than you should have had (education shouldn’t put you in debt imo), but definitely far less than other people have. Particularly people studying in the last few years. .com is long gone, prices have gone up and the margin between net income and cost of living is slim for many people.
That’s my whole point: you shouldn’t downplay the impact that being thrown out can have if you’ve got $50k in student loan debt. If you cheated, sure, that’s on you. But if you were falsely accused by a detector that turns up false positives for one or two students per class, on average, that just sucks like an open window in space.
That’s not good enough. Just a thought experiment: Only 7% of humans died because their work got flagged incorrectly
this is just spherical cow fallacy combined with purity culture slop
no methodology is perfect, it just needs to be practically valid, using ai != killing people for using ai
unless you are for killing people who use AI, which wouldn’t surprise me one bit about Anti-AI horde
Tell that to the student who gets expelled because they’re writing was falsely identified as AI and then takes their own life.
Tell that to the person falsely identified with facial recognition and is arrested and loses their job.
Using AI to detect AI is as bad as using AI for any important decision.
This isn’t hypothetical. It happens. People will lose their livelihood or die because of it. 7 in 100 is not accurate enough when any penalty is applied to detection.
Removed by mod
Removed by mod
To be fair it still is. It’s even worse if your expelled and have to pay back the loan any ways.
Removed by mod
Dude if my code had an error rate of 7%, my boss would not say that this “good enough”.