The AI Score Answers the Wrong Question
AI can help with prose. The author still has to supply the judgment and own the result.
On July 21st, Substack introduced a new “Scan for AI text” feature that allows readers to scan posts to see an estimate of how much of it was written by a human, or with AI assistance. The announcement immediately sparked a wave of comments on Substack, X and Reddit, some of it applauding this feature, some of it condemning it.
Substack’s goal is understandable: it wants to keep the platform from being flooded with polished text that contains little human thought behind it. My concern is that a score designed to detect AI assistance can easily be read as a verdict on value, placing writers who use AI thoughtfully for research, editing, or clarity under the same suspicion as those who outsource the work itself.
At first I wanted to stay out of the conversation as writing is not something I typically cover on my newsletter, however, after thinking more deeply on the problem, I realized that I was wrong.
The very same conversation happening now between Substack and its writers, is a conversation long due in the AI world, it’s a conversation of motives and trust, about moving from generation to judgement, and that’s what this newsletter is in part about.
I’ll leave the accuracy debate to people who understand AI text detection better than I do. My concern is that even a perfect score would answer the wrong question.
A perfect score could tell us that the author used AI to generate some text, or to polish sentences, which is useful information about its process, but doesn’t say anything about the value of its contents. The fact that an author used AI doesn’t prove that the content isn’t worth reading.
And here’s where I differ with some of the anti-AI sentiment, I make a strong differentiation between AI slop, and content with actual value, where the author used AI to generate the text based on their opinion, their experiences, their knowledge or research. So if the author is able to use their judgment, and pass their views and essence while AI produces the polished result, so what?
The relevant distinction is not human vs AI, but rather assistance vs delegation.
The power of Substack as a platform compared to other social networking sites, is that substack promotes a relationship between the readers and the authors, a relationship that’s built on trust. A reader may find your notes, or come cross your article, at first there’s not trust, but curiosity, that sparks to learn more about the author, or read more of their content. Readers subscribe to an author’s judgment: what they notice, verify, recommend, and are willing to correct.
Here’s the problem that I have with the score, it doesn’t talk about the quality of the output, it only speaks about how the presentation was generated, though that may not be 100% clear to readers.
It sends the wrong message to new readers that may discard the content because the tools the author decided to use or even worse, imagine being a long subscriber from a publication, you always loved it, and now you find out the author has been using AI all along, how do you feel about that? Did the author betray your trust? Should authors be more explicit about their use of AI?
In my view, and what it comes to this publication, I use AI to assist my research, to seek for inspiration, and to validate my drafts. I write from my own experience, my own opinions and my own judgment. I don’t let AI do my thinking.
I respect every one of my reader’s time, so what I share in here is genuine contribution, it’s honest, and all I publish, I publish under my name, so I own it and I’m accountable for it. I’ve been writing before AI was a thing, at first in Wordpress, then in Markdown, used many apps, used Grammarly, now I use AI. The tools I use may change, AI makes it a lot easier, but the essence of my content will stay truly mine.
Today is a bit of a different focus, but at the end, it’s all about trust. Create a PR using AI, but own the changes. Write an essay with AI, but own what it says. If your name is on the result, you should be able to explain it, defend it, and fix it.
Thanks to all for being a reader, a subscriber, a friend!
Part of the reason some people like the score is that they don't trust writers to disclose, or to accurately disclose, their process. Ironically, the score itself is generated by a language model.
I totally agree, I wrote this a few days ago, and I am glad someone else is laying out a similar point: Pangram arriving on Substack was inevitable. This is not a good or bad thing. People were already scouting (policing) each other’s writing for AI signals—it’s just that now the thing they hate so much has become the easy-access tool to validate their suspicions. I use AI plenty (although not in this note), but apparently all I have to do, when I do use it on Substack, is provide you with a “How I make this” statement to explain my ridiculous process of wrestling with my self-doubt and the big three AI text editors, so I can “set expectations” for readers.
As if procedural disclosure (fighting the self-made three-model crucible over which edits to keep, revert, or synthesize into something I’ll put my name on) is going to reveal anything the writing itself cannot.
What they don't see (yet) is how the AI-detection tool will be turned against us. In the name of protecting us from AI slop, our capacity to read for understanding—to recognize meaning, or to judge at all—is now outsourced to yet another tool of administration.
The problem is not the writers but the readers. I am neither an AI technophiliac nor an AI technophobe. I refuse to submit to either position because I see both extremes as expressions of the larger problem of authoritarianism. In either case, we are granting technology an autonomous, alienated power before which the subjective freedom of human reflective judgment can only submit or retreat.