Tracking how often I use AI in my writing

In my post “In the know”, I talked about how I want to be transparent in my use of large language models (LLMs) in my writing:

I’ll admit LLMs are a great tool for helping with writing. Feed them enough examples of your writing and tell them what you want to talk about and they’ll do a reasonable job of doing the work for you in a style that can be nearly indistinguishable from your own. But they’re also thinking for you, and this is dangerous (and a bit sad). In my writing, I’ll be transparent when I use an LLM to help me.

Just like spelling and grammar checkers, tools like autocorrect and autocomplete, large language models can help improve our writing. By tracking how much of my work is created or modified by AI, I can monitor myself and share the results in my posts.

I used a combination of the Q Developer CLI and Claude 4.0 to help me plan, iterate, and develop the code. It takes the shape of a browser extension you can load into Chrome, and borrows functionality from the way Google Docs lets you add comments to a doc. I can now highlight content that I used AI to modify or generate, then mark it as “AI Assisted.” It tracks the total words in the document and total “AI words” and gives me a percentage.

The source code and installation instructions are available on my GitHub repo at https://github.com/jbnunn/substack-ai-transparency-tracker.

---