# LLM tells in code, commit messages, comments and READMEs > Source-anchored research for LLM tells in code, commit messages, comments and READMEs. Run status: COMPLETED. Parent: [Writing and Documentation](https://llms-explorer.com/tree/writing-and-documentation/) · 7 facets · 29 facts · page: https://llms-explorer.com/tree/ai-isms-code-and-commit-tells/ ## Definitions - The characteristic narration tell is a comment that restates the adjacent code, adding maintenance burden and burying the few comments that matter. (low confidence) — [source](https://github.com/takahirox/my-ai-employee/issues/76) - Comment-based detection is a weak, proxy signal: the repository study calls its results proxy-based and warns detectors both under- and over-estimate LLM use, and other work reports commercial detectors are affected by comments and unreliable. (medium confidence) — [source](https://arxiv.org/html/2607.01867v1) ## Structure and components - Agent guidance for avoiding the tell: do not add comments by default; add one only for non-obvious why (rationale, constraints, invariants, trade-offs, surprising external behavior); prefer better names, types and structure over narration. (low confidence) (verified as of 2026-09-04) — [source](https://github.com/takahirox/my-ai-employee/issues/76) - In one study of security-related AI-generated PRs, 70.4% of 2,823 commit messages were high quality (contain both Why and What) and 29.6% omitted one or both. (low confidence) — [source](https://arxiv.org/pdf/2604.19965) - Practitioner signature list of AI-grown bloat: verbose scaffolding layers, partial fixes, ceremonial state layers mirroring other layers, non-functional placeholder subsystems left wired in, debug harness code in shipping source, and defensive abstractions that accumulate each agent turn; one Flutter app shrank 31.7% with tests passing. (low confidence) (verified as of 2026-06-01) — [source](https://dev.to/maximsaplin/debloating-the-ai-grown-codebase-2om) - The readme-ai generator emits a fixed skeleton: header with badges and logo, project introduction, features table, project structure, project index, getting started (install, usage, tests), community and contributing, license; it offers emoji theme packs prefixed to section titles. (medium confidence) (verified as of 2026-09-23) — [source](https://github.com/eli64s/readme-ai) ## How it works - Wikipedia's AI-cleanup editors state the listed patterns are only potential signs of a problem, not the problem itself, and that text with these features is not necessarily AI-generated because LLMs are trained on human writing. (high confidence) — [source](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing) ## Measurements and reference values - LLM comments are not uniformly poor: in a 415-comment Java evaluation, human experts judged 58.8% to match originals and 27.7% to surpass them; automated metrics such as BLEU correlated negatively with human judgment. (low confidence) — [source](https://arxiv.org/html/2408.14007v3) - A classifier on AIDev PRs identified which of five agents (Codex, Copilot, Devin, Cursor, Claude Code) authored a PR at 97.2% F1; commit-message features dominated, with multiline commit ratio the strongest single signal (44.7% importance). (low confidence) — [source](https://arxiv.org/html/2601.17406v1) - Per-agent signatures differ: Codex extensive multiline commits, Copilot longer PR descriptions, Cursor bullet points and hyperlinks, Devin multiline messages. Claude Code recall was only 57% in that study, so its tells are weaker. (low confidence) — [source](https://arxiv.org/html/2601.17406v1) - Across four LLMs, generated code had on average 63.34% more code smells than reference solutions (73.35% implementation smells, 21.42% design smells), with larger increases on more complex tasks. (low confidence) — [source](https://arxiv.org/abs/2510.03029) - Detection tools for AI text score poorly in independent testing (all under 80% accuracy, five over 70%), so a formatting tell in a README is grounds to review, not proof of AI authorship. (medium confidence) — [source](https://arxiv.org/pdf/2306.15666) ## Problems, failure modes and limitations - Search-result snippets (not fetched) report opposite security-study findings: LLM code lacking defensive constructs, and one model with error handling in only 3.3% of scripts. Excess defensiveness is not established as a general LLM tell. (low confidence) — [source](https://arxiv.org/abs/2409.19182) - Population-level word statistics cannot identify an individual LLM-written text, and cannot separate direct LLM use from humans unconsciously adopting LLM-preferred vocabulary; a single flagged word is weak evidence. (medium confidence) — [source](https://pmc.ncbi.nlm.nih.gov/articles/PMC12219543/) ## Comparisons and alternatives - GitClear's analysis of 211M changed lines (2020-2024) reports copy/pasted lines rising from 8.3% to 12.3% of changed lines while moved (refactored) lines fell from 25% to under 10%, read as assistants adding new code rather than reusing existing functions. (low confidence) (verified as of 2026-01-01) — [source](https://www.gitclear.com/ai_assistant_code_quality_2025_research) - Excess-vocabulary analysis of 15M+ PubMed abstracts found 2024 excess words were mostly style words (verbs ~66%, adjectives ~14%) such as delves, intricate, pivotal, unlike COVID-era excess words which were content nouns; it estimates at least 13.5% of 2024 abstracts were LLM-processed. (high confidence) (verified as of 2025-07-02) — [source](https://pmc.ncbi.nlm.nih.gov/articles/PMC12219543/) - A Stanford study of seven GPT detectors found about 61% of non-native English writers' essays flagged as AI; detectors penalise plain, predictable wording, so clean simple prose is a false-positive risk rather than a tell. (high confidence) — [source](https://themarkup.org/machine-learning/2023/08/14/ai-detection-tools-falsely-accuse-international-students-of-cheating) ## Facts and statements - A 2021-2025 study of eight GitHub repositories found comments flagged as likely LLM-generated concentrated in Meta and Explanation categories, and remained relatively stable over time. (medium confidence) (verified as of 2026-07-01) — [source](https://arxiv.org/html/2607.01867v1) - Claude Code by default appends a Co-Authored-By: Claude trailer to commit messages and a 'Generated with Claude Code' line to PR bodies; cloud sessions also add a claude.ai link. The attribution.commit, attribution.pr and attribution.sessionUrl settings change or hide them. (high confidence) (verified as of 2026-09-23) — [source](https://www.ssdnodes.com/learn/claude-code-commit-attribution) - Trailer tells are easily removed (git commit --amend, force-push, commit-msg hook), so their absence does not show human authorship. (low confidence) — [source](https://www.ssdnodes.com/learn/claude-code-commit-attribution) - Agent-authored PRs show message-code inconsistency: descriptions can claim functionality with no corresponding implementation, or describe changes incompletely. A description's confident summary therefore needs checking against the diff. (medium confidence) — [source](https://arxiv.org/pdf/2601.04886) - Agent-generated code shows procedural bloat (Long Method dominant) on simple tasks and architectural decay (God Class, superficial file separation) in complex systems; total lines of code correlates with architectural degradation at rho=0.94. (medium confidence) — [source](https://arxiv.org/html/2605.02741v1) - More capable models can produce messier code as they try to handle every edge case, yielding excessively long methods; detailed prompts and few-shot examples did not reduce defects in that study. (low confidence) — [source](https://arxiv.org/html/2605.02741v1) - Google's developer documentation style guide prescribes active voice, second person, conditions before instructions, and friendly-but-not-frivolous tone; these are human-authored norms that clean LLM output can also satisfy, so following them is not itself a tell. (medium confidence) — [source](https://developers.google.com/style/highlights) - The Wikipedia guide's stronger tells are content-level: undue emphasis on significance and legacy, vague attribution ('experts argue'), 'not just X but Y' parallelism, copula avoidance, and rigid 'challenges and future prospects' sections; these signal padding, not correctness of grammar. (medium confidence) — [source](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing) - Human judgement alone performs no better than chance at telling AI from human text per the Wikipedia guide, and writers who fear accusation may distort their style; human and LLM writing are also converging. (low confidence) — [source](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing) - Wikipedia's AI-writing guide lists emoji used as formatting, Markdown syntax, excess boldface, vertical lists with inline headers, and skipped or heading-only-heading structures as signs of AI text, and says they are signs of a possible problem, not the problem itself. (high confidence) — [source](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing) - LLM changelog generation can produce entries that are fluent, correctly formatted and false, such as inferring an extra change that never shipped or inventing a value like a cache TTL; the recommended control is human review of the draft. (low confidence) — [source](https://dev.to/nazar-boyko/ai-agents-for-release-notes-and-changelog-automation-kia) - The avoid-ai-writing skill (74 pattern categories, deterministic detector, CI and pre-commit integration) treats generic conclusions, chatbot artifacts and formatting inconsistencies as tells that can be linted in docs. (low confidence) (verified as of 2026-09-23) — [source](https://github.com/conorbronsdon/avoid-ai-writing)