Why LLMs converge on the same words: mode collapse and post-training lexical effects
Parent: LLM Alignment and Post-Training · Published reference · snapshot 2026-09-23 · skill ai-llm-model-layer/references/llm-lexical-convergence.md
↓ Facts as markdownall context files
Source-anchored research for Why LLMs converge on the same words: mode collapse and post-training lexical effects. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Definitions
- Verbalized Sampling is a training-free prompt that asks the model to verbalize a probability distribution over multiple responses (e.g. 'Generate 5 jokes about coffee and their corresponding probabilities') to circumvent mode collapse. (high confidence) [source]
- Homogeneity is a distributional property, not a lack of individual quality: in the PNAS Nexus study LLMs matched or slightly beat humans on individual originality for two of three tasks, while population-level diversity was much lower. The Hivemind paper likewise finds comparable overall quality but poor calibration of LMs, reward models and LM judges on generations that draw differing idiosyncratic human preferences. (medium confidence) [source]
How it works
- Follow-up work reports that alignment training (RLHF, DPO) substantially increases focal-word overuse relative to base models, so pretraining alone is not the sole cause. Both papers share the same authors, so this is not independent replication. (medium confidence) [source]
Measurements and reference values
- Post-training methods sharpen the output probability distribution and reduce diversity of generated responses; DivPO counters this by picking rarer, higher-quality responses as preferred and common lower-quality ones as rejected, reporting +45.6% persona attribute diversity and +74.6% story diversity with comparable quality. (medium confidence) [source]
- Verbalized Sampling, a training-free prompt asking the model to verbalize a distribution of responses with probabilities, reports 1.6-2.1x diversity gains in creative writing, with larger gains for more capable models. (low confidence) (verified as of 2026-07-15) [source]
- The popular explanation that 'delve' is overused because OpenAI's outsourced RLHF annotators in Nigeria favor Nigerian formal English was proposed by Alex Hern and is framed as a plausible theory; circumstantial support is PubMed 'delve' usage rising 10-100x, with no direct causal evidence. (medium confidence) [source]
Problems, failure modes and limitations
- Scope limits on the homogeneity findings: the PNAS Nexus study covers only publicly hosted, aligned LLMs and three divergent-thinking tasks. The longitudinal study cannot separate distillation, shared training data or shared techniques, samples one response per model per prompt, and notes some Infinity-Chat prompts themselves request similar outputs. (medium confidence) [source]
- Current lexical-shift studies cannot test ChatGPT directly because it lacks a public base variant, and they use mid-size open models on English scientific text, limiting generalization to frontier chat assistants. (low confidence) [source]
Comparisons and alternatives
- VS authors attribute mode collapse to typicality bias in preference data and report 1.6-2.1x creative-writing diversity over direct prompting without hurting factual accuracy or safety; results are author-reported. (medium confidence) (verified as of 2026-07-01) [source]
- VS gains are larger for more capable models than for smaller mini/flash variants. (medium confidence) [source]
- A reanalysis (Schaeffer et al., June 2025) finds min-p does not beat baseline samplers on quality, diversity or their trade-off once hyperparameter count is controlled, cites omitted human-eval data and inconsistent LLM-judge reporting, and notes adoption figures were unsubstantiated and removed. (medium confidence) [source]
- In a PNAS Nexus study of 22 LLMs and 102 humans, LLM response variability was lower than human variability on all three divergent-thinking tasks: Alternative Uses 0.459 vs 0.699, Forward Flow 0.518 vs 0.642, Divergent Association 0.632 vs 0.791 (p<0.01). LLM responses mirror other LLM responses more than humans mirror other humans. (medium confidence) [source]
- RLHF significantly reduces output diversity compared with SFT across several measures, while generalising better out of distribution; the authors frame this as a generalisation-diversity tradeoff (tested on two base models, summarisation and instruction following). (high confidence) [source]
- Top overused lemmas in instruction-tuned models versus human abstracts include suggest, highlight, enhance, furthermore, additionally, crucial, potential, strategy; many are ordinary scientific vocabulary, so models amplify already-typical wording rather than inventing novel words. (low confidence) [source]
- Base models also diverge from human word usage (misalignment scores 0.0329-0.0345 vs human baseline 0.0304), so not all lexical drift is post-training; content-word alignment trends are weakened or reversed after instruction tuning. (low confidence) [source]
Facts and statements
- Min-p sampling scales the truncation threshold by the top token's probability; the original paper claims better quality and diversity than top-p at high temperature and reports adoption in Hugging Face Transformers and vLLM. (medium confidence) [source]
- The Artificial Hivemind paper defines two effects: intra-model repetition (one model repeats similar outputs) and inter-model homogeneity (different models independently converge on similar ideas with minor phrasing variations). It uses Infinity-Chat, 26K real-world open-ended queries with 31,250 human annotations (25 annotators per example). (high confidence) [source]
- A longitudinal study of 68 models from 12 providers (March 2023 to July 2026) reports cross-family semantic distance falling over time: on the Alternate Uses Task from about 0.50 to below 0.40, and on Infinity-Chat100 from about 0.34 to just above 0.32. All 1,000 resampling iterations gave negative slopes. (medium confidence) (verified as of 2026-09-23) [source]
- The Hivemind paper measures similarity with cosine similarity of OpenAI text-embedding-3-small embeddings; secondary summaries note this is sensitive to surface phrasing and could overstate homogeneity, though verbatim-overlap examples suggest it is not a pure artifact. (low confidence) [source]
- Verbalized Sampling paper identifies typicality bias in preference data (annotators systematically favour familiar text) as a data-level driver of mode collapse, formalised theoretically and checked on preference datasets; it claims collapse occurs even with a perfect reward model. (medium confidence) [source]
- Aligned (RLHF) Llama-2 models show lower token-prediction entropy, distinct clusters in embedding space, and 'attractor states', indicating constrained diversity; study covers only the Llama-2 family. (medium confidence) [source]
- A 2026 study of 42,000 PubMed abstract continuations across six open model families (Falcon, Gemma, Llama, Mistral, OLMo, Yi; 4B-8B) defines curation-free metrics (Lexical Alignment Score, Triangulated Preference Shift) to isolate overuse relative to human text and shifts attributable to preference-learning stages. (medium confidence) [source]
- Instruction tuning does not uniformly change lexical diversity: in Qwen2.5-7B, Mistral-7B and Llama-3.1-8B on three multiple-choice benchmarks, cross-rationale similarity diversity (1-SelfBLEU) consistently fell while surface Unique-2 diversity moved in both directions, so 'post-training narrows vocabulary' is model- and metric-dependent. (medium confidence) [source]
- Markdown-heavy training data leaves stylistic fingerprints such as em-dash frequency; post-training can modify but does not fully erase these markers. (low confidence) [source]
- Juzek and Ward emulated learning from human feedback on Llama and found participants systematically preferred text variants containing focal words, consistent with human-feedback training contributing to lexical overuse. (medium confidence) [source]
- The Nigerian-annotator theory (outsourced RLHF raters favour Nigerian formal English, hence 'delve') was circulated as speculation, called 'a pretty solid theory' by Willison, without a causal test. (medium confidence) [source]
- Juzek and Ward state that evaluators appeared to treat 'delve' differently from other focal words, so human preference does not cleanly explain that word, and factors beyond RLHF and pretraining data contribute. (low confidence) [source]
- A 2026 preprint argues markdown-heavy pretraining text leaves fingerprints such as em dashes, and reports em-dash rates differing by post-training recipe (GPT-4.1 ~10.6 per 1,000 words, Claude Opus ~9.1, Llama ~0), implying both stages shape the tic. Rates come from a search snippet, not verified in the full text. (low confidence) (verified as of 2026-09-23) [source]
Children
- Annotator dialect hypothesis (frontier)
- Annotator dialect hypothesis for 'delve' (frontier)
- Attractor states in aligned models (frontier)
- Base-model vs aligned-model sampling (frontier)
- Distillation as a homogeneity confounder (frontier)
- Diversity-restoring decoding and prompting
- Human-feedback emulation experiments (frontier)
- Infinity-Chat benchmark (frontier)
- Inter-model homogeneity in open-ended generation
- KL-regularised optimal policy sharpening (frontier)
- Markdown and em-dash fingerprinting (frontier)
- Markdown and em-dash fingerprints (frontier)
- Mode collapse and typicality bias in preference tuning
- Post-training effects on LLM vocabulary
- Preference-stage lexical shift metrics (frontier)
- Pretraining versus alignment attribution of lexical tics
- Temperature and truncation sampler tuning (frontier)
Frontier under this node: Annotator dialect hypothesis, Annotator dialect hypothesis for 'delve', Attractor states in aligned models, Base-model vs aligned-model sampling, Distillation as a homogeneity confounder, Human-feedback emulation experiments, Infinity-Chat benchmark, KL-regularised optimal policy sharpening, Markdown and em-dash fingerprinting, Markdown and em-dash fingerprints, Preference-stage lexical shift metrics, Temperature and truncation sampler tuning