AI Writing Skills and the Performance Plateau: Analyzing the Stagnation of Large Language Models in Creative and Professional Content Generation

The rapid evolution of generative artificial intelligence has fundamentally transformed the landscape of content creation, yet a growing consensus among professional writers, linguists, and software engineers suggests that the writing capabilities of leading Large Language Models (LLMs) may have reached a period of stagnation. While these models continue to make exponential leaps in mathematical reasoning, logical deduction, and computer programming, the "art" of prose—characterized by stylistic nuance, original voice, and rhetorical depth—appears to be plateauing. This phenomenon has significant implications for businesses that have integrated AI into their marketing and communications workflows, raising questions about the long-term viability of current AI-driven writing strategies.

The Quantitative Evidence: Benchmarks and the Writing Plateau

Public leaderboards and standardized benchmarks serve as the primary metrics for evaluating LLM performance. These tests typically employ pairwise comparison, where the outputs of two different models are judged against one another, either by a superior "judge" model or by human evaluators. The resulting data is processed through Elo-style rating algorithms, similar to those used in competitive chess, to rank models based on perceived quality.

Data from the Arena Leaderboard, specifically within the "Writing, Literature, and Language" category, reveals a telling trend. When plotting the Elo scores of the three dominant model families—Anthropic’s Claude, Google’s Gemini, and OpenAI’s GPT—against their respective release dates, the trajectory of improvement shows a distinct flattening. While there was a vertical surge in quality between the releases of GPT-3.5 and early iterations of GPT-4, subsequent models have shown marginal gains or, in some cases, regression.

For instance, Claude 3 Opus, initially hailed for its "human-like" warmth, has seen its successors focus more on speed and reasoning efficiency rather than prose refinement. Similarly, the Hemingway Bench, which utilizes professional writers to conduct pairwise comparisons specifically for business writing criteria, suggests a pointed slowdown. These findings indicate that while models are becoming more "intelligent" in a computational sense, they are not necessarily becoming better writers.

Technical Barriers: Why AI Writing Remains "Mid"

The term "mid"—internet slang for mediocre—has increasingly been applied to AI-generated prose. Nathan Lambert, a prominent researcher in the field of AI alignment and training, has identified several core technical reasons why LLMs struggle to transcend basic competence in writing.

First is the issue of Reinforcement Learning from Human Feedback (RLHF). While RLHF is essential for making models safe and helpful, it often has a "homogenizing" effect on style. Because human trainers tend to reward clarity, politeness, and neutrality, the models are effectively trained to avoid the risks associated with bold, idiosyncratic, or provocative writing. This results in a "consensus style" that is functional but lacks soul.

Second, the underlying architecture of LLMs relies on tokenization, where words and parts of words are converted into numerical values. This process can sometimes disconnect the model from the rhythmic and phonological aspects of language that human writers use to create flow and "voice."

David Johnston, Chief Data and AI Scientist at Navalia, notes that the training process itself is inherently backward-looking. "AI’s opinions are weighted by the frequency of occurrence among recognized sources," Johnston explains. "Creators are worried about the models giving fringe or conspiratorial opinions, so they steer toward the safety of consensus. This is why AI has not made any inroads to the cutting edge of thought leadership."

Comparative Analysis: Mainstream Models vs. Specialized Platforms

To determine if specialized "orchestration layers" or dedicated writing tools could overcome these limitations, a series of human evaluations were conducted to compare the outputs of frontier models against popular AI writing platforms. The test utilized a 500-word prompt on the dangers of artificial intelligence, requiring a conversational style, factual accuracy, and a clear point of view.

The evaluation criteria focused on five key pillars:

1. Substance and Accuracy: The density of information and factual reliability.
2. Overall Coherence: The logical progression and structural integrity of the piece.
3. Style and Control: Clarity, sentence variety, and the avoidance of repetitive "AI-isms."
4. Strength of Point of View: The ability to move beyond a neutral summary of existing opinions.
5. Task Fulfillment: Adherence to the specific constraints of the brief.

The Baseline: Mainstream Models

Frontier models like GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro established the baseline. These models scored consistently high in coherence and logic (averaging 32-34 out of 50). They demonstrated a superior ability to follow complex instructions when provided with detailed audience and brand guidelines. However, they still exhibited a tendency toward "both-sidesism," struggling to commit to a truly provocative stance.

The Orchestrators: Wrappers and Tools

Platforms such as Jasper, Anyword, and Writesonic operate as "orchestration layers," essentially wrapping mainstream LLMs in additional code designed to provide better context, SEO research, and brand alignment. Despite the added features, these tools often scored lower than the raw frontier models in terms of prose quality (averaging 18-23 out of 50). While their interfaces for adding brand voice guidelines are highly developed, the actual output frequently felt more constrained and formulaic than the outputs from the models they were built upon.

The Specialists: Custom LLMs

A third category includes tools like Writer, which uses its own "Palmyra" family of models, and creative-focused tools like NovelAI and Sudowrite. Writer performed admirably in business contexts, matching the scores of mainstream models by focusing on clarity and professional register. NovelAI and Sudowrite, being trained on narrative fiction rather than just internet data, produced the most "original" and stylistically diverse prose, though they were less suited for factual, data-driven business reporting.

A Chronology of the AI Writing Evolution

The perception of AI writing has moved through several distinct phases:

• 2020–2022 (The Curiosity Phase): GPT-3 introduces the world to long-form generation. The quality is erratic, often leading to "hallucinations," but the novelty is high.
• Early 2023 (The Breakthrough Phase): GPT-4 sets a new standard for reasoning and coherence. Businesses begin mass-adopting AI for content production.
• Late 2023–Early 2024 (The Refinement Phase): Claude 3 and Gemini 1.5 Pro introduce higher "emotional intelligence" and longer context windows, allowing for more consistent tone across long documents.
• Mid-2024–Present (The Stagnation/Plateau Phase): New model releases show significant gains in "agentic" capabilities (the ability to use tools and browse the web) but marginal or zero gains in the quality of creative prose.

Broader Impact and Implications for Business

The flattening of the AI writing curve suggests that the competitive advantage for businesses will no longer come from simply having access to these tools, but from how they are "steered."

The findings indicate that "wrappers" or orchestration platforms may offer diminishing returns for sophisticated users. Because frontier models from OpenAI and Anthropic now support robust project context and style files, businesses can often achieve better results by prompting these models directly with their own proprietary research and guidelines rather than paying for an intermediary service.

Furthermore, the inability of AI to form a genuine "Point of View" (POV) remains its greatest weakness. In an era where the internet is being flooded with AI-generated "consensus content," human-led thought leadership that takes risks and offers original insight will likely increase in value. AI is currently an excellent editor and a competent structural assistant, but it remains a poor substitute for an author with a unique perspective.

Conclusion: The Path Forward for AI-Assisted Writing

As the industry moves forward, the focus is shifting from "generation" to "contextualization." The most successful AI writing workflows currently involve creating a custom-built set of project context files—containing brand guidelines, audience personas, and historical examples—and feeding them into the latest frontier models.

While the "stochastic parrot" nature of LLMs may limit their ability to ever truly "write" in the human sense, their increasing proficiency in reasoning and logic makes them invaluable tools for the drafting and editing phases. For businesses, the takeaway is clear: do not rely on AI for the "soul" of your content. Use it to build the skeleton and the muscle, but ensure that the final voice and the strategic direction remain firmly in human hands. The plateau in writing quality is not a sign of AI’s failure, but rather a clarification of its role as a sophisticated tool rather than a creative replacement.