Siti Muinah
As global enterprises transition from experimental conversational AI to mission-critical analytical systems, a subtle but pervasive failure mode has emerged: the tendency for Large Language Models (LLMs) to perform flawless reasoning over incorrectly selected context. While the industry has historically focused on model performance, the challenge for 2026 and beyond is not simply making models "smarter" but ensuring they are grounded in the correct analytical evidence. When an enterprise AI system provides an unreliable answer, the standard diagnosis suggests the model was insufficient, leading to calls for larger context windows, longer prompts, or more sophisticated retrieval-augmented generation (RAG) pipelines. However, research presented at the DataHack Summit 2026 suggests that the most common failures are not mathematical or logical, but rather failures of "context resolution."
In these scenarios, the arithmetic is correct and the explanation is coherent, yet the final answer is wrong. The system chooses the wrong evidence, follows an incorrect relationship between data tables, or resolves an ambiguous business term into a definition the user never intended. This distinction is vital as businesses move away from simple chatbots toward systems that support high-stakes decision-making. An answer can be logically consistent and fully traceable to a source while still being analytically void because the system selected evidence that did not match the core intent of the question.
The Problem of the Undefined Denominator
To understand the gravity of context failure, one must look at seemingly trivial business questions. A common inquiry such as "On average, how many hours does an employee work per day?" appears simple, yet the word "day" lacks a functional definition in most enterprise data estates. It could refer to calendar days, scheduled workdays, or days where work was actually recorded. Each interpretation produces a defensible number, but only one reflects the user’s specific intent.
If a system divides total hours by calendar days, it will produce a lower average than if it divides by days actually worked. A conventional output check might confirm that the totals reconcile and the arithmetic is sound, but it would fail to detect that the system selected the wrong denominator. This highlights the gap between validating a calculation and validating the analytical decisions preceding it. Validating the latter requires a rigorous check of the population, filters, denominators, and the relationships between disparate data sources.
The stakes of these "compressed analytical programs" rise exponentially when questions span multiple metrics, reporting periods, and respondent groups. For example, a request to identify which brand "improved trust" this quarter requires a series of unresolved decisions: Which questionnaire item represents trust? Is it a top-box score or a derived KPI? Which survey waves constitute "this quarter"? What is the scope of the respondent base? Are the results weighted? Without resolving these parameters, the system essentially guesses, often settling on the most "similar" table in a vector database rather than the "correct" one.
The Scalability Paradox and Context Rot
As enterprise data estates grow, the problem of selecting the right context becomes harder, not easier. In a small dataset, the evidence is usually obvious. In a massive enterprise environment, several tables may carry a "trust" metric, multiple survey waves may resemble the requested period, and various crosstabs may exist for different segments and weighting schemes. The system faces a dual challenge: discovery (finding where the answer might be) and resolution (deciding which candidate matches the user’s intent).
Recent research supports the idea that larger models and longer context windows are not panaceas. The Spider 2.0 benchmark, an enterprise text-to-SQL evaluation released in late 2024, revealed that even the most advanced systems—including agents built on OpenAI’s o1-preview—solved only about 17% of real-world workflow problems. While vendor-submitted scores have since climbed to over 96% on specific tracks, these improvements have come from heavily engineered agentic systems rather than bare models. These systems succeed because they are built to recognize ambiguity before attempting a calculation.
Furthermore, a study by Chroma Research titled "Context Rot" found that models became significantly less reliable as input length grew, even on simple tasks. Similar-looking distractors in the context window exacerbated the problem, leading the model to lose track of the relevant
evidence. This suggests that "long context" is not a shortcut; giving an LLM more documents can increase information volume without making the right evidence easier to identify.
Redefining Context Engineering: From Piles to Paths
To combat these failures, enterprise leaders are shifting toward "context engineering," a discipline that goes beyond prompt engineering. While Anthropic defines context engineering as managing an agent’s entire context state, the analytical challenge requires a more structured approach: treating context as a sequential path rather than an unordered pile.
In a "context pile" approach, the system provides the model with multiple definitions and candidate sources and asks it to infer the best combination. This often leads to "hallucinations of logic," where the model justifies a wrong choice with plausible but incorrect reasoning. In contrast, a "context path" approach establishes the dependencies that make a source valid before the model uses it.
A reliable five-step workflow for resolving a business question follows this sequence:
- Identify the Study Intent: Establish the broad goals of the research or data collection.
- Resolve the Metric: Define exactly what a term like "trust" or "improvement" means in this specific context.
- Determine Scope and Period: Identify the markets, segments, and timeframes involved.
- Select Eligible Evidence: Match the resolved constraints against available data tables.
- Execute Calculation: Perform the math only after the analytical path is verified.
Each decision in this chain narrows the next. Instead of asking the model to reconstruct analytical logic from a collection of documents, the system supplies the relationships and constraints needed to reach the correct evidence.
Principles of Governable AI Analysis
For this structured path to be effective, it must be governable and inspectable. Experts in the field suggest five core principles for designing reliable analytical AI:
1. Sequential Context Building: Ensure that later definitions depend on earlier ones. A question label must depend on the study objective, and a crosstab description must depend on the weighting and market rules.
2. Durable Feedback: When a human corrects a definition (e.g., clarifying that "purchase propensity" and "intent to buy" are identical for a specific project), that approval should be versioned and retained. This prevents vocabulary drift and ensures the system doesn’t make the same mistake twice.
3. Upstream Human Review: Errors should be corrected at the layer where they originated. If a metric is defined incorrectly, fixing the final answer is a temporary patch. Returning to the "definition layer" and correcting it there ensures all future calculations using that metric are accurate.
4. Tool-Based Execution: Models should not invent the mechanics of data access. They should use narrow, authorized tools to retrieve values and compute changes. Every reported number should be traceable to a specific tool result.
5. Uncertainty Thresholds: A sound system must know when to stop and ask for clarification. If a retrieval system finds two nearly identical tables with different weighting schemes, it should pause rather than guess.
The Broader Impact: Evaluating the Route, Not the Result
The shift toward context engineering necessitates a change in how AI is evaluated. Most current AI evaluation focuses on the final answer, which is insufficient when the primary risk is misrouting. A model can reach the right figure through the wrong analytical path simply by luck—for instance, if two different tables happen to contain the same value for different reasons.
The unit of evaluation must therefore become the "decision trajectory." Organizations should test interpretation, evidence selection, and execution separately. This involves using representative questions with known definitions and deliberately ambiguous wording to see if the system identifies the ambiguity.
The implications for the enterprise are profound. Traceability is no longer just a compliance requirement; it is a prerequisite for trust. An enterprise system must be able to show the analytical decisions behind an answer—recording the source table, metric definition, filters, weighting scheme, and reporting period.
In conclusion, the lesson for the next generation of AI implementation is clear: model capability cannot compensate for an undefined analytical path. When intent is distributed across complex schemas and hundreds of near-identical tables, the engineering focus must shift. Before replacing a model or increasing its size, enterprises must inspect the context path. Reliable AI begins before generation; it turns intent into constraints, constraints into a route, and the route into an auditable sequence of operations. In the high-stakes world of business intelligence, the path to the answer is just as important as the answer itself.
Dwi Wanna
Ali Ikhwan
