Every major data platform now ships an agent that answers business questions in plain language. It writes the SQL, runs it, and hands back a number. The demos are genuinely impressive, and the capability is on by default in tools you already pay for.
Then it meets your warehouse.
The agent is asked what churn looked like last quarter. It picks a customers table from the wrong system. It doesn't know that "churn" in your organisation means cancelled-within-30-days, not simply inactive. It doesn't know which of three customer_id columns refers to the same human being. It returns a fluent, well-formatted, confidently wrong answer — and nobody in the room can tell.
In this article, we’d like to talk about the grounding problem that is currently the single largest factor separating organisations that get value from conversational data agents from those that don’t.
There is a useful way to see the size of this gap: look at how the same class of AI system performs on clean academic test data versus real enterprise data.
On the standard research benchmarks, text-to-SQL is close to solved. On BIRD, the most widely used of them, leading systems score in the low eighties against a human baseline of 92.96% (leaderboard as of August 2026). But when an MIT-led team — with colleagues at Harvard, Intel and AWS — built BEAVER, a benchmark drawn from the actual query logs of private enterprise data warehouses across 812 tables and 19 domains, state-of-the-art agentic frameworks using a frontier model achieved 10.8% accuracy. The same methods score 62.9% on Spider 2.0, itself already considered the hard benchmark. (BEAVER, arXiv, Sept 2024, rev. May 2026)
That is not a small gap. It is roughly a sixfold drop, and it appears the moment you swap clean, documented, purpose-built test data for the messy reality of any warehouse that grew organically over years.
For thirty years, every consumer of enterprise data was a human being who could tolerate ambiguity. An analyst who saw a column called rev_amt_cur knew to ask a colleague. A BI developer knew which of the four date columns was the one Finance actually reports on. The context was never encoded anywhere, because the consumer always carried it in their head.
The models are getting the questions wrong because nobody ever wrote down what the data means. Agents cannot stop mid-task to browse, cannot ask the analyst two desks over, and cannot pattern-match their way past a vague metric definition. They either receive the right context before they reason — or they produce something plausible and wrong at the speed of inference.
Grounding is the practice of connecting an AI agent to your actual, authoritative data and to the business meaning behind it, so its answers rest on your facts rather than on patterns learned during training.In practice, that means three things:
Grounding is a discipline, and it resolves into a set of factors you can assess, invest in, and improve. Here are the key factors to make it right.
The first instinct when grounding an agent is to give it access to everything. This is a mistake, because access is not the same as authority. Most enterprises have three or four systems that could answer a revenue question and no written record of which one Finance actually reports from. Point an agent at all of them and it will pick one — but who knows if it’s the right one?
The work here is unglamorous and decisive: for each business domain you want an agent to serve, establish which source is canonical, who owns it, and how fresh it is. A narrow agent grounded in the right source is worth more than a broad agent grounded in all of them. So scope is a feature.
Column descriptions have always been the documentation everyone agreed was important and but rarely had the time to write. When the only consumer was a human who could ask around, that was survivable. In the agentic era, it no longer is.
Thin or stale metadata is the most common single deficit we see in client environments: descriptions missing, table comments describing a schema from two migrations ago, synonyms unmapped.
The encouraging news is that large language models can now scan a schema, infer the likely meaning of a column from its name, contents and usage, and propose a description for a human to approve. The human role shifts from author to reviewer, which is a different order of effort entirely.
Factor 3 — Codifying the language of your business
This is the factor most often underestimated, and the one that most often caps promising pilots.
"Customer" means an account with a signed contract in Sales, anyone who logged in to Product, and anyone who has filed a ticket to Support. All three are correct but an agent asked "how many customers do we have?" has no basis for choosing between them.
The same applies to metrics. “Everyone knows” that revenue excludes refunds, so it’s not documented anywhere. An agent that doesn't know it produces technically valid SQL that overstates revenue by several percent, and that number reaches a board deck before anyone catches it.
The fix is a semantic layer: canonical definitions, the filters and joins your team actually trusts, and the terms that mean different things in different contexts. The measured effect is substantial. Snowflake compared Claude 3.5 Sonnet answering questions directly against the same model working through a semantic model, and saw average accuracy across four datasets move from 57% to 78% — a ~21-point lift from context alone, with no change of model. (Snowflake Engineering, March 2025) Worth noting that this test ran on public benchmark data, which is cleaner than most real warehouses; on your own estate the starting point is likely lower and the gap wider.
Worth noting: semantic layers help enormously within the scope they model, and not at all outside it. That is an argument for modelling one domain properly, not for modelling everything thinly.
A grounded agent must know two things about every question: whether it is in scope, and who is asking.
In-scope means the agent answers only from the tables it has been grounded in — and, critically, declines when a question falls outside them. An agent that says "I don't have the data to answer that" is far more valuable than one that finds something adjacent and answers anyway. Refusal is a trust feature, and in our experience it is rarely configured deliberately.
Who is asking matters just as much. Grounding has to respect the access controls you already enforce, so the agent sees only what that user is entitled to see. This is not only a compliance requirement; it is what allows an agent to be deployed to the business at all rather than to a small pilot group.
The gap here is real and widely acknowledged. In IBM's 2026 study of 2,000 senior technology executives, 77% reported that AI adoption is outpacing their governance, and only 11% described themselves as completely prepared for the scale of agent deployment ahead. (IBM Institute for Business Value, June 2026)
Every organisation runs on rules that exist nowhere in code. Exclude cancelled orders from revenue. Use subscription start date, not order date, for recurring revenue. Don't count free-tier users in customer totals for board reporting, etc.
Senior analysts absorb these rules over years and they are the difference between SQL that is technically valid and SQL that matches what Finance actually reports. Again, these rules are invisible to an agent unless someone writes them down as machine-readable filters attached to the metrics they protect.
Luckily, there is a good way to find them: mine the queries your team already runs. The joins, filters and date logic that appear again and again in trusted, production reporting are your business rules. Extracting and codifying them is faster and more accurate than interviewing people about what they think the rules are.
The final factor is the one that decides whether trust is ever earned.
Without an evaluation harness, no one can say where the agent is strong and where it fails. You have anecdotes: it got that one right, it got this one embarrassingly wrong. Anecdotes do not support a decision to scale, don’t encourage usage, and do not survive contact with a sceptical CFO.
What good looks like is a curated set of "golden" question–answer pairs — real questions from your business, with answers your team has verified — and a harness that grades the agent from simple to complex. Two things get measured. Faithfulness: is the answer grounded in the data actually returned, or did the model invent a number? And SQL correctness: did it call the right tables, joins and filters, checkable against a verified query?
This matters more than it sounds, because inaccuracy has become a first-tier risk in its own right. In McKinsey's March 2026 survey on AI trust, 74% of respondents identified inaccuracy as a highly relevant risk of agentic AI, alongside 72% for cybersecurity — putting it level with the risk every enterprise already budgets and staffs for. (McKinsey, March 2026) You cannot manage a risk you have never measured.
Two signs suggest grounding is moving from good practice to table stakes.
The first is that every major platform vendor has now converged on the same answer. Google Cloud built grounding into its Agentic Data Cloud in April 2026; Microsoft announced Fabric IQ at Build in June, with its ontology layer still in preview; Databricks brought business semantics in Unity Catalog forward at Data + AI Summit the same month; Snowflake published the architecture of its own internal context layer in August. Four companies that agree on very little have independently concluded that the missing piece is not the model — and, tellingly, several of these layers are still shipping, which is a fair signal of how early this is. Gartner's own 2026 prediction goes further: by 2030, it expects universal semantic layers to be treated as critical infrastructure, alongside data platforms and cybersecurity. (Gartner, March 2026)
The second sign is the returns gap. McKinsey's August 2026 global survey found 44% of organisations now report AI scaling across the enterprise, up from 38% — while the share attributing any EBIT impact to AI stayed essentially unchanged at 37%. (McKinsey, August 2026) Deployment is accelerating; but somehow value is not following it automatically. The organisations closing that gap are doing something the others aren't, and the evidence points squarely at the foundation.
MIT CISR put a number on the payoff in a May 2026 briefing, drawing on its data monetisation survey. Only 21% of executives rated their organisation's data curation practices as somewhat or very well developed — but those that did were three times as likely to be effective at implementing value-realising data and AI initiatives, and twice as likely to report a meaningful competitive advantage from them. (MIT CISR, May 2026)
The six factors above are not a maturity model, but they point at one. Read back through them and you will already have a rough sense of where you are on the spectrum.
The practical next step is smaller than most people expect: a focused readiness assessment on a single domain. Inventory what's in scope, grade the signals honestly, and evaluate an agent against a handful of your real questions. That produces a scorecard specific to your data, and it replaces opinion with evidence.
The agents are ready. The open question for the next eighteen months is whether the foundations underneath them are. Luckily, that question is answerable.
Want the full picture? Our 2026 benchmark report, Talking to Your Data, sets out where enterprises stand on conversational data agent readiness. It draws roughly four-fifths of its content from independent research published in 2024–2026, and includes a five-stage maturity model and a seven-signal self-assessment you can run against your own environment.
[Download the benchmark report →]
Aliz is a Google Cloud Premier Partner with specialisations in Data Analytics, Machine Learning, Generative AI and Infrastructure.
.png){kind=small}
.png){kind=small}
.png){kind=small}
.png){kind=small}
