A Wrong Answer Has Never Looked So Good

Executive Summary:An agent doesn’t have a gut check, so it treats a wrong answer as fact and can’t catch it before building the next question on top of it, and accuracy drags down fast as a conversation goes on. The fix is a universal semantic layer: it’s machine-readable, it can compute the answer, and any tool or agent can reach it, unlike a catalog or glossary that only describes the data. AI is actually making this layer easier to build, since a code-based modeling language now lets it draft and test semantic models under human review. Getting there takes a Center of Excellence with real governance, built on open standards so adopting it never turns into new lock-in.Key Takeaways:• Fluency and correctness are uncorrelated in AI: a well-written answer isn’t necessarily a right one, and agents don’t have the human “gut check” that used to catch bad numbers.
• 66% of organizations call a semantic layer critical, but only 19% have one. The gap isn’t about cost, it’s that the business hasn’t agreed on its own definitions.
• Of six ways an agent could get an answer (metric store, BI semantic model, glossary, catalog, knowledge graph, universal semantic layer), only a universal semantic layer is machine-readable, able to compute the answer, and reachable outside a single vendor’s tool.
• Building one takes people and governance as much as technology: a cross-functional Center of Excellence, a hub-and-spoke ownership model, PR-style review for new metrics, and continuous regression testing against certified, known-correct answers.

Fluency and correctness are completely uncorrelated in AI. Ask a generative AI tool a business question, and it will give you a well-written answer. It will probably generate a fine-looking chart. And it has no relationship to whether the number is right.

I spent an hour on a panel with Donald Farmer, research fellow at TDWI (Transforming Data With Intelligence), on a question I’ve heard in different forms for as long as I’ve worked in this industry: why do two-thirds of organizations say a semantic layer is critical, while fewer than one in five actually have one?

Agents don’t get a gut check

Historically, BI worked because of a social safety net: someone sees a number on a dashboard, and something in them says, “that’s off.” I learned this early in my career: I handed my boss a report I’d worked hard on, and within five seconds he told me which number was wrong. He was right, and I hadn’t even thought to ask the question that would have caught it.

An agent doesn’t have that reflex. It takes an answer, treats it as fact, and builds its next question on top of it. Take a system that’s 90% accurate, let it chain five follow-up questions using its own prior answers as context, and you’re down to roughly 59% accuracy by the end. Donald named part of what’s happening here: regression to the model. The more conversational the interaction gets, the more the LLM drifts back toward its generic training knowledge instead of staying anchored to your specific enterprise context.

The data backs this up. On the BIRD benchmark, which tests LLMs on realistic, multi-turn business questions against messy schemas, the best model on the leaderboard scores 38% on getting the correct answer (as of mid-September 2026). Accuracy for genuine conversational analytics drops to 24% when you strip out tasks that are really database management work rather than analysis. And that’s with the full schema, knowledge base of every metrics and every column definition handed to the LLM up front. Superintelligence doesn’t fix this. A probabilistic system asked to traverse ten thousand tables and compute the same answer twice reliably needs something deterministic underneath it, no matter how capable the model gets.

Why does AI need a semantic layer?

Because fluency and accuracy are two different things, and generative AI only guarantees the first one. A model can produce a confident, well-written answer and a clean-looking chart with no guarantee the underlying number is correct. When agents start chaining questions on top of each other’s answers, those small errors compound fast. A semantic layer puts a deterministic, governed source of truth underneath the probabilistic model, so an agent asking for a number gets the same, correct answer every time instead of a fluent guess.

The gap, and why it’s not about cost

Donald cited 2026 TDWI research that found 66% of organizations say a semantic layer is critical. Only 19% have one. AI governance is even further behind data governance, and most organizations are giving AI direct database access or fragmented API access instead of a governed layer, which is why fewer than half trust their own structured data.

I don’t think the gap is about cost. The ROI case is easy to make. The real issue is the business doesn’t agree on its own definitions. Most enterprises already have four or five semantic layers, each tied to a specific BI tool or platform. What’s missing is a universal one. Semantics have always been welded to either the consumption layer or the platform underneath it, never treated as their own independent thing. Once you see it that way, the real question is why enterprises won’t consolidate the ones they already have.

AI turned out to be part of the answer

For most of my career, the biggest obstacle to a universal semantic layer was people, specifically the rare person who understands the database well enough to model it and the business well enough to know what the model should mean. I used to call them unicorns because they were so hard to find.

What’s changed is that a code-based semantic modeling language gives an LLM enough structure to do a meaningful share of that work itself. Given the right context, AI can write a semantic model, publish it, test it against real queries, and fix it when it breaks, and I’ve watched that self-correction loop happen in ways that would have taken a human team weeks or even months to do by hand. A person still validates the result, but the work has changed completely. AI created the pressure to finally build a universal semantic layer, and it’s proving to be one of the best tools for building it.

One of our customers doing this well assigns an owner to every metric in the business, tracked through their catalog and semantic layer together. When a definition needs to evolve, the provenance is clear: who owns it, who changed it, and why. That’s not a technology decision. It’s a people and process decision the technology has to support.

What should an agent actually query?

When an agent needs an answer, should it read from a metric store, a BI product, a glossary, a catalog, a knowledge graph, or a semantic layer? Run any of those through three questions: 

• Can a machine read it?
• Can it compute the answer?
• Can anything outside one vendor’s tool reach it?

Option	Machine readable?	Can it compute the answer?	Reachable outside one vendor?	What it’s genuinely good for
Metric store	Yes	Yes	No. It works inside its own walls.	Serving metrics to the tools that one vendor supports.
BI platform semantic model	Yes, through that vendor’s own interface	Yes	Partial at best. It’s welded to one consumption layer.	Fast, governed reporting for a team standardized on one BI tool.
Business glossary	Yes	No. It holds definitions, not logic.	Yes	Telling people and LLMs what a term is supposed to mean.
Data catalog	Yes	No. It describes data, it doesn’t query it.	Yes	Discovery, lineage, and ownership, and context that feeds the model.
Knowledge graph or ontology	Yes	Partial. It traverses relationships, it doesn’t compute governed business metrics.	Yes	Relationship context that should shape how you build the semantic model.
Universal semantic layer	Yes	Yes	Yes	The one place an agent can ask for a number and get the same answer twice.

Only one option answers yes three times. Catalogs, glossaries, and knowledge graphs are real assets, they’re just context for building the model rather than the thing an agent should query for a number.

Think of the semantic model as the map and the agent as the self-driving car. Without a map, the car doesn’t know where to go, no matter how good its sensors are. 

Open standards, so the switching cost doesn’t become the new lock-in

The other question that came up is whether open standards make the semantic layer a commodity. I don’t think they do, any more than ANSI SQL made the database a commodity. SQL gave the industry a common baseline, while every vendor kept competing hard on the engine underneath it. 

AtScale has been contributing our work on SML, our code-based, YAML-based semantic modeling language, into the Apache Ossie initiative, aiming at a shared way to describe semantic models across platforms. The goal is to make switching between semantic engines closer to a database migration than a full rebuild.

How to build a semantic layer for generative AI

If I were doing this from scratch again, I’d start with people. Stand up a cross-functional Center of Excellence with both business and technical ownership. Use a hub-and-spoke model: let finance, sales, and marketing each own their own semantic models in their own Git repositories, while the Center of Excellence owns the shared dimensions, the handful of things every domain needs to agree on, that the spokes inherit. That way finance’s revenue and marketing’s revenue are the same number when someone puts them side by side.

Govern it the way you’d govern software. A new metric goes through something like a pull request and a review before it publishes, which is the only reliable way I’ve seen to prevent semantic drift, even inside a single platform. Don’t try to solve everything before you start. Governance is a process that grows alongside your AI implementation, and none of this works without an executive mandate behind it.

When you’re ready to look at tech that might work for your organization, review our Context’s Buyer’s Guide for a deep dive into the competitive landscape.

Auditing the thing you built

Before you deploy a semantic model, build a set of verified business questions with known-correct answers, and run them continuously, regression testing, borrowed directly from CI/CD. When a model change breaks a certified answer, that’s your signal something drifted, and you catch it before a customer does. Layer in lineage, so users can see where an answer came from, and feedback signals like thumbs up and down, and you start building a growing corpus of certified answers to check every future change against. 

Agents also need well-annotated semantic models, with clear descriptions, synonyms, and AI-specific context, so when several models exist, the agent picks the right one instead of guessing from a name.

Learn more about TDWI’s framework for the semantic and governance capabilities needed to take generative BI from pilot to production. Download the report.

The takeaway

Conversational analytics is genuinely democratizing. People who never had a BI tool license or the skills to use it can now just ask a question. That’s real progress, and it’s also the problem, because the people furthest from the data are the ones least equipped to catch a wrong answer. Now add agents to the mix. They amplify mistakes at machine speed across every subsequent question.

The fix is a universal semantic engine: deterministic where the computation happens, governed by people and process, and open enough that adopting it doesn’t lock you in.

Check out my full conversation with Donald in the TDWI webinar here.

Reviewed by: Sarah Mulcahy