Ask your AI what churn was last quarter, and it will give you a number. It won’t tell you whether that’s logo churn or revenue churn, or whether it includes voluntary cancellations. A human analyst filters all of that without thinking. An LLM picks one and states it as settled fact.
I joined Fern Halper (TDWI) and Josh Klahr (Snowflake) on a webinar last week to talk about what that trust gap means for data and analytics leaders. We discussed findings from TDWI’s new Q3 Blueprint Report, Next-Generation Analytics: From Semantic Layers to Generative BI. Bottom line: interest in generative analytics is real, but the discipline to make it trustworthy needs work.
Adoption Is Ahead of Readiness
One-third of respondents told TDWI they’re still researching or planning their generative BI strategy. Only 17% have it running across the enterprise. Everyone else is somewhere in between: piloting, evaluating, and trying to figure out what’s real enough to trust.
What stood out to me is why organizations are doing this at all. Productivity improvement topped the list of business drivers at 48%, followed by faster insight generation and better decision-making. Reduced dependency on analysts came in dead last, at 14%. This is about giving analysts superpowers. Instead of clicking and dragging your way to a chart, you ask a question in plain English and let AI analyze the results and dig deeper.
Why Speed Without Judgment Backfires
When a person works through an analysis, they catch their own mistakes along the way. A number looks off, and you stop to check it. An LLM asking a chain of follow-up questions treats a wrong answer as fact and builds the next three questions on top of it. Josh called out that agents don’t have the context to know when something’s wrong, so a small error can quickly multiply into a conclusion that’s badly off.
Go back to that churn example from the top. A customer success manager who’s been at the company for two years applies that filter without a second thought. An LLM has to guess, and nothing in its answer tells you whether it guessed right.
One Tier 1 bank’s LLM guessed at a metric twenty-one thousand times over, work that a single governed call would have returned once.
Anthropic ran into this themselves. Without any semantic context, their own models answered business questions correctly about 20% of the time. Layering in a set of skill files got them to roughly 80%. Still, 80% isn’t good enough when you’re computing churn, and maintaining thousands of lines of skill files isn’t either. You need AI that’s right every time, backed by semantic meaning that scales.
Getting to 100% takes two things: something that pins down which metric the question means, and an engine that computes it the same way every time.
A semantic layer’s job is to let the AI be creative in how it explores the data and asks follow-up questions, while keeping the metric computation itself deterministic. Creativity in the question, certainty in the answer.
The Semantic Layer Maturity Gap
TDWI’s data shows most companies haven’t gotten there yet. Only 15% of respondents said their semantic layer supports AI and agentic systems or is treated as strategic infrastructure. Another 37% have a semantic layer, but it’s scoped to a single BI platform or shared across a handful of tools. Close to 30% have no semantic layer strategy at all. Meanwhile, the same survey found the organizations further along on semantic maturity were substantially more likely to have generative BI in production and to describe their AI outputs as trustworthy.
Getting there means treating your semantic models like software. Define your metrics in a language you can version-control. Route every change through the standard code approval practice of a Git pull request and run automated tests before it is promoted to production.
Unstructured Data Adds Context, Not Compute
We also spent time discussing unstructured data: two-thirds of respondents said both structured and unstructured sources matter to their generative BI work. My view here is simple. Numbers should never come from documents, but documents explain the numbers. A contract or a support ticket can tell you why churn increased. It should never be what computes your churn rate. The two connect through the entities they share: a customer, a supplier, a product, and a semantic layer that computes and certifies the metric. That’s where the semantic layer earns its keep. It lets you attach the descriptive richness of your documents to the certified metrics your business runs on, without ever letting the documents become the source of the math.
What This Means for Data Leaders
None of this gets solved by picking a more powerful LLM. The organizations TDWI found succeeding are building real infrastructure: governed semantics, prepared unstructured data, evaluation and testing that catch regressions before a customer does. It’s less flashy than “we shipped an AI copilot,” but it’s what holds up when someone asks your AI a hard question twice and expects the same answer both times.
Read TDWI’s full Q3 Blueprint Report for the complete survey data and the framework Fern lays out for building this the right way.
A Few Questions from the Webinar
It’s the part of the stack that actually computes a metric the moment someone asks for it, rather than just describing what the metric means. A semantic layer defines “revenue” once; the engine underneath does the math, every time, across whatever dimension someone slices it by. Most catalogs and ontologies merely tell you where the definition lives.
First, a governed set of business definitions, like “revenue,” “churn,” or “active customer.” Second, a dimensional model that slices those metrics by region, time, or product without breaking the definition. Third, a compute layer that serves the result consistently to AI agents, notebooks, and BI tools.
The same way governance always has, except the stakes are higher now because AI acts on the answer instead of a person double-checking it first. Governance earns trust when it’s tied directly to computation, not just documentation: one certified definition of a metric, version-controlled like code. That’s the treat-your-semantics-like-software idea above. Do that, and generative analytics output becomes a number the organization can stand behind.
An ontology maps meaning and relationships: what a “customer” is, how it relates to an “account” or a “product,” which terms are synonyms. It’s context. A semantic layer goes a step further and computes answers: it takes a metric definition and calculates the same number every time. Take the ambiguity in “revenue”: booked, recognized, ARR, and TCV are all “revenue” and all different numbers. At one company, ‘revenue’ had twenty different definitions across teams and tools. Every BI report and every agent was quietly answering a different question. An ontology can help an LLM identify the right metric, but a semantic layer computes the revenue number. LLMs need both: the ontology narrows the question, the semantic layer answers it correctly.
First is standard ETL and ELT work. Raw data first gets integrated and cleaned. Then it gets modeled and transformed into something queryable, such as dimensional models and star schemas. From there it typically branches three ways: a catalog step to document lineage, ownership, and quality; a knowledge graph or ontology step to capture relationships and meaning; and a semantic layer step to define and compute the metrics.
Reviewed by: Sarah Mulcahy
SHARE
Whitepaper | Enterprise Semantics for Power BI