“What was revenue in the Northeast region last quarter?”
At first blush, the query looks simple enough. It reads like something you’d ask an analyst on your team and get an answer back in five minutes.
But an LLM can’t read an analyst’s mind, and it didn’t hear the news that the Northeast region just got reorganized to include New York. It doesn’t remember that your last quarter started in June, not July.
AI wants to please you, so it resolves your vague question with whatever context it can find. When it’s stuck, it guesses. Each guess can be wrong, and each one costs Anthropic or OpenAI compute.
Here’s what’s happening inside an AI model, from the prompt to the answer, decision by decision, the way you’d audit any process that produced a wrong number.
Decision #1: What does “revenue” mean?
First, the model has to solve the business-definition problem before it can touch a warehouse. The prompt seems obvious enough: we want to know what revenue was.
But “revenue” is a surprisingly vague concept. It could mean recognized revenue, booked revenue, or, for a subscription business, annualized or total contract value. Which is it?
Here’s how an LLM makes that call. First, it looks for a “Revenue” column in the database it has on hand. That figure might live in three or four different tables, one for bookings, one for recognized revenue, another for ACV and TCV, and the agent just picks one. It might decide whether revenue is gross or net. Should it back out returns and discounts? What about intercompany adjustments? Is the figure in local currency, or does it need conversion, and if so, at what rate and on what date?
None of that lives in the schema. A column named revenue_amt doesn’t say whether it’s gross or net. The model samples some rows, notices a pattern, and commits to an interpretation. It isn’t analyzing your data yet. It’s reconstructing institutional knowledge your finance team already has, by guessing.
Decision #2: What is “Northeast”?
Ask a human analyst what “Northeast” means and they’ll answer immediately. They’ve absorbed years of context that’s not in a data dictionary: which sales-territory codes map to which geography, and which of those mappings broke when the last reorg moved the lines.
The LLM doesn’t have any of that context. It looks for a region column and finds two candidates: region_cd in the sales fact table and territory_name in the sales_territory_hierarchy dimension table. It queries the dimension table for distinct values and gets back something like NE-01, NE-02, MATL-03, MATL-04, DC-METRO.
New York is obviously NE-01. But Pennsylvania splits across MATL-03 and MATL-04, and DC-METRO doesn’t say whether DC belongs to the Northeast or the Mid-Atlantic. The model writes a CASE statement mapping five territory codes to “Northeast,” its best guess, with no way to check that guess against the reorg that moved six accounts out of Mid-Atlantic two fiscal years ago.
None of that lives in the schema either. The sales_territory_hierarchy table doesn’t flag which mapping is current. The model samples distinct values, notices the NE- and MATL- naming patterns, and commits to a boundary. It’s guessing at something geography, finance, and sales already agreed on, then encoding that guess as a CASE statement nobody asked it to write.
Decision #3: What counts as “last quarter”?
Now enterprise semantics gets even more complicated.
“Last quarter” assumes a calendar, but whose? Your fiscal year might not start in January. Your reporting periods might run on a rolling 13-week basis instead of calendar quarters. And if the current month is incomplete, that changes whether “last quarter” means the one that just closed or the one before it.
The LLM doesn’t know any of that, so it looks for a date column, finds order_date on the fact table, and reaches for its own math instead of your calendar. It writes something like DATE_TRUNC(‘quarter’, CURRENT_DATE – INTERVAL ‘3 months’), which assumes a January-start calendar year. It never notices the fin_calendar dimension table sitting three joins away, the one with fiscal_qtr and fiscal_year columns that would’ve told it your fiscal year starts in June, so “last quarter” is actually Q4 FY26, not calendar Q1. It also can’t know today is the 3rd of the month, so this quarter is nine-tenths empty and “last quarter” probably means the one that just closed, not the one before it.
None of that lives in the schema. A column named order_date doesn’t say which calendar governs it, and a table named fin_calendar sitting unjoined doesn’t get consulted unless something tells the model to look there. The model picks the convention it can compute without asking, calendar-year quarters, rounds to the nearest month boundary, and writes the SQL with complete confidence and zero accountability for being wrong.
Only now does SQL happen
The model has made three separate guesses about the business before it writes a query: what revenue means, what Northeast covers, and what quarter counts as “last.” Generating the SELECT statement is mechanical from there. The expensive work already happened inside the model.
Every hour spent polishing prompt templates or fine-tuning text-to-SQL accuracy is aimed at the wrong end of the problem. You can make the SQL generation flawless and still get the wrong number.
Why every new question repeats the same work
LLMs don’t accumulate enterprise knowledge the way a new analyst does over their first six months on the job. Every prompt starts over from zero.
Ask the Northeast revenue question today, and the model rediscovers the table, works out the definition, guesses the region, and rebuilds the calendar. Ask a similar question tomorrow, and it does the whole thing again, often landing on slightly different guesses than it did the first time.
The warehouse pays for it every time, because nothing in the unguided path remembers what it figured out five minutes ago.
No matter how much warehouse capacity you throw at it, the guessing doesn’t stop. Even if you try to encode the logic in skills files or instructions, there’s no guarantee the LLM will use them. And maintaining pages of skills files is brittle at best and impossible at worst.
Where the fix lives
The real question is why we keep asking models to rediscover business knowledge we’ve spent decades defining. Your finance team already knows what revenue means, and your ops team already knows how the sales territories map. The model just can’t reach that knowledge.
The fix is a semantic layer between the model and the warehouse, one that already knows what revenue means, how the territories map, and which calendar governs “last quarter,” so nothing downstream has to guess it again.
We ran a full production benchmark to measure exactly what that guessing costs, query by query, over a real day inside a real bank’s warehouse. Everything in this post is what’s underneath every number in that benchmark. If you want to see what the guessing actually costs in production, download it now.
Reviewed by: Mark Palmer
SHARE
Whitepaper | Enterprise Semantics for Power BI