Any AI performance conversation runs the same three tracks: token costs, inference latency, and model accuracy, and all three numbers matter. But once an AI system reaches enterprise data, a fourth system starts determining both cost and performance: the data warehouse underneath it, and whatever stands between the model and that warehouse.
A commercial banking division of a multinational Tier 1 bank just proved this with a real production workload. The cost gap was 21,903x, for the same five questions, with the same right answers, on the same warehouse.
Gartner validated this premise earlier this year: organizations with successful AI initiatives invest up to 4 times more, as a share of revenue, in foundational areas like data quality and governance, context included, than organizations with poor AI outcomes. Context (semantics and metadata) is now critical infrastructure for AI.
The diagnosis is right. What I haven’t seen is anyone actually pricing what “context” is worth in warehouse dollars and compute time. So we went looking for a real answer.
Here’s the story of one benchmark that tried to answer a simple question: what happens when an LLM runs production analytics against an enterprise warehouse, with and without a semantic layer in front of it?
Path 1 vs. Path 2
The bank compared two architectures.
Path 1: The LLM discovers the warehouse on its own. It samples table schemas, guesses which columns contain the answer, and writes SQL based on its best inference of the business logic.
Path 2: The LLM uses a semantic engine that already knows what business terms mean and routes the query to the cheapest, correct path.
The business questions were identical in both paths. That’s the whole point of a controlled test. Same five questions, same warehouse, same expected answers. The only variable: whether something between the LLM and the warehouse understood the business before the query ran.
Many AI benchmarks rely on synthetic prompts written to make a point. This one didn’t. It ran against a live commercial banking workload, using questions an analyst actually asks. Four metrics were captured per query: total compute time, wall-clock duration, bytes processed, and bytes billed.
That combination matters. An independent study of LLM-generated SQL costs on BigQuery found that execution time and query cost aren’t tightly correlated (r = 0.16). A query can return quickly and still be expensive, because speed reflects parallelization, not the volume of data scanned to achieve it. Measure only latency, and you miss where the money goes.
Each question was routine for a commercial banking analyst: customer segmentation across a category hierarchy, the age distribution of the customer base, gender breakdown by product holding, transaction volume by channel, and transaction value and customer counts by product line and region over the trailing 12 months.
Where the Cost Was Hiding
Simple-looking questions often required massive scans, and that’s where the money hid. The transaction-channel question, a monthly breakdown across branch, online, mobile, ATM, and agent, sounds routine. Unguided, it meant scanning the transaction ledger directly, because the LLM didn’t have a way to know which channel taxonomy the business used, or whether a pre-aggregated view already existed.
236.61 GB scanned unguided, versus 10 MB guided: a 64,656x gap in compute time, on a question any analyst asks routinely.
The biggest gains showed up on high-cardinality, multidimensional questions. The trailing-twelve-month question, three measures (transaction value, transaction count, and active customer count) sliced by product line and region, is exactly the kind of multi-measure, multi-dimension question that breaks unguided agents, the kind no naive query survives. It requires joining the full transaction ledger to customer and product masters and computing three separate aggregations, including a distinct count, over a year of data. Unguided, that scanned 2.09 TB. Guided, 104 MB.
The semantic engine changed how much work the warehouse had to do to produce answers. Both paths returned the same numbers. They just didn’t cost the same to get there. The guided path just didn’t have to rediscover what “active customer” or “transaction channel” meant every time someone asked.
An LLM without a governed business context re-derives structure from scratch, and that’s expensive.
Compute Time, Bytes Billed, Dollars
Across the full benchmark:
- 11,425x reduction in compute time
- 21,903x reduction in bytes billed
- 99.995% lower cost at BigQuery’s on-demand pricing, for the same five questions ($17.93 unguided versus roughly eight-hundredths of a cent guided)
- Same business answers returned, in both paths
What that translates to: less warehouse contention, lower autoscaling costs, more concurrent workloads on the same reservation, and faster response times for everyone else querying the same warehouse at the same time.
What This Means for AI Economics
For a few years now, AI budget conversations have centered on models and GPUs. This benchmark points at a layer that hasn’t gotten equal attention: query planning. Enterprise AI economics are determined by how intelligent the model is and by how efficiently the system between the model and the warehouse turns a business question into warehouse execution. That’s the layer this benchmark measures.
That’s consistent with Gartner’s call that context is infrastructure. Put an LLM in front of a warehouse without something that understands the business, and it may get the right answer at a cost that scales with how many times it has to guess.
Your Real AI Bill Is Hiding Here
A single bank ran this against real transactions. Before your next AI budget conversation, ask one question: how many times this month did your model re-derive what “active customer” or “transaction channel” means from scratch, instead of already knowing? That number’s your real AI cost driver.
Download the full case study for the numbers behind every figure in this piece, and you’ll want them before your next AI budget conversation.
Reviewed by: Mark Palmer
SHARE
Guide: How to Choose a Semantic Layer