Your AI Isn’t Expensive. It’s Guessing.

Inside a Tier 1 bank’s $9-million-a-year AI rediscovery tax, and the architectural fix that cut compute costs by as much as 21,903x.

Every time an analyst at a Tier 1 bank asks, “What was revenue in the Northeast last quarter?” The LLM has to guess what those words mean to the business. The answers are often expensive. And often wrong.

One bank measured the damage: nearly $9 million a year, letting an LLM rediscover metrics the business already knew, and getting the answer wrong about a third of the time. We call it the AI rediscovery tax. And you should stop paying it. 

The exemption is legal, audited, and sitting on the shelf. It’s called a semantic layer.

Your bloated AI bill is a scanning problem, not a token problem.

The bank’s team ran five everyday analyst questions through their Google BigQuery warehouse in two ways. The first way is what most companies do today: hand the AI your data sources and let it figure them out. The AI doesn’t know how Finance defines “revenue,” how Sales drew “Northeast,” or where the fiscal calendar starts and ends. So it makes up its own version, scans trillions of rows, and bills you for the work. We call this the rediscovery path. The second way, the guided path, routes each question through a semantic layer with the bank’s metrics already defined.

The rediscovery path cost $17.93 and scanned 3.15 terabytes. The guided path costs less than a tenth of a cent and reads 144 megabytes from a pre-built summary. The semantic layer used 21,903 times less warehouse compute.

That 21,903x is the lookup case, where the answer is pre-built. When a question is novel, and the semantic layer has to compute it live, it still wins. It knows the data model and writes the right SQL instead of guessing. The bank’s broader testing showed 95% to 99% less data scanned, even in this live-compute case, still a 20x to 100x cost cut.

Your AI accuracy problem isn’t an LLM problem. It’s a semantic problem.

Ask an unguided LLM the same business question twice, and you can get two different answers. Not because the model is bad. Because it has to make up a definition each time. “How many young customers do we have?” One LLM pass picks ages 25-34; the next picks 26-35. Neither matches what Marketing actually uses.

The semantic layer holds the definitions in one place. Every query for “young customers” uses Marketing’s bracket. Every quarter matches Finance’s. Every customer count ties back to the customer master. The model can’t invent the wrong number when the right one is already defined. AI accuracy on curated metrics goes from an industry-average 70% to 100%.

The path to 100% accuracy runs through semantics, not tokens.

Stop fixing the wrong layer.

The bank’s semantic layer is in production, handling more than 6,000 AI-driven questions a day, with two-thirds answered in under a second. Multiply that out across a year: nearly $9 million in compute they no longer spend on rediscovery, and tens of thousands of wrong answers that never impact a business decision. They stopped paying the rediscovery tax with a single move: putting their definitions into a semantic layer. They didn’t swap their LLM. They swapped the layer underneath it, and the exemption is sitting on the shelf for you, too.

What to do on Monday

Ask your data team three questions:

1. Are our top business metrics defined once, in one place, where AI can use them?
2. When AI runs a query, is it guessing and scanning, or using our pre-agreed definitions?
3. Can we see what each AI query costs us?

If those answers are all no, you’re paying the rediscovery tax. The bank stopped paying it the day they put their metrics in a semantic layer, and the bill dropped 21,000x.

---

>> Download the complete benchmark report, with full methodology and per-query SQL.

Source: production benchmark conducted by the UK commercial banking division of a global Tier 1 bank, measuring five representative analytical queries on BigQuery with and without an AtScale semantic engine in front. The nearly-$9 million annual figure applies the unguided per-query cost ($3.59) to the bank’s reported daily AI-query volume (over 6,000) served through the semantic-layer path; in practice, the unguided architecture would cap query throughput well below that level, so the figure represents the cost the bank would face if it tried to serve the same daily workload through the unguided path. The roughly 650,000 wrong-answer figure applies the public BIRD-SQL benchmark’s ~30% text-to-SQL error rate to that same daily query volume.