Every architecture diagram I see draws the same three boxes.
Sometimes it’s Model + RAG + MCP. Sometimes it’s Model + MCP + Tools. The labels move around, but the shape never changes.
The diagram describes how an agent reaches your data. It says nothing about whether the answer it computes once it gets there is right, or what that answer costs to produce. Access and computation are two different problems.
A new production benchmark shows exactly what it’s costing enterprises. The same five questions, answered by two different architectures, cost $17.93 one way and less than a tenth of a cent the other. Same answer. Twenty-one thousand times the cost.
Enterprise AI needs a third leg, a compute engine that delivers governed answers instead of guesses.
Leg one: LLM
LLMs are genuinely good at what they’re built for: reasoning, planning, summarizing, and orchestrating a multi-step task.
But an LLM has never seen your fiscal calendar. It doesn’t know that your gross margin definition excludes freight allowances, or that “Northeast” in your region_cd column includes Pennsylvania but not DC.
MIT Sloan Management Review’s “The Working Limitations of Large Language Models” makes the underlying point well: these models can simulate the appearance of logical, rule-based reasoning without reliably chaining that logic together to produce and verify a complex conclusion. Ask a model to have a conversation about your business, and it’s excellent. Just don’t expect it to consistently compute your business metrics.
Leg two: MCP
MCP deserves real credit. Before it, every agent-to-tool connection was a bespoke integration. MCP is why agents can plug into a warehouse, a CRM, or a ticketing system without needing a custom adapter for each one.
People have started calling it the USB-C for AI, meaning everything plugs into the same port. But that doesn’t mean every device on the other end speaks the same language once it’s plugged in.
MCP tells an agent how to call a tool. It doesn’t translate what “EBITDA” means to your finance team, or how “active customer” is defined. Standardizing the protocol or interface layer still leaves the harder governance question open: where does reasoning actually happen, and how do you audit it?
Leg three: Semantic Compute Engine
This is the leg that gets consistently skipped. The word “semantic layer” gets used for a wiki, a column glossary, anything documentary. This leg runs on something different: a semantic compute engine, something that already knows your metrics, dimensions, hierarchies, fiscal logic, security rules, and aggregates, and that can act on that knowledge rather than just document it. AtScale calls this the AI Computation Engine (ACE).
With a semantic compute engine, the agent retrieves a governed definition of “revenue” that’s already been signed off on, rather than discovering it by sampling tables and guessing. ACE plans an efficient execution path against structures built for exactly that question, instead of generating a fresh, expensive SQL query against billions of rows every time. And when the next analyst asks something similar, ACE reuses what it already built rather than recomputing the same aggregation from scratch.
Why architecture beats model size
A commercial banking division inside a multinational Tier 1 bank ran a real production comparison. They compared two architectures that answered the same five questions a working analyst asks every day, such as “what was revenue in the Northeast region last quarter?”
Architecture A sent the model straight to the warehouse. With no semantic layer to consult, the LLM had to discover the schema on the fly: sample tables, guess which of several candidate tables held revenue, infer which region codes counted as “Northeast,” and improvise what “last quarter” meant against a fiscal calendar that doesn’t end in March. It got an answer, but had no way to tell whether the answer was right.
Architecture B put AtScale’s ACE in between the model and the warehouse. Revenue was already a defined measure. Northeast was already a dimension member. Last quarter was a time-intelligent calculation ACE already knew how to resolve. No guessing or re-deriving required.
Both paths returned the same answer, but the work to get there was substantially different. The same five queries cost $17.93 through the unguided path and less than a tenth of a cent through the guided one. That’s up to 21,000x less compute, which works out to roughly 21,900x lower cost, measured across a full representative day of production traffic.
The efficiency gains come down to architecture. Model choice doesn’t move the number.
Why this gets worse as agents multiply
The benchmark above measured one query per question, the easy case.
Agentic workflows compound complexity. They investigate, compare, validate, retry, ask follow-up questions, and hand work off to other agents. What starts as a single user question can fan out into dozens, sometimes hundreds, of warehouse queries as an agent works through its own verification loop. The bank behind the benchmark above already lives this: 6,000-plus AI queries a day, and on the unguided path that works out to roughly $9 million per year spent re-deriving answers the warehouse had already computed correctly once.
If the model is discovering your schema and guessing at your business logic on query one, it’s doing that same guesswork on queries two through two hundred. Agentic AI just multiplies architectural inefficiency.
Stop calling it metadata
Many vendors describe “semantics” as documentation: a catalog, an ontology, or a glossary of column definitions. Those are useful, but they don’t execute anything.
A catalog can tell an agent where a metric might live. It cannot compute the metric, choose the right aggregate, or enforce the definition that finance actually signed off on. That work still falls to the LLM, which needs to understand the warehouse at full cost every single time.
Closing the accuracy gap takes an execution framework that plans and runs the query correctly, not a catalog of definitions.
ACE computes the metric, optimizes the execution, chooses the aggregate, and governs access. It serves as the deterministic arm of the probabilistic LLM.
Where the stack goes next
The first generation of enterprise AI was mostly about making the LLM smarter. The next generation will be about making the entire stack more efficient and accurate.
LLMs will keep improving. MCP will keep maturing. Neither replaces the third leg, the part of the stack that understands what your business means and knows how to compute it efficiently, at governed accuracy, every time an agent asks.
MCP solved access. The third leg is what turns that access into a governed answer, correctly and at a fraction of the cost.
This is the architecture, backed by a production benchmark, and it’s worth seeing in full, including exactly how those numbers were measured, query by query.
Download “Semantic Layers and the Economics of AI on the Warehouse” to see the complete benchmark.
Reviewed by: Mark Palmer
SHARE
Guide: How to Choose a Semantic Layer