A Skill or README Isn’t a Semantic Layer

Why not just give the agent a skill? A bunch of people asked this at this week’s Databricks Data + AI Summit. Write down what “revenue” means, drop the skill next to your data, and skip the semantic layer.

I understand the appeal of this hack. I’ve written dozens of skills, and they’re powerful. But for analyzing corporate data, they aren’t deterministic, dependable, or cost effective. A skill describes your business. It can’t enforce it. It can’t make the model compute a metric correctly, keep the wrong number from the wrong person, or stop it rediscovering math your finance team settled years ago. A skill tells. A semantic layer governs.

Where a skill document stops

A skill is a static instruction, and a model treats it as a suggestion, not a contract. Ask a real question, like API revenue, token consumption, and active workspaces by model and plan tier for the trailing twelve months, and the model writes its own SQL to go get it. It joins the usage logs to the account and product tables, picks its own trailing-twelve-month cutoff, and runs a COUNT(DISTINCT) across a year of usage. Maybe it gets it right. Run it again and the active-workspace count comes back different, because nothing pins the fiscal rules down. You find out when the board deck is wrong.

Three things a skill can’t do:

1. Enforce. “Active workspace” and “trailing twelve months” have to compute one way, every run, for every tool. The skill can’t make them, which is why the count just drifted.
2. Govern. Revenue by plan tier isn’t for everyone. A skill doesn’t know who’s asking, and that access rule doesn’t live in prose.
3. Control cost. To answer, the model scans a year of raw usage logs for that COUNT(DISTINCT) instead of reading a maintained aggregate, and you pay for the scan every time someone asks. (A maintained aggregate is just a running total the layer keeps current, so the number is already tallied and waiting. It’s the scoreboard, not a recount of every ticket stub.)

You can’t run a business on something that won’t enforce a number, control who sees it, or predict what it costs. A semantic layer can. It’s a running engine between the agent and the warehouse, not a file the model skims. It holds one definition, carries your governance, and routes every query to the cheapest correct path.

21% to 95% accuracy, and 21,000x less cost

Anthropic builds frontier models for a living, and it published its findings about how they went about making their internal reports accurate. On raw data its agent was right about 21% of the time; with the AtScale semantic layer, without prompting guidance, accuracy went to  95%. With proper prompt engineering, results approached 100%. 

The fix wasn’t a better model. It was semantics. If a skill were enough, the team with the best models on earth would have written one and stopped. They didn’t.

Accuracy is only half the bill. A Tier 1 bank benchmarked hand-crafted prompts pointing an LLM at the warehouse, the exact thing you’d put in a skill, against a semantic engine. The engine cut compute by up to 21,000x. Point a model at your data and it tokenmaxxes: it discovers the schema, writes its own SQL, and burns tokens rediscovering metrics you defined years ago, roughly $9 million a year at this bank. A skill doesn’t change that. Better prompting improves the guess. It doesn’t remove it.

Author once, depend on it everywhere

Define revenue once, in an open, governed layer outside any single tool. The controller in Excel, the analyst in Power BI, and the engineer wiring up an agent through MCP all reach the same definition. Same number, same governance, and the agent rediscovers nothing.

Write the skill with a semantic layer as its foundation

So write the skill. Then put a semantic layer underneath it. Before you settle for “just give the agent a skill,” ask three questions: 

• Does it enforce one definition at query time? 
• Does it carry your security and governance? 
• Does it keep the model from rediscovering your schema on every question?

If the answer is no, you don’t have a semantic layer. You have an elegant mess: ungoverned, tokenmaxxing instructions that deliver answers 21% accurate, blow a $9M hole in your budget, and cause confusion and inefficiency.

Use the SKILL + semantic layer strategy, and you approach 100% accuracy, control costs, and scale AI intelligence across the organization. That’s not a hack. That’s infrastructure.