Book a demo

Blog

Topics
Topics
Platforms / Tools
Platforms / Tools
Industries
Industries
Content Type
Content Type
Clear all

/ 9 MIN READ

BIRD-Interact, explained: the conversational text-to-SQL benchmark

What is BIRD-Interact? BIRD-Interact is an independent benchmark that tests how well AI systems complete database tasks through conversation. It evaluates whether a system can clarify a vague request, explore the database schema, write correct…

Posted By: Ryan Siegler

/ 7 MIN READ

Stop Writing Your Semantic Model. Start Reviewing It.

Executive summary AI tools, like AtScale’s CoModel, can now draft most of a semantic model, and that will change the data modeler’s job. Claude Sonnet 5, working through a semantic layer that AI generated and…

Posted By:

/ 8 MIN READ

How to Achieve AI Accuracy: Buy the Foundation, Rent the Model

Executive summary AI leaderboard rankings measure a model working alone, on tasks that look little like your warehouse. On the 410 business questions in the BIRD-Interact benchmark, what sat under the model made a bigger…

Posted By:

/ 10 MIN READ

A Wrong Answer Has Never Looked So Good

Executive Summary: An agent doesn’t have a gut check, so it treats a wrong answer as fact and can’t catch it before building the next question on top of it, and accuracy drags down fast…

Posted By: Dave Mariani

/ 9 MIN READ

The Missing Step in Text-to-SQL

Executive Summary: I asked how many units we sold in Q2 2008 against an AdventureWorks tutorial dataset and got 25,427 units. Correct. A second calendar in the same schema, a retail 4-4-5, returned 24,639. Also…

Posted By: Ryan Siegler

/ 7 MIN READ

Test the Math in Your AI Answers

Executive summary AI evals test what an answer says and how it says it. They rarely test whether the number inside it is right, and that number is what the business acts on. A wrong…

Posted By: Mark Palmer