The Universal Semantic Layer. More Important than Ever.
A little more than a year ago, my colleague and co-founder, Matt Baird, wrote a great article called “What Is A Universal Semantic Layer? Why Would You Want One?”. It’s been one of our most popular posts so clearly people are interested in the concept. I went back and read it recently and concluded that not only is it still highly relevant, it’s more relevant than ever.
As a reminder, this is Wikipedia’s pretty good definition of a semantic layer:
Add the adjective “Universal” to this definition and you’ll see why it’s a pretty popular topic these days. In this post, we’ll dig a little deeper and talk about how a Universal Semantic Layer (USL) should be a critical element of your modern analytics stack.
What A Difference a Year Makes
In little more than a year, the data warehousing space has changed drastically. Now, cloud data lakes and cloud data warehouses have become well-accepted data platform architectures. According to the 2020 Big Data & Analytics Maturity Survey, 61% of respondents currently operate cloud data platforms, and 48% plan on deploying them in the near future. In the meantime, Hadoop didn’t become the end all data solution but, rather, just one solution for managing data. What does this all mean? It means that data architects are increasingly becoming more comfortable with data living in different locations and in different platform architectures. While “choosing the best tool for the job” is a mantra I personally subscribe to, the challenge of managing data access and quality across multiple silos is a big issue for IT. This is why a Universal Semantic Layer is becoming an even more critical piece of your data platform strategy.
Location, Location, Location
It’s clear that more and more enterprises are moving to the cloud. While they do so, their analytics stack becomes more complicated (not less) because their on-premise data stacks live on throughout the transition. In fact, many of these on-premise data platforms may never go away. How can IT bridge their business users across these old and new worlds? A Universal Semantic Layer is a great equalizer, hiding the physical complexity from users by presenting them with understandable business terms and business-friendly data instead of database schemas and raw SQL.
As you can see in the architecture below, with a proper USL, you get a centralized repository of business terms and data virtualization that makes data access ubiquitous for any analytics consumer.
Where a Universal Semantic Layer Fits in the Data Stack
This sort of abstraction layer makes data locale and data format invisible to end users.
Who, What, Where?
It’s not just that data has become more dispersed. The types of analytics consumers and their respective tool sets have proliferated too. While the needs of a data scientist and a business intelligence user may seem quite different, they both need simple and secure access to clean, understandable data. With today’s self-service architectures, we’ve forced our analytics consumers to become data wranglers and data engineers. In fact, the average data scientist spends more than 80% of their time preparing data rather than modeling it. Besides being a colossal waste of time, by asking business users and data scientists to program their own metrics and business terms, we’ve created a recipe for chaos and inconsistency. Again, the USL is an excellent solution to this problem as well. By defining business metrics, data access and transformations in one place, analytics consumers are almost guaranteed to speak the same language, regardless of their use case of tool sets.
Even better, by creating a single point for data access, the USL also serves as a central governance gateway across the enterprise. IT can secure the data and control its access once and for all. As you can see from the chart below, 79% of enterprises rank cloud security and governance critical to their success in the cloud.
Source: 2020 Big Data & Analytics Maturity Survey
Keep it Universal and Make it Simple
I’m a firm believer in the design principle of KISS (Keep it Simple Stupid). By removing multiple, moving parts, you can simplify a design drastically and improve resiliency.
So, why add another moving part to the analytics stack you ask? Actually, by adding a Universal Semantic Layer to your architecture, you can drastically simplify your stack, not complicate it. To start with, you can retire multiple, proprietary and conflicting semantic layers that are tough to maintain and impossible to keep in sync. The key is that a semantic layer is useless and counterproductive unless it’s universal. All tool and cloud vendors want to convince you to stay within their walled gardens with their tool-specific semantic layers.
By investing in a stand-alone Universal Semantic Layer, you can free yourself from vendors’ proprietary chains and create the flexibility you’ll need as new data platforms and tools inevitably continue to proliferate. Best of all, with a USL, everyone will be speaking the same language and playing by the same rules of data access.
To learn more about how the AtScale Universal Semantic Layer can work for you, download the “Achieve Data-Driven Insights With AtScale Cloud OLAP” whitepaper.
To learn more about where enterprises are investing, download the “2020 Big Data & Analytics Maturity Survey Results” report.
To learn more about how AtScale can scale your cloud data warehouse and save you money, download the “AtScale Cloud Data Warehouse Benchmark Report” report.