Self-service Business Intelligence (BI or SSBI) is the capability for insights creators and consumers to create their own reports and analyses without requiring direct assistance from technical resources, including from IT skills such as data engineers, data modelers, data architects, platform architects and business intelligence engineers, and supported by easy to use centralized data infrastructure and governance. Self Service (e.g. Do it Yourself or Do it With Me) BI consists of the following capabilities, using no code or low code tools:
- Ad hoc query
- Data visualization
- Dashboard design
- Report generation capabilities
- Data Preparation
- Metric Creation
- Data Modeling / Dimensionalized Data Creation and Integration
- Semantic Layer Modeling
Full Service BI (Do it For Me) relies on IT resources to create and manage all or most activities required to turn data into insights. In this process, business users (sales, finance, and HR, among others) store their data and ask analysts or IT colleagues to query the data to generate reports and analyses. This traditional method required IT to have much more control over data quality, but often created bottlenecks due to resource constraints that would throttle a business’s ability to generate actionable insights at scale.
Instead, SSBI focuses on enabling the end user – an insights creator or insights consumer such as business users or data analysts – to be more involved creating and managing their data product creation, usage, analysis and presentation. With SSBI, IT teams instead focus on how data is being ingested and governed within the organization, freeing analysts to apply their expertise to dig deep into the data via data mining or preparing and analyzing data to address both pervasive and ad hoc reporting requests.
Deploying platforms to do this is just one crucial part of BI. Self-service BI doesn’t mean training everyone to be data analysts; it also doesn’t mean taking responsibility from your IT department. Instead, SSBI means encouraging and educating your business teams to understand and interact with the data they generate throughout their work.
The purpose of Self Service BI (SSBI) is to transfer key activities involved with data preparation and insights creation from IT or Technical resources to business-oriented users such as data analysts, data scientists and insight consumers.
Typical benefits of implementing self service BI are as follows:
- Faster Speed-to-Insights – SSBI enables the business user, who has greater domain knowledge to have more control over insights creation, thus enabling the business user to create relevant insights, both pervasive / repetitive and ad hoc faster by eliminating multiple hand-offs with IT and eliminated technical debt / wasted effort that comes with not being able to iterated faster (learn what the data is saying) and having someone else not as familiar creating reports and analyses that might miss the mark.
- Scalability – SSBI enables more insights creation – more data products and insights to be created because business users are not limited by IT resource availability and fighting centralized enterprise priorities.
Primary Uses of a Self Service BI
Self Service Business Intelligence is used to improve business performance through better business awareness, decisions and actions based on insights created from data. Business Intelligence supports a broad set of users and use cases, potentially addressing the needs of most every user in a business enterprise to address business questions better based on insights created from data. Business Intelligence addresses two major uses: reporting and analysis as follows:
Self Service Business Intelligence addresses two major uses: reporting and analysis as follows:
- Reporting – Providing structured output of a consistent set of data in common format for a repeated, scheduled presentation to a standard set of users. Reporting typically focuses on the “what” is happening.
- Analysis – Providing insights from data to address specific business questions, often focusing on the “why” more than the “what”. Analysis activity is focused on creating queries of the data that are more tailored and often ad-hoc to specific situations, users and uses, whereas reporting queries are more standardized and repeated and delivered to a wider audience.
Benefits of Well-Executed Self Service BI
The benefits of self service business intelligence are to deliver improved business performance through improved awareness, insights, plans, decisions and actions using data. Key impact drivers are being able to make decisions and take actions with greater relevance, speed, clarity, alignment and confidence as follows:
- Relevance – Insights delivered from data are relevant to the business subject and user needs. Data are available, accurate, timely, understandable and comprehensive to address the business users’ needs.
- Speed – Insights created from data enable actions to be taken faster, because the insights are structured to address business questions more timely and effectively.
- Clarity – The insights are seen as being clear, compelling and accurate in such a way that conclusions reached from the data are consistent across the audience of users, and data are sufficient to address questions posed from multiple interpretations of the data and insights.
- Alignment – Because the data is relevant, timely, accurate and comprehensive and the insights are presented in a clear, consistent way that is easily understood, interpretation of the data is consistent, supporting improved alignment regarding decisions and actions to be taken.
- Confidence – The insights created from data are trusted, and performance of decisions and actions are measured using data such that the relationship between insights and effective, impactful decisions and actions is direct, positive and improving.
Common Roles and Responsibilities – Self Service BI
Self Service BI involves the following key roles:
- Insights Creators – Insights creators (e.g. data analysts) are responsible for creating insights from data and delivering them to insights consumers. Insights creators typically design the reports and analyses, and often develop them, including reviewing and validating the data. Insights creators are supported by insights enablers.
- Insights Enablers – Insights enablers (e.g. data engineers, data architects, BI engineers) are responsible for making data available to insights creators, including helping to develop the reports and dashboards used by insights consumers.
- Insights Consumers – Insights consumers (e.g. business leaders and business analysts) are responsible for using insights and analyses created by insights creators to improve business performance, including through awareness, plans, decisions and actions.
- Insights Consumers – Insights consumers (e.g. business leaders and analysts) are responsible for using insights and analyses created by insights creators to improve business performance, including through improved awareness, plans, decisions and actions.
- Data Engineers – Data engineers create and manage data pipelines that transport data from source to target, including creating and managing data transformations to ensure data arrives ready for analysis.
- Analytics Engineers – Analytics engineers support data scientists and other predictive and prescriptive analytics use cases, focusing on managing the entire data to model ops process, including data access, transformation, integration, DBMS management, BI and AI data ops and model ops.
- Data Modelers – Data Modelers are responsible for each type of data model: conceptual, logical and physical. Data Modelers may also be involved with defining specifications for data transformation and loading.
- Technical Architect – The technical architect is responsible for logical and physical technical infrastructure and tools. The technical architect works to ensure the data model and databases, including source and target data is physically able to be accessed, queried and analyzed by the various OLAP tools.
- Data Analyst / Business Analyst – Often a business analyst or more recently, data analyst are responsible for defining the uses and use cases of the data, as well as providing design input to data structure, particularly metrics, topical and semantic definitions, business questions / queries and outputs (reports and analyses) intended to be performed and improved. Responsibilities also include owning the roadmap for how data is going to be enhanced to address additional business questions and existing insights gaps.
Key Business Processes Associated with Self Service BI
The processes for delivering self service BI include the following:
- Access – Data, often in structured ready-to-analyze form and is made available securely and available to approved users, including insights creators and enablers.
- Profiling – Data are reviewed for relevance, completeness and accuracy by data creators and enablers. Profiling can and should occur for individual datasets and integrated data sets, both in raw form as was a ready-to-analyze structured form.
- Preparation – Data are extracted, transformed, modeled, structured and made available in a ready-to-analyze form, often with standardized configurations and coded automation to enable faster data refresh and delivery. Data is typically made available in an easy to query form such as database, spreadsheet or Business Intelligence application.
- Integration – When multiple data sources are involved, integration involves combining multiple data sources into a single, structured, ready-to-analyze dataset. Integration involves creating a single data model and then extracting, transforming and loading the individual data sources to conform to the data model, making the data available for querying by data insights creators and consumers.
- Extraction / Aggregation – The integrated dataset is made available for querying, including, including aggregated to optimize query performance.
- Analyze – Process of querying data to create insights that address specific business questions. Often analysis is based on queries made using business intelligence tools using a structured database that automate the queries and present the data for faster, repeated use by data analysts, business analysts and decision-makers.
- Synthesize – Determine the key insights that the data are indicating, and determine the best way to convey those insights to the intended audience.
- Storytelling / Visualize – Design of data storyline / dashboards and visuals should be prepared and then developed based on the business questions to be addressed and the queries implemented. It is important to think about how the data will be presented so that the insights are understood and addressed.
- Publish – Results of queries are made available for consumption via multiple forms, including as datasets, spreadsheets, reports, visualizations, dashboards and presentations.
Common Technologies Associated with Self Service BI
Technologies involved with Self Service BI are as follows:
- Data Engineering – Data engineering is the process and technology required to move data securely from source to target in a way that it is easily available and accessible.
- Data Transformation – Data transformation involves altering the data from its raw form to a structured form that is easy to analyze via queries. Transformation also involves enhancing the data to provide attributes and references that increase standardization and ease of integration with other data sources.
- Data Preparation – Data preparation involves enhancing it and aggregating it to make it ready for analysis, including to address a specific set of business questions.
- Data Modeling – Data modeling involves creating structure and consistency as well as standardization of the data via adding dimensionality, attributes, metrics and aggregation. Data models are both logical (reference) and physical. Data models ensure that data is structured in such a way that it can be stored and queried with transparency and effectiveness.
- Database – Databases store data for easy access, profiling, structuring and querying. Databases come in many forms to store many types of data.
- Data Querying – Technologies called Online Analytical Processing (OLAP) are used to automate data querying, which involves making requests for slices of data from a database. Queries can also be made using standardized languages or protocols such as SQL. Queries take data as an input and deliver a smaller subset of the data in a summarized form for reporting and analysis, including interpretation and presentation by analysts for decision-makers and action-takers.
- Data Warehouse – Data warehouses store data that are used frequently and extensively by the business for reporting and analysis. Data warehouses are constructed to store the data in a way that is integrated, secure and easily accessible for standard and ad-hoc queries for many users.
- Data Lake – Data lakes are centralized data storage facilities that automate and standardize the process for acquiring data, storing it and making it available for profiling, preparation, data modeling, analysis and reporting / publishing. Data lakes are often created using cloud technology, which makes data storage very inexpensive, flexible and elastic.
- Data Visualization – Data visualization is a method for visually depicting slices of data formed through queries. Data visualizations are typically available in Query tools (OLAP / BI Applications), as standalone-applications and as libraries.
Trends / Outlook for Cloud Data Warehouses
Key trends to watch in the Self Service BI arena are as follows:
- Semantic Layer – The semantic layer is a common, consistent representation of the data used for business intelligence used for reporting and analysis, as well as for analytics. The semantic layer is important, because it creates a common consistent way to define data in multidimensional form to ensure that queries made from and across multiple applications, including multiple business intelligence tools, can be done through one common definition, rather than having to create the data models and definitions within each tool, thus ensuring consistency and efficiency, including cost savings as well as the opportunity to improve query speed / performance.
- Real-Time Data Ingestion and Streaming (CDC) – Create and manage data as live streams to enable and support modern cloud-based analytics and microservices. Extend enterprise data into live streams to enable modern analytics and microservices with a simple, real-time and universal solution.
- Data Governance – Implementing effective, centrally managed data governance, including data usage policies, procedures, risk assessment, compliance and tools for managing secure access and usage such as semantic layer, metric stores, feature stores and data catalogs.
- Automation – Increase emphasis is being placed by vendors on ease of use and automation to increase speed-to-insights. This includes offering “drag and drop” interfaces to execute data-related preparation activities and insights creation / queries without having to write code, including reusing activities and processes, both for repeating use as well as sharing.
- Self-service – As data grows, availability of qualified data technologists and analytics are very limited. To address this gap and increase productivity without having to lean 100% on IT resources to make data and analysis available, Self-service is increasingly available for data profiling, mining, preparation, reporting and analysis. In addition tools like the Semantic Layer offered by AtScale, Inc are also focused on enabling business users / data analysts to model data for business intelligence and analytics uses.
- Transferable – Increased effort is also underway to make data easier to consume, and this includes making data available for publishing easier, including using api’s and via objects that store elements of the insights.
- Observable – Recently, a host of new vendors are offering services referred to as “data observability”. Data observability is the practice of monitoring the data to understand how it is changing and being consumed. This trend, often called “dataops” closely mirrors the trend in software development called “devops” to track how applications are performing and being used to understand, anticipate and address performance gaps and improve areas proactively vs reactively.
AtScale and Self Service BI
AtScale is the leading provider of the Semantic Layer – to enable actionable insights and analytics to be delivered with increased speed, scale and cost effectiveness. Research confirms that companies that use a semantic layer improve their speed to insights by 4x – meaning that a typical project to launch a new data source with analysis and reporting capabilities taking 4 months can now be done in just one month using a semantic layer.
AtScale’s semantic layer is uniquely positioned to support rapid, effective self-service BI: AtScale provides the ability to ensure that data used for AI and BI are consistently defined and structured using common attributes, metrics and features in dimensional form, including automating the process of data inspection, cleansing, editing and refining as well as rapidly adding additional attributes, hierarchies, metrics / features, and extracting / delivering ready-to-analyze data automatically for multiple BI tools, including Tableau, Power BI and Excel. Moreover, this work only requires one resource who understands the data and how it is to be analyzed, eliminating the need for complexity and resource intensity. This approach to self-service BI and data operations automation eliminates multiple data hand-offs, manual coding, the risk of duplicate extracts and suboptimal query performance.