Request a Demo

Consider This When Managing Snowflake

In a recent webinar, Mark Stange-Tregear of Rakuten Rewards and I discussed how to  control costs and manage your cloud data warehouse environment. Mark made some excellent suggestions for how to better manage your  Snowflake environment. 

A lot of the cloud data warehouse vendors insist that they already have fast performance and endless capacity, leading you to believe that you have nothing to worry about. But, buyer beware. There’s always a catch. When it comes to thinking about managing a cloud data warehouse efficiently, you should think about the following four dimensions: 

Query Performance

Query Performance: How fast can the cloud data warehouse return a single?

Each cloud data warehouse has its own query latency for returning query results. If your end users require OLAP style, instant queries, not all cloud data warehouses will fit the bill. Snowflake actually does a pretty good job in this department by leveraging their query cache. However, if a query hasn’t been cached, you’re likely to see queries run for several seconds.

User Concurrency

User Concurrency: How do multiple users running simultaneous queries affect performance and stability? 

You would think that with the cloud’s endless compute capacity, that user concurrency wouldn’t be a problem. But, that’s just not the case. If your cloud warehouse is undersized for a spike in user query activity, query latency increases. As Mark mentioned, the way that cloud data warehouses typically deal with too much concurrency is by queuing. While queuing maintains the health of the cluster, queries will stack up and wait for their slot to run. This can result in unpredictable query runtimes and frustrated users. 

Compute Costs

Compute Costs: How do query workloads and cluster configuration impact your monthly bill?

Mark talked a lot about compute costs because that is something that needs to be planned and managed before you get that surprise monthly bill (we’ve all been there).. At Rakuten Rewards, Mark built a cost management system by leveraging Snowflake’s system activity tables so his CFO can see all cloud costs by department and function. No surprises.

Semantic Complexity

Semantic Complexity: How difficult is it to write the query to answer the business question? 

Mark and I talked a lot about the importance of data modeling to make your cloud data warehouse consistent and easy to use. By creating a semantic layer on top of Snowflake’s raw tables and views, Mark made sure that his analysts and data scientists were all speaking the same language and saved them from the drudgery of ETL and data engineering.

What does the Cloud analytics stack look like? 

The Rakuten Rewards team restructured their data infrastructure by moving from an on-premises Hadoop cluster to a Snowflake cloud data warehouse on Amazon Web Services (AWS) with AtScale providing universal semantic layer to optimize queries, manage costs and make the data easy to work with.. Mark’s centralized  BI team provides his internal customers with access to data through Tableau Server dashboards, ad hoc analysis using Tableau Desktop as well as hand written SQL queries. 

The Cloud Analytics Stack

The Benefits of the Universal Semantic Layer

What makes all this possible is the Universal Semantic Layer.  AtScale, you get a full multi-dimensional engine that provides a rich business friendly interface for users while ensuring consistency for key business metrics and definitions.

The Cloud Analytics Stack with AtScale

In addition to the power of a semantic layer, AtScale’s single point of entry delivers a one stop data governance shop. You can apply your data governance policies at a logical or physical level while virtualizing and hiding the physical implementation of the data.

Snowflake’s elastic, scalable resource model helps Rakuten Rewards manage their analytics infrastructure with the flexibility and scalability the business demands. AtScale’s Universal Semantic Layer providing labor-saving automation and is making Rakuten’s data easier and safer to use.

Related Reading: 

More Articles

Five Ways to Improve Your Analytics ROI on Snowflake

For practical reasons, it is increasingly difficult—and sometimes impossible—to perform large-scale analytics with an on-premises data warehouse. Scaling physical hardware to the needs of your customers is so cost intensive that you might well find yourself running a data center rather than performing analytics on your data. If you’d rather keep your focus on your core business, you may have migrated, or be planning a migration, to a cloud data warehouse such as Snowflake. What’s more, the cloud data warehouse is infinitely more scalable. You don’t have to blend data from different levels of granularity across different systems—instead, all the…

Read More

Join us to Learn How to Get More Value from your Snowflake Investment

Want to improve the ROI on your Snowflake Cloud data warehouse? Join the experts from Rakuten Rewards and AtScale on a live webinar this Thursday, July 23rd and learn how. In an hour, they will explain how you can reduce unnecessary costs and boost BI performance on the Snowflake platform. A Sneak Peak… Snowflake’s cloud data platform comes with the promise of cost-savings. But, if you aren’t careful, cloud compute usage can sneak up on you and leave you with runaway costs. We’ve called in some of the experts in the area, including our own Dave Mariani, Co-Founder and Chief-Strategy…

Read More