See AtScale in Action
Request a Demo
Close

Request a Demo

How An Adaptive Analytics Fabric Helps Conquer the Top 5 Cloud Migration Pitfalls

Cloud migration: The good, the bad, and the ugly

Cloud migration is the process of moving data and application hosting infrastructure from on-premise data centers to a public cloud hosting service such as Amazon Web Services (AWS), Google Cloud Platform (GCP) or Microsoft Azure.

The cloud has rapidly evolved to being the new normal, and is integral for the digital transformation of organizations of all sizes. In fact, according to Forrester Research, global public cloud spending will top $200 billion in 2019.

Moving to the cloud can mean lower operating costs, the flexibility and scalability companies need to grow, and the agility to adapt to changing business landscapes. Cloud technologies provide the tools and infrastructure necessary to access, combine and analyze vast amounts of data living across the organization in disparate systems.

But migrating your enterprise data to the cloud can be challenging. The cloud is a different operating environment, with new integrations, pricing models, security controls and optimization tactics. It is a complicated endeavor to thoroughly assess the cloud-suitability, operational impact, security risks and business benefits of migrating each system or process. The best way to realize the promised benefits and business advantages of migrating data to the cloud is to leverage the right technology to get there.

Read on to explore how an adaptive analytics fabric can help eliminate five common pitfalls that organizations encounter while undergoing a cloud migration.

Pitfall 1

1. Higher than expected (and less predictable) operating costs

Lower costs are almost always touted as one of the biggest benefits of the cloud. But many organizations that migrate to the cloud are soon dismayed by “cloud sticker shock.”

Migrating to a cloud or hybrid-cloud environment promises cost savings in storage and maintenance, and is frequently a decision driver. However, cost savings aren’t realized unless resource usage is well managed.

For example, cloud providers often offer consumption-based plans, where the rate you pay is based on the bandwidth and services you use. Paying only for what you use is a no brainer, right?

It turns out that even though the cloud may offer more flexibility, predicting monthly charges is extremely difficult on a consumption-based plan.

A malformed query, for example, can end up costing thousands of dollars in unexpected and unplanned usage.

Overlooking the steps of optimizing queries and data scans can result in some expensive surprises.

Cloud providers also offer fixed pricing plans with a predetermined amount of usage available. But that same malformed query can eat up all your available resources if you’re on a fixed pricing plan. Just like going over your allotted minutes on a fixed monthly cell phone plan, you’ll have to pay additional fees for continued usage.

What you can do to make cloud operating costs more predictable

Pricing models will vary by cloud platform. Depending on the platform you’re using, you may be charged for the size of your database, the number of queries you run, the data being moved in a query, the number of rows in a query or a number of other variables. Therefore, before migrating your data to the cloud, you need to have a resource governance plan.

In some cases, you can manage your database directly. Snowflake, which offers a data warehouse built for the cloud, has a cool feature that creates separate data warehouses with different resourcing levels without duplicating data. This helps manage costs at a fine granularity, depending on the priority of the function or group. Caching technology and a central data governance platform is another way of managing costs.

In others, such as Google BigQuery, the amount you’re charged is proportional to the size of your database. But with an adaptive analytics fabric, you can build acceleration aggregates on your database, so you’re only charged for the data you use, not the size of the entire database. At AtScale, we’ve saved tens of millions of dollars for our customers by helping them avoid redundant full-table scans through our acceleration structure technology.

Pitfall 2

2. Finding and using data in multiple locations

It’s a challenge for business analysts and data scientists to find and use the data they need. Throughout the self-service analytics revolution, we’ve asked them to become data janitors—they need to learn how to decipher data in all the various data platforms, dialects, types and formats before they can do their actual jobs.

In fact, some data scientists spend 80% of their time just preparing data for their models.

The cloud era has made their jobs even harder. Data is now scattered across organizations in disparate formats and systems, making it difficult to locate, access, and integrate for analysis. Some of it may be in the cloud, some of it may be in on-premise servers, and it is often in different formats and governed by different policies and security practices.

What you can do to minimize the impact of disparate data

The most obvious solution to having data in disparate locations is to consolidate it in one place in the cloud, using enterprise data warehouse technologies like Snowflake and Google BigQuery. But moving all of your data to a single platform is a long-term goal. And even then, there will always be new platforms that may be even more advantageous for you to move your data to.

An adaptive analytics fabric ensures all of your data across all of your systems is represented in a single common business language and readily accessible from a single virtualized source, even while the data remains distributed across multiple platforms.

Not only will your cloud migration be accelerated and simplified, but your data will be optimized for insights and decision-making.

Moreover, with an adaptive analytics fabric, you can put acceleration structures in any database, and the fabric will automatically decide where to put data based on where it will enable the best performance. This empowers you to leverage many different databases that each have different advantages and strengths, producing optimizations that a human might not be able to conceive of. The variability of all your different database types has traditionally been a liability; now, with an adaptive analytics fabric, that liability is turned into a strength.

Pitfall 3

3. Impact on report and application migrations

For any database migration, applications and reports that have been written for a legacy database dialect need to be ported to the new platform. Unfortunately, these data models and dialects are often baked directly into hundreds or even thousands of reports and dashboards.

Porting these reports is a Herculean task, so often the existing reports are left on the older platform while new reports are written against the new data platform. Unfortunately, this just ensures an enduring presence for the legacy platform and sets up the same scenario for future migrations, complicating the stack for everyone.

What you can do to avoid the impact of cloud migration on reports and applications

An adaptive analytics fabric can reverse engineer the queries and data models used to create legacy reports. It can determine which data sets were used, and what queries were run, so you don’t have to rebuild data models or queries and you can keep using the same report — if your TPL report is from the old system, you can still read it in the new system.

An adaptive analytics fabric can also translate your old queries into new formats. Even if it was run on old data, a report can still be translated and run on the new system.

Pitfall 4

4. Security integration

In an on-premise environment, you’re used to owning 100% of your security, networking and compute infrastructure. Single sign-on (SSO) technologies such as AD and LDAP, VPNs, firewalls and the rest are locked down and under your control.

But when you move to the cloud, you now have to integrate your data center security stack with your cloud vendor’s stack. For enterprise data warehouses, you may need to deal with an entirely different authentication protocol.

For example, Google BigQuery relies on the Google Identity Platform for authentication, which means you need to figure out how to sync your Active Directory (AD) with Google’s sign-in directory. Now your carefully-crafted SSO strategy that you worked so hard to deliver over the past decade is essentially worthless.

What you can do to properly orchestrate security policies

An adaptive analytics fabric enables companies to leave data where it is. That means all of the existing security solutions and policies governing your data remain in place as well.

While your data may be readable to all of your users and a multitude of different BI tools, your permissions and policies are not changed. Security and privacy information is preserved all the way to the individual user by tracking the data’s lineage and the user’s identity. The user’s identity is also preserved and tracked, even when using shared data connections from a connection pool.

When users are working with multiple databases that may have different security policies, policies are seamlessly merged, and global security and compliance policies are applied across all data.

Pitfall 5

5. User retraining

It’s already bad enough that you have to retrain your users on where to get their data, but if you also need to convince them to learn new tools for visualization and/or modeling, your chances of success drop dramatically.

Everyone has their own personal preference when it comes to their tools. Whether it’s an individual choice or the standards of each department, most enterprise-level companies have already invested a considerable amount into BI tools. Traditionally, moving to a new enterprise data platform requires that you either switch to new tools or that you re-engineer your data models to fit the new architecture.

What you can do to make cloud migration seamless for BI users

Your prospective cloud vendor will likely try to convince you to trade out your existing BI tools for ones compatible with their platform. There is no reason for you to do this.

With adaptive analytics, data scientists and business users can use any BI tool they want, because the differences in data formats are automatically normalized with a semantic translation layer. No longer do you have to bend all users to a single standard for BI software. Disparate datasets can be accessed, integrated and analyzed with any BI tool, and data and queries will always return consistent answers.

Now your cloud journey is clear

Avoiding the pitfalls and challenges that plague so many organizations and gaining the expected benefits of cloud migration in a matter of weeks rather than months or years will make your organization more agile and competitive in an ever-accelerating digital landscape.

Whether you’re considering a migration to a cloud or hybrid-cloud environment, in the midst of a migration, or have already made the transition, an adaptive analytics fabric provides the confidence you need to move data from legacy systems, avoid disruptions in operations and unexpected costs, integrate all of your data into a single view, keep using your preferred BI tools, and ensure all of your existing security solutions and policies governing your data remain in place.

Let the others stumble into cloud migration pitfalls while you propel your organization forward with an adaptive analytics fabric.

More Great Content

ABOUT ATSCALE

AtScale is the leading provider of adaptive analytics for data architecture modernization, empowering citizen data scientists
to accelerate and scale their business’ data analytics and science capabilities and ultimately build insight-driven enterprises.
For more information, visit us at atscale.com.

Get your free copy