Industry’s Most Comprehensive Big Data Maturity Survey Reveals Surprising State of Hadoop, Dramatic Rise of Big Data in the Cloud

Big Data Maturity Survey offers insights into Business Intelligence landscape, Spark’s position, and increased concerns for governance, security and accessibility

SAN MATEO, CA, December 14, 2016 – AtScale, the first company to provide enterprises with a fast and secure self-service analytics platform for Big Data announced today the results of the 2016 Big Data Maturity Survey. Based on 2,550+ responses from big data professionals at 1,400 companies across 77 countries, the report reveals new & unexpected insights about Cloud, Big Data, Business Intelligence, Hadoop and Spark.

A complimentary copy of the report can be downloaded here.

Big Data’s future is Cloudy

Amongst many other insights, the survey reveals a surge in Cloud deployments for Big Data. More than half of respondents deploy Big Data in the Cloud today and 72% plan on doing so in the future.

“There’s been a clear surge in use of Big Data in the Cloud over the last year and what’s perhaps as interesting, is the fact that respondents are far more likely to achieve tangible value when their data is in the Cloud,” explains AtScale CTO and co-founder Matt Baird.

Mature and savvy respondents

“The maturity of respondents in this survey is a key consideration,” says Thomas Dinsmore, Big Data analytics industry analyst and author of the book Disruptive Analytics. “One in 5 respondents has more than 100 nodes and 74% of them are in production, indicating double-digit growth year over year.”

Business Intelligence soaring on Big Data

Another key trend observed in this report is the continued momentum of Business Intelligence for Big Data. Last year’s survey indicated that Business Intelligence had overtaken ETL and Data Science as the predominant workload planned for Big Data. This trends continues moving forward: Business Intelligence is the #1 workload for Big Data today and 75% of respondents are planning on using BI on Big Data. In addition, 97% of respondents indicated that they would do as much or more with Big Data over the next 3 months…. 2017 might very well be the year of Business Intelligence on Big Data!


Mature and Experienced Respondents

  • Close to 70% have been using Big Data for more than a year (vs. 59% last year)
  • 74% of respondents have more than 10 nodes, close to 20% have over 100 nodes
  • 95% of respondents have achieved positive value or are anticipating they will

Big Data Adoption Soaring

  • 97% will do as much or more with Big Data over the next 3 months
  • More than 53% of respondents are using Cloud for their Big Data deployment today
  • 14% of respondents have ALL their Big Data in the Cloud
  • 72% of respondents plan on deploying Big Data in the Cloud
  • Two-thirds (66%) view Big Data as “Strategic” or “Game Changing”, while only 19% of respondents consider it “Experimental”

#1 Opportunity: BI on Big Data

  • Business Intelligence is the #1 workload for Big Data with 75% of respondents planning on using BI on Big Data.
  • Self-service access on Big Data grew by 15% year over year, however, most companies (53%) still suffer from a lack of self-service access to Big Data.
  • Accessibility, security and governance have become the fastest growing areas of concerns year-over-year, with worries related to governance growing the most at 21%.
  • The majority of organizations use Spark for educational purposes, while 73% of respondents are now in production with Hadoop (vs. 65% last year).
  • Organizations who have deployed Spark in production are 85% more likely to achieve value.
    The survey was conducted in collaboration with Cloudera, Hortonworks, MapR, Cognizant, Trifacta and Tableau.

While many of the results are surprising, the survey is representative of what AtScale has observed across its customer base, which includes leaders across the financial services, insurance, healthcare, telecommunication, online and retail industry. Just this past week, Ventana awarded the Macy’s AtScale deployment with its Business Technology Leadership Award for Big Data, showcasing the maturity and savvy such enterprises have developed in the BI on Big Data field. AtScale recently announced the newest version of its platform will provide a universal semantic layer that provides BI on Big Data across traditional data warehouses (like Teradata), today’s data platforms (like Hadoop) and future cloud based services (Google BigQuery, Microsoft Azure, etc).

Learn More

You can request a complimentary copy of the survey

About AtScale

AtScale makes BI work on Big Data. With AtScale, business users get interactive and multi-dimensional analysis capabilities, directly on Big Data, at maximum speed, using the tools they already know, own and love – from Microsoft Excel and Power BI to Tableau Software to QlikView. Built by Big Data veterans from Yahoo!, Google and Oracle, AtScale is already enabling the BI on Big Data revolution at major corporations across healthcare, telecommunications, retail and online industries. To see how AtScale can help you, go to