What’s in store for big data in 2016? Expect updates in machine learning, real-time data-as-a-service, algorithm markets, Spark, and more.
The hype around big data and analytics has gone through cycles over the past couple of years, starting with excitement about how much data we have and the potential for it. That moment was followed by that let-down, “now what?” feeling after organizations put the storage and tools in place and found themselves wondering what to do with it. There are so many technologies and trends to track — machine learning, AI, advanced analytics, predictive analytics, real-time analytics, Hadoop, Spark, other Apache Foundation projects, open source, cloud-based-as-a-service offerings, self-service, and more.
This past year was no exception. Everybody talks about the promise and the potential of big data. Yet there’s a sense of disenchantment as CIOs search for use-cases to inspire change inside their own companies. They want to be shown, not told. They want the signal and not the noise.
We noticed that 2015 was a noisy year, and 2016 seems like it will be equally as loud. It’s not something that CIOs can afford to tune out. With digital transformations and pure-play startups disrupting established industries — Uber is the example everyone mentions first — the pressure is on to leverage data in new ways for competitive advantage. CIOs need to straddle two different worlds — satisfying their existing customer base while moving fast to deliver instant, data-driven services to customers, or they risk losing ground to market upstarts.
But how do you get from point A (lots of data) to point B (amazing business insights that give your organization a competitive advantage)? Will we get any closer to realizing that in 2016?
1. Rise Of The Chief Data Officer
As enterprises grapple with the onslaught of changes and the need for a real strategy in the wake of digital-native competitors, more will put weight and effort into titles such as chief data officer (CDO). Somebody needs to drive the strategy.
“Chief data officers will gain power, prestige, and presence … for now,” wrote Brian Hopkins, VP and principal analyst for enterprise architecture at Forrester Research, in a blog post. “But the long term viability of the role is unclear. Certain types of businesses, like digital natives, won’t benefit from appointing a CDO.”
2. Power To The Business Users
Driven by a shortage of big data talent and the ongoing gap between needing business information and unlocking it from the analysts and data scientists, look for more tools and features that expose information directly to the people who use it. For instance, Microsoft and Salesforce both recently announced features to let non-coders create apps to view business data.
3. Embedding The Intelligence
No-coding-required apps are one way companies are making it easier for business users to get the information they need now. But something else has been going on behind the scenes, too. Organizations have been embedding pieces of analytics functions directly in the applications where they are needed. IDC predicts that by 2020 half of all business analytics software will incorporate prescriptive analytics built on cognitive computing functionality.
On a mass scale, Gartner identifies “autonomous agents and things” as one of the up-and-coming trends, which is already marking the arrival of robots, autonomous vehicles, virtual personal assistants, and smart advisers.
“Over the next five years we will evolve to a post-app world with intelligent agents delivering dynamic and contextual actions and interfaces,” said David Cearley, Gartner Fellow and VP, in a statement. “IT leaders should explore how they can use autonomous things and agents to augment human activity and free people for work that only people can do.”
4. The Talent Shortage — Is It Over?
Looking for a data scientist? So is everyone else. A recent report from business consultancy A.T. Kearney showed that 72% of market-leading global companies reported that they had a hard time hiring data science talent.
However, the International Institute for Analytics predicts that the talent crunch may ease somewhat in 2016 as companies employ new tactics.
“Large corporations won’t talk about the talent crunch as much anymore,” the organization said in its predictions and priorities report. “Instead, they have taken a number of approaches to solving the crunch, relying on new university programs to boost recruiting and creating internal programs to train existing staff on analytics and data science. As a result, companies that are serious about getting data analytics are able to do so.”
Meanwhile, IDC reports that the staff shortage will extend from data scientists to data architects and experts in data management. That will drive big-data-related professional services to grow at a compound annual growth rather of 23% through 2020.
5. Machine Learning Gains Momentum
Machine learning, basically creating algorithms that enable computers to learn from experience, is attracting more attention and interest among organizations looking to automate jobs that used to require human interventions. Analyst firm Ovum predicts that machine learning will become “a checklist item for data preparation and predictive analytics” in 2016.
And Gartner identified a next stage, naming advanced machine learning as a top strategic trend for 2016. The analyst firm said that an advanced form of machine learning called deep neural nets will create systems that can autonomously learn to perceive the world on their own. “This area is evolving quickly and organizations must assess how they can apply these technologies to gain competitive advantage.”
6. Everybody Loves Spark
Ovum said that SQL “reigns supreme” for big data analytics, but Spark is growing fast. “Spark will be complementary to SQL by providing additional paths to insights, such as through streaming of graph analysis, which can then be queried using language that enterprise database developers are very familiar with,” said Tony Baer, principal analyst at Ovum, in a blog post.
7. Data-As-A-Service Business Model Is Coming
IBM’s acquisition of the Weather Company — with all its data, data streams, and predictive analytics — highlighted something that’s coming. Companies will be packaging data-streams-as-a-service as a new business model. Other companies are looking to package and sell their data, too. Forrester predicts that some companies will succeed with this strategy, while “most will sputter. Despite the promise, most companies will struggle to master the intricacies of protecting personal information and the appropriate business models,” Forrester VP Brian Hopkins said in his blog post.
8. Real-Time Insights
Forrester predicts that streaming ingestion of data and analytics will become a must-have for digital winners in 2016.
“The window for turning data into action is narrowing. The next 12 months will be about distributed, open source streaming alternatives built on open source projects like Kafka and Spark,” Forrester VP Brian Hopkins wrote.
9. The Start Of Algorithm Markets
This is another prediction from Forrester. “Firms will recognize that many algorithms can be acquired rather than developed. Just add data,” Forrester’s Brian Hopkins wrote, giving several examples of services available today, including Algorithmia, Data Xu, and Kaggle.
Source (informationweek): ubm.io/1MVci1g