“Google, Facebook are really big data companies, not software companies. They collect data, process it and sell it back with value added extensions. They don’t have better algorithms. They simply have more data.” — Anonymous
The convergence of cloud, social, mobile and connected computing has sparked a data revolution. More than 90 percent of the world’s data has been generated over the last two years . And with a projected 50 billion connected “things” by 2020 , the volume of data available is expected to grow exponentially. This proliferation of data has created a vast ocean of potential insights for companies, allowing them to know their customers in a whole new way.
Data is valuable. Data is plentiful. Data is complex. Data is in flux. Data is fast moving. Capturing and managing data (Cloud, On-Premise, Hybrid IT) is challenging. It’s a paradox of the information age. The glut of information that bombards us daily too frequently obscures true insight.
Help people uncover, see, understand and visualize data presents a broad and momentous market opportunity….call this user-driven discovery. Take for instance, Facebook (like Amazon.com) builds a custom Web page every time you visit. It pores over all the actions your friends have taken—their postings, photos, likes, the songs they listen to, the products they like—and determines in milliseconds which items you might wish to see, and in what order. Is this the future for every firm…..
The opportunity is simply getting bigger by the day. Every customer interaction is generating a growing trail of data (“data exhaust”). Every machine that services the customer is generating data. Every conversation, transaction, engagement, touchpoint location, offer, response is a potential digital bread-crumb of opportunity.
Now let’s flip the context. A typical mobile user check their phone interface 150 times a day for updates. A Gen Y or Millenial user obviously much more than a Gen X user. The consumption patterns for information are changing continuously. Facebook style real-time updates which were revolutionary 5 years ago seem outdated in the mobile world. We live in an “attention deficit economy” where attention is the new basis for competition. The firms that create the evolving experience using data which can grab/hold your attention will attract marketing and ad $$.
As a result, the buzz and hype around data…small data, big data, machine data, social data, mobile data, wearables data….is relentless. As a result there are a lot of new initiatives and companies. I have been asked repeatedly by a lot of entrepreneurs and strategy teams about analytics market size and opportunity size. Product and services firms are also interested in opportunity sizing as they create new offerings in the data rich world.
I thought i would share a mashup of industry and market sizing data i have collected so far.
- How big is the overall market for Analytics, Big Data?
- How big is the market for Digital Customer Interaction or Engagement?
- How big is the market for Mobile and Social Intelligence?
- How big is the market for Wearables?
- What is growing fast, faster and fastest?
All good questions as services firms think about digital strategy, analytics and future state. You always want to be in the “hot” area… selling is easier, valuations are richer, revenue growth percentages exponential.
Everyone has data, but the more elusive goal is getting value out of that data The growing challenge in corporations is how to organize for “data as a platform.” What is the right organizational structure that will help monetize data?
John Wanamaker, considered a pioneer in modern advertising, said: “Half the money I spend on advertising is wasted; the problem is I don’t know which half.” Today, we can say the same of enterprise investment in business intelligence (BI), analytics, and big data.
Even after doing their best for over 20 years to build centralized, scalable information architecture, I found that only a small percentage of organizations’ data is actually converted to useful information in time to leverage it for better insight and decisions.
At both strategic and tactical levels, much of this gap can be explained by the fundamental disconnect in goals, objectives, priorities, and methods between IT professionals and the business users they should ideally serve.
The other challenge facing leadership is the rapid evolution of the data platform (see below.) How do you create strategies that adapt to a changing landscape?
How do you become a world-class data-driven firm? What portfolio of projects do you execute to mature the capabilities?
If you’re an executive, manager, or team leader, one of your toughest responsibilities is managing and organizing your BI, Reporting or Analytics initiative. While the nuances – skillsets, toolsets and datasets — are different for each initiative, the fundamentals of managing, organizing and structuring are pretty much the same.
Almost every Fortune 1000 company’s management is increasingly focused on monetizing small data, big data or fast data, and how to gain a real-time competitive edge from their information. How can firms achieve positive returns on their analytic investments by taking advantage of the growing amounts of data?
So what’s the right organizational model that will help them achieve the “ten second advantage”? Competency Centers, Centers of excellence (CoE) or Shared Services models are execution models to enable the corporate or strategic vision to create an enterprise that uses data and analytics for business value.
The goal of every World-class CoE is the same – enable the right combination of toolsets, skillsets, mindsets and datasets for better, faster, cheaper and more repeatable analytics, reporting or platform development.
Evolution of BI/Reporting/Analytics
- Data is Growing Faster than Budgets
- Demand is Growing, Speed to Insight is Crucial
- Modifying large, existing applications is NOT the path forward.
- Skills are lagging.. New tooling
As a result, Enterprise BI and Analytics strategies need to evolve. The evolution tends to happen in 3 phases:
- Department Solutions – Many companies deploy Analytics (and BI) applications as departmental solutions, and in the process, accumulate a large collection of disparate BI technologies – SAP Business Objects, IBM Cognos, Microstrategy, Oracle OBIEE, Microsoft, Qlikview, Tableau, Spotfire etc. – as a result. Each distinct technology supported a specific user population and database, within a well-defined “island of analytics.” At first, these dept islands satisfied the initial needs of the business, but early success in departmental deployment sowed the seeds for new problems as the applications grew.
- Successful applications and platforms always expand. The second phase of Analytics (and BI) is where there is tremendous growth and platform solutions are longer isolated islands. Instead, they overlap in user populations, data access, and analytic coverage. As a result, organizations are now faced with an untenable situation. The enterprise is getting conflicting versions of the truth through the multiple disparate BI systems, and there is no way to harmonize them without an extraordinary ongoing manual effort of synchronization, validation and quality checks. Equally problematic is the fact that business users are forced to use many different BI tools depending on what data they want.
- The third phase of Analytics (and BI) is one where the executives had enough. They simply make a decision to rationalize to a single platform or a centralized model that is sold as a “magic nirvana” solution…delivers one version of the truth (golden source of data) to all people across the enterprise. It can access all of the data, administer all of the people, eliminate repetitive data access, reduce the administrative effort, and reduce the time to deploy new BI applications.
“Time to decisions, scope of decisions, disconnected toolsets and cost of decisions” is deemed unacceptable within & across functional areas. This typically drives a new phase… centralized BI, Reporting or Analytics CoE.
For example, at a Fortune 500 company, costly self-service environment, static reports, departmental solutions and other issues (shown below) forced them to re-think and re-engineer their enterprise BI solution. The firm set new target objectives…(1) Shorter time to insights; (2) Greater leverage for analytics team; (3) Accelerated product innovation and (4) 20% reduction in BI support costs.
While centralization of BI, Reporting and Analytics can enable organizations to reduce their IT delivery costs by up to 40%. However, a failure to align the level of BI, Reporting and Analytics centralization closely to long-term business and IT strategic goals and to manage the transition to centralized delivery carefully can not only erode expected savings from centralization, it can increase the cost of delivering IT services by up to 30-45% compared to a pre-centralization baseline. This where good management can make a big difference.
BI CoE Elements for Faster, Better, Cheaper Execution
BI CoE (could be Analytics CoE, Big Data CoE or Integration CoE) is an organizing mechanism to align People, Process, Technology and Culture. The target benefits include:
- Better collaboration between Business and IT
- Increased adoption and use of BI and Analytics in the lines of business.
- Better data management, quality and reporting
- Cost savings from eliminating redundant functions
CoE elements include:
However, it took until 1980s when decision support systems (DSS) became popular and mid 1990s for BI started to emerge as an umbrella term to cover software-enabled innovations in performance management, planning, reporting, querying, analytics, online analytical processing, integration with operational systems, predictive analytics and related areas.
Gartner 2014 magic quadrant shows the key players in the BI market. The different players are differentiated based on five abilities— ability to handle large volumes of data, ability to deal with data velocity, variety (structured and unstructured), visualization capabilities and domain/vertical specific accelerators.
Analytics is becoming three different markets. First of all, there is the BI market which is actually going through quite a bit of change itself. This is a more consolidated market than we have seen in the past and there is a tremendous amount of work being done by Oracle, SAP, IBM and others to kind of retool it for the next generation of BI. So it is a growing market, lots of upgrade, replatform, modernization demand, lots of clients who are finally realizing that the tools (visualization etc.) are ready to give them some of the capability that they have historically cared about.
The second part of the market is what is called Advanced Analytics. Here you need PhD level data scientists who have backgrounds in machine learning, industry specific domain modeling, and different types of data science who can apply that in a very specific way to specific industry problems. This is a rapidly growing part of IT Services. Also, there are just not enough data scientists to go around.
The third part of the market is Analytics as a Service. This is about leveraging software-as-a-service platforms as opposed to on-premise. This is about a business model that is more like Business Process Outsourcing (BPO). Clients buy business outcomes; they don’t buy transactions and FTEs.
The analytics market has thousands of boutique consultants who are specialists in particular industries or specific technologies. It includes all the major technology providers, who are all trying to advance their business and capabilities that they are bringing to the market. And then there are vendors who are just bringing sheer capacity of data science skills to the market and they are coming in from a completely different angle of basically just renting the expertise of their data scientists into the market.
The market is incredibly fragmented. We are in the early stages of growth in the market. Every single one of our clients is building this capability internally and they are looking for more services from vendors, because the opportunity to apply analytics is in every single one function whether it is a customer analytics, industrial Internet, e-commerce platform, is growing. Analytics is embedded into literally every single business interaction.
BI, Analytics [and Big Data] Market Sizing
More recently to support a new generation of cost cutting and growth initiatives, corporations are investing heavily to gain near real-time actionable insights (historical and predictive), and from a mix of disparate spreadsheets and myriad of systems (legacy, internal silos, customer facing, suppliers, partners, etc.).