Skip to content

Archive for

18
Feb

Big Data is Entering the Trough of Disillusionment


hype cycleBig data, data lakes and becoming data-driven is no longer news or novel. Hype is not enough anymore.

The challenge today for leaders in every enterprises is (a) how to monetize data? (b) how to create enterprise class platforms instead of sandboxes? Basically, how can data, analytics and insight drive digital operations and digital transformation?

The paradox is interesting.  While leadership is struggling with value creation, the long term data trends are favorable: (1) data continues to grow exponentially and outstrip our ability to convert into insight;  (2) data consumption is evolving… Consumers and employees have more interactions with data through mobile apps than they do through desktop browsers; (3) Analytics – predictive and prescriptive – is gaining traction in several industries and business processes.

In the first wave of excitement around big data, there is a massive amounts of investment in stand-alone pilots and sandboxes.  Some experiments worked, but many failed to deliver.  The linkage between analytical projects and the everyday business applications (systems of record, systems of engagement, systems of insight) have mostly been missing.

In the next wave, we are seeing a tighter alignment or foundational underpinning between analytics (even machine learning) and traditional business applications. Take for instance Salesforce. Machine learning has become increasingly important to Salesforce, which has acquired PredictionIORelateIQ and Tempo AI, among other companies.

This implies a massive transformation wave (and upgrade cycle) across existing:

  • Systems of insight  (Reporting, BI, Analytics platforms)
  • Systems of engagement (CRM, SFA)
  • Systems of record (ERP)

The bigger transformation challenge is around how to systematically clear the bottlenecks in each of the above so that end-users can (1) access real-time data;  (2)  slice and dice the data for actionable insights from any device, anywhere; (3) convert the insights into guided decisions.

The directional strategy is clear, but can leadership get behind it and implement it swiftly and effectively?

This impending transformation is both exciting and daunting at the same time.  Application development and delivery (AD&D) teams are overwhelmed.  Leadership (in most corporations) in terms of vision or directional clarity is often weak or missing.  Strained relationships and misaligned business and technology teams is unfortunately the norm, not the exception.

Something is out of whack.

What I have observed in multiple engagements and research is that every year, billions of dollars are being spent with the consulting industry on establishing a corporate data strategy. Millions of hours of leadership time is invested in the strategy effort.

Even more billions are being spent on the core foundational “data lake” strategies – using Hadoop to create a large scale data dump. The more advanced form of this is to add Master Data Management (MDM) on top to create a “single golden view” of customers, employees, products or accounts. The hope (and prayer) is that this will enable companies to derive a more accurate picture of their business.  This is the hypothesis behind many “data lake” initiatives. The results and business value have been sketchy so far. So, is the strategy wrong or execution flawed?

Executing the data/analytics strategy around systems of engagement, record and insight is where the pain and the largest costs to the organization come into play. This is where discipline and talent becomes critical, and where competitive advantages are either won or lost.  But most firms starve the application development and delivery (AD&D) teams — limited budgets, basic talent to do cutting edge solutions,  unrealistic deadlines, constantly changing requirements. Few invest in product or program management to ensure the entire organization is involved, aligned, and ready to make it happen?

As a result, most corporations today have an awfully poor batting average when it comes to analytics projects or data informed business processes. This also may lead to the frequency of failure of most large-scale data-driven corporate change initiatives.  

The hype cycle is not a new phenomenon, but one that repeats itself with each innovation that somehow captures people’s imagination.  We have to be patient in the next 2-3 years and dig ourselves out of the big data Trough of Disillusionment 🙂

I am bullish on data long term.  The future is becoming more data-driven everything. Every industry (financial services, healthcare, retail, industrial) is going digital powered by data.

Evolution-of-Data

Source: It’s Time To Upgrade Business Intelligence To Systems Of Insight Supercharge BI With Agility, Big Data, And Insights To Drive Action by Boris Evelson, July 20, 2015

 

9
Feb

Big Data Evolving Landscape – 2016


In 2017, Big Data as a term is pretty much dead.   The market – VCs, Startups and F500 companies – have stopped using Big Data as a term to describe their programs/projects and have moved to Machine Learning (ML) and AI. Everything Big Data is now AI.

2015 was a year of significant change in the “Art of the Possible” with Analytics.   Norms about Analytics are evolving, and as they do, leading to wholesale business model transformation and cultural change at some workplaces. This change is driven not only by fast-moving technology, but also by new techniques to get value from data.

2016 is turning into the year of “democratization of data”, AI and Deep Learning. The state of the art is around Deep learning (also known as structured learning, hierarchical learning or machine learning), which are essentially efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.

Customer intelligence is the hot space. Commoditization pressures and shifting consumer expectations have inspired many companies to creatively use data and analytics to enrich their core products and services, a phenomenon called “wrapping”. Companies wrap offerings with information to differentiate them and to add value to customers. The best companies build distinctive competencies for wrapping.

However, there’s a lot of technology in the world, my friend.  The 2016 edition of Matt Turck & Jim Hao Big Data landscape supergraphic is shown below.

bigdatalandscape2016v12

 

Additional References:

A post by Matt Turck, a partner at FirstMark Capital, with a previous version 2014 –   Big Data Landscape v. 3.0.

Click here for a High-Res version of original

%d bloggers like this: