Skip to content

January 15, 2013


Big Data Fatigue and Company Shakeout?

by Ravi Kalakota

hype cycleBig Data is the latest “next big thing” transforming all areas of business, but amid the hype, there remains confusion about what it all means and how to create business value.

Usually when there is so much hype…there is an inevitable boom-bust-boom cycle. Hence my question:  Is the Big Data shakeout inevitable?

Are we in a big data tech bubble? If you are an enterprise customer, how do you prepare for this? What strategies do you adopt to take advantage of the situation? Can you move from lab experiments to production deployments with confidence?

The sheer number of companies that are chasing “the pot of big data gold” is astounding (see below).  While the innovation has accelerated the ability of the typical Fortune 1000 enterprise to absorb and assimilate has not. They tend to be 5-10 years behind the curve. As a result, many big data startups are either running out of cash or they are being folded by VCs into other firms.  This boom-bust cycle is a typical pattern in innovation.


Source: Big Data Universe v3.. Matt Turck, Sutian Dong & FirstMark Capital

The Case of Drawn to Scale

Drawn to Scale, the four year-old startup behind Spire, shut down recently. Co-founder and CEO Bradford Stephens announced the news in a blog post. Drawn to Scale raised .93M in seed funding.

Spire is a real-time database solution for HBase that lets data scientists query Hadoop clusters using SQL. According to Stephens, the system has been by deployed by American Express, Orange Flurry, and four other companies.

Drawn to Scale showed that its technology was viable in enterprise environments and established a “presence against  competitors who raised 10-100x more cash,” but even that wasn’t enough to save the startup from its financial woes.

As Hadoop evolves and different layers of the data analytics stack get commoditized, specialized vendors like Drawn to Scale will have problems surviving.   SQL-on-Hadoop was a unique feature set…but over time it has become a must-have feature, that is becoming embedded in the stack – e.g., Impala in Cloudera CDH stack.  As a result, firms like Drawn to Scale once unique functionality becomes difficult to monetize.

Startup to Viable Ventures

The Big Data ecosystem is exploding with exciting start-ups, new divisions and new initiatives from established vendors.  Everyone wants to be the vendor/platform of choice in assisting firms deal with the data deluge (Data growth curve: Terabytes -> Petabytes -> Exabytes -> Zettabytes -> Yottabytes -> Brontobytes -> Geopbytes), translate data to information to insight, etc.

In both U.S and Europe, several billion dollars of venture money has been invested in the past three years alone in over 300+ firms.  Firms like Splunk had spectacular IPOs. Others like Cloudera and MapR have raised gobs of money. In the MongoDB space alone – a small market of less than 100M total revenue right now, over $2 Billion is said to have been invested in the past few years.

An interesting sampling of start-up vendors and the amount of venture capital each has raised, including lead investors is listed at Wikibon (we reproduced the list below).   This list includes only Big Data pure-plays delivering products and/or services in one of the following markets: Hadoop, NoSQL, Next Generation (MPP) Data Warehousing, predictive analytics and/or advanced data visualization.

A narrow slice of the Big Data Market but illustrates the vibrant big data startup activity taking place. This list doesn’t include all the Social Intelligence and Analytics firms that are living off Facebook or Twitter data that represent a different slice of Big Data.  Several hundred firms easily in that segment alone.

The temporal clustering of major innovations under the banner of big data is definitely one of the catalysts for the next wave of innovation and economic growth.  We have seen this pattern before where advances in technology have combined to bring about a series of coordinated technological transformations that are correlated with waves of investment and business efficiency. (Joseph Schumpeter’s studied these business cycle patterns in the 1930s. These were later labeled creative destruction in innovation management circles.)

Most recently we saw this innovation pattern in the late 1990s around e-commerce.  Thousands of new companies were created, bought, and merged during the 1997 – 2000. At the end of the cycle we saw a massive creative destruction period with a washout that lasted from late 2000 to end of 2003.

My hypothesis is that in 2013/2014 we are going to see a similar shakeout pattern around Big Data and Social Intelligence/Analytics.  The evidence… we have too many startup companies chasing customers.  Most of the projects are low-cost ($100K or less) or free pilots.  Enterprise customers who are innovative are piloting technologies to understand the business value but are having a hard time moving these into production deployments.

The shakeout will start slowly but will pickup pace towards the end of this year.  The catalyst for creative destruction is always the same – lack of next round of funding, lack of enterprise customers,  declining valuations that prompt investors to pull back, and finally big established firms like Oracle, SAP and others moving to protect their turf by creating fear, uncertainty and doubt.

So what does this mean if you are a Big Data startup firm?   It means managing your funding aggressively and making sure it lasts till you get paying customers.   A simply analogy – a car with gas in the tank will win always against another car that is running out of gas.

What does this mean if you are an enterprise customer?  Create a roadmap and continuously learn. If you are in the experimental mode, it’s ok to do several pilots. But make sure you are learning something and bringing this knowledge back into the organization.  I see a lot of companies that are doing interesting pilots but have no plans to assimilate, scale or leverage the insights.  So lot of effort is wasted.

What does this mean if you are a CIO? investing in cutting edge technologies could potentially lead to incredible rewards, but also come with risks. Project failure could be fatal in some high profile cases. CIOs are putting their reputation on the line everytime a new product is rolled out the enterprise.

Birth and death of firms is a natural phenom in entrepreneurship.  The boom-bust-boom cycle is not a question of if but when. The survivors in Big Data will be those that are actively planning for the impending shakeout and acquiring assets – customers, technology, IP, patents — from the weaker players.

Big Data Start-up Ecosystem

Innovation in Big Data – Hadoop, NoSQL, Next Generation (MPP) Data Warehousing, predictive analytics and/or advanced data visualization… How many of these will survive?

Consolidation via mergers/acquisitions has already started.  Vertica (HP),  Kitenga (Dell), (Buddymedia, Radian6), Oracle (Vitrue), EMC (Greenplum), IBM (spent over $15bln+ so far in aquisitions).

If you know of other firms that should be added to this list…

Big Data Start-up Funding by Vendor (adapted from Wikibon)

Vendor Founded Funding (in $US mil.) # of Institutional Rounds Investors
Palantir 2004 $301 7 SAC Capital, The Founders Fund, Glynn Capital, In-Q-Tel, Reed Elsevier Ventures, Ulu Ventures, Youniversity Ventures and Jeremy Stoppelman
Cloudera 2008 $146 5 Accel Partners, Greylock Partners, Ignition Partners, In-Q-Tel and Meritech Capital Partners
Mu Sigma 2004 $133 2 General Atlantic and Sequoia Capital
Opera Solutions 2004 $84 1 Silver Lake Sumeru, Accel-KKR, Invus Financial Advisors, JGE Capital and Tola Capital
10gen 2008 $81 6 Intel Capital, Red Hat, New Enterprise Associates, Sequoia Capital, Flybridge Capital and Union Square Ventures
Guavus 2006 $78 3 Investor Growth Capital, QuestMark Partners, Intel Capital, Artiman Ventures and Sofinnova Ventures
ParAccel 2005 $73 5 Amazon, Menlo Ventures, Mohr Davidow Ventures, Bay Partners, Walden International, Tao Venture Capital Partners and Silicon Valley Bank
Talend 2005 $61.6 5 Silver Lake Partners, Balderton Capital, Galileo Partners and IDInvest Partners
GoodData 2007 $53.5 3 Andreesen Horowitz, General Catalyst, O’Reilly AlphaTech Ventures, Windcrest Partners, Tenaya Capital and Next World Capital
Splunk 2003 $40 3 Ignition Partners, August Capital, JK&B and Sevin Rosen Funds
DataStax 2010 $38.7 3 Meritech Capital, Lightspeed Venture Partners, Sequoia Capital and Crosslink Capital
1010data 2000 $35 1 Norwest Venture Partners
Couchbase 2009 $31 3 Ignition Partners, Accel Partners, Mayfield Fund, and North Bridge Venture Partners
MapR 2009 $29 2 Redpoint Ventures, Lightspeed Venture Partners and NEA
Tidemark 2011 $28 2 Andreesen Horowitz, Redpoint Ventures and Greylock Partners
Factual 2007 $27 1 Andreesen Horowitz and Index Ventures
Platfora 2011 $25.7 2 Battery Ventures, Andreessen Horowitz, Sutter Hill Ventures and In-Q-Tel
MetaMarkets 2010 $23 2 Khosla Ventures, IA Ventures, AOL Ventures, Neu Venture Capital, Joshua Stylman, Village Ventures and True Ventures
Hopper 2007 $22 3 Atlas Venture, OMERS Ventures and Brightspark Ventures
Lattice Engines 2006 $21.6 2 Battery Ventures and New Enterprise Associates
SumoLogic 2010 $20.5 2 Sutter Hill Ventures, Greylock Partners, Shlomo Kramer
Hortonworks 2011 $20 1 Benchmark Capital, Yahoo and Index Ventures
RainStor 2004 $19.2 3 Storm Ventures, Doughty Hanson Technology Ventures, Informatica, Rogers Venture Partners and The Dow Company
DataXu 2009 $18.8 2 Menlo Ventures, Atlas Venture and Flybridge Capital Partners
Datameer 2009 $17.8 3 Kleiner Perkins Caufield & Byers and Redpoint Ventures
Revolution Analytics 2007 $17.6 2 North Bridge Venture Partners and Intel Capital
Hadapt 2010 $16.2 1 Atlas Venture, Norwest Venture Partners and Bessemer Venture Partners
Lucid Imagination 2007 $16 2 Shasta Ventures, Granite Ventures, In-Q-Tel and Walden International
Continuity 2011 $12.5 2 Andreessen-Horowitz, Ignition Ventures, Battery Ventures, Data Collective and Amplify Partners
Connotate 2000 $12.3 2 Castile Ventures, Prism VentureWorks and .406 Ventures
ClearStory Data 2012 $12.25 1 Kleiner Perkins Caufield & Byers, Andreessen Horowitz, Google Ventures and Khosla Ventures
Karmasphere 2005 $11 2 Presidio Ventures, Hummer Winblad and US Venture Partners
Loggly 2009 $10 1 True Ventures, Trinity Ventures, Matrix Partners
AgilOne 2012 $10 1 Mayfield Fund
Oragami Logic 2012 $8 1 Accel Partners
Alpine Data Labs 2010 $7.5 1 Sierra Ventures, Mission Ventures, Sumitomo Corporation Equity Asia and Stanford University
SpaceCurve 2009 $7.5 1 Triage Ventures, Reed Elsevier Ventures and Divergent Ventures
ParStream 2008 $5.6 1 Khosla Ventures, Baker Capital, Crunch Fund, Data Collective and Tola Capital
SpaceCurve 2011 $5.2 2 Reed Elsevier, Divergent Ventures, and Triage Ventures
MemSQL 2011 $5 1 First Round Capital, SV Angel, Y Combinator, IA Ventures and Ashton Kutcher
WibiData 2010 $5 1 New Enterprise Associates, SV Angel, Mike Olson and Eric Schmidt
InsightSquared 2010 $4.5 1 Atlas Venture, Bessemer Venture Partners, NextView Ventures and
Chartio 2010 $4.4 1 Avalon Ventures, Bullpen Capital, Y Combinator, Crosslink Capital and Jeff Hammerbacher
Trifacta 2012 $4.3 1 Accel Partners, X/Seed Capital, Data Collective LLC, Dave Goldberg, Venky Harinarayan and Anand Rajaraman
Digital Reasoning 2000 $4.2 2 In-Q-Tel and Silver Lake Sumeru
SiSense 2008 $4 1 Opus Capital, Genesis Partners and Eli Farkash
Calpont 2000 $3.27 1 Austin Ventures and GF Private Equity
StackIQ 2006 $3 1 Anthem Venture Partners and Avalon Ventures
Zettaset 2009 $3 1 Draper Fisher Jurvetson and Epic Ventures
Energy analytics platform
2012 $3 `1 Khosla Ventures 2012 $6.5 million in series A 1 Khosla Ventures, True Ventures and Romulus Capital.
GridGain 2005 $2.5 1 RTP Ventures
NGDATA 2011 $2.5 1 ING, Sniper Investments, Plug and Play Ventures
Sqrrl 2012 $2 1 Atlas Venture
Feedzai 2008 $2 1 Espirito Santo Ventures and Novabase Capital
Nodeable 2011 $2 1 True Ventures and Matrix Partners
RelateIQ 2012 $1.25 1 Accel Partners, Morgenthaler and SV Angel
Zoomdata 2012 $1.1 1 Hemang Gadhia
AppEnsure 2011 $1 1 Citrix Accelerator, TiE, Ignition Partners
DataHero 2012 $1 1 Neu Venture Capital, The Foundry Group, David G. Cohen and Tasso Argyros
Drawn to Scale 2009 $0.93 1 RTP Ventures, IA Ventures, and SK Ventures
Openera is a Canadian big data startup in the storage management Dec 2012 250K 1

Multi-Year Opportunity for Services vendors

Business Analytics is the one of the fastest growing opportunities in the IT market. Few areas that enable new revenue streams from expanded solutions portfolio and higher margin deals through services opportunities. Opportunity exists to help drive unprecedented growth and demand for service offerings and delivers customer ROI.

The demand for analytics is certainly there.  See chart below.


While the Big Data startup market will undoubtedly go through some consolidation over the coming 18 to 36 months, there is reason to be optimistic. The Big Data phenomenon is a long-term digital transformation trend that will affect how we live, work and enjoy.  It continue to impact enterprise customers across nearly all industries. Vendors who emerge victorious from this period of disillusionment will find an enormous growth market for the next ten years or more as corporations begin to embrace and push the envelope of Big Data  in improving their businesses.

3 Comments Post a comment
  1. Jan 31 2013

    Hi Ravi,

    Thank you for your blog entry. You’ve really detailed all the startups in big data and the exciting new turn for storage. Good job!

    At the moment, I’m looking for bloggers and contributors for a storage and big data website. Would you perhaps be interested in contributing your past and future blog articles? We want this website to be a thriving community of experts generating conversations on big data, cloud computing and storage virtualization.

    It’s free to join, and only the title and the first few sentences of your blog entries will be published on the website. We want readers to engage with your content and be directed to your blog for the full article. This way, you’ll get traffic! 🙂

    If you’re interested or have any questions, please send me an email at tinajin [at] with “Tech” in the subject line. I’ll be glad to answer any questions and get you started on being an expert contributor!

    Tina Jin



Trackbacks & Pingbacks

  1. Big Data Company Shakeout in 2013? | Business Analytics 3.0 | Big Data Cloud
  2. Big Data Bubble: Surveillance and the Coming Market Shakeout - Insider Surveillance

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Note: HTML is allowed. Your email address will never be published.

Subscribe to comments

%d bloggers like this: