Skip to content

Posts tagged ‘Hadoop’

3
Mar

Analytics and ML Use Case – Robo-Advisors in Wealth Management


New data driven FinTech business models built on Hadoop, Spark and Machine Learning are rapidly emerging and disrupting wealth management.  Here is my recent posting from disruptivedigital.wordpress.com about one such use case… Robo-Advisors.

We are in the early stages of a generational shift in wealth management, especially “plain vanilla” investing for the mass affluent and millennial segment.  Until recently, you had only two options when investing:

  • Do-it-yourself (DIY)
  • Hire a registered investment advisor (RIA)

Now there is a third option.  Robo-advisors are new class of personal financial advisors that provides online, algorithm based portfolio management with minimal human intervention. Robo-Advisors going after the low-end of brokerage/RIA business with automated asset allocation using Modern Portfolio Theory.

The Robo-Advisors market leaders who are serving the mass affluent include are:

  • Wealthfront (with over USD 2.6bn in assets under management (AuM) and 20,000 investors);
  • Betterment (with over USD 1.4bn in AuM and 70,000 investors); and
  •  FutureAdvisor (With over $600 million in AUM).

The timing for this market shift coincides with three trends: consumerization, digital tools, and disillusionment with status-quo investment advisors.  The gyrating stock market driven by program trading is increasingly bringing Robo-Advisors, algorithmic portfolio management to the forefront.  Investors are getting disillusioned with traditional investment advisors who simply track the market indices (SPY, QQQ or Russell 2000) by purchasing ETFs at best.

Many banks and brokerage firms over the years have shifted their focus to serve ultra high net worth (UHNW) and high net worth (HNW) investors, leaving an opportunity for firms to target the “mass affluent” investors, or those with less than $1 million in investable assets. Younger investors are increasingly interested in online digital advice (trial-and-error bets), as opposed to hiring an adviser.

wealthfront3 Read more »

25
May

Big Data Analytics Use Cases


Are you data-flooded, data-driven, data informed? Are you outcome oriented, insight driven or hindsight driven?

Are you a firm where executives claim – “Data is our competitive advantage.” Or sprout analogies like, “data is the new oil”.

The challenge I found in most companies is not dearth of vision… everyone has a strategy and a 100,000 ft general view of the importance or value of data. Every executive can parrot the importance of data and being data-driven.

The challenge is the next step….so, how are you going to create new data products? How are you going to execute a data driven strategy? How are you going to monetize data assets? What are the right business use cases to focus on? How to map the use case to underlying models and data requirements? What platform is a good long-term bet?  The devil is in these details.

Everyone is searching for new ways to turn data into $$$ (monetize data assets). Everyone is looking for new levers to extract value from data.  But data ingesting and modeling is simply a means to an end. The end is not just more reports, dashboards, heatmaps, knowledge, or wisdom. The target is fact based decisions, guided machine learning and actions. Another target is arming users to do data discovery and insight generation without involving IT teams…so called User-Driven Business Intelligence.

In other words, what is the use case that shapes the context for “Raw Data -> Aggregated Data -> Intelligence -> Insights -> Decisions -> Operational Impact -> Financial Outcomes -> Value creation.”  What are the right use cases for the emerging hybrid data ecosystem (with structured and unstructured data)?

Read more »

24
Feb

Security Analytics – Big Data Use Case


Another day, another data breach.  Just received another “We’re sorry you got hacked”…letter.  

This is the fifth letter I have received in the past 3 months:  Forbes.com, Target, Neiman Marcus, credit card company and a previous employer.  What is going on?

Why aren’t firms investing in beefing up their predictive ability to spot the cyber-security intrusion threats? What’s taking them so long to identify?  Why is the attack signature – sophisticated, self-concealing  malware – so difficult to spot?   Do firms need to invest in NSA PRISM type threat monitoring capabilities?

The three impediments to discovering and following up on attacks are:

  • Volume, velocity and variety – Not collecting appropriate security data
  • Immaturity and not identifying relevent event context (event correlation)
  • lack of system awareness and vulnerability awareness

Obviously… where there is pain…there is opportunity for entrepreneurs see below – data from IBM).  There is a growing focus on big data use case for security analytics after all the breaches we are seeing.  General Electric announced it had completed a deal to buy Wurldtech, a Vancouver-based cyber-security firm that protects big industrial sites like refineries and power plants from cyber attacks.

securityanalytics3

 

Here are three recent examples that I was personally affected by – Forbes,  Target, Neiman Marcus.  

Read more »

15
Jan

Big Data Fatigue and Company Shakeout?


hype cycleBig Data is the latest “next big thing” transforming all areas of business, but amid the hype, there remains confusion about what it all means and how to create business value.

Usually when there is so much hype…there is an inevitable boom-bust-boom cycle. Hence my question:  Is the Big Data shakeout inevitable?

Are we in a big data tech bubble? If you are an enterprise customer, how do you prepare for this? What strategies do you adopt to take advantage of the situation? Can you move from lab experiments to production deployments with confidence?

The sheer number of companies that are chasing “the pot of big data gold” is astounding (see below).  While the innovation has accelerated the ability of the typical Fortune 1000 enterprise to absorb and assimilate has not. They tend to be 5-10 years behind the curve. As a result, many big data startups are either running out of cash or they are being folded by VCs into other firms.  This boom-bust cycle is a typical pattern in innovation.

http://www.bigdata-startups.com/open-source-tools/

BigDataUniverse

Source: Big Data Universe v3.. Matt Turck, Sutian Dong & FirstMark Capital

The Case of Drawn to Scale

Drawn to Scale, the four year-old startup behind Spire, shut down recently. Co-founder and CEO Bradford Stephens announced the news in a blog post. Drawn to Scale raised .93M in seed funding.

Spire is a real-time database solution for HBase that lets data scientists query Hadoop clusters using SQL. According to Stephens, the system has been by deployed by American Express, Orange Flurry, and four other companies.

Drawn to Scale showed that its technology was viable in enterprise environments and established a “presence against  competitors who raised 10-100x more cash,” but even that wasn’t enough to save the startup from its financial woes.

As Hadoop evolves and different layers of the data analytics stack get commoditized, specialized vendors like Drawn to Scale will have problems surviving.   SQL-on-Hadoop was a unique feature set…but over time it has become a must-have feature, that is becoming embedded in the stack – e.g., Impala in Cloudera CDH stack.  As a result, firms like Drawn to Scale once unique functionality becomes difficult to monetize.

Startup to Viable Ventures

The Big Data ecosystem is exploding with exciting start-ups, new divisions and new initiatives from established vendors.  Everyone wants to be the vendor/platform of choice in assisting firms deal with the data deluge (Data growth curve: Terabytes -> Petabytes -> Exabytes -> Zettabytes -> Yottabytes -> Brontobytes -> Geopbytes), translate data to information to insight, etc.

In both U.S and Europe, several billion dollars of venture money has been invested in the past three years alone in over 300+ firms.  Firms like Splunk had spectacular IPOs. Others like Cloudera and MapR have raised gobs of money. In the MongoDB space alone – a small market of less than 100M total revenue right now, over $2 Billion is said to have been invested in the past few years.

Read more »

19
Jun

Organizing for BI, Analytics and Big Data: CoE, Federated or Departmental


  • Data-as-a-Service:  Data Provisioning, Management, Lineage, Quality
  • Reporting-as-a-Service:  Dashboards, KPIs, Drilldowns/Aggregates…. Descriptive
  • Analytics-as-a-Service:  Predictive  Modeling and BI… Prescriptive analytics
  • Information-as-a-Service:  Threshold based Alerts, Exceptions, Mobile Prompts
  • Insights-as-a-Service:   ML/AI based…automated learning – ambient intelligence, Next best Offer/Action

Which strategy are you implementing?

Data is valuable. Data is plentiful. Data is complex. Data is in flux. Data is fast moving. Capturing and managing data is challenging.

So, if you are a senior leader in a Fortune 2000 company.  How do you structure your group to deliver effective BI, Analytics or Big Data projects? Do you have the right structure,  toolset, dataset, skillset and mindset for analytics and Big Data?

Organizing for effective BI, Analytics and Big Data is becoming a hot topic in corporations.  In 2012, business users are exerting  significant influence over BI, Analytics and Big Data decisions, often choosing analytics and visualization platforms and products in addition to/as alternatives to traditional BI platform (reporting and visualization tools).

Read more »

19
Jan

Multi-channel to Omni-channel Retail Analytics: A Big Data Use Case


MULTI-CHANNEL is simply having multiple channels through which you buy, market, sell, and fulfill.

CROSS-CHANNEL has the ability to see all of a customer’s information across all channels enables more personalized offers based on their brand relationship.

OMNICHANNEL weaves all the touchpoints of the products and services of the brand into a  seamless fabric  of all phases of the  customer’s brand experience.

Which one are you?

Let’s face it – The old uni-channel retail model is dying in some cases and changing in others.  E-commerce is driving nearly all retail growth. Digital customers want simple, consistent, and relevant experiences across all channels, touchpoints, mobile screens, smart watches and other devices.

Read more »

19
Dec

Big Data Investment Theme – Fidelity Investments


Fidelity Investments put out an interesting analysis on Big Data as a Macro Investment Themes for clients.  Since everyone has an underperforming investment portfolio in this current market, I reproduced the article here to generate some ideas.

Fidelity Investments

Key Takeaways

  • New types of large data sets have emerged because of advances in technology, including mobile computing, and these data are being examined to generate new revenue streams.
  • More traditional types of business data have also expanded exponentially, and companies increasingly want and need to analyze this information visually and in real time.
  • Big data will be driven by providers of Internet media platforms, data amalgamation applications, and integrated business software and hardware systems.

Investment Theme – Big Data

The concept of “big data” generally refers to two concurrent developments. First, the pace of data accumulation has accelerated as a wider array of devices collect a variety of information about more activities: website clicks, online transactions, social media posts, and even high-definition surveillance videos.

A key driver of this flood of information has been the proliferation of mobile computing devices, such as smartphones and tablets. Mobile data alone are expected to grow at a cumulative annualized rate of 92% between 2010 and 2015 (see Exhibit 1, below). Read more »

6
Nov

What is a “Hadoop”? Explaining Big Data to the C-Suite


Keep hearing about Big Data and Hadoop? Having a hard time explaining what is behind the curtain?

The term “big data” comes from computational sciences to describe scenarios where the volume of the data outstrips the tools to store it or process it.

Three reasons why we are generating data faster than ever: (1) Processes are increasingly automated; (2) Systems are increasingly interconnected; (3) People are increasingly “living” online.

DataEvolutionAs huge data sets invaded the corporate world there are new tools to help process big data. Corporations have to run analysis on massive data sets to separate the signal from the noisy data.  Hadoop is an emerging  framework for Web 2.0 and enterprise businesses who are dealing with data deluge challenges – store, process, index,  and analyze large amounts of data as part of their business requirements.

So what’s the big deal? The first phase of e-commerce was primarily about cost and enabling transactions.  So everyone got really good at this. Then we saw differentiation around convenience… fulfillment excellence (e.g., Amazon Prime) , or relevant recommendations (if you bought this and then you may like this – next best offer).

Then the game shifted as new data mashups became possible based on… seeing who is talking to who in your social network, seeing who you are transacting with via credit-card data, looking at what you are visiting via clickstreams, influenced by ad clickthru, ability to leverage where you are standing via mobile GPS location data and so on.

The differentiation is shifting to turning volumes of data into useful insights to sell more effectively. For instance, E-bay apparently has 9 petabytes of data in their Hadoop and Teradata cluster. With 97 million active buyers and sellers they have 2 Billion page view and 75 billion database calls each day.  E-bay like others is racing to put in the analytics infrastructure to (1) collect real-time data; (2) process data as it flows; (3) explore and visualize. Read more »

3
Oct

Wanted: CIO – BI/Analytics


In a tough economy, a new tech-fueled BI and analytics arms race is on to create the next competitive advantage.

Everyone is beginning to look beyond the status quo in BI, analytics, Big Data, Cloud Computing etc to fundamentally change how they discover fresh insights, how they can make smarter decisions, profit from customer intelligence and social media, and optimize performance management.

The headache for corporations is not the technology aspects but the leadership side. Who is going to lead this effort, corral the vendors and formalize and execute a more structured program.  

Who is going to lead the effort to create the right toolset, dataset, skillset and mindset necessary for success?

As BI and Analytics moves from “experiment and test” lab projects to commercial deployments, companies are going to need more leadership and program management capabilities.  They need leadership that can provide strategic, expert guidance for using powerful new technologies to find patterns and correlations in data transactions, event streams, and social media.

Some firms are making moves.  In insurance, AIG – Chartis Inc. unit appointed Murli Buluswar to the new post of chief science officer.  This aims to enhance Chartis’ focus on analytics… he “will be responsible for establishing a world-class R&D function to help improve Chartis’ global commercial and consumer business strategies and to deliver more value for customers.”  This focus on analytics involves “asking the right questions and making science-driven decisions about strategies—whether it’s related to underwriting decisions, product innovation, pricing, distribution, marketing, claims or customer experience—with the end result of improving the scope of what Chartis delivers for customers”.

As a result of where we are in the maturity cycle and to support the business units better, we are seeing a new emerging role “CIO – BI” that is dotted lined to the global CIO or a shared services leader.  Let’s look at a representative job posting from GE Capital, which always seems to be a step ahead of most companies.   Read more »

13
Aug

Analytics-as-a-Service: Understanding how Amazon.com is changing the rules


“By 2014, 30% of analytic applications will use proactive, predictive and forecasting capabilities”  Gartner Forecast

“More firms will adopt Amazon EC2 or EMR or Google App Engine platforms for data analytics. Put in a credit card, by an hour or months worth of compute and storage data. Charge for what you use. No sign up period or fee. Ability to fire up complex analytic systems. Can be a small or large player”    Ravi Kalakota’s forecast 

—————————-

Big data Analytics = Technologies and techniques for working productively with data, at any scale.

Analytics-as-a-Service is cloud based… Elastic and highly scalable, No upfront capital expense. Only pay for what you use, Available on-demand

The combination of the two is the emerging new trend.  Why?  Many organizations are starting to think about “analytics-as-a-service” as they struggle to cope with the problem of analyzing massive amounts of data to find patterns, extract signals from background noise and make predictions. In our discussions with CIOs and others, we are increasingly talking about leveraging the private or public cloud computing to build an analytics-as-a-service model.

Analytics-as-a-Service is an umbrella term I am using to encapsulate “Data-as-a-Service” and “Hadoop-as-a-Service” strategies.  It is more sexy 🙂

The strategic goal is to harness data to drive insights and better decisions faster than competition as a core competency.  Executing this goal requires developing state-of-the-art capabilities around three facets:  algorithms, platform building blocks, and infrastructure.

Analytics is moving out of the IT function and into business — marketing,  research and development, into strategy.  As result of this shift, the focus is greater on speed-to-insight than on common or low-cost platforms.   In most IT organizations it takes anywhere from 6 weeks to 6 months to procure and configure servers.  Then another several months to load, configure and test software. Not very fast for a business user who needs to churn data and test hypothesis. Hence cloud-as-a-analytics alternative is gaining traction with business users.

Read more »

%d bloggers like this: