“Facebook today cannot exist without AI. Every time you use Facebook or Instagram or Messenger, you may not realize it, but your experiences are being powered by AI.” — Joaquin Candela, Facebook’s Applied ML team leader
“Machine learning is a core transformative way by which we are rethinking everything we are doing. Google Brain is the way we are embedding this in everything we do.” Sundar Pichai (CEO Google)
Salesforce CEO Marc Benioff said at a recent conference: “This [AI, ML] is a huge shift going forward, which is that everybody wants systems that are smarter, everybody wants systems that are more predictive, everybody wants everything scored, everybody wants to understand what’s the next best offer, next best opportunity, how to make things a little bit more efficient.”
Machine Learning (ML), Deep Learning and AI powering “Systems that Learn at scale” are at the bleeding edge of data science, deep learning and predictive search today.
Every market leader is jumping on this AI enabled engagement trend in retail, banking and healthcare.
Machine learning is central to Facebook’s future. Creating the extreme personalized experience (“individual equation based on 1000s of attributes – your preferences, predilections, conversations and transactions “) is the killer app.
FaceBook – AI/ML Case Study
Facebook is a bleeding edge case study of where AI/ML are being used to augment user engagement and experiences. I am starting to see many leading firms investing in ML Accelerators and Platforms as part of their data science strategy.
Big Data has been replaced by Machine Learning and AI as the next “must have” trend. Machine learning has caught the attention of venture capitalists.
Chief Data Officers, Chief Analytics Officers, Chief Data Science Officers and Chief Digital Officers are everywhere. The job is to leverage the latest in predictive analytics, data science, machine learning, and multi-tenant cloud architecture to bring innovation to traditional processes.
This is a pivotal moment in data driven business models (“systems that learn”) but there is no getting around the inherent difficulties associated with either altering organizational behavior, data ownership politics or managing transformation of the data infrastructure. And while the challenges are real, many firms are getting closer to achieving a data science and data management environment.
What are data and analytics officers overseeing… A variety of foundational & plumbing strategies:
- Data-as-a-Service: Data Provisioning, Management, Lineage, Quality
- Reporting-as-a-Service: Dashboards, KPIs, Drilldowns/Aggregates…. Descriptive
- Analytics-as-a-Service: Predictive Modeling and BI… Prescriptive analytics
- Information-as-a-Service: Threshold based Alerts, Exceptions, Mobile Prompts
- Insights-as-a-Service: ML/AI based “Systems that Learn”…automated learning – augmented intelligence, Next best Offer/Action
At core of all these, Data Management and Data Science tools are core technical and business capabilities. Some firms are more mature and further along than others.
Why Mature Data Management as a Function
Organizations live or die by the quality of their data.
Data is an underlying factor of input into business operations and essential in order to facilitate process automation, digitize operations, support financial engineering and enhance customer facing analytical capabilities.
An effective data management program requires a planned strategic effort
- Integrate multi-discipline efforts
- Inculcate a shared vision and understanding
- Data is a ‘thing’ – vital infrastructure element foundation of the n-tier architecture
- Not a project, more than a program…it’s part of the core foundation
There is no question about it – the foundational levels of people, process, governance and technology required to establish data management on a sustainable basis are coming together under the CDO umbrella.
What does a Chief Data Officer (CDO) do?
“Looking to the future, the next big step will be for the very concept of the “device” to fade away. Over time, the computer itself—whatever its form factor—will be an intelligent assistant helping you through your day. We will move from mobile first to an AI first world.” — Sundar Pichai, CEO Google
- A global oil and gas company has trained software robots to help provide a prompt and more efficient way of answering invoicing queries from its suppliers.
- A large US-based media services organization taught software robots how to support first line agents in order to raise the bar for customer service.
Software agents or Robotic process automation (RPA) is becoming a mainstream topic at leading corporations. I have seen a massive uptick in corporate strategy work in this area as C-Suite execs look at new ways to do more with less.
Software robots ∼ Conversational-AI products like Apple Siri, Microsoft Cortana, IBM Watson, Google Home, Alexa, drones and driverless cars ∼ are now mainstream. What most people are not aware of is the rapidly advancing area of enterprise robots to create a “virtual FTE workforce” and transform business processes by enabling automation of manual, rules based, back office administrative processes.
This emerging process re-engineering area is called Robotic process automation (RPA).
Machine Learning (ML) and graph processing are becoming foundations for the next wave of advanced analytics use cases. Speech recognition, image processing, language translation have gone from a demo tech to everyday use in part because of machine learning. Machine learning models, e.g., in driverless cars, teaches itself how to discover relevant things like a stop sign with snow partially obscuring the sign.
The market opportunity of artificial intelligence has been expanding rapidly, with analyst firm IDC predicting that the worldwide content analytics, discovery and cognitive systems software market will grow from US$4.5 billion in 2014 to US$9.2 billion in 2019, with others citing these systems as catalyst to have a US$5 trillion – US$7 trillion potential economic impact by 2025.
RPA – What?
“Robotic automation refers to a style of automation where a machine, or computer, mimics a human’s action in completing rules based tasks.” – Blue Prism
RPA is the application of analytics, machine learning and rules based software to capture and interpret existing data input streams for processing a transaction, manipulating data, triggering responses and driving business process automation around enterprise applications (ERP, HRMS, SCM, SFA, CRM etc.).
RPA is not a question of “if” anymore but a question of “when.” This is truly the next frontier of business process automation, enterprise cognitive computing, predictive analytics and machine learning. To make a prediction, you need an equation and parameters that might be involved.
Industrial robots are remaking blue-collar factory and warehouse automation by creating higher production rates and improved quality. RPA, simple robots and complex learning robots, are revolutionizing white-collar business processes (e.g. customer service), workflow processes (e.g., order to cash), IT support processes (e.g., auditing and monitoring), and back-office work (e.g., data entry).
I strongly believe that as cognitive computing slowly but surely takes off, RPA is going to impact process outsourcers (e.g., call center agents) and labor intensive white collar jobs (e.g., compliance monitoring) in a big way over the next decade. Any company that uses labor on a large scale for general knowledge process work, where workers are performing high-volume, highly transactional process functions, will save money and time with robotic process automation software.
Business Impact of RPA – Where?
Everyone knows that the retail industry is being transformed by digital, analytics and big data. Winning requires continual data-driven experimentation and transformation.
Shortened time from idea-to-app is a constant challenge.
Evidence of this “digital disruption” by category are mounting every day. Wal-Mart closes 269 stores as it retools portfolio to compete with online natives like Amazon.com. Macy’s said that it will shutter over 36 stores as store traffic declines faster than expected, and Finish Line said that it would close 150 stores by 2020. Gap, J.Crew, American Apparel, Sears and Kmart are all facing similar headwinds.
Starbucks CEO Howard Schultz laid out his thoughts on the future prospects for retail business, “three years ago we began to envision that there would be a seismic change in consumer behavior, and that seismic change was due in large part to e-commerce, omni-channel and smartphone shopping.”
It’s fascinating to watch retailers trying to shift tech/platform strategies to deal with digital disintermediation, showrooming, asset-light models, physical-to-digital channel integration, mobile shoppers, same-day delivery/fulfillment, programmatic targeting, online native models and now the new buzz.. virtual and augmented reality.
Several retailers have invested in Big Data and Hadoop platforms to mine massive volumes of structured transactional, operational data and unstructured data—web logs, clickstream data, geo-location data, social interactions and sensor data.
While most retailers understand the mega-shift and seems to know what to do….they are unable to execute consistently or effectively. A talent gap in many cases. A platform gap in others. Others are hindered by legacy IT systems or don’t have the necessary technology capabilities in place.
I think the digital induced pain is going to get worse in 2016 and 2017. Consumers will continue to diversify their retail activity across channels in search of the best value, forcing retailers to spread out their digital investments. This puts additional stress on execution and leadership.
The challenge today for leaders in every enterprises is (a) how to monetize data? (b) how to create enterprise class platforms instead of sandboxes? Basically, how can data, analytics and insight drive digital operations and digital transformation?
The paradox is interesting. While leadership is struggling with value creation, the long term data trends are favorable: (1) data continues to grow exponentially and outstrip our ability to convert into insight; (2) data consumption is evolving… Consumers and employees have more interactions with data through mobile apps than they do through desktop browsers; (3) Analytics – predictive and prescriptive – is gaining traction in several industries and business processes.
In the first wave of excitement around big data, there is a massive amounts of investment in stand-alone pilots and sandboxes. Some experiments worked, but many failed to deliver. The linkage between analytical projects and the everyday business applications (systems of record, systems of engagement, systems of insight) have mostly been missing.
In the next wave, we are seeing a tighter alignment or foundational underpinning between analytics (even machine learning) and traditional business applications. Take for instance Salesforce. Machine learning has become increasingly important to Salesforce, which has acquired PredictionIO, RelateIQ and Tempo AI, among other companies.
This implies a massive transformation wave (and upgrade cycle) across existing:
- Systems of insight (Reporting, BI, Analytics platforms)
- Systems of engagement (CRM, SFA)
- Systems of record (ERP)
The bigger transformation challenge is around how to systematically clear the bottlenecks in each of the above so that end-users can (1) access real-time data; (2) slice and dice the data for actionable insights from any device, anywhere; (3) convert the insights into guided decisions.
The directional strategy is clear, but can leadership get behind it and implement it swiftly and effectively?
This impending transformation is both exciting and daunting at the same time. Application development and delivery (AD&D) teams are overwhelmed. Leadership (in most corporations) in terms of vision or directional clarity is often weak or missing. Strained relationships and misaligned business and technology teams is unfortunately the norm, not the exception.
Something is out of whack.
What I have observed in multiple engagements and research is that every year, billions of dollars are being spent with the consulting industry on establishing a corporate data strategy. Millions of hours of leadership time is invested in the strategy effort.
Even more billions are being spent on the core foundational “data lake” strategies – using Hadoop to create a large scale data dump. The more advanced form of this is to add Master Data Management (MDM) on top to create a “single golden view” of customers, employees, products or accounts. The hope (and prayer) is that this will enable companies to derive a more accurate picture of their business. This is the hypothesis behind many “data lake” initiatives. The results and business value have been sketchy so far. So, is the strategy wrong or execution flawed?
Executing the data/analytics strategy around systems of engagement, record and insight is where the pain and the largest costs to the organization come into play. This is where discipline and talent becomes critical, and where competitive advantages are either won or lost. But most firms starve the application development and delivery (AD&D) teams — limited budgets, basic talent to do cutting edge solutions, unrealistic deadlines, constantly changing requirements. Few invest in product or program management to ensure the entire organization is involved, aligned, and ready to make it happen?
As a result, most corporations today have an awfully poor batting average when it comes to analytics projects or data informed business processes. This also may lead to the frequency of failure of most large-scale data-driven corporate change initiatives.
The hype cycle is not a new phenomenon, but one that repeats itself with each innovation that somehow captures people’s imagination. We have to be patient in the next 2-3 years and dig ourselves out of the big data Trough of Disillusionment 🙂
I am bullish on data long term. The future is becoming more data-driven everything. Every industry (financial services, healthcare, retail, industrial) is going digital powered by data.
Source: It’s Time To Upgrade Business Intelligence To Systems Of Insight Supercharge BI With Agility, Big Data, And Insights To Drive Action by Boris Evelson, July 20, 2015
In 2017, Big Data as a term is pretty much dead. The market – VCs, Startups and F500 companies – have stopped using Big Data as a term to describe their programs/projects and have moved to Machine Learning (ML) and AI. Everything Big Data is now AI.
2015 was a year of significant change in the “Art of the Possible” with Analytics. Norms about Analytics are evolving, and as they do, leading to wholesale business model transformation and cultural change at some workplaces. This change is driven not only by fast-moving technology, but also by new techniques to get value from data.
2016 is turning into the year of “democratization of data”, AI and Deep Learning. The state of the art is around Deep learning (also known as structured learning, hierarchical learning or machine learning), which are essentially efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.
Customer intelligence is the hot space. Commoditization pressures and shifting consumer expectations have inspired many companies to creatively use data and analytics to enrich their core products and services, a phenomenon called “wrapping”. Companies wrap offerings with information to differentiate them and to add value to customers. The best companies build distinctive competencies for wrapping.
However, there’s a lot of technology in the world, my friend. The 2016 edition of Matt Turck & Jim Hao Big Data landscape supergraphic is shown below.
A post by Matt Turck, a partner at FirstMark Capital, with a previous version 2014 – Big Data Landscape v. 3.0.
- Where do customers abandon the shopping process? Is it the same in every geography?
- Audience of One…. Who are your fans versus haters in the marketplace?
- How do customers feel about your products? How engaged are customers with your brand versus your competitors’ brands across social media and web channels?
Fortune 500 companies are making large investments around Programmatic Marketing, Sales and Service (“marketing that learns”). Programmatic marketing is the application of automated technology through which media buyers and sellers may align organizational processes in support of ongoing, channel-agnostic customer engagement (and to allow for the continuous optimization of that effort as business strategies evolve) in order to drive revenue.
One of most often implemented use case in Programmatic Marketing is customer journey mapping and analytics.
Why? Because, deciphering the nuts-and-bolts” of individual customer journeys online (and deducing intent) is core to improving customer experience and driving brand loyalty.
Specifically, the objectives are:
- Visualize and map the end-to-end customer journey by personas
- Optimizing on the right journey attributes to increase yields by >30% lift… Uncover the right combination of web, mobile and physical channels, content and experiences that best achieves the target goals
- Enable marketers to identify journey bottlenecks for individuals and aggregates
- Leverage actual behavior data to enhance and personalize the experience for each individual customer
More data + Better models + More accurate metrics + Better approaches & architectures = Lots of room for improvement!
There are clearly massive foundational shifts taking place around big data. I am not sure how large conventional Fortune 500 firms can innovate and keep up with what’s going on. I have run into CIOs who have not heard of Hadoop in some cases.
It’s also fascinating to see how data-driven “bleeding” edge firms like NetFlix are pushing the envelope. Netflix stats are amazing: 1/3+ Internet traffic (NA / peak); 100+ Million hours per day; 65+ Million members / 50+ countries; 500 Billion Events / Day.
NetFlix is clearly reinventing Television and targeting 90 million potential subs in the US market alone. Binge-watching, cord-cutting are now part of our everyday lingo. What most people don’t realize is how data-driven Netflix is…. from “giving viewers what they want” to “leveraging data mining to boost subscriber base”.
Viewing -> Improved Personalization -> Better Experience is the virtuous circle.
Here is a glimpse at how their BI landscape has evolved in the past five years as they integrate 5 million to 6 million net adds for several years now. The figures are from a presentation by Blake Irvine, Manager Data Science and Engineering.
BI tools @ NetFlix pre-Hadoop
New Technologies | New Possibilities
As a C-level executive, it’s becoming clear to me that NoSQL databases and Machine Learning toolsets like Spark are going to play an increasingly big role in data-driven business models, low-latency architecture & rapid application development (projects that can be done in 8-12 weeks not years).
The best practice firms are making this technology shift as decreasing storage costs have led to an explosion of big data. Commodity cluster software, like Hadoop, has made it 10-20x cheaper to store large datasets.
After spending two days at the leading NoSQL provider MongoDB World event in NYC, I was pleasantly surprised to see the amount of innovation and size of user community around document centric databases like MongoDB.
Data Driven Insight Economy
It doesn’t take genius to realize that data driven business models, high volume data feeds, mobile first customer engagement, and cloud are creating new distributed database requirements. Today’s modern online and mobile applications need continuous availability, cost effective scalability and high-speed analytics to deliver an engaging customer experience.
We know instinctively that there is value in all the data being captured in the world around out…no question is no longer “if there is value” but “how to extract that value and apply it to the business to make a difference”.
Legacy relational databases fail to meet the requirements of digital and online applications for the following reasons: