Machine data (Internet of Things) or “data exhaust” analysis is one of the fastest growing segments of “big data”–generated by websites, applications, servers, networks, mobile devices and other sources. The goal is to aggregate, parse and visualize this data – log files, scripts, messages, alerts, changes, IT configurations, tickets, user profiles etc – to spot trends and act.
Machine data comes in many forms. Take for instance, what the Bosch Group is doing in Germany and Schnieder Electric in France.
The Bosch Group has embarked on a series of initiatives across business units that make use of data and analytics to provide so-called intelligent customer offerings. These include intelligent fleet management, intelligent vehicle-charging infrastructures, intelligent energy management, intelligent security video analysis, and many more. To identify and develop these innovative services, Bosch created a Software Innovations group that focuses heavily on big data, analytics, and the “Internet of Things.”
Similarly, the Schneider Electric focuses primarily on energy management, including energy optimization, smart-grid management, and building automation. Its Advanced Distribution Management System, for example, handles energy distribution in utility companies. ADMS monitors and controls network devices, manages service outages, and dispatches crews. It gives utilities the ability to integrate millions of data points on network performance and lets engineers use analytics to monitor the network.
By monitoring and analyzing data from customer clickstreams, transactions, log files to network activity and call records–and more, there is new breed of startups that are racing to convert “invisible” machine data into useful performance insights. The label for this type of analytics – operational or application performance intelligence.
In this posting we cover a low profile big data company, Splunk. Splunk has >3500 customers already. Splunk’s potential comes from its presence in the growing cloud-analytics space. With companies gathering incredible amounts of data, they need help making sense of it and using it to optimize their business efficiency, and Splunk’s services give users the opportunity to get more from the information they gather.
Next best offer, next best action, interaction optimization, and experience optimization typically have similar architecture. Machine learning and multivariate statistical analysis are at the heart of these cutting edge Behavioral Analytics strategies. Typically firms use statistical tools for segmentation models, behavioral propensity modeling, and market basket analysis.
The bleeding edge in next best offer is increasingly around:
- Applying machine learning to find connections between product tastes and different affinity statements
- Developing low-latency algorithms that help show the right product at the right time to a customer
- Developing rich customer affinity profiles through a variety of feedback loops as well as third-party data source (e.g. Facebook user demos and taste graph)
Targeted Offer Solutions
“Running a company is an endless quest to find out things you don’t know“
– Jeff Immelt, CEO GE
What will 2012 bring? Recently, I attended the CIO Executive Leadership Summit in Greenwich, Connecticut. I was particularly intrigued by the presentation by the new CIO of IBM, Jeanette Horan where she presented the projects she was tackling and how IBM is thinking about business analytics.
IBM is making a bet that “true leaders” will develop the capabilities required for making good and timely decisions in unpredictable and stressful environments.
IBM is adapting to this new data analytics reality by a rapid-fire acquisition strategy: Cognos, Netezza, SPSS, ILog, CoreMetrics, Algorithmics, OpenPages, Clarity Systems, Emptoris, DemandTec (for retail). IBM also has other information management assets like Watson, DB2 etc. They are building a formidable capability around the value chain: “Raw Data -> Aggregate Data -> Intelligence ->Insight -> Decisions” . They see this as a $20Bln opportunity. Read more
- How to convert Lookers to Bookers…
- How to create unique and effective Digital Experiences that impact probability of purchase or likelihood of return.
- What offers might result in higher “take rates”
The change in consumer behavior and expectations that e-commerce, mobile and social media are causing is hugely significant – big data and predictive analytics will separate brand/retail winners from losers. This won’t happen overnight but the transformation is for real.
Retail Industry makes up a sizable part of the world economy (6-7%) and covers a large ecosystem – E-commerce, Apparel, Department Stores, Discount Drugstores, Discount Retailers, Electronics, Home Improvement, Specialty Grocery, Specialty Retailers and Consumer Product Goods suppliers.
Retail is increasingly is looking like a barbell – a brand oriented cluster at the high-end, a very thin middle, and a price sensitive cluster at the low end. The consumerization of technology is putting more downward pricing pressure in an already competitive “middle” retail environment. The squeeze is coming from e-commerce and new “point, scan and analyze” technologies that give shoppers decision making tools — powerful pricing, promotion and product information, often in real-time. Applications in iPhones and Droid, like Red Laser can scan barcodes and provide immediate price, product and cross-retailer comparisons. They can even point you to the nearest retailer who can give you free shipping (total cost of purchase optimization). This will lead to further margin erosion for retailers that compete based on price (a sizable chunk of the market in the U.S, Europe and Asia).
Data analytics is not new for retailers. Point of sale transactional data obtained from bar-codes first appeared in 1970s. A pack of Wrigley’s chewing gum was the first item scanned using Universal Product Code (UPC) in a Marsh Supermarket in Troy, Ohio in 1974. Since then, retailers have been applying analytics to get even smarter and speedup the entire industry value chain.
More recent use cases of retail analytics include: Read more
Apple with its iCloud offering is attacking the consumer facing digital content big data problem. Big Data is challenging on many fronts from the insights (e.g., analytics and query optimization), to the practical (e.g., horizontal scaling), to the mundane (e.g., backup and recovery).
On June 6th, 2011 Apple Inc. launched its new purpose built digital locker service called iCloud for its 225 million iTunes accounts that frees the end-user from the tyranny of the device. The iCloud service is a cloud offering that would allow users to store digital files such as photos, MP3 music, videos and documents in the cloud and access them from Internet-connected devices like iPhones, iPads, iPods, iMacs and others.
So, what’s the big deal? They are addressing a classic BI data management problem: How to free up data trapped in “device and application jails” in a user-friendly way. The “scan and match” concept is quite applicable to large scale Enterprise Datawarehouses which suffer from data integrity issues as edge data capture and consumption devices proliferate.
Data ingestion, governance and management is a huge problem facing large organizations. As data volumes double every year, not having a basic data management strategy will become an Achilles heel. Most organizations unfortunately don’t know what data assets they have, where these assets are, how they are organized and how well they are secured. Apple shows a neat way to address the Big Data problem in personal cloud management.
Marshall McLuhan‘s enigmatic phrase – medium is the message- from the sixties gives him credit for predicting the World Wide Web 30 years ago. He could have just as well have been talking about Data Visualization for Business Analytics. While information management technology has grown at a blistering pace, the human ability to process and comprehend numerical data has not.
Visualization opens up the channel of communication between the technologists who create the data and the business people who act upon it. Data visualization tools, such as mashups, executive dashboards, KPI and performance scorecards and other data visualization technology, are becoming more popular and necessary to deal with mind numbing charts and exponential data growth.
However, the C-Suite has heard about the promise of dashboards and interactive scorecard for a few decades now and is typically dissatisfied with what they get from IT and the speed at which they get it. The big difference is that visualization technologies have finally advanced to a level where they can give actionable intelligence to the right people at the right time at the right place.
Lets take for instance an a mobile BI solution using a tool such as an Apple iPad. This gives the business executive the ability to manipulate the data with the ease of reading an e-book. The visualization library that you can draw upon to create an interactive experience on the iPad includes:
There are three critical business requirements addressed by such a solution. These are: Read more