Skip to content

Posts from the ‘Data Visualization’ Category

26
Mar

IoT, Machine Data and Cloud Analytics: Splunk


Machine data (Internet of Things) or “data exhaust” analysis is one of the fastest growing segments of “big data”–generated by websites, applications, servers, networks, mobile devices and other sources. The goal is to aggregate, parse and visualize this  data – log files, scripts, messages, alerts, changes,  IT configurations, tickets, user profiles etc – to spot trends and act.

Machine data comes in many forms. Take for instance, what the Bosch Group is doing in Germany and Schnieder Electric in France.

The Bosch Group has embarked on a series of initiatives across business units that make use of data and analytics to provide so-called intelligent customer offerings. These include intelligent fleet management, intelligent vehicle-charging infrastructures, intelligent energy management, intelligent security video analysis, and many more. To identify and develop these innovative services, Bosch created a Software Innovations group that focuses heavily on big data, analytics, and the “Internet of Things.”

Similarly, the Schneider Electric focuses primarily on energy management, including energy optimization, smart-grid management, and building automation. Its Advanced Distribution Management System, for example, handles energy distribution in utility companies. ADMS monitors and controls network devices, manages service outages, and dispatches crews. It gives utilities the ability to integrate millions of data points on network performance and lets engineers use analytics to monitor the network.

IndustrialInternet

By monitoring and analyzing data from customer clickstreams, transactions, log files to network activity and call records–and more, there is new breed of startups that are racing to convert “invisible” machine data into useful performance insights.  The label for this type of analytics – operational or application performance intelligence.

In this posting we cover a low profile big data company, Splunk. Splunk has >3500 customers already.  Splunk’s potential comes from its presence in the growing cloud-analytics space. With companies gathering incredible amounts of data, they need help making sense of it and using it to optimize their business efficiency, and Splunk’s services give users the opportunity to get more from the information they gather.

Read more »

20
Mar

Next Best Offer Design: Solution Architecture


Next best offer, next best action, interaction optimization, and experience optimization typically have similar architecture.  Machine learning and multivariate statistical analysis are at the heart of these cutting edge Behavioral Analytics strategies. Typically firms use statistical tools for segmentation models, behavioral propensity modeling, and market basket analysis.

The bleeding edge in next best offer is increasingly around:

  • Applying machine learning to find connections between product tastes and different affinity statements
  • Developing low-latency algorithms that help show the right product at the right time to a customer
  • Developing rich customer affinity profiles through a variety of feedback loops as well as third-party data source (e.g. Facebook user demos and taste graph)
In this blog posting we examine a more traditional next best offer solution architecture.

Targeted Offer Solutions

Read more »

2
Nov

IBM CIO Study: BI and Analytics are #1 Priority for 2012/2013


“Running a company is an endless quest to find out things you don’t know“

– Jeff Immelt, CEO GE

What will 2012 bring?  Recently, I attended the CIO Executive Leadership Summit in Greenwich, Connecticut. I was particularly intrigued by the presentation by the new CIO of IBM, Jeanette Horan where she presented the projects she was tackling and how IBM is thinking about business analytics.

IBM is making a bet that “true leaders” will develop the capabilities required for making good and timely decisions in unpredictable and stressful environments.

IBM is adapting to this new data analytics reality by a rapid-fire acquisition strategy:  Cognos,  Netezza, SPSS, ILog, CoreMetrics, Algorithmics, OpenPages, Clarity Systems, Emptoris, DemandTec (for retail).  IBM also has other information management assets like Watson, DB2 etc.  They are building a formidable capability around the value chain: “Raw Data -> Aggregate Data -> Intelligence ->Insight -> Decisions” . They see this as a $20Bln opportunity. Read more »

26
Sep

Sentiment Analytics, Twitter, Federal Reserve and Consumer Pyschology


Social media captures consumer sentimentWhat do these have in common:  The Federal Reserve Bank, Text Analytics, Facebook, Statistical Computations, Big Data and Keyword/Phrase/Boolean searching?

Interestingly these are more related than you think.

The Federal Reserve wants to develop a next generation Consumer Listening Platform based on social media sentiment analytics (or opinion mining) to know what people are saying and commenting about the economy.

The goal for the Fed is to better understand which way consumer confidence is trending. Microeconomics and psychology have always been interlinked. With social media, a real-time opportunity exists to monitor local, national and even global consumer psychology. And, coupled with analyzing e-commerce transactions, insightful linkage between consumer psychology and behavior (what they are spending money on and where) is possible. Read more »

3
Aug

Big Data, Analytics and KPIs in E-commerce and Retail Industry


  • How to convert Lookers to Bookers…
  • How to create unique and effective Digital Experiences that impact probability of purchase or likelihood of return.
  • What offers might result in higher “take rates”

The change in consumer behavior and expectations that e-commerce, mobile and social media are causing is hugely significant – big data and predictive analytics will separate brand/retail winners from losers. This won’t happen overnight but the transformation is for real.

Retail Industry makes up a sizable part of the world economy (6-7%) and covers a large ecosystem –  E-commerce, Apparel, Department Stores, Discount Drugstores, Discount Retailers, Electronics, Home Improvement, Specialty Grocery, Specialty Retailers and Consumer Product Goods suppliers.

Retail is increasingly is looking like a barbell – a brand oriented cluster at the high-end, a very thin middle, and a price sensitive cluster at the low end. The consumerization of technology is putting more downward pricing pressure in an already competitive “middle” retail environment. The squeeze is coming from e-commerce and new “point, scan and analyze” technologies that give shoppers decision making tools — powerful pricing, promotion and product information, often in real-time. Applications in iPhones and Droid, like  Red Laser can scan barcodes and provide immediate price, product and cross-retailer comparisons. They can even point you to the nearest retailer who can give you free shipping (total cost of purchase optimization). This will lead to further margin erosion for retailers that compete based on price (a sizable chunk of the market in the U.S, Europe and Asia).

Data analytics is not new for retailers. Point of sale transactional data obtained from bar-codes first appeared in 1970s. A pack of Wrigley’s chewing gum was the first item scanned using Universal Product Code (UPC) in a Marsh Supermarket in Troy, Ohio in 1974.  Since then, retailers have been applying analytics to get even smarter and speedup the entire industry value chain.

Consumer Goods Value Chains

More recent use cases of retail analytics  include: Read more »

12
Jul

Are you one of these — Data Scientist, Analytics Guru, Math Geek or Quant Jock?


“The sexy job in the next ten years will be statisticians…”
‐ Hal Varian, Google

Analytics Challenge — California physicians group Heritage Provider Network Inc. is offering $3 million to any person or firm who develops the best model to predict how many days a patient is likely to spend in the hospital in a year’s time. Contestants will receive “anonymized” insurance-claims data to create their models. The goal is to reduce the number of hospital visits, by identifying patients who could benefit from services such as home nurse visits.

The need for analytics talent is growing everywhere. Analytics touches everyone in the modern world. It’s no longer on the sidelines in a support role, but instead is driving business performance and  insights like never before.

Job posting analysis indicate that market demand for data scientists and analytics gurus capable of working with large real-time data sets or “big data” took a huge leap recently.  The most common definition of “big data” is real-time insights drawn from large pools of data. These datasets tend to be so large that they become awkward to work with using on-hand relational database tools, or Excel.

It’s super trendy to be labeled “big data” right now – but that doesn’t mean the business trend’s not real.  Take for the instance the following scenario in B2B supply chains. Coca-Cola Company is leveraging retailers’ POS data (e.g., Walmart) to build customer analytical snapshots, including mobile iPad reporting, and enable the CPFR (Collaborative Planning, Forecasting, and Replenishment) process in Supply Chain. Walmart alone accounts for $4 bln of Coca-Cola company sales.

Airlines, hotels, retail, financial services and e-commerce are industries that deal with big data. The trend is nothing new in financial services (low latency trading, complex event processing, straight thru processing) but radical in traditional industries.  In trading, the value of insights depends on speed of analytics.  Old data or slow analytics translate into losing money.

As data growth in business processes outpaces our ability to absorb, visualize or even process, new talent around Business Analytics will have to emerge. New roles such as Data Scientists, Analytics Savants, Quant Modelers are required in almost every corporation for converting the growing volumes of data into actionable insights.

Look at these data stats.

Read more »

6
Jun

Apple iCloud Service – Lessons for “Big Data” and BI Architects


Apple with its iCloud offering is attacking the consumer facing digital content big data problem. Big Data is challenging on many fronts from the insights (e.g., analytics and query optimization), to the practical (e.g., horizontal scaling), to the mundane (e.g., backup and recovery).

On June 6th, 2011 Apple Inc. launched its new purpose built digital locker service called iCloud for its 225 million iTunes accounts that frees the end-user from the tyranny of the device.  The iCloud service is a cloud offering that would allow users to store digital files such as photos, MP3 music, videos and documents in the cloud and access them from Internet-connected devices like iPhones, iPads, iPods, iMacs and others.  

So, what’s the big deal? They are addressing a classic BI data management problem:   How to free up data trapped in “device and  application jails” in a user-friendly way. The “scan and match” concept   is quite applicable to large scale Enterprise Datawarehouses which suffer from data integrity issues as edge data capture and consumption devices proliferate.

Data ingestion, governance and management is a huge problem facing large organizations.  As data volumes double every year, not having a basic data management strategy will become an Achilles heel. Most organizations unfortunately don’t know what data assets they have, where these assets are, how they are organized and how well they are secured.  Apple shows a neat way to address the Big Data problem in personal cloud management.

Read more »

13
Mar
Courtesy National Oceanic and Atmospheric Administration

Data Visualization- The Medium is the Message


Marshall McLuhan‘s enigmatic phrase – medium is the message-  from the sixties gives him credit for predicting the World Wide Web 30 years ago.  He could have just as well have been talking about Data Visualization for Business Analytics. While information management technology has grown at a blistering pace, the human ability to process and comprehend numerical data has not.

Visualization opens up the channel of communication between the technologists who create the data and the business people who act upon it.  Data visualization tools, such as mashups, executive dashboards, KPI and performance scorecards and other data visualization technology, are becoming more popular and necessary to deal with mind numbing charts and exponential data growth.

However, the C-Suite has heard about the promise of dashboards and interactive scorecard for a few decades now and is typically dissatisfied with what they get from IT and the speed at which they get it. The big difference is that visualization technologies have finally advanced to a level where they can give actionable intelligence to the right people at the right time at the right place.

Lets take for instance an a mobile BI solution using a tool such as an Apple iPad. This gives the business executive the ability to manipulate the data with the ease of reading an e-book.  The visualization library that you can draw upon to create an interactive experience on the iPad includes:

There are three critical business requirements addressed by such a solution. These are:   Read more »

%d bloggers like this: