Skip to content

September 26, 2011


Sentiment Analytics, Twitter, Federal Reserve and Consumer Pyschology

by Ravi Kalakota

Social media captures consumer sentimentWhat do these have in common:  The Federal Reserve Bank, Text Analytics, Facebook, Statistical Computations, Big Data and Keyword/Phrase/Boolean searching?

Interestingly these are more related than you think.

The Federal Reserve wants to develop a next generation Consumer Listening Platform based on social media sentiment analytics (or opinion mining) to know what people are saying and commenting about the economy.

The goal for the Fed is to better understand which way consumer confidence is trending. Microeconomics and psychology have always been interlinked. With social media, a real-time opportunity exists to monitor local, national and even global consumer psychology. And, coupled with analyzing e-commerce transactions, insightful linkage between consumer psychology and behavior (what they are spending money on and where) is possible.

Consumers are said to be 70% of the U.S economy so listening to what they are saying; and how they are influencing others makes some sense. This is yet another sign that the consumers’ behavior and expectations conveyed via social media is having a dramatic impact.

The Problem/Situation Summary

Social media platforms are changing  peer-to-peer communications and the way organizations are communicating to the public.  Conversations are happening all the time and everywhere.

Now even the Federal Reserve Bank of New York (FRBNY) is getting into the mix.  They issued an RFP for a social listening platform that allows them to monitor discussions across Facebook, Twitter, Blogs, Youtube, Forums, Associated Press Content, Google News Aggregated content, Subscription based news sources and Other News sources (CNN, WSJ).

According to the RFP: “There is need for the FRBNY  Communications Group to be timely and proactively aware of the reactions and opinions  expressed by the general public as it relates to the Federal Reserve and its actions on a variety of subjects.”

The Federal Reserve is clearly changing how it listens to consumers. It’s RFP states that the highly sophisticated listening system will gather text comments and will compute what our feelings about the economy are.  This almost gives them a real-time consumer survey and pulse of the consumer emotional state on a 24×7 basis.

What are Social Listening Platforms?

Social media listening platforms are solutions that gather data from various social media outlets and news sources.  They monitor billions of conversations and generate text analytics based on predefined criteria.  They can also determine the sentiment of a speaker or writer with respect to some topic or document.

The information gathered can guide the organizations public relations group in assessing the effectiveness of communication strategies.

Here are some of the services social listening platforms can offer:

  • Track reach and spread of messages and press releases
  • Handle crisis situations
  • Continuously monitor conversations
  • Identify and reach out to key bloggers and influencers
  • Spot emerging trends, discussions themes and topics
Sentiment or opinion analytics firms include: LexalyticsSAS; and Mindshare Technologies. Sentiment analytics platforms are enabling a number of  applications such as  reputation management (the problem every marketing person faces), “voice of customer” (listen to how they’re saying what they say, don’t constrain them to closed-ended questions), eDiscovery (was there a wave of negative emails before a certain crisis hit?) and financial services (automated trading, better information to traders).

Sentiment Analytics or Opinion Mining

Generally speaking, sentiment analysis aims to determine the attitude of a speaker or a writer with respect to some topic in a social media forum.

Sentiment scoring algorithms rate the positive or negative assertions that are associated with a document or entity.  The scoring of sentiment (or tone) from an unstructured document is a problem that uses a variety of methods such as natural language processing.

How does this work?  A basic task in sentiment analysis is classifying the polarity of a given text at the document, sentence, or feature/aspect level — whether the expressed opinion in a document, a sentence or an entity feature/aspect is positive, negative, or neutral. Advanced sentiment classification looks, for instance, at emotional states such as “angry,” “sad,” and “happy.”

The Federal Reserve is not unique.  Many hedge funds and algorithmic high frequency traders have been leveraging sentiment analytics to trade. They trade on the basis of proprietary algorithms that find a strong predictive correlations and relationships between financial news feed, Internet chatter, message boards and the historical behavior of the stock market.

Monitoring and predicting consumer confidence is simply critical to the Federal Reserve.

The banking system is build on a foundation of consumer confidence.  In Mary Poppins, there is a central incident when the children are refused repayment of a penny from their bank account and thereby trigger a bank run.  This scenario exposes the inherent fragility of every capitalist financial system. What is happening today, with banks falling like ninepins in America and Europe, has nothing to do with collapsing property prices, bad loans or greedy bankers.  It is simply an old-fashioned loss of confidence.

Welcome to the next generation of Big Data Analytics – Consumer Psychology Analytics for Macroeconomic Policy Setting.


Notes and References

  1. The Federal Reserve Bank of New York has issued a “Request for Proposal” to suppliers who may be interested in participating in the development of a “Sentiment Analysis And Social Media Monitoring Solution”.  The full RFP is available here
  2. The Federal Reserve apparently is not the only one thinking about Twitter mining. Not surprisingly, the intelligence side of government is interested in low-cost methods for tracking people’s every move on the Internet. According to Businessweek: “The intelligence agencies have jumped in to fund analytics startups—especially those using a kind of software called Hadoop—and are some of the biggest buyers of the technology. The spooks are also building their own analytics tools. Two years ago the National Security Agency talked publicly about creating a new type of technology that would grab “essentially every kind of data there is.” In July, the Defense Advanced Research Projects Agency, the military’s R&D arm, issued a Social Media in Strategic Communication proposal that called for $42 million to go toward better social networking analysis. The idea is to track the spread of ideas on networks such as Facebook, find people participating in “persuasion campaigns,” and develop countermeasures. “We must eliminate our current reliance on a combination of luck and unsophisticated manual methods,” the agency said.”
  3. For a high-level description of behavioral economics see
  4. For a high-level description of Sentiment Analytics see…..
  5. Great article on how to go about mining Twitter Feeds — Datasift Architecture: Realtime data mining at 120,000 tweets per second…  —

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Note: HTML is allowed. Your email address will never be published.

Subscribe to comments

%d bloggers like this: