As a data engineer and scientist, I have been following the NSA PRISM raw intelligence mining program with great interest. The engineering complexity, breadth and scale is simply amazing compared to say credit card analytics (Fair Issac) or marketing analytics firms like Acxiom.
Some background… PRISM – “Planning Tool for Resource Integration, Synchronization, and Management” – is a top-secret data-mining “connect-the-dots” program aimed at terrorism detection and other pattern extraction authorized by federal judges working under the Foreign Intelligence Surveillance Act (FISA). PRISM allows the U.S. intelligence community to look for patterns across multiple gateways across a wide range of digital data sources.
PRISM is unstructured big data aggregation framework — audio and video chats, phone call records, photographs, e-mails, documents, financial transactions and transfers, internet searches, Facebook Posts, smartphone logs and connection logs – and relevant analytics that enable analysts to extract patterns. Save and analyze all of the digital breadcrumbs people don’t even know they are creating.
The whole NSA program raises an interesting debate about “Sed quis custodiet ipsos custodes.” (“But who will watch the watchers.”) Read more
Over the past seven years, we’ve seen a massive regulatory overhaul and an industry-wide push to enhance trust and confidence and encourage investor participation in the financial system.
To roadmap Wall Street regtech priorities, we have been having ongoing meetings with MDs and leading architects in global banks and investment services firms. RegTech (e.g., regulation as a service) is a subset of FinTech. Companies include
- Fintellix offers a data analytics platform allowing banks to convert internal data into regulatory reporting formats
- Suade offers banks “regulation as a service” interpreting real time regulatory knowledge so that banks can better manage and respond to regulation
- Sybenetix combines machine learning with behavioral science to create a compliance and performance tool for traders
No longer business as usual. It is clear that banks are devoting more resources to Know Your Customers (KYC), Anti-Money Laundering (AML), fraud detection and prevention, Office of Foreign Assets Control (OFAC) compliance. FINRA is at the beginning stages of the process for building the Consolidated Audit Trail, or CAT for trading surveillance.
To enable compliance with variety of Risk/Regulatory initiatives, AML and KYC initiatives…the big RegTech related investments are:
- Strengthening the Golden Sources – Security Master, Account Master and Customer Master.
- Standardized, common global business processes, data, systems and quantitative solutions that can be leveraged and executed across geographies, products, and markets to manage delinquency exposures, and efficiently meet Regulatory requirements for Comprehensive Capital Analysis and Review (CCAR), FDIC Reporting, Basel, and Stress Loss Testing.
- Various enterprise data management initiatives – Data Quality, Data Lineage, Data Lifecycle Management, Data Maturity and Enterprise Architecture procedures.
Regulatory reporting improvements via next generation Enterprise Datawarehouses (EDW) (using Oracle, IBM, NoSQL or Hadoop)– Reporting on top of EDW addresses the core problems faced by Finance, Risk and Compliance when these functions extract their own feeds of data from the product systems through which the business is conducted and use differing platforms of associated reference data in support of their reporting processes.
Lot of current investments are in the areas of Finance EDW which delivers common pool of contracts, positions and balances, organized on an enterprise wide basis and completed by anointed “gold” sources of reference data which ensure consistency and integration of information.
Crawl, walk, Run seems to be the execution game-plan as the data complexity is pretty horrendous. Take for instance, Citi alone….has approximately 200 million accounts and business in 160+ countries and jurisdictions. All risk management is made incredibly complex by the numerous banking mergers that took place over the past 3-4 decades.
The type of data challenges global banks like Citigroup, Goldman, Wells Fargo, Bank of America and JP MorganChase are wrestling with include: Read more
Big Data emphasizes the exponential growth of data volumes worldwide (collectively, >2.5 Exabytes/ day).
Big Data incorporate the following key tenets: diversification, low latency, and ubiquity. In parallel, the emerging field of data science introduces new terms including, predictive modeling, machine learning, parallelized and in-database algorithms, Map Reduce, and data monetization.
A variety of infographics have been published around Big Data, Data Scientists. Here is a compendium of some very interesting ones.
The Real World of Big Data (Click image to see a larger version and article)
|Big Data Big Opportunity||A Data Scientist Study|
At the Analytics Executive Forum, I facilitated a session on Omni-channel analytics. It struck me how every leading consumer facing firm seems convinced that mobile is becoming the dominant B2C interaction channel. Mobile is the gateway to insight based marketing and the “always addressable customer”….
Insight-based interactions – The company knows who you are, what you prefer, and communicates with relevant, timely messages, using the power of analytical intelligence to detect patterns, decode strands of information and create meaningful offers and value.
The “always addressable customer.” This is a consumer who fits the bill on three fronts simultaneously: (1)
- Owns and personally uses at least three connected devices; (2)
Goes online multiple times throughout the day; (3)
- Goes online from at least three different physical locations
The opposite of insight-based is “spray-and-pray” marketing – The company has very limited knowledge about who you are, forgets what you prefer, and tries to reach you with off-target communications that alienate you – based on fragmented data, poor data quality and inadequate integration, resulting in confusing, chaotic interactions. A good example: “I have 2 million frequent flyer miles with your airline and still do not get any recognition, respect or value from this loyalty.”
As companies architect new insight based mobile use cases I suggest that they look at what is coming next. With IOS 7, Apple is delivering several new features – Passbook, Beacon.
Retailers, banks and other customer facing firms/brands better pay attention. 100+ million iPhones are automatically getting this feature with the new OS upgrade making this a mega-disruptor in the coveted target segment everyone is chasing. Read more
- Data-as-a-Service: Data Provisioning, Management, Lineage, Quality
- Reporting-as-a-Service: Dashboards, KPIs, Drilldowns/Aggregates…. Descriptive
- Analytics-as-a-Service: Predictive Modeling and BI… Prescriptive analytics
- Information-as-a-Service: Threshold based Alerts, Exceptions, Mobile Prompts
- Insights-as-a-Service: ML/AI based…automated learning – ambient intelligence, Next best Offer/Action
Which strategy are you implementing?
Data is valuable. Data is plentiful. Data is complex. Data is in flux. Data is fast moving. Capturing and managing data is challenging.
So, if you are a senior leader in a Fortune 2000 company. How do you structure your group to deliver effective BI, Analytics or Big Data projects? Do you have the right structure, toolset, dataset, skillset and mindset for analytics and Big Data?
Organizing for effective BI, Analytics and Big Data is becoming a hot topic in corporations. In 2012, business users are exerting significant influence over BI, Analytics and Big Data decisions, often choosing analytics and visualization platforms and products in addition to/as alternatives to traditional BI platform (reporting and visualization tools).
Interested in slicing, dicing, measuring, and analyzing data for customer and business insights?
According to a recent survey by Bloomberg, 97% of companies with revenues of more than $100 million are using some form of business analytics, up from 90% just two years ago.
While businesses have embraced the idea of fact-based decision-making, a steep learning curve remains. Only one in four organizations believes its use of business analytics has been “very effective” in helping to make decisions. Data is not just ignored but often discarded in many organizations as the business users can’t figure out how to extract signal from data noise.