HC2013-big-data.pdf

(270 KB) Pobierz
BIG DATA
19 JUNE 2013
Torjus Jensen
HORIZON CONFERENCE
Luxembourg, 19 June 2013
Agenda
1
2
3
4
2
Big Data Examples & Perspectives
Big Data Usage in Finance
Data Project Success Criteria
Key Take-Aways
Horizon Conference | Big Data
HORIZON CONFERENCE
Luxembourg, 19 June 2013
Data Examples & Perspective
•
•
•
•
•
•
•
•
•
•
•
•
3
First known data recording - 10,000 year ago (clay jars): 10s of items per day
All books ever written - ~500 TB
Daily new Facebook content: ~2.7 billion items (500+ TB)
Daily NYSE transaction volume: ~900 million trades
Customer transactions handled by Walmart per hour: 1 million
Genetic information in the human genome: ~3 billion
Daily Google search volume: ~3.5 billion
Text messages generated worldwide annually: 6.1 trillion
Daily internet traffic: 1.1 exabytes (1 exabyte = 1 billion gigabytes)
NSA Utah Center storage capacity: est. 5 zettabytes (5,000 exabytes)
1986: 1% of human information stored in digital format.
2013: more than 97% of human information stored in digital format.
Horizon Conference | Big Data
HORIZON CONFERENCE
Luxembourg, 19 June 2013
Examples of Big Data Uses in Finance
Use Case
Traditional Approach
Big Data Approach
Automatic pattern matching of data from
many sources
Results of data mining and mathematical
engines on real-time market data, portfolio
analysis, pattern recognition and parties
relationships
Single source of data with centralized
reporting
Intra-day enterprise risk calculation at
lower cost
Trading based on near-instant analysis of
enormous set of prices and analytics
across multiple markets.
Transactional data, geolocation data,
merchants promotions, global discounts
and cross sells
Automatic customer reaction analysis
based on social media, digital slipstream
Trade surveillance
Use
Selected Use Cases 1/2
mostly transactional data for unusual
activities detection
Recommendation Engine
General recommendation and intuition
Regulatory and Financial
Reporting
Risk Analysis And Management
Program & HF Trading
Every report requires integration with data
sources
Daily risk calculation and aggregation
Limited Capabilities
Geolocation - based Mobile
Banking
Customer Sentiment Analysis
4
Horizon Conference | Big Data
Limited Capabilities
Direct customer calls and surveys
HORIZON CONFERENCE
Luxembourg, 19 June 2013
Success Criteria for Data Projects
Clear Business Goals
• Data collection for its’ own sake not very helpful
• Technology and analysis requirements differ by project – e.g. HF trading heavily dependent on very rapid market access
as well as numerical analysis skills and vector-oriented data sets; customer behavior analysis more dependent on
comparative data and logistical analysis
Adopt New Technologies and Techniques
• Big Data database technologies moving away from traditional SQL databases to specialized technologies & concepts
such as BigTable, Hadoop, Accumulo, Hive, NoSQL, Scoop, parallel streaming ETL, and others.
• Big Data requires strong statistical analysis and potentially numerical analysis packages and skills. The best pre-built tolls
come from SAS, MaLab, Mathematica.
• Highly scalable infrastructure with parallel/clustered processing capabilities and dedicated storage.
Potential Organizational Changes
• New skills/talent required such as experts in statistics/numerical techniques, data scientists, technologists with parallel
processing and advanced data storage and management expertise
• Potentially restructure away from siloed analytics to federated analytics, CDO
5
Horizon Conference | Big Data

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika:

Zgłoś jeśli naruszono regulamin