Big data : a crucial challenge for energy players

Big Data: A crucial challenge for energy players

July 2014

www.chappuishalder.com Twitter : @ch_retail

2

Big Data revolution is in motion in the energy sector

Given recent evolutions on energy markets, remaining competitive implies to be able to process a growing flow of data.

We strongly believe that the keys to success are two-fold when it comes to designing a successful Big Data strategy:

• A structured and robust framework

• A continuous upgrade of hardware and infrastructure to stick to volume of data and complexity of analyses

Big Data represents a source of business opportunities for energy players…

… from rethinking client relationship to crafting tomorrow’s risk management.

Quality

Depth

Exhausti-veness

Centrali-sation

« Real time » data

« Data storage »

Exogen-ous data

Big Data

3

Evolutions within energy markets imply a growing flow of data to be processed…

Energy players strengthen their international footprint as more regions reach crucial liquidity to generate significant trading volumes …

… and multi-commodity players are becoming the rule

Extension to 24 hours/7 days opened derivatives and spot exchanges • Pan-European Gas Market Exchange (PEGAS)

will be one, as of July (still no exchange in Spain)

• New products emerge that can modify supply and hedging options and strategy

Numerous parameters required to optimize portfolio management (power plants)

Providers increase real-time flow on meteorological data

Development of smart metering increases the volume and frequency of data collection: • Giving suppliers/ shippers a better view on

how energy is consumed (at grid or user level)…

• And enabling operators (transmission and distribution systems, LNG Terminal & Storage) to publish real time data and forecasts

Supply problems or political and

regulatory decisions must be known rapidly and taken into account in order to be exploited on several time horizons

… as business conduct gets more complex … and more information becomes available

Implementation of a successful Big Data strategy requires a structured and robust framework, from acquisition of info to decision-making process

Acquire Analyze Organize Decide 1 3 4 6

Control 2 Report 5

Structured data: • Ex. market data gathered

both on supply and demand side

Unstructured data: • Non-parametric statistics

and unconventional data • Ex.: Twitter content

A dataset clean of errors

• Wrong data can cause you to make wrong decisions

• Warning mechanism and auto correction replacing wrong data with substitutes

Process collected data:

• Reduce -> Aggregate -> Enrich -> Structure

Create datasets:

• Use relational technology to assemble and match data

Data modeling: • Fast read and write speeds • Build statistical models

comparing data sets’ variables and datasets

Find correlations: • Bring additional value to

classical market analysis • Machine learning

approach to capitalize on past events knowledge

Sort through to extract the useful and stay synthetic

• Give it meaning, and make it exploitable,

• Show the big picture and focus on a few key points

Anticipate • Predict market trends / price • Estimate likelihood of external

factors (weather, media impact …)

Make predictive decisions • New market positions to

enhance margins • Ex: new capacity planning,

capacity expansions

5

Big Data requires investing in new forms of data processing…

Integrated framework

Dynamic framework

Static framework

3

4

5

6

2

1 0

Data Collection Data warehouse IT implementation & Infrastructure

Data Storage Historical data

Data Cleaning Data quality management Data transformation

Data Statistical Description Mean, Median, standard deviation Histogram…| VaR 95,99%

Data Analysis Clustering & segmentation Automatic classification | Factorial analysis

“X factor” Mining Web mining (behavior…) Image mining (face recognition) |

Text mining

Data Mining & Big Data Prediction Ranking / discrimination Anticipation & simulation

Today’s average position

Past

Future

Present

7 Artificial Intelligence Self-Learning models (auto

efficient) Multi crossing data set

Integrated framework

Dynamic framework

Static framework

6

… that will necessitate continuous hardware and infrastructure upgrades to adapt to the volume and complexity of data to be managed

Flexibility & Adaptability

Change the way you treat the data according to your continuous experience

You always need to handle more data, more frequently and with agility, implying great storage capacities

Power & Speed

Windows of opportunity may close up quickly therefore your calculation speed should be optimized ASAP

As long as you keep manual steps, you will not reach optimum -> automation must be your motto

Modeling Computation Storage

Self-detection, auto upgrading and efficient models

Optimized computation capacity

Centralized, unlimited data storage capacity

Homogenisation of modeling practices, business incentives for Data modelling technique development

Good computation capacity

Aggregation of data sources (Finance, Risk, Marketing, Sales…)

R&D Development Limited computation capacity

Limited storage capacity

Integrated

Dynamic

Static

7

Numerical transformation induced by Big Data has root causes and objectives that transcend sectors, and the energy industry is no exception

Source : Ventana Research | 2013

Source : Analytics | IBM Institute | 2012 Refocus on customer (CRM)

55%

Process optimisation incl. cost optimisation

4% New business

model 15%

Risk management/ Financial reporting

23%

Collaborative working mode

2%

Cost reductions

Reduce and limit manual steps

Produce daily results evermore precisely

Increase the compute speed

Store and analyze even more data

But in addition to these ‘standard’ objectives, players on energy markets will focus on specific issues…

8

On-demand data mining to dig into meta-data Risk and P&L indicators (calculation and reporting) P&L explain – detection of abnormal variations - VaR back-testing

Data preparation for EMIR, Basel II/III, MIFID, REMIT, audits …

To support KYC, rogue trading, AML or anti-fraud process …

Pre-trade decision support (ex. locational/ geographical spreads, time spreads on storage etc.)

To help identify trades from various systems to avoid missed or duplicated trades

… with business or process orientations

Automatically executed quantitative processes or High-Frequency-Trading

Real-time optimization of day-ahead and intra-day position coverage

Optimize client consumption forecasts (short – medium – long-term) Optimize production forecasts (generation)

Bu

sin

ess

-ori

en

ted

is

sue

s P

roce

ss-o

rien

ted

issu

es

Data Tagging

Trading Analytics

Watch Tower

Regulation

Financial Data Management

Forecasting

Hedging Strategy

Systematic Trading

9

Don’t let yourself be overrun by competitors …

Hardware and infrastructure upgrades induced by Big Data (storage, calculation capacity, etc.) must not overshadow the necessity of investing in human capital.

Players on energy markets that will best ride the Big Data wave will not only get their heads above water in a harsh competition context…

Controlling the three V’s of data (Variety, Volume and Velocity) creates an alternative information edge, which is:

• A potential new source of uncorrelated excess returns

• Advanced techniques for valuing clients and deals

• Most helpful in risk and performance management

• Easing data management for internal purposes

10

CH&Cie at a glance

Management Consultancy ... … for Financial Services & Commodities

Retail Banking

Private Banking Corporate & Investment Banking

Insurance Commodities Customer

Experience

Risk & Finance

IT & Operations

Business Development

11

8 offices around the world, in major trading and financial places…

… 100+ consultants, with strong academic backgrounds and experience

Business school 60%

Engineering school 30%

Others 10% In average, CH&Cie consultants have 7 years of

experience within consulting firms, Financial Services and Commodities.

12

Your contacts for this offer

Director - Head of CH&Cie Commodities + 33 6 40 56 21 71 [email protected] Paris Office

CEO & Partner + 44 78 34 55 03 98 + 33 6 12 41 64 06 [email protected] London Office

Partner + 44 203 427 3559 + 33 7 87 68 81 77 [email protected] London Office

Manager + 33 6 65 02 80 07 [email protected] Paris Office

Geneva Office

Rue de Lausanne 80

CH 1202 Genève, Suisse

London Office

50 Great Portland Street

London W1W 7ND

Paris Office

25 rue Alphonse de Neuville

75017 PARIS