Technology in support of utilities challenges

<Insert Picture Here>

Technology for Utilities Transformation and new Challenges: Integration, Big Data, Extreme Performance.Aitor IbañezEnterprise Architect for Utilities

2


Contents

• Utilities Business Transformation• Need for Technology• Extreme Performance• No-limit Integration• Applications: Architectural Challenges

3


Utilities Business Transformation

4

Data Growth defines new Challenges…

4

DATA GROWTH IMPACT

• Multiple Devices: Meters, Apps, SCADAs,…

• Info Processed by multiple Departments: Risk Mgmt., Billing, Demand Analysis, Network.

• Growth of Corporate DW for Reports and Trend Analysis.

• Agility for Business Processes on Transactions between Devices and Applications

• New Scalable Architecture supporting new Analysis and Transaction Growth Reqs.

• New Data Mgmt Reqs. for Storage, Master Data, Synchronization & Sharing of Information

Fuente: Accenture “Challenges for Utilities” 2010, Oracle Insight

IT CHALLENGES

5

Growing Market Share

Defense against Competency

Cost Savings

Growing Sales

Managing efficiently growing Data Volumes

Improving Customer Care & Go-to-market

Processes

Simplifying Storage and Servers

On-demand growing, Business Continuity

Business Processes Automation

Scalable Infrastructure

More Power

Improving Performance and

security of Transactions

Reducing Operational Costs and improving

Energy Efficiency

Faster processing to improve Billing,

Demand Analysis, …

5

Improving AvailabilityReal-time Data Access, Load

balancing

External Pressure Strategy

Business Perspective

Levers Implications

Ma

na

gin

g D

ata

Ex

plo

sio

n

wit

ho

ut

co

st

inc

rem

en

ts

IT Perspective

Challenges that arise new Biz & IT Needs

Utilities need New Technology to face up New Challenges

6

… Needs translated into new Requirements

• Scalable Infrastructure• Horizontal growing

• Business Processes Automation• Faster deployment of processes• Process Governance• Transactions

• Real-time Data Access, Load Balancing• Fast response to data needs• Grid computing

• On-demand growing, Business continuity• Right-sizing, growing infrastructure as the Business demands• SLAs Compliance

• Simplifying Storage and Servers• Reducing HW Installed base: best platform for every purpose• Energy Savings

• More Power• Unprecedented Processing Power• Driving new Business opportunities

6

Performance

Scalability

Manageability

Protecting Investment

7


Need for Technology

8

Smart Grid ChallengesManaging the Exponential Growth of Data Volumes

New devices in the homeenabled by the smart meter

You are here.

AMI Deployment

PCTs Come On-line

Distribution Management Rollout

Mobile Data Goes Live

RTU Upgrade

GIS System Deployment

OMS Upgrade

Advanced Distribution Automation

Substation Automation System

Workforce Management Project

Time

200 TB

400 TB

600 TB

800 TB

Expected Growth of a Tier 1 Utility’s Data Volume


You are here.

AMI Deployment

PCTs Come On-line



RTU Upgrade


OMS Upgrade




Time

200 TB

400 TB

600 TB

800 TB


You are here.

AMI Deployment

PCTs Come On-line



RTU Upgrade


OMS Upgrade




Time

200 TB

400 TB

600 TB

800 TB

Expected Growth of a Tier 1 Utility’s Data Volume

Source: EPRI

Much more data… Much more processing…

Processing Power required for Interval based billing

ScalableSystem

Break Point System

Co

st

Meters

Costinviable

Scalability break points: storage, scalability cost

9

Integration beyond BoundariesResponding to events and information from a variety of sources

10

So… Need for Technology!

• What do we need?

Extreme Performance

Massive Data Volume

Unstructured Data Analysis

No-Limit Integration

Biz Processes Automation

Services Architecture

Engineered Systems

Events Processing

Data Integration

BPM, SOA, EDA

11


Extreme Performance

12

Exadata Database Machine

Database Grid Intelligent Storage Grid

InfiniBand Network

• Redundant 40Gb/s switches• Unified server & storage network

• 14 High-performance low-cost storage servers

• 8 Dual-processor x64 database servers

OR

• 2 Eight-processor x64 database servers

• Scalable Grid of industry standard servers for Compute and Storage • Eliminates long-standing tradeoff between Scalability, Availability,

Cost

• 100 TB High Speed disk, or336 TB High Capacity disk

•5.3 TB PCI Flash

• Data mirrored across storage servers

Copyright © 2010, Oracle Corporation and/or its affiliates – 12 –

13© 2008 Oracle Corporation – Proprietary and Confidential – 13 –

Traditional Vs Smart Scan Processing

I/Os Executed:

1 terabyte of data returned to hosts

DB Host reduces

terabyte of data to 1000 customer names that are returned to client

Rows Returned

SELECT

customer_nameFROM calls

WHERE amount > 200;

Table

Extents Identified

I/Os Issued

2MB of data

returned to server

Rows Returned

Smart Scan

Constructed And Sent To Cells

Smart Scan

identifies rows and columns within

terabyte table that match request

Consolidated

Result Set Built From All

Cells

SELECT

customer_nameFROM calls

WHERE amount > 200;

Traditional Scan processing Exadata Smart Scan processing

14

Exadata Storage Software Unique Features

• Exadata Smart Scans• 10X or greater reduction in data sent

to database servers

• Exadata Storage Indexes• Eliminate unnecessary I/Os

• Hybrid Columnar Compression• Efficient compression increases

effective storage capacity and increases user data scan bandwidths by a factor of up to 10X

• Exadata Smart Flash Cache• Breaks random I/O bottleneck by

increasing IOPs by up to 20X• Doubles user data scan bandwidths

• I/O Resource Manager (IORM) • Enables storage grid by prioritizing

I/Os to ensure predictable performance

• Quality of Service (QoS)• Actively meet and maintain SLAs• Memory Guard to protect existing

current transactions from memory-based failures

Copyright © 2010, Oracle Corporation and/or its affiliates – 14 –

15

Proof of Concepts Results

0

500

1000

1500

"01" "07" "OF " Queries

Referencia “As IS” Exadata Optimizacion

47 veces más rápido




16

Oracle Exalogic Elastic CloudApplication Grid, Integrated Compute, I/O, Networking and Storage

• Central storage for system images

• Clustered for HA• 40 TB SAS disk• 4 TB read cache• 72 GB write cache

Integrated Storage

• 40 Gb/sec links• 10 Gigabit Ethernet

connectivity to datacenter

I/O Fabric

EL X2-2

• 30 x86 compute nodes• 360 Xeon cores (2.93 GHz)• 2.8 TB DRAM• 960 GB SSD

X2-2

Copyright © 2011 Oracle Corporation - Proprietary and Confidential

EL X2-2

Coherence

Enterprise M

anager Oracle Linux and/or Solaris

Exalogic Elastic Cloud Software

JRockit and HotSpot

WebLogic Server

Exalogic Elastic Cloud Hardware

Tuxedo

17

Oracle Exalogic and Exadata TogetherDesigned for Seamless Interoperability, Unmatched Performance

Exalogic Exadata

960 Gigabits/second(Maximum: 24x InfiniBand QDR)

• Exclusive: direct InfiniBand integration!

• Extreme performance, reliability, security

• Simple to deploy and manage

Active GridLink for RAC

• Resource-aware load balancing

• Instantaneous connection failover

• Transaction affinity to RAC nodes

• SQLnet optimized for InfiniBand SDP

Up to

3X OLTP Performance

Copyright © 2011 Oracle Corporation - Proprietary and Confidential

18

Technologies in SupportExaMDM Concept: Technical Architecture

10 M Meters Benchmark• ½ Exadata + ½ Exalogic + Oracle MDM + CC&B

• 500.000 bills 1 hour• Daily meter reads upload 2,6 hours• VEE processing 4 hours• Usage Aggregations 30 min.• Daily Total: 8 hours.

•Scale: ½ Full ExaMDM = 4 hours

19

Oracle Exalytics – Under the Hood

For information purposes only, not a commitment to the development, release, and timing of any features or functionality.

In-Memory Analytics Software

Essbase

TimesTen for Exalytics

Adaptive In-Memory Tools

1 TB RAM40 Processing Cores

High Speed Networking

In-Memory Analytics HardwareOracle Business Intelligence Foundation Suite

InfiniBand

Speed-of-thought enterprise Business Intelligence platform

Extreme performance via Exadata

20

Oracle ExalyticsExtreme Performance for Enterprise Analytics

• Speed-of-Thought Analysis at Enterprise Scale• 20x better response times• Sub-second performance enables speed-of-thought analytics• Grow to 50,000 enterprise users on one appliance

• What-if Analysis and Forecasting• 16x better response times • More accurate – use more detail, include more scope, evaluate wider operations• More scale – more users and higher volumes at busy periods• Greater agility – do daily instead of monthly

• Enterprise Planning• Reduce planning cycle times with 5x better response time and 6x better throughput• Run more planning models with finer grained operational detail to improve accuracy• Extend beyond Finance to every LOB, with 10,000 Planning users on one

appliance

21

Big Data for Smart GridAnalysis from unstructured data

• Unstructured data coming from• Smart Meters & Devices, Domotic,…• Info coming from Network & Generation SCADAs• Substations, other equipments,…• Meteo Data.

• BigData Appliance• 18 nodes, 216 CPU cores.• 864 GB RAM, 648 TB storage• 40 Gb/s Infiniband

• Extreme Performance for Unstructured Data processing.

22

Big Data for Smart GridThink about the Future…

• Driving Immediate Analysis with unprecedented processing power• Social Networks: mentions [ourCompany]

• Sentimental Analysis, Complaints• Social Networks: mentions #Meter, [ourCompany], #power Trending

Topics.• Targeted programs for each consumer

• Understanding Customer: CRM, Meter load, Internet logs, Twitter,…

• Power Demand Planning: processing trends and actual consumption by geography/concentrator.

• Prediction of Network failures or bottlenecks.• Geographical & Meteo Data

23

Divided Solution Spectrum

Acquire AnalyzeOrganize

MapReduceSolutions

DBMS (DW)

DBMS (OLTP)

Advanced Analytics

DistributedFile Systems

Transaction (Key-Value)

Stores

ETL

NoSQL Flexible

SpecializedDeveloper

Centric

SQL TrustedSecure

Administered

DataVariety

InformationDensity

Unstructured

Schema

24

DataVariety

InformationDensity

Unstructured

Schema

Oracle’s Big Data Solution

Acquire AnalyzeOrganize

OracleDatabase

(DW)

OracleDatabase

(OLTP)

In-DBAnalytics

“R”Mining

TextGraphSpatial

OracleBI EE

Oracle NoSQL DB

HDFS Hadoop

OracleData Integrator

Oracle Loader for Hadoop

Big Data Appliance• Hadoop

• NoSQL Database

• Oracle Loader for hadoop

• Oracle Data Integrator

Oracle Exadata• OLTP & DW

• Data Mining & Oracle R

• Semantics

• Spatial

Exalytics• Speed of

ThoughtAnalytics

25

Smart Metering Architecture DecisionsBig Data

Business Intelligence

meter Head-End

CIS

Work & Asset

Mobile

Distribution management

Outage Management

Self Service / portal

MDM

TBs

SOA Framework

Middleware

& Bus. Orchestration

SOA Framework

Middleware

& Bus. Orchestration

AM

I S

ervi

ces

AM

I S

ervi

cesNRT

26


No-Limit Integration

27

Oracle GoldenGateThe Solution for Enterprise-wide Real Time Data Needs

Real-time information

Oracle GoldenGate delivers real-time access of real-time information, enabling companies to dramatically improve the availability, reliability, and performance

of critical data across enterprise systems.

Database and applications, Mixed sources, distributed systems, legacy,

OLTP, OLAP

Mission Critical Applications & Data, Business

Intelligence, Reporting for Customers, Partners & Employees

Real-time Access

28

Oracle Data Integrator

• E-LT Transformation• Declarative Design• No-intrusive• Change Data Capture• Architecture hot-

changes• Multi-DB support:

Knowledge modules

Data Integrator

Transactional

RDBMSCDC

Data Integration Solution

Logs

Realtime Data Hubs

Business Intelligence

Enterprise Warehouse

SOA | BAM | CEP

Oracle Data Integrator

Events

Services

E-LT

CDC

Logs

Rea

l Tim

e D

ata

Flo

w

Bat

ch D

ata

Flo

w

CDC

Logs

29

Supply Real Time Business IntelligenceNon-invasive Capture + E-LT Processing

Application Real Time BI(using Data Copy)

Analytic BI(Facts & Dims)

E-LT(Mini-Batch + Transforms)

Consistency Window

30


Applications: Architectural Challenges

31

Technologies in Support Oracle Integration Platform for Utilities

Objective:Provide a comprehensive, consistent, and flexible integration approach across the Utilities Suite

Approach:• Built on Oracle’s Fusion Middleware • Leveraging AIA Methodology• Expand AIA Foundation Pack for Utilities:

–Common Objects and Enterprise Services–Compliant with utility industry standards–Deliver best-practice process models for key

processes across utility segments (water, gas, electric, waste)

Benefits:• Provide a common integration approach across

all Oracle applications. • Accelerate deployments and provide flexibility

for project-specific extensions.

Oracle Utilities Integrated Solution Suite-

SCADA

CRMML / AMIMWMWAM

MDMGIS / Spatial

ModelNMS

CC&B

QM

SOA Suite, AIA (ESB + FP for CIM and MultiSpeak)

Reference Process Model

Industry Standard

s

Common Object Model & Services

AIA Foundation Pack for Utilities

32

Oracle Utilities Integration StrategyOracle AIA Integrations Benefits

AIA Centers of Excellence

AIA Partner Co-development

UGBU Partners to certify with SAP + other 3rd Parties

Open EcosystemOpen Ecosystem

Pre-built integrations using leading Oracle Fusion middleware

Lower TCO

Supported products

Designed for next generation apps

compatibility

Reusability

ValueValue$$

Industry best practices

Prescriptive methodology

Industry standards adoption such as

CIM for Utilities

Proven by Oracle

MethodologyMethodology

Loose coupling of apps through SOA

Focus on configurations not

rebuilds

Successfully deployed in complex, high-

volume environments

ExtensibilityExtensibility

33

Business Process ManagementProcesses Life Cycle

Dashboards, tools and historical trends

Business Analyst

Process Developmentand Systems Integration

Process Developer

Business

Owner

Process Execution and Management

Users Interaction Business

ParticipantsCorporate

Applications

And Databases

Process Modeling, simulating and Documentation

34

Oracle Technologies in support Event-Driven Architecture

Complex Event Processor

BPM Processes

event

JMS

closed loop

Applications

event

Databases

Streams

event

event

Messaging

Event Sources

BAM Dashboards

BAM Alerts

BPM/BPEL Processes

Response Services

Coherence Real Time Listeners

event • Event sourcing

• Continuous query language to define event

• Correlation

• Sequencing

• Window of time evaluation

• Pattern matching

• Event publishing

• Real time Java infrastructure

35

Monitor business processes & services in real-time Key Performance Indicators (KPIs) Service-Level Agreements (SLAs)

Analyze events as they occur Correlate events & KPIs Identify trends as they emerge Alert users to bottlenecks & solutions

Act on current conditions Event-driven alerts Real-time dashboards BPEL processes & web services integration

Business Impact of Events, Oracle BAMMeaningful, Event-driven Intelligence for End-Users

36

Questions?

Technology in support of utilities challenges

Technology

big data

mb of data ios

new technology

smart grid challenges

new business opportunities6

storage redundant

new biz

new challenges5