IP/Ethernet Drivers and Landscape: Smart Grid in Iberdrola Europe 2013 - Miguel Angel... · IP/Ethernet Drivers and Landscape: Smart Grid ... • Just Ethernet services ... Switch

IP/Ethernet Drivers and Landscape: Smart Grid

in Iberdrola NETWORK BUSINESS SPAIN

Control Systems and Telecommunications

Miguel. A. Sánchez Fornié

Alberto Sendín

July 2013

2

Table of Contents

• Iberdrola

• Communication Network and Architecture in

Iberdrola

• Evolution towards Ethernet

• MPLS as Extended LAN basis

3

Index

Iberdrola

4

Iberdrola

http://www.iberdrola.com, May 2013,

August 2011

5

USA

Brazil

UK

Spain

Iberdrola

Coordinated strategies and architectures across Iberdrola Group

6

Index

Communication

Network and

Architecture in

Iberdrola

7

• Utilities rely on assets that makes them different from many other companies when facing Telecommunication challenges.

Communications in Iberdrola. Infrastructure = opportunity

Poles and ducts for fiber optic cables Poles for Base Stations

Substations as nodal premises

Radio station sites

Carriers’ carrier model in place

8

• Telecommunication Services are fundamental for the Operational activities of the company. However, not all “telecommunications” are well suited for Operational activities.

• Not all operational communication services have the same criticallity:

– Data is not the same as voice.

– Cost of the solution is biased by initial investment.

• Not all data related communications in operational areas have the same relevance and requirements:

– Remote control is critical to improve electricity grid availability.

– Remote metering of certain points of supply is not time critical.

√ Operational activities have more demanding requirements than Corporate ones:

− A network that is prepared for operational services is ready for corporate use (but not necessarily the other way around!)

Communications in Iberdrola. Mission Critical

9

Communications in Iberdrola. Network Perspective

Type Quantity

Substations (Primary –ST&STR) Thousands

(1,100, IBD, Spain)

Distribution Substations

(Surface&Pole mounted)

Tens of

Thousands

(36,000 + 26,000, IBD,

Spain)

Distribution Substations

(Underground)

Tens of

Thousands

(26,000, IBD, Spain)

Points of Supply Millions

(10,800,000, IBD,

Spain)

10

Communications in Iberdrola. Architecture

Secondary Substation

LV

PRIME -Millions -Not IP (narrowband)

ACCESS NETWORK - Thousands

- Ethernet/IP

BACKBONE - Hundreds - Ethernet

Telecommunications Network

The Smart Grid …

11

• And how does the Smart Grid affect us in terms of telecoms?

Pervasive reach: due to the distributed nature of electricity assets.

Bandwidth: depending on the services needed (i.e. Smart Grids), bandwidth will be different.

Two way comms. and interactivity: the communications have to be from the system to the devices, and vice versa.

Scalability and flexibility: subject to changing demands and concepts, solutions must be flexible.

Reliability, Security and Safety: the critical nature of the assets need such an approach.

Electricity utilities own assets that configure a privileged situation for the provision of telecommunication services

However …

Communications in Iberdrola. Challenges

• Who is going to pay for the Remote Management?

• Who is going to pay for the Smart Grid?

• Which regulatory changes will affect the evolution of Smart Grid?

• Technologies almost fine-tuned. Integration is the biggest concern.

• Large amounts of devices from manufacturers.

• Changes are going to take place in all parts of the network, and day-to-

day activities will be modified. Changes in processes.

• Deployment works will be taking place in a non-greenfield

environment. Economic deviations must be closely controlled.

12

Which elements condition telecom technologies planning and programming:

Communications in Iberdrola. Concerns

13

IP free

Communications in Iberdrola. Integration approach

Head End

Private, Fixed / Mobile

Network

S

S

PLC: PRIME

PLC: MV (BPL)

PUBLIC NETWORK

Public Network

Public Network

Public Network

Fixed/Mobile Public Network

PLC: PRIME

PLC: PRIME

SS

SS

SS

Private, Fixed / Mobile

Network

MPLS/SDH/FO

√ Efficiency (€) is the key when selecting the proper solution

14

• IP (Internet Protocol) plays a fundamental role in Communications Systems, due to its wide adoption.

• However, there are some communication environments (narrowband PLC channels, narrowband VHF&UHF radio channels) where IP might not be the best solution:

Because channels are usually narrowband: IP was born in a broadband world.

Because its overhead both in terms of payload, and in terms of message exchange, cannot be supported and at the same time provide the best performance.

Because IP protocol might not be providing any added value (just cost).

• In these cases, end to end IP solutions are not probably the most cost efficient solutions. Communication Systems can provide gateways and/or adaptation layers to external networks using IP:

There is no need to use IP internally in some systems as adaptation layers can get interoperability.

Narrowband systems can avoid the unnecessary burden of the broadband-engineered IP protocol message exchange.

Communications in Iberdrola. IP everywhere?

MV

BPL

15

ACCESS

NETWORK

Telecommunications Network

Communications in Iberdrola. Access Technologies

Eth / IP Based

16

Communications in Iberdrola. Access Techn: BPL vs. Public Radio

L2 (BPL)

L3 (GPRS)

17

Communications in a Utility. Technology Comparison

• BPL vs. GPRS comparison:

• Performance example: – Firmware Update (FW). Data Concentrator:

FW image = 1,5 MB; Minimum throughput GPRS = 20 min.; BPL = 2 min.

Maximum throughput GPRS = 10 min.; BPL = 6 s.

– Data Concentrator web page:

Web page = 0,05 MB; Minimum throughput GPRS = 40 s.; BPL = 4 s.

– Telecontrol status update (with data) :

Minimum throughput / Maximum Latency GPRS = 4,88 s.; BPL = 0,81 s.

Maximum throughput / Minimum Latency GPRS = 2,64 s.; BPL = 0,33 s.

GPRS BPL

Throughput 10 kbps – 20 kbps 100 kbps – 2 000 kbps

Latency 400 – 700 ms 80 – 150 ms

(x 10, x 100)

18

Communications in Iberdrola. Increasing Data Volumes

Data exchange

ME

TE

RIN

G &

DS

M

NE

TW

OR

K

IT &

AP

PL

ICA

TIO

NS

PLC Home Area Network Detail

Telecomm.

RTU IP

Camara Sensors

Actuators

Secondary Substation

kbps

Mbps

Gbps

Mbps

Scada Planning

Serv Comm. Serv Comm.

Datawarehouse

Meter Managemnt

Points of Supply

TSO

Maintenance

Mobile Wf

Retailers

Telecommunication Network

19

Index

Evolution

towards

Ethernet

20

Evolution towards Ethernet. Transport

• Backbone technology evolution: Transport

SDH: STM 16 (2.5 Gbps)

STM 64 (10 Gbps)

PDH: 34 Mbps DWDM: 40 λ (1.6 Tbps)

λ =40, 10, 2.5, 1 Gbps

21

PDH

SDH

DWDM

TYPES OF SERVICES

TELECONTROL

TELEPHONE

TELEPROTECTION

OTHERS, non critical services

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Total 520 107 115 191 516 532 912 344 335 219 202 112 104 98

0

100

200

300

400

500

600

700

800

900

1000

Number of provisioned services per year

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

External 31 120 178 205 137 167 57 61 42 44 16

Internal 21 37 53 107 120 235 434 369 339 302 363

0

100

200

300

400

500

600

Num

ber

of s

ervic

es

Number of provisioned services per year

2008 2009 2010 2011 2012

External 0 18 20 18 0

Internal 6 6 4 8 4

0

5

10

15

20

25

30

Nu

mb

er

of

se

rvic

es

Number of services per year

2005

2010

2008

Evolution towards Ethernet. Transport Circuit Evolution

A huge investment to take into consideration: Ethernet transport

22

Protocol towers

Time

Fibre Fibre Fibre Fibre Fibre Fibre

Level 2

Level 1 Fibre

Evolution towards Ethernet. Protocols

23

Evolution towards Ethernet. Switching

• Backbone technology evolution: Switching

Point to point: 34 Mbps Frame Relay/ATM: 155 Mbps

15 years

MPLS: 622 Mbps, 1/2/10 Gbps

6 years

24

VPLS Services

Evolution towards Ethernet. “Extended” Local Area Networks

25

• MPLS Network Management: Transport-like system

Evolution towards Ethernet. Management

26

– Layer 2 based MPLS core.

– Types of service offered:

• VPLS (Virtual Private LAN Service) Ethernet Multipoint Wide Area Network.

• VLL (Virtual Leased Line): Ethernet point to point circuit using MPLS network.

2004 2005 2006 2007 2008 2009 2010 2011 2012

VPLS Services 12 2 35 28 40 42 90 81 395

VPLS Access Point 46 19 124 485 904 1062 949 1279 988

0

200

400

600

800

1000

1200

1400

Num

ber

of V

PLS

and

Acc

ess

Number of Access Points and VPLS

Evolution towards Ethernet. MPLS Services

27

Index

MPLS as

Extended

LANs basis

28

MPLS and Extended LANs

IBERDROLA MPLS NETWORK

29

• Direct Ethernet connectivity: Fiber Optics and Ethernet over SDH or DWDM.

• Ethernet extension from the core to the border: EthoSDH, and optical switches.

• Iberdrola matches VLANs with VPLS (administrative purposes): each VLAN is a service (VPLS). Simplification

• MPLS allows QoS. Iberdrola supports QoS:

– Quality to specific Services.

– Mapping with DSCP (IP field) and Quality (internal customers).

• Single device type: spares, training, …

• Just Ethernet services (no PDH, RS232 over Eth encapsulation, …).

• L2 MPLS provides the provision simplicity we need.

MPLS and Extended LANs. Keep it simple

30

Main and Backup Systems

– Criteria: Infrastructure for Redundancy

MPLS in Iberdrola. Control Systems

31

Corporate Systems

– Data Center Connectivity

Substation Connectivity

Generation Control Centers Connectivity

Renewable Premises Connectivity

…

MPLS in Iberdrola. Data Centers

32

Switching and IP network:

– Switches: (LANs are straight-forward to provision)

– Routers: (Smart Grid derived concentrations: scalability reasons)

Switches Routers

Teldat 1452

Ruggedcom 764 0

ZIV 1146 2178

NSN 1251

0

500

1000

1500

2000

2500

3000

3500

4000

Nu

mb

er

of

de

vice

s

Switches and Routers

40%

36%

24%

Switch Providers

NSN ZIV Ruggedcom

60%

40%

Router Providers

ZIV Teldat

MPLS in Iberdrola. MPLS to the Border?

33

MPLS in Iberdrola. And … what next?

EthoSDH,oTDM & L2MPLS Transprt & Reuse & Simple Oper Network Evolution

Transport & L2

Transport & Fiber Optic

Network Evolution

34

Conclusions

• Iberdrola takes profit from carriers’ carrier model activities: (1) see “outside the box” & (2) develop private infrastructure.

• MPLS has been the technology that, together with transport capabilities, has allowed Iberdrola to provide a simple implementation of Ethernet service provision anywhere:

– Iberdrola soon realized that PDH platforms, even if appropriate for low speed services, would not allow to take benefit from mass market prices. This fact drove us to install SDH, and soon NGN-SDH (Eth).

– Iberdrola soon realized that Switching platforms would allow to simplify operational models and service provision: low number of premises, but widely spread.

• There is a big question mark on the evolution of MPLS networks together with the evolution of Transport Systems.

• Smart Grids are changing the status quo with tens of thousands of premises, and millions of smart meters.

• Cost is a critical concern when we need to change the scope from hundreds, to thousands of premises.