Page 1
Horizon 2020 – LCE-2017 - SGS
FLEXCoop
Democratizing energy markets through the introduction of innovative
flexibility-based demand response tools and novel business and market models
for energy cooperatives
WP5 – Open Demand Response Optimization Framework and
Tools for Aggregators
D5.6 – FLEXCoop Flexibility Forecasting,
Segmentation and Aggregation Module –
Final Version
Due date: 31.05.2020 Delivery Date: 18.09.2020
Author(s): Valalaki Katerina (Hypertech), Makris Stefanos (Hypertech), Ververidis
Christoforos (Hypertech)
Editor: Valalaki Katerina (Hypertech)
Lead Beneficiary of Deliverable: Hypertech
Contributors: Hypertech, Suite5, CIMNE, SomEnergia, MERIT
Dissemination level: Public Nature of the Deliverable: Demonstrator
Internal Reviewers: Peder Bacher (DTU), Dimitris Panopoulos (Suite5)
Page 2
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 2 of 22
FLEXCOOP KEY FACTS
Topic: LCE-01-2016-2017 - Next generation innovative technologies
enabling smart grids, storage and energy system integration with
increasing share of renewables: distribution network
Type of Action: Research and Innovation Action
Project start: 01 October 2017
Duration: 36 months from 01.10.2017 to 30.09.2020 (Article 3 GA)
Project Coordinator: Fraunhofer
Consortium: 13 organizations from nine EU member states
FLEXCOOP CONSORTIUM PARTNERS
Fraunhofer Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
ETRa ETRA INVESTIGACION Y DESARROLLO SA
HYPERTECH HYPERTECH (CHAIPERTEK) ANONYMOS VIOMICHANIKI
DTU DANMARKS TEKNISKE UNIVERSITET
GRINDROP GRINDROP LIMITED
CIRCE FUNDACION CIRCE CENTRO DE INVESTIGACION DE RECURSOS
Y CONSUMOS ENERGETICOS
KONCAR KONCAR - INZENJERING ZA ENERGETIKUI TRANSPORT DD
SUITE5 SUITE5 DATA INTELLIGENCE SOLUTIONS Limited
S5 SUITE5 DATA INTELLIGENCE SOLUTIONS Limited
CIMNE CENTRE INTERNACIONAL DE METODES NUMERICS EN
ENGINYERIA
RESCOOP.EU RESCOOP EU ASBL
SomEnergia SOM ENERGIA SCCL
ODE ORGANISATIE VOOR HERNIEUWBARE ENERGIE DECENTRAAL
Escozon ESCOZON COOPERATIE UA - affiliated or linked to ODE
MERIT MERIT CONSULTING HOUSE SPRL
Disclaimer: FLEXCoop is a project co-funded by the European Commission under the Horizon
2020 – LCE-2017 SGS under Grant Agreement No. 773909.
The information and views set out in this publication are those of the author(s) and do not
necessarily reflect the official opinion of the European Communities. Neither the European
Union institutions and bodies nor any person acting on their behalf may be held responsible for
the use, which may be made of the information contained therein.
© Copyright in this document remains vested with the FLEXCoop Partners
Page 3
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 3 of 22
EXECUTIVE SUMMARY
This deliverable is the outcome of the T5.2 “Demand Segmentation, Clustering and
Classification Methods and Module” and it is a demonstrator deliverable. It comprises of the
design and implementation of an analytics platform, a tool for the Aggregator. Its main focus
is the integration of forecasting, segmentation and clustering functions into the aggregator
operations, while satisfying the goal of reduced complexity that comes from organizing profiles
into clusters presenting similar behaviours. The clustering is based on individual household
parameters, taking into account information related to the building infrastructure and DER
device availability, the location, contracts with aggregator, etc. The main objective is to
facilitate the management of the consumer demand and to provide the aggregator important
information, which will help to decide and forecast upon:
the participation on specific Demand Response programs
optimal break down of Demand Response signals
Clustering in this sense is intended as a pre-processing and data reduction step as a basis for
sub-sequent prosumer-level feature extraction and segmentation. More specifically, the goal is
to identify a diverse set of typical responses that can be described by a cluster representing
different device characteristics and prosumer-to-prosumer consumption schedules and
flexibility potential.
The module designed and developed in this task comprises a powerful tool for aggregators
(energy cooperatives) to facilitate the management of the consumer demand and flexibility
profiles in order to forecast and decide upon the optimal breakdown of DR signals to device
control actions. This would be done while satisfying the goal of reduced complexity that comes
from organising profiles into clusters of homogeneous behaviour. This FLEXCoop module will
cluster and segment residential buildings / assets based on various criteria defined by the
aggregator and forecast the aggregated demand flexibility incorporating information about
building infrastructure and use by occupants. This approach leverages the existing tool structure
(segmentation, clustering, forecasting) since a number of indirect estimations are replaced by
real measurement-based calculations.
The module allows for multi-parameter criteria analysis over aggregators’ portfolios,
addressing technical as well as business potential. The tool allows for the historical analysis of
demand flexibility data (as acquired from the FLEXCoop pilot sites) enabling the realization of
forecasting, segmentation and clustering functions as part of the aggregator operations
introduced in the project.
Page 4
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 4 of 22
Table of Contents
FLEXCOOP KEY FACTS ................................................................................................................................... 2
FLEXCOOP CONSORTIUM PARTNERS ....................................................................................................... 2
EXECUTIVE SUMMARY ................................................................................................................................... 3
LIST OF FIGURES .............................................................................................................................................. 5
LIST OF TABLES ................................................................................................................................................ 5
CODE LISTINGS ................................................................................................................................................. 5
ABBREVIATIONS ............................................................................................................................................... 6
1. INTRODUCTION ............................................................................................................................................. 7
2. THE DEMAND SEGMENTATION, CLUSTERING AND CLASSIFICATION IN THE FLEXCOOP
ARCHITECTURE ................................................................................................................................................ 7
3. VERSIONING/VERSION OF THE SOFTWARE DEMONSTRATOR ..................................................... 8
4. RELEASE DATE .............................................................................................................................................. 8
5. RELEVANT LICENCES USED IN THE DEMONSTRATOR ................................................................... 8
6. DESIGN AND ARCHITECTURE .................................................................................................................. 8
7. PROGRAMMING LANGUAGE .................................................................................................................. 11
8. CONTENTS OF THE CURRENT RELEASE ............................................................................................. 11
9. SOURCE CODE OF THE RELEASE .......................................................................................................... 19
10. RELATED DOCUMENTATION ................................................................................................................ 19
11. INSTALLATION GUIDE ............................................................................................................................ 19
12. USER GUIDE ................................................................................................................................................ 19
13. INTERFACES WITH OTHER COMPONENTS AND THEIR INTEROPERABILITY ..................... 19
14. REQUIREMENTS COVERAGE ................................................................................................................ 21
15. DEVELOPMENT AND INTEGRATION STATUS .................................................................................. 22
16. CONCLUSION.............................................................................................................................................. 22
17. BIBLIOGRAPHY ......................................................................................................................................... 22
Page 5
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 5 of 22
LIST OF FIGURES
Figure 1: FLEXCoop Conceptual Architecture Design ............................................................. 7
Figure 2: Flexibility Forecasting, Segmentation and Aggregation Module Architecture Design
............................................................................................................................................ 9
Figure 3: Sequence diagram showing interactions between constituent parts ......................... 10
Figure 4: Virtual Power Plant (VPP) view of the aggregator UI ............................................. 16
Figure 5: Grouping devices based on location and capability in participating in specific markets
.......................................................................................................................................... 17
Figure 6: Calculate the aggregated flexibility of a cluster of devices ...................................... 18
Figure 7: Create a cluster with the most reliable devices ......................................................... 18
Figure 8: Sequence diagram showing the interaction between the different modules of the
FLEXCoop architecture. .................................................................................................. 20
Figure 9: SwaggerUI – FFSA derClusterResponse endpoint ................................................... 21
LIST OF TABLES
Table 1: Requirements Coverage ............................................................................................. 22
Table 2: Development and integration status ........................................................................... 22
CODE LISTINGS
Table 1: Requirements Coverage ............................................................................................. 22
Table 2: Development and integration status ........................................................................... 22
Page 6
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 6 of 22
ABBREVIATIONS
aFRR Automatic Frequency Restoration Reserve
API Application programming interface
CO Confidential, only for members of the Consortium (including the Commission Services)
D Deliverable
DER Distributed Energy Resources
DR Demand-Response
DSO Distribution System Operator
DoW Description of Work
EC European Commission
EU European Union
EV Electric Vehicle
FFSA Flexibility Forecasting, Segmentation and Aggregation
G2V Grid-to-Vehicle (G2V)
UI User Interface
H2020 Horizon 2020 Programme
ICT Information and communication technology
MOM Message Oriented Middleware
OSB Open Smart Box
PV Photovoltaic
RES Renewable Energy System
TSO Transport System Operator
V2G Vehicle-to-Grid
VPP Virtual Power Plant
WP Work Package
Page 7
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 7 of 22
1. INTRODUCTION
The FLEXCoop Flexibility Forecasting, Segmentation and Aggregation (FFSA) module has
been designed as the component responsible for the analysis, correlation and efficient
management of prosumer flexibility profiles and response capacity to DR signals.
The module’s key role is to seek for the most efficient Distributed Energy Resources (DERs)
for each service request based on criteria defined by the aggregator through the aggregator User
Interface (UI). To achieve this, the module performs portfolio segmentation based on pre-
defined criteria like the location of the building, the individual consumption of the consumers
and the capability of different loads to provide different flexibility services (e.g. participation
in aFRR market, self-consumption, wholesale market). In practice, the aggregators are offered
flexibility-based portfolio clustering services, according to the service/market that they want to
participate. The module is implemented as a cloud-based service, allowing for easy deployment
and maintenance procedures.
Considering that this deliverable is a demonstrator, the main delivered product is a piece of
software. This deliverable is a descriptive document with the objective of providing details
about the functionalities, software architecture and respective interfaces used for the design and
development of this component as well as with its communication with other components
(through the middleware). It also provides the endpoints for accessing the services provided by
the FFSA module.
2. THE DEMAND SEGMENTATION, CLUSTERING AND CLASSIFICATION IN THE FLEXCOOP
ARCHITECTURE
With respect to the FLEXCoop Architecture shown in Figure 1, the FFSA is located at the
Aggregator side (indicated with a red frame). It is a back-end system whose results can be
visualized in the Visualization – Aggregator Toolkit module, while its functionalities will be
used by the concerned components using the related endpoints (described in Section 13).
Figure 1: FLEXCoop Conceptual Architecture Design
Prosumer Portal
IEC 61850 Server/ DER
Management System
Open Marketplace ViewDER Registry View
Visualization / Prosumer Toolkit
District DER Systems
(Generation, Storage, EV)
External Data (Wholesale
market, Weather)
Electricity / Gas/ DH Network
Operator System
USEFWeb-
services
Proprietary field area equipment smart devices
OSBHVAC, Light, DHW
controlAmbience Sensing
OneM2M, Zigbee,
Z-wave, etc.
GDM View Open Marketplace ViewDER Registry View Other Information Services
Flexibility forecasting, segmentation and
aggregation
Global Demand Manager
DR Settlement / Remuneration
Local Demand Manager
DER Registry
Open Marketplace
Certificate Service
OAuth2 /OpenID
Message Oriented Middleware
DNS Resolver
VTES
ModuleContext-aware
flexibility profilingEV flexibility
module
Local flexibility analysis and forecasting
Prosumer energy behavior and comfort
Demand Flexibility Profiling
Visualization - Aggregator Toolkit
Page 8
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 8 of 22
3. VERSIONING/VERSION OF THE SOFTWARE DEMONSTRATOR
This is the final version of the FFSA component, according to the Description of Work (DoW).
4. RELEASE DATE
The final version of the Flexibility Forecasting, Segmentation and Aggregation (FFSA) Module
was released at 30.05.2020.
5. RELEVANT LICENCES USED IN THE DEMONSTRATOR
The Flexibility Forecasting, Segmentation and Aggregation Module has been implemented using
software packages with Open Source licenses. The relevant software packages and libraries used
are listed below:
Spring MVC framework v4
Java v1.8 / J2EE
Hibernate ORM v5
WEKA library: Weka provides implementations of learning algorithms that can be applied
to a dataset. It also includes a variety of tools for transforming datasets, such as the
algorithms for discretization and sampling. Pre-processing of a dataset, feeding into a
learning scheme, and analyzing the resulting classifier and its performance can be
performed using this library.
MySQL v8
6. DESIGN AND ARCHITECTURE
In this part of the deliverable, we concentrate on the technical aspects of the module, namely
its design and implementation. To this end, we first reproduce the component diagram (updated
to reflect the final status) of the FFSA engine in the figure below, as was defined in Deliverable
2.6 “FLEXCoop Framework Architecture including functional, technical and communication
Specifications - Preliminary Version” and updated in D2.9 “FLEXCoop Framework
Architecture including functional, technical and communication Specifications - Final
Version”.
Page 9
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 9 of 22
Figure 2: Flexibility Forecasting, Segmentation and Aggregation Module Architecture
Design
The FFSA contains three basic submodules, activated upon request by its interface with the
Middleware and have the following functionalities:
Segmentation Module: This submodule when triggered by the Aggregator in the
Aggregator toolkit (through the Middleware) takes as input the DER related data
obtained by the Data Management Layer (see Figure 2) and activates the segmentation
and clustering services based on specific pre-defined criteria. The criteria that are
available for this final version of the demonstrator are:
o Location
o Service type (i.e. aFRR, self-consumption, wholesale)
o Customer reliability (i.e. customer’s response in previous DR events)
Page 10
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 10 of 22
o Contract type and validity (i.e. active, canceled. On hold)
Forecasting Module: This submodule, when there is a request executes the forecasting
functionality taking as input DERData obtained by the Data Management Layer. In
particular, provides a forecast of aggregated flexibility (and a baseline) for all the assets
/ devices that belong to specific segments as these have been defined by the
segmentation module based on the criteria as defined above. The forecasting period (e.g.
24 hours), the start date and time (e.g. 15/09/2020 00:00:00) and timestamp (e.g. 15
min) is determined in the relevant request.
Outliers Engine: In the same manner, this submodule is responsible for the outliers’
detection. This can be for example the identification of a specific type of assets that for
some reason (e.g. occupant is on holidays) cannot provide flexibility for a long period
of time.
The relevant services provided by the FFSA shown in Figure 2 are described in more detail
Section 8 while the relevant endpoints are provided in Section 13.
In the following sequence diagram, we illustrate interactions between module’s constituent
parts.
:Aggregator :Segmentation
Module
Request portfolio segmentation
for a DR service based on defined criteria
:Forecasting
Module
Request DER device information
:Data Management
Layer:MOM
Middleware
:Aggregator UI
:Outliers Engine
Request flexibility forecasting for the
identified segments
Identify outliers
Request device information
Portfolio segmentation
Store information temporarily
Retrieve Segment Information
Forecasting
Figure 3: Sequence diagram showing interactions between constituent parts
The FFSA module includes some further exploratory tools for the aggregators to perform
pattern analysis over specific metrics. The analysis of the results can assist towards the
definition of suitable DR strategies. The additional parameters include spatial grouping and
time-dependent similarity clustering. This allows for capturing typical dynamic characteristics
that are observed in specific prosumer segments, thus, enable the aggregator to identify
similarities between different groups of prosumers belonging to his portfolio. Aggregator may
need to perform such clustering for a full period (e.g. a whole day) or instead to conduct
customer segmentation for different adjacent periods (e.g. towards evaluating the noon peak
shaving potential of each prosumer).
Page 11
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 11 of 22
7. PROGRAMMING LANGUAGE
The entire backend module has been developed using Java and the whole framework consists
a J2EE application.
8. CONTENTS OF THE CURRENT RELEASE
As mentioned above, the current version enables the dynamic and spatio-temporal segmentation
and clustering of devices for the selection of appropriate aggregated demand side-based VPPs
to provide specific DR functions (considering flexibility profiles and their suitability to provide
different grid services, e.g. balancing).
The module embeds functionalities for properly segmenting, classifying, clustering and
managing demand and demand flexibility profiles in order to establish optimal VPP
composition for the delivery of different services to the aggregator. Its main innovation is that
rather than matching the assumed demand profile of a specific building to a generic class and
then extracting demand flexibility estimations, this tool clusters and segments residential
prosumers based on specific criteria defined by the aggregator and then calculates the actual,
locally estimated and context-aware demand flexibility (based on real-time data coming from
the WSN capturing contextual characteristics i.e. indoor temperature and occupant thermal
comfort) incorporating detailed information about building infrastructure, use by prosumers
and more importantly prosumer preferences.
Workflow
In this section, we provide information and workflows that can be performed exploiting the
functionalities of the FFSA module.
In an explicit DR program, the aggregator opts to concentrate a considerable volume of
aggregated dispatchable load in order to bid it in energy markets as it were a potential generation
unit. The FFSA module is a component that analyses different categories of dispatchable loads
with purpose to address them in different energy markets according to their properties. For
example, in ancillary services and balancing markets fast response of load dispatching is more
crucial than the total capacity of demand reduction. On the other hand, in wholesale energy
markets, the aggregator can bid significant volumes of consumer demand against high-cost
production units. The main objective of the aggregator is always to bid in lower prices
comparing to the bid price of the generation unit that clears the energy market. Under these
circumstances, the aggregator will be profitable and will distribute the incomes to each one of
the individual contracted consumers.
In this section, the methodology used for the development and implementation of the FFSA
module is presented. As mentioned before, the FFSA module is a component that concentrates
the demand and demand flexibility from each one of the individual consumers in the
aggregator’s demand response portfolio and supports the definition of the aggregator’s strategy
in response to the energy markets or the DSO/TSO demands. Therefore, the engine will have
to administrate and appropriate categorize a portfolio of different loads and facilitate the
optimal scheduling that is finally performed by the Global Demand Manager for load
dispatching with final purpose to maximize aggregator’s profits.
Page 12
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 12 of 22
Before we proceed with the module description, we provide some main assumptions of our
methodology taken for this first current version of the demonstrator:
All consumers and their building assets are already contracted with the aggregator and
can provide information about the flexibility potential at the appropriate time interval.
In general, explicit DR programs require strictly defined Service-level agreement
between the consumer and the aggregator while consumers have to give direct
information and, if it’s required, load control of their energy resources to aggregators
for demand profile modification in response to grid requests under exceptional
circumstances.
The aggregator will directly control the buildings’ devices in a Direct Load Control
fashion (automated DR). This approach yields more predictable and reliable results,
but also places constraints on the availability of tele-command infrastructure [1].
Building assets must be registered to Aggregator’s asset portfolio so that the aggregator
is able to build its services upon a strong and robust portfolio.
The operation of the component is based upon the following methodology.
The Aggregator signs a framework contract with all consumers which are willing to participate
in explicit demand response programs. This framework contract should define the operating
conditions for the explicit demand response service executed by the aggregator and corresponds
with the conditions agreed between the aggregator and the consumer. Aggregator’s contract
with each one of the consumers should define the operational modalities, e.g.
minimum/maximum quantity, delivery period, activation time and number of repetitions. For
more detailed information on the contract mechanism can be found in the D6.6 “FLEXCoop
Open Marketplace for flexibility Pooling and Sharing – Final Version”. The fields from the
marketplace that are used by the FFSA are: if the contract includes PV, list of devices included
in a contract, type and status of the contract (for more specific information of the FFSA inputs/
outputs and the respective data model please refer to the D4.7 “FLEXCoop Common
Information Model – Final Version”). Based on this agreement and the information coming
from the registered DER devices (for more detailed information please refer to the D6.5
“FLEXCoop Semantically Enhanced DER Registry – Final Version”), the FFSA module has a
detailed description of aggregator’s assets portfolio. The fields from the DER Registry that are
used by the FFSA are: the id identifying each device, the ven_id identifying the OSB, the
aggregator id and the location of the device (for more specific information of these
inputs/outputs of FFSA and the respective data model please refer to the D4.7 “FLEXCoop
Common Information Model – Final Version”).
The aggregator has a few options regarding the beneficial exploitation of the aggregated loads.
Aggregator’s strategy may select to bid the dispatching loads in wholesale markets (day-ahead
as in the Spanish case examined in the project or intraday) or to bid them in balancing markets
(as in the Dutch case examined in the project). As a consequence, the first approach of the
engine would be to analyze all aggregator’s assets based on load dispatching characteristics and
categorize them according to the aforementioned demands. The grouping of devices into types
has been based on [2] and adopted to cover FLEXCoop use cases.
Page 13
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 13 of 22
In FLEXCoop, we consider as potential demand response sources only residential buildings.
From DER devices considered for the households we can identify the following categories (see
also D3.5 “Local Demand Manager Specifications and Intra-building Optimization
Algorithms”):
Inductive Thermal loads: In general, thermal loads from HVACs can be a significant
source for demand response due to the huge total amount of electrical power consumed
in residential buildings for heating and cooling. Such thermal loads are usually not
steady and they vary during the operational cycle of the thermal device depending on
the operation mode. Controlling a big number of thermal assets can resort to a massive
aggregated load capacity capable to participate in energy markets.
Resistive Thermal loads: Purely resistive loads, such as electric heaters used either for
space heating or for Domestic Hot Water production, offer immediate control
capabilities, but are less common. They can be utilized for participation in most services
and markets.
Lighting loads: Lighting devices can deliver an easily controllable load with almost
immediate response but the aggregated load is significantly low comparing to thermal
loads. In this case, the delivered flexibility is extracted by dimming or switching off
lighting devices but always taking under consideration consumers comfort boundaries.
This type of loads can be aggregated and participate in ancillary services and balancing
markets where fast response is the most crucial demand while the requested load
dispatching quantities are particularly low.
Battery storage loads: Installed battery packs as building assets are without doubt the
ideal devices for explicit demand response services. Battery storage has the fastest
activation time and they are the most reliable among others load devices. High capacity
battery packs are a high-value asset in aggregator’s portfolio and they can participate in
every potential energy market and service due to fast response together with high
capacity and reliability. The main drawback is the high cost of their implementation,
especially in residential buildings, and the necessity of investment depreciation in a
relatively small time. Aggregators should opt to contract consumers with battery storage
assets and give them incentives to participate in explicit demand response programs.
Electric Vehicles (EVs): An important support in Demand Response (DR) programs
can be provided by EVs. EVs may behave as a load to the grid, a supplier of electricity
to the grid or an energy storage device. Two types of interactions are possible between
an EV and the power grid, Grid-to-Vehicle (G2V) and Vehicle-to-Grid (V2G). In G2V,
an EV battery can be charged from the grid using stored electricity originating from
external power sources, i.e., the power flow is always unidirectional. In V2G, the power
flow is bidirectional, i.e., from the grid to an EV while charging and from an EV to the
grid while discharging. V2G-enabled EVs earn incentives while discharging power to
the grid, and make payments while charging batteries from the grid. Thus, V2G-enabled
EVs can facilitate the supply/demand balance by discharging during peak hours (peak
shaving) and charging during off-peak hours (valley filling)
Page 14
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 14 of 22
Rooftop PVs. PV production will be monitored and taken into account in the overall
optimization to demonstrate the FLEXCoop use cases and business scenarios but they
will not be controlled in pilot demonstration phase. The production of rooftop PVs (that
are found in the pilot users) will be continuously measured and the measurements will
be used for PV production forecasting that will be further included in the overall
FLEXCoop optimization mechanism. However, curtailment at local level is not possible
and, in any case, it is out of the scope of the project itself. PV monitoring and production
forecasting will be used for the self-consumption scenario in Spain.
Based on this classification aggregator can use the FFSA module to cluster portfolio assets
according to the energy market that each individual asset opts for. For the aggregator to be able
to participate in more than one energy markets, there should be always available an assets
estimation based on a first generic classification.
Furthermore, aggregator may want to select assets for participation in demand response
campaigns that are the most promising ones for a specific campaign. This can be facilitated by
a methodology that will segment assets into categories for facilitating the selection of the best
assets to utilize in a particular campaign. The segmentation will be performed based on
applicable criteria which can indicatively include their amount of flexibility, location and
reliability. This procedure can be utilized every time there is a change in aggregator’s portfolio
or whenever is deemed necessary by the aggregator. In the current implementation, the
aggregator UI updates this information with a request in the middleware every day.
In addition, a classification that may be performed in order to deliver an explicit demand
response service can be based depending on the capability of an asset to provide (indicatively):
Peak Power Reduction: This is related to the peak shaving of electricity loads and the
peak power reduction at different time periods. If this is the scope, the FFSA module
should find the assets that can deliver the maximum peak power reduction (based on an
analysis over historical data). For example, the performance of peak power should be a
priority when the aggregators want to optimise their portfolio for participating in
Ancillary Services and Balancing Markets like the Dutch case (which requires a specific
load dispatching for relatively small time periods).
Energy Savings: This is related to the volumes of energy involved to the demand
response program in a time period. It assesses energy efficiency and reduction of
electricity consumption. Energy saving is related to the environmental benefit in terms
of greenhouse gas emissions.
User engagement: This assesses the user engagement to the explicit demand response
program and they are actually a measure of reliability. It includes an assessment of how
frequently consumers have delivered their demand flexibility and what time periods
consumers perform utmost.
Having described the basics of the workflow and the methodology considered in the realisation
of the FFSA, we proceed in describing the exact functionality delivered with this final version
of the FFSA demonstrator.
Page 15
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 15 of 22
The engine receives from the demand flexibility profiling engine (through the middleware) as
an input the demand flexibility calculations for building assets periodically (once per day). The
propagated information is actually timeseries of demand flexibility calculations for every
predefined time period. In addition, the FFSA module obtains historical data of assets
performance from participation in past demand response campaigns. Then, depending on
historical data, the engine evaluates the reliability of each asset (see more information of the
algorithmic framework presented in the D5.2 “FLEXCoop Flexibility Forecasting,
Segmentation & Aggregation Module – Preliminary Version “). Reliability refers to the level
of response (in percentage) at previous demand response requests.
This information can be utilized by the aggregator to group consumer’s assets for every
predefined time period. For example, we may assume the case where the FFSA requests for
day-ahead demand flexibility as a timeseries for identically divided time periods. Clustering of
important criteria per consumer can be performed independently on different time periods
during the day (when requested). This quantization of time enables the aggregator to focus on
the desired time period and assess the benefits of the various assets at the period of interest.
This eliminates the chance that asset characteristics that do not apply in the specific period are
taken into account (e.g. a home with significant flexibility during the evening cannot deliver
any during mid-day when all occupants are away and all-important loads are switched off). Per
time period the clusters will probably comprise different assets.
Concluding the FFSA module will perform the following procedures:
Cluster consumers and their assets based on their potential participation in wholesale
energy markets or in ancillary services and balancing markets or other criteria defined
by the aggregator (i.e. location, contract validity, customer reliability)
Classify consumers and their assets based on their demand flexibility calculations and
historical data for participating in a demand response campaign at a specific time period.
For every time period, the engine will present a list with consumers and assets as a pool
for demand flexibility.
Provide demand flexibility forecasts for assets that belong to requested clusters as well
as aggregated flexibility estimation of requested segments. The forecasting period (e.g.
24 hours), the start date and time (e.g. 15/09/2020 00:00:00) and timestamp (e.g. 15
min) is determined in the relevant request.
Perform outliers detection
Although this is a back-end component, part of its functionality can be visualised in the
FLEXCoop aggregator UI (please refer to the D5.9 “FLEXCoop Aggregator real-time
Monitoring and Control Platform - Final Version” for more detailed information). As detailed
in D5.9, the aggregator GUI can be used accessing its public URL:
https://flexcoop.etra-id.com/
As regards the credentials for logging in, please refer to the D5.9.
For visualising the results coming from the functionality of the FFSA components, the users
should visit the Virtual Power Plane (VPP) tab, as shown in the Figure 4 below.
Page 16
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 16 of 22
Figure 4: Virtual Power Plant (VPP) view of the aggregator UI
As described in D5.9, the Virtual Power Plants tab visualises the grouping of the entire portfolio
in clusters to specify the criteria of the aggregator. This information is updated once a day (this
is the message shown at the bottom side of the screen).
This screen shows different tables containing information about the assets (devices) in relation
to the data reliability (indicated on the header). The information presented on each table is:
Name of the cluster: A human readable identifier for the group of assets contained on
that cluster.
Number of devices: How many different devices are contained on that cluster.
Number of OSBs: How many different OSBs are contained on that cluster (it has to be
reminded that an OSB can control more than one device).
Total upwards flexibility: The forecasted upwards flexibility for the presented day on
that group.
Total downwards flexibility: The forecasted upwards flexibility for the presented day
on that group.
The actual flow for updating this information is shown in the sequence diagrams below. Within
the current implementation, the flow will be followed once per day providing meaningful
information for the identified clusters for the next day. However, the FFSA implementation
allows for providing the respective information in different time horizons and intervals (if
requested).
This means that each day, the aggregator will be provided through the aggregator UI shown in
Figure 4 with:
The different clusters that will include devices able to provide different services to an
aggregator. Services examined in the FLEXCoop project are:
o Balancing services and in particular aFRR market
Page 17
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 17 of 22
o Self-consumption at individual prosumer level
o Wholesale market
Timeseries of aggregated flexibility (upwards / downwards) that is forecasted for the
next day as an aggregation of the flexibility of individual device flexibility profiles
The average reliability of a reliability cluster including the most reliable devices (as this
is extracted from their performance during the past DR campaigns)
Although the visualisation provided in the aggregator presents a more high level information in
order for providing a quick user-friendly overview of the portfolio to the aggregator, the
functionality of the FFSA with the detailed listing of devices is provided to the Global Demand
Manager (GDEM) for supporting also its functionality for VPP formulation and participation
of devices in DR Campaigns.
:Aggregator :Flexibility forecasting,
segmentation..
Request for clustering
based on various criteria
Provide Contracts
Return a cluster including a list of devices
Request Contracts
Search for registered DERs and their characteristics
Provide registered DER-related data
:MOM
Middleware:MOM
Middleware
:Aggregator UI
Identify devices covering the criteria
Figure 5: Grouping devices based on location and capability in participating in specific
markets
Page 18
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 18 of 22
:Aggregator :Flexibility forecasting,
segmentation..
Request for aggregated flexibility
forecast
Return an aggregated flexibility timeseries
:MOM Middleware:MOM
Middleware
:Aggregator UI
Provide Individual Flexibility Profiles
Request for individual flexibility profiles
Aggregate Flexibility Forecast
Figure 6: Calculate the aggregated flexibility of a cluster of devices
:Aggregator :Flexibility forecasting,
segmentation..
Request for reliability clusters
Return a reliability cluster with a list of the most
reliable devices
:MOM Middleware:MOM
Middleware
:Aggregator UI
Provide forecasted device flexibility
Request of the forecasted device flexibility
Calculate Reliability
Request of device flexibility delivered
Return device flexibility delivered
Create a Reliability cluster based on the intial
aggregator's request
Figure 7: Create a cluster with the most reliable devices
Page 19
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 19 of 22
9. SOURCE CODE OF THE RELEASE
Git has been used for source code versioning which is hosted at Hypertech’ s premises git
repositories. The engine provided has been strongly based on Hypertech’ s background
knowledge and thus, it is not an open source software.
10. RELATED DOCUMENTATION
The current demonstrator has followed the functional and technical specifications as these have
been described in the D2.6 “Framework Architecture including functional, technical and
communication specifications – Preliminary Version” and updated in the D2.9 “Framework
Architecture including functional, technical and communication specifications – Final
Version”.
This demonstrator implements the functionalities specified on the DoW in addition with the
results of the Task 3.5 “Prosumer-centric local optimization strategies definition” and Task 7.1
“Detailed Pilot Evaluation, Impact Assessment and Cost-Benefit Analysis Framework”.
The algorithmic framework used is provided in the preliminary version of the demonstrator in
the D5.2 “FLEXCoop Flexibility Forecasting, Segmentation & Aggregation Module –
Preliminary Version”.
As mentioned before, partial demonstration of its functionality is visualised in the aggregator
UI as detailed in the D5.9 “FLEXCoop Aggregator real-time Monitoring and Control Platform
- Final Version”.
Information regarding the applicable data model and relevant endpoints used to provide the
FFSA services to the middleware can be also found in the D4.3 “FLEXCoop Common
Information Model - Preliminary Version” and it updated version D4.7 “FLEXCoop Common
Information Model - Final Version”.
As regards the integration of the FFSA with the middleware and its functionality testing within
the relevant FLEXCoop chain of components more information can be found in the D6.8
“FLEXCoop Integrated DR Optimization Framework and Pre-validation results - Final
Version”.
11. INSTALLATION GUIDE
No installation guide is needed. The service has been installed at Hypertech’ s servers and its
services can be used through the provided public endpoints.
12. USER GUIDE
No user guide is needed. The service is already installed at Hypertech’ s servers and its services
can be used through the provided public endpoints.
13. INTERFACES WITH OTHER COMPONENTS AND THEIR INTEROPERABILITY
The FLEXCoop FFSA module will interface through the Message Oriented Middleware:
Page 20
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 20 of 22
With Aggregator UI for data visualisation
With DER registry and Open marketplace for searching registered DERs and their
contractual characteristics
With the Demand Flexibility Profiling module for providing DER flexibility forecast
With the Global Demand Manager to facilitate the real time DSS optimization and
selection of Prosumers to participate in DR campaigns (VPP Configuration, DR
monitoring and dispatch module)
The details about the interoperability interfaces with the other components of the FLEXCoop
solution have been provided in D2.6 “FLEXCoop architecture including functional, technical
and communication specifications”, D2.9 “FLEXCoop Framework Architecture including
functional, technical and communication Specifications – Final Version” and D6.8
“FLEXCoop Integrated DR Optimization Framework and Pre-validation results - Final
Version”. In the Figure below, we provide an updated sequence diagram as this was initially
provided in the D2.6 and further updated in the preliminary version of the FFSA module
(described in the accompanying documentation D5.2 “FLEXCoop Flexibility Forecasting,
Segmentation and Aggregation Module - Preliminary Version”).
:Aggregator :Flexibility forecasting,
segmentation..
Request for aggregation / clustering /
forecasting based on various criteria
Provide Contracts Details
Return clusters based on specific criteria
Request Contracts Deatils
:Marketplace:DER
Registry
Search for registered DERs and their characteristics
Provide registered DER-related data
:MOM
Middleware:MOM
Middleware
:Aggregator UI
:Demand Flexibility
Profiling
Request DER historical data
Provide DER historical data
Request Forecasred Flexibility
Provide Forecasted Flexibility
Figure 8: Sequence diagram showing the interaction between the different modules of the
FLEXCoop architecture.
Furthermore, the FFSA module will request historical data on energy consumption and DR
campaigns by the middleware whenever required based on reliability clustering requests.
The respective endpoints that can be used for providing the different functionalities are listed
below:
http://adsl.hypertech.gr:81/flexcoop/services/derClusterResponse
The endpoint provides a cluster of devices matching the criteria requested
http://adsl.hypertech.gr:81/flexcoop/services/derClusterForecastingResponse
The endpoint calculates the aggregated flexibility forecast of a cluster of devices
specified in the request
Page 21
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 21 of 22
http://adsl.hypertech.gr:81/flexcoop/services/reliabilityClusterResponse
The endpoint calculates the realizability and provides a list of devices matching
specific reliability criteria set in the request
http://adsl.hypertech.gr:81/flexcoop/services/outliersResponse
The endpoint identifies any outliers in a pool of devices based on criteria
specified in the request
The FFSA offers build-in documentation of its endpoints via SwaggerUI / OpenAPI that can
be found here. An example is shown in the Figure 9 through a screenshot of the Swagger UI.
Figure 9: SwaggerUI – FFSA derClusterResponse endpoint
14. REQUIREMENTS COVERAGE
The following table summarises the requirements’ coverage by the demonstrator.
ID Description Status
1 Flexibility forecasting, segmentation and aggregation module shall enable the
dynamic segmentation, classification and clustering of residential building
assets
2 Flexibility forecasting, segmentation and aggregation module shall enable the
spatio-temporal segmentation, classification and clustering of assets
3 Flexibility forecasting, segmentation and aggregation module shall enable
aggregators to set criteria for clustering
Page 22
HORIZON 2020 –773909 - FLEXCoop D5.6 – FLEXCoop Flexibility
Forecasting, Segmentation & Aggregation Module –
Final Version
WP5 – Open Demand Response Optimization
Framework and Tools for Aggregators FLEXCoop Consortium Page 22 of 22
4 Flexibility forecasting, segmentation and aggregation module shall provide
multidimensional analysis for consumers' portfolios as a response to a service
request
5 Flexibility forecasting, segmentation and aggregation module shall perform
aggregated flexibility forecasting
6 Flexibility forecasting, segmentation and aggregation module shall
incorporate the SEAC framework of FLEXCoop
Table 1: Requirements Coverage
15. DEVELOPMENT AND INTEGRATION STATUS
Current Status Final demonstrator
Development status Final version finished
Pending development
actions
Maintenance / debugging during actual demonstration (if
required)
Integration status Finished
Pending integration
actions
Maintenance / debugging during actual demonstration (if
required)
Table 2: Development and integration status
16. CONCLUSION
This demonstrator provides the final version of the FLEXCoop FFSA module. The document
provides all the information required in order for its functionality to be described and
demonstrated.
17. BIBLIOGRAPHY
[1] K. Stenner, E. R. Frederiks, E. V. Hobman, and S. Cook, “Willingness to participate in
direct load control: The role of consumer distrust,” Appl. Energy, vol. 189, pp. 76–88,
Mar. 2017.
[2] D. Olsen, S. Kiliccote, M. Sohn, L. Dunn, and M. A. Piette, “Taxonomy for Modeling
Demand Response Resources,” 2014.