ASISTO: An integrated intelligent assistant system for power plant operation and training

ASISTO: An Integrated Intelligent Assistant System

for Power Plant Operation and Training

Alberto Reyes, Pablo H. Ibarguengoytia, Francisco Elizalde, Liliana Sanchez and Alondra Nava

Instituto de Investigaciones Electricas, Mexico

Email: {areyes, pibar, fef, liliana.sanchez, alondra.nava}@iie.org.mx

Abstract—

In this paper we present ASISTO, an intelligent assis-tant system for power plant operation and trainingbased on probabilistic graphical models. Its mainadvantage is that it provides on-line guidance in theform of ordered recommendations, sensor validationcapabilities, and explanation features, all for uncertainenvironments. The system allows dealing with abnor-mal situations, non-expected events, or the occurrenceof process transients. The different modules of the sys-tem are based on Markov decision processes, Bayesiannetworks, and knowledge representation using theobject–oriented paradigm. Functional results for eachcomponent of ASISTO using a power plant simulatorare also presented.

Index Terms—Intelligent Assistant Systems, power plants,probabilistic graphical models.

I. INTRODUCTION

Modern power plants are following some clear tendencies.

First, they are very complex processes working close to their

limits, with problems and unexpected disturbances. Second,

they are highly automated and instrumented, leaving the oper-

ator with very few decisions. Third, plant operators are faced

with a large amount information based on which they have

determine the state of the process. However, there still exist

some maneuvers that require the experience and ability of the

operator. To support such decisions, automatic assistants exist

that i) provide operators with a list of suggested commands

[1], ii) detect possible failures in process instrumentation [2],

and iii) explain the suggestions provided by the system for

training purposes [3].

Intelligent Assistant Systems (IAS) are knowledge-based

systems for the decision support. They should provide users

with accurate information at the right moment, and do sug-

gestions and criticisms during the decision making process

[4]. The basic functions of an IAS are: knowledge acquisition

and representation, simulation, test case generation, problem

solving, and knowledge transfer through explanations to users

[5]. Among the most representative work in the field of

intelligent assistants, the following systems can be found:

ASTRAL [6], which is a simulator-based assistant for power

operator’s training. SOCRATES [7] is a real time assistant

for control center operators in alarm processing and energy

restoration. SART [8] is a traffic control support system for

the french subway system, which implements an intelligent

tutor.

In this work we present ASISTO, an intelligent assistant

system for power plant operators and training based on prob-

abilistic graphical models. The paper is organized in two big

blocks: i) module description and ii) functional results. The

last section establishes some conclusions and future work.

II. PROBLEM DOMAIN

In order to illustrate how important the decisions of a human

operator are, we have selected the steam generation process

of a combined cycle power plant as a problem domain.

In this domain, a heat recovery steam generator (HRSG)

recovers residual energy from the exhaust gases of a gas

turbine to generate high pressure (Pd) steam in a special tank

(steam drum). The recirculation pump is a device that extracts

residual water from the steam drum to keep a water supply

in the HRSG (Ffw). The result of this process is a high-

pressure steam flow (Fms) that keeps running a steam turbine

to produce electric energy (g) in a power generator. The main

control elements associated are the feed-water valve (fwv) and

main steam valve (msv). A simplified diagram of the steam

generation system is shown in Fig. 1.

During normal operation, the conventional feedwater control

system commands the feedwater control valve to regulate the

Fig. 1. Simplified diagram of the steam generation system showing its maincomponents, control devices and instrumentation. The gas turbine connectionis not shown.

steam drum level. However, when a partial or total electric load

rejection is presented this traditional control loop is not longer

capable to stabilize the drum level. In this case, the steam-

water equilibrium point moves, causing an enthalpy change

of both fluids (steam and water). Consequently, the enthalpy

change causes an increment in the water level because of a

strong water displacement to the steam drum. The control

system reacts closing the feedwater control valve. However a

water increament is needed instead of a feedwater decrement.

Under these circumstances, the participation of a human

operator is necessary to help the control system to decide the

actions that should be taken in order to overcome the transient.

On the other hand, the process instrumentation in the

system, i.e. valve positioners, level or pressure sensors, could

produce noisy information or simply fail during normal op-

eration such that this would affect the operator performance

when making decisions.

III. ASISTO

ASISTO is an intelligent assistant system (standing for

Operation ASSISTant in Spanish) for training and on-line

assistance in the power plant domain. The intelligent system is

composed of the following modules: i) recommender system,

ii) diagnosis system, and iii) explanations system. It has two

different operation modes: operation assistant and operators

training. In the operation assistance mode, it uses the rec-

ommendation and the diagnosis modules. The training mode

functionally is a replica of the operation assistance with the

difference that it includes an instructor console conected to

a plant simulator aimed to set failures and change operation

conditions with learning purposes.

A. Recommender system

The decision making process in a control room can be

modelled as a sequential decision problem under uncertainty

with an optimization criteria such as the maximation of electric

power or the security in the operation of certain equipment.

In a sequential decision problem a decision maker executes

and action to produce a state transition. In the resulting new

state the decision maker again executes another action and so

on until getting an absorbing state. Thus, we decided to use

the factored Markov decision processes (MDP) approach as it

allows generating an optimal policy function that maximizes

the expected utility in the form of textual recommendations.

Formally, a factored MDP is represented as a finite set of

factored states (state variables), a finite set of actions, an im-

mediate reward function, and a transition model that denotes,

as a probabilistic distribution, the impact of actions on the state

transition. For further details about the theory under the MDP

approach and factored representations please refer to [9], [10]

In total, the ASISTO’s recommender system is able to

use sets of 5 or 9 actions. The possible actions are to

open or close fwv and msv, their combinations, and do

nothing (null action). A factored state is an instantiation of

the ffw, pd, fms, g, and d variables. The transition model

for each action is represented by a dynamic Bayesian network

Fig. 2. The user interface is the link between the recommender system andthe operator. It includes supervision and manual control capabilities.

which was approximated using data for minimum, medium

and nominal load conditions. The reward function awards

operation zones that match with those recommended by the

steam drum manufacturer, and also awards those operation

conditions where the plant produce a maximum electric power.

The transition model was implemented in Elvira [11] (which

was extended to compute Dynamic Bayesian Networks) and

the reward function was implemented using Weka [12]. The

transition and reward functions are effectively learned using

the K2 algorithm available in Elvira, and the reward function

using a Java implementation of the C4.5 algorithm available

in Weka (J4.8). A proprietary planning tool uses these models

and, through its inference algorithms, builds an optimal policy

plant state-recommendation. A more detailed description about

how to learn factored MDP models might be found in [13].

The recommender system obtains a plant state and then

it queries the policy function to obtain a recommendation.

Both, current state and recommendation, are shown graphically

through the user interface to the operator who finally decides

whether or not to execute the recommended command. The top

part of figure 2 shows the recommendation “open feedwater

valve FWV” in a text area.

B. Diagnosis system

The diagnosis system in ASISTO was mainly defined as a

sensor validation system, i.e., a system that detects deviations

of the normal behavior of the sensors in the domain. In fact,

the sensors in this diagnosis module might be the sensors

themselves, the feedback signals of actuators and other process

variables. The term sensor is used in this paper to represent

undistinguishably all these variables.

The sensor validation system utilizes a probabilistic model

to represent the relations among all the sensors [14]. Specif-

ically, this model is a Bayesian network learned through

process historical data and experts advice. In this project, the

historical data are obtained using a simulator.

Fig. 3. Diagnosis module’s user interface. The color code allows user aneasy identification of those instruments under an abnormal behavior.

The sensors are validated through the estimation of each

variable from the readings of other sensors as evidence, and

propagating probabilities to estimate the real variable value.

This estimation is compared with the value read, to detect

a deviation with respect to its normal behavior. However, a

fair question arises: what happen if an unexpected reading is

caused by a deviation of the normal behavior in the process,

and not a malfunction in the sensor?

The diagnosis system in ASISTO represents an extension

of the sensor validation in the diagnosis of the complete

process, including actuators validation. The hypothesis in this

work is that, when an abnormal behavior is detected, one or

more sensors report a real fault. Thus, assuming that a sudden

malfunction of multiple sensors is very unlikely, the presence

of multiple faulty sensors at certain time represent a process

deviation. Consequently, a simple classification layer is needed

to relate a set of faulty sensors with a real process failure.

The main advantage of this diagnosis approach is that only

one model of the process working properly is required to

construct the probabilistic model. The model is formed by

a Bayesian network that can be constructed using historical

data with machine learning algorithms, using human experts’

advice, or a combination of both. When a failure is detected,

a very simple classification procedure is applied to identify if

the failure is located in a sensor or there is a failure in the

system.

Figure 3 shows the diagnosis module’s user interface as

implemented in ASISTO. The first two columns display the

sensor identifier with a brief description. In the third column,

the probabilities of failure from 0 to 100% are shown. In order

to make the failure identification easier to users, a color code

denoting no failure (green), alarm (yellow), and failure (red)

was also implemented. Section IV describes the experiments

conduced and the results obtained in the diagnosis module.

For a more detailed description about the diagnosis system

please refer to [15].

C. Explanation system

The explanation system is aimed to provide new operators

with background information during a training session. The

automatic generation explanation mechanism used by ASISTO

is composed of two main stages. In the first stage, the most

relevant variable is obtained by analyzing the MDP model

used by the recommender system. This relevant variable is

defined as the factor that has the greatest impact on the utility

given certain plant state and recommendation, and it represents

a key element in the explanation generation mechanism. In

the second stage, an explanation is generated by combining

the information obtained from the MDP analysis, and dis-

played in the form of a general template containing domain

knowledge represented as a class hierarchy. The current state,

the recommended action generated by the MDP, and the

resulting relevant variable are then used as pointers to query

the knowledge base and extract the relevant information to

fill–in the explanation template. A more detailed description

of the explanation mecanism can be found in [3], [16].

ASISTO generates an explanation for every user level:

novice, intermediate and advanced. Advanced users do not

require a well detailed explanation so that they are provided

with essencial information only. However, novices might need

more specific explanations and a template with more complete

information might be displayed. Figure 4 shows an example

of explanation templates for advanced users. In general, the

template contains to their left side the optimal action, say

why it is important to perform the recommended action,

the component related to the optimal action, and a brief

description of the current plant state. On the right side of the

relevant variable there is a diagram of the process associated

with the action executed by the user.

The KB has been implemented using a software tool for

object-relational mapping (ORM). This technology allows to

convert data from an object-oriented programming language

into a relational database. Thus, the interrelations between the

Fig. 4. This template is composed of 3 main parts: (i) the recommendedaction and the relevant variable in the current situation; (ii) a graphicalrepresentation of the process highlighting the relevant variable, and (iii) averbal explanation.

elements of a combined cycle plant represented in the KB

profit the advantages of the data base models. This format

allows to represent explanations for operators training and

stores information such as variables, additional components

of the recommended actions and procedures involved (fig-

ure 5). Through this model, it is possible to have the domain

knowledge centralized so that an easy and efficient knowledge

manipulation can be possible for a specific user.

Fig. 5. The KB, represented as a class hierarchy, is divided in 3 parts: (i)actions, (ii) components, and (iii) variables.

IV. EXPERIMENTAL RESULTS

ASISTO has been tested using a simulator which is capable

to partially reproduce the operation of a combined cycle power

plant, in particular, the steam generation process described in

section II. The simulator is provided with controls for setting

power conditions in the gas and steam turbines (nominal load,

medium load, minimum load, hot standby, low speed, and

start-up). Besides, it can emulate white noise in sensors and

adjust the simulation speed for a better proccess perspective.

It also provides an operation panel to set load demands, unit

trips, shutdowns, and other high-level operations in different

subsystems. It includes a visualization tool for tracking the

behavior of user-selected variables in time, and recording

historical data.

We evaluated the functionality of the operation assistant

and the training modes with users through their main former

components: the recommender system, the diagnosis system,

and the explanation system.

A. Recommender system

The recommender system tests consisted in showing the

behavior of the expected utility as a user follows or not the

recommendations displayed by the system. To do this, we

plotted the expected utility under two different scenarios. In

the first one, the plant state is modified by a simulated process

transient and the user manually has to set the plant into optimal

conditions. In a second scenario, a user manually avoids a

decrement of the expected utility by executing the suggested

actions under a load rejection (disturbance).

The tests were performed by operator trainees of different

progress level. The novices operated the control valves at

minimum load conditions, the intermediate users made it at

medium load, and the advanced users maneuvered the plant

under nominal load conditions. In all cases, the opening step

of valves msv and fwv was 2%.

Figure 6 shows how the expected utiliy (red line) decreases

after an induced transient (first drop) under medium load

conditions while figure 7 shows how it does it after a load

rejection under minimum load conditions. The little blue dots

represent the recommended command and the little green

triangles donotates the executed actions (right vertical axis).

In both scenarios is demonstrated that the operator manage to

restore the utility until a maximum by following the suggested

actions (recommended commands matching with the executed

action). Everytime the user did not follow the instructions

(recommended commands differing to the executed action),

the utility decreased again. Notice that the decrement of utility

after a transient is not as significant as after a load rejection.

This is because of the difference in load conditions and due

that a disturbance produces a higher drop in the utility than

a simple transient. In these two examples, the impact of

following the recommendations is evident and significant.

Fig. 6. Expected utility response after a process transient, recommendedactions and executed actions under a medium load condition.

Fig. 7. Expected utility response after a load disturbance, recommendedactions and executed actions under a minimum load condition.

Fig. 8. Response of the diagnosis system when the power plant hasexperienced a load rejection.

B. Diagnosis System

The diagnosis module in ASISTO considers that, using the

sensor validation algorithm, it is possible to detect failures

in sensors and deviations of the normal behavior [15]. The

following experiments were executed:

• normal operation. This is to show that there are not false

positives failures.

• noise in sensors. This is to show the presence of devi-

ations in the normal value of a sensor. We assume only

single noisy sensors.

• load rejection. This is to show an abnormal event that

happens in the power plants when the electric load is

disconnected from the supplier plant. This event is caused

when there are faults in the transmission or distribution

networks.

• unit trip. This is to show an abnormal event that happens

when the operator or the control system stops abruptly the

whole process.

Experiments were conducted including three operational

modes. First, when the plant is working at minimum load. This

means that there is not high demand of energy. Second, when

the plant is working at medium load and finally, when the plant

was generating at maximum power (nominal load). When only

one sensor is declared faulty, the diagnosis recognizes this

as a real sensor failure. When multiple sensors are suddenly

declared faulty, a simple classification scheme can define the

abnormal event presented in the process.

As an example of the results, consider the case of load

rejection (figure 8). If an unexpected interruption of the

consumption is suffered, then a drastic interruption of the

generation is needed. The horizontal axis represents the time of

the experiment. The right axis represents the generated power

in kilo Watts, and the left axis represents the probability of

failure of all sensors. Notice the WET3 signal at the upper

left side of the graph. It shows that the generation reaches

almost 21 mega Watts. When the load reject event occurs, the

generation drops to less than a mega watt, and at the end of

the event rises again to reach more than 6 mega watts. Several

reactions occur in the equipment and in the control system.

When the load is rejected, many sensors show values that are

not related with others. That is why many sensors increase

their probability of failure. In a few seconds, the control

system runs the appropriate corrections and a stable situation is

presented with only failures reported in XZ1, XZ2, and WET3

sensors. They are, the main steam valves and the generation

sensors. Finally, when the load rejection event finishes, then

the generation signal indicates a recovering of the generation,

and only the generation sensor indicates a failure until the rest

of the signals recover their normal value corresponding to a

situation where the generation is above 6 mega watts.

C. Explanation System

The explanation system was tested with 9 users of different

progress levels: 3 novices, 3 intermediate, and 3 advanced-

level students. The tests were performed with and without

explanations to evaluate its impact. In all cases, the tests were

run with the recommender system on. The tests consisted

of subjecting a student to a series of training sessions. A

session is composed by 10 actions to execute by the user.

The goal is to complete a session with no errors after 5

attempts, measuring the advance progress according to the

relation progress = ac/10 ∗ 100, where ac are the first

consecutive correct actions.

Figures 9 and 10 show the performance obtained by 3

novice students after running experiments with and without

explanations. In the case of not using explanations, during the

first three attempts, the students got a progress lower than 35%.

In the last two attempts, two of them managed to achieve a

progress higher than 80% to finally obtain an average group

progress of 46.333%. In the case of using explanations, users

1 and 2 achieved the goal (no errors) in sessions 2 and 4

respectively, to get an average group progress of 68.0%. In

these two examples, users without explanations needed more

attempts to achieve the goal than users with explanations.

In general, the progress rate for non explained students was

always significantly lower.

Fig. 9. Progress obtained by a novice student (minimum load) withoutexplanation aids

Fig. 10. Progress obtained by a novice student (minimum load) assisted withexplanations

V. CONCLUSIONS AND FUTURE WORK

This paper introduced ASISTO, an integrated intelligent

assistant for operation and training. We have integrated, in

one demostration system, capabilities of recommendation,

diagnosis and explanation. The academic software for planing,

learning and knowledege representation used in ASISTO is

also a well known and robust platform for research in artificial

intelligence.

The recommendation system allows to guide the operator’s

decisions towards a better operation practices. The results

demonstrated that it is possible to maximize utility and security

in a power plant by following the ASISTO’s recommended

commands. We are currently extending the recommender

system to consider partial observability. POMDPs [17] extend

the MDP framework with an observation function, which

stochastically relates observations to states. No global changes

to the ASISTO architecture have to be made.

The diagnosis system shows the feasibility of using the

sensor validation algorithm to find not only failures in sensors,

but finding deviations in the process with respect to its normal

behavior. This paper shows two abnormal events, one in the

power plant and other in the classification of faulty sensors

detected. Future events need to be identified to complete a case

base with the faulty sensors signature for a specific event.

When running experiments, operators trainees experienced

improvements in their general performance after using the

explanation system. We plan to conduct additional user study

tests using the explanation module in order to demonstrate

that the quality of explanations generated automatically is of

a very high standard when compared against those given by

an domain expert.

ASISTO more than a system is a demonstration of the

application of probabilistic graphical models to a power plant

domain. So, it might be used in related domains with existance

of uncertainty and optimization requirements.

ACKNOWLEDGMENTS

This work has been supported by the Instituto de In-

vestigaciones Electricas-Mexico (project no. 13773) and the

International Cooperation Programme for the Promotion of

Scientific and Tecnological Research of the European Union

and Mexico (FONCICYT project no. 95185).

REFERENCES

[1] A. Reyes, L. E. Sucar, E. Morales, and P. H. Ibarguengoytia, “AbstractMDPs using qualitative change predicates: An application in powergeneration,” in Planning under Uncertainty in Real-World ProblemsWorkshop. Neural Information Processing Systems (NIPS-03), Vancou-ver CA, Winter 2003.

[2] E. Morales and P. Ibarguengoytia, “On-line diagnosis using influencediagrams,” in Advances in Artificial Intelligence - MICAI 2004, LNAI2313, G. A. L.E. Sucar R. Monroy and e. H. Sossa, Eds. BerlinHeidelberg: Springer-Verlag, 2004, pp. 546–554.

[3] F. Elizalde, L. E. Sucar, M. Luque, J. Diez, and A. Reyes, “Policyexplanation in factored Markov decision processes,” in Proceedings of

the 4th European Workshop on Probabilistic Graphical Models (PGM

2008), Hirtshals, Denmark, September 2008, pp. 97–104.[4] A. Aamodt and M. Nygard, “Different roles and mutual dependencies

of data, information and knowledge: An AI perspective on their inte-gration,” Data and Knowledge Engineering (North-Holland Elsevier),vol. 16, pp. 191–222, 1995.

[5] P. Brezillion and E. Cases, “Cooperating for assisting intelligentlyoperators,” in Proceedings of COOP-95. INRIA Ed., 1995, pp. 370–384.

[6] M. Caimi, C. Lanza, and B. Ruiz-Ruiz, “An assistant for simulator-basedtraining of plant operator,” Maria Curie Fellowships Annual Report,Tech. Rep., 1999.

[7] Z. Vale, C. Ramos, A. Silva, L. Faria, J. Santos, F. Fernandez, C. Rosado,and A. Marques, “SOCRATES, an integrated intelligent system forpower system control center operator assistance and training,” in Pro-

ceedings of the International Conference on Artificial Intelligence and

Soft Computing (IASTED), 1998, pp. 27–30.[8] P. Brezillion, C. Gentile, I. Saker, and M. Secron, “SART: A system for

supporting operators with contextual knowledge,” in Proceedings of the

International and Interdisciplinary Conference on Modeling and Using

Context (CONTEXT-97), Rio de Janeiro, Brasil, February 1997.[9] M. Puterman, Markov Decision Processes. New York: Wiley, 1994.[10] C. Boutilier, T. Dean, and S. Hanks, “Decision-theoretic planning: struc-

tural assumptions and computational leverage,” Journal of AI Research,vol. 11, pp. 1–94, 1999.

[11] E. Consortium, “Elvira: an environment for creating and using proba-bilistic graphical models,” U. de Granada, Spain, Tech. Rep., 2002.

[12] I. Witten and E. Frank, Data Mining: Practical Machine Learning Toolsand Techniques, 2nd Ed. San Francisco, USA: Morgan Kaufmann,2005.

[13] A. Reyes, M. T. J. Spaan, and L. E. Sucar, “An intelligent assistant forpower plants based on factored MDPs,” in IEEE Int. Conf. on Intelligent

System Applications to Power Systems: ISAP 2009, Curitiba, Brazil,November 2009.

[14] P. H. Ibarguengoytia, S. Vadera, and L. Sucar, “A probabilistic modelfor information and sensor validation,” The Computer Journal, vol. 49,no. 1, pp. 113–126, January 2006.

[15] P. H. Ibarguengoytia and A. Reyes, “On-line diagnosis of a powergeneration process using probabilistic models,” in 16th Intl. Conf.on

Intelligent Systems Application to Power Systems, ISAP-2011. Her-sonissos, Crete Greece: IEEE PES, 2011.

[16] F. elizalde, E. Sucar, J. Noguez, and A. Reyes, “Integrating probabilisticand knowledge-based systems for explanation generation,” in Proceed-

ings of 3rd International Workshop on Explanation-aware Computing,

Exact08, vol. CEUR Workshop Proceedings, 2008, pp. 25–36.[17] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and

acting in partially observable stochastic domains,” Artificial Intelligence,vol. 101, pp. 99–134, 1998.