BayesiaLab Knowledge Elicitation Environment

©2010 BAYESIA SASAll rights reserved. Forbidden reproduction in whole or part without the Bayesia’s express

written permission

Plan

Modeling by Brainstorming

BAYESIALAB 5.0Knowledge Elicitation Environment

BayesiaLab’s Knowledge Elicitation Environment

An innovative Brainstorming Tool

Dr. Lionel JOUFFE

May 2010

1

Plan




written permission2

All models are wrong; the practical question is how wrong do they have to be to not be useful (Box&Draper 87)

MODELING BY BRAINSTORMINGMODELING BY BRAINSTORMING

Plan




written permission3

Designing a Model for Decision Support

Every Company is faced to complex decisions that need to be rationally supported

Sometime, there are too few data available, or no data at all, to allow using data mining and data analysis technics to automatically build a Decision Support System

Experts have gathered invaluable Tacit Knowledge through their experience

We need to Convert this Tacit Knowledge into Explicit Knowledge and use it to build a model

We want actionable models to allow What-if scenarios (simulation and/or diagnosis), drivers analysis, ...

Bayesian Belief Networks (BBNs) are ideal models for such problematics: their graphical representation allows a manual design by using expert knowledge, and their probabilistic engines offer powerful simulation capabilities

Plan




written permission4

BBNs are made of Two Distinct Parts

StructureDirected Acyclic Graph (DAG), i.e. no directed loop

Nodes represent the variables

Each node has a set of exclusive states (e.g.: Young, Adult, Aged)

Arcs represent the direct probabilistic influences between the variables (possibly causal)

ParametersProbability distributions are associated to each node, usually by using tables

CONDITIONAL PROBABILITY DISTRIBUTION

A smoker has a 60% of risk of suffering from a Bronchitis, whereas the risk of

a non smoker is 30% only

MARGINAL PROBABILITY DISTRIBUTIONWe consider a population made

of 40% of Adults

Plan




written permission5

BBNs are Powerful Inference Engines

We get some evidence on the states of a subset of variables: Hard positive and negative evidence, Likelihoods, Probability distributions, Mean values

We take these findings into account in a rigorous way to update our belief on the states of all the other variables

Probability distributions on their values

Multi-Directional Inference (Simulation and/or Diagnosis)

The evidence on Smoker (a new probability distribution) allows to update the probability distribution of Age (Diagnosis) and Bronchitis (Simulation)

Prior Distribution Posterior Distribution

Plan




written permission6

BBN Modeling by Brainstorming

Clear definition of the BBN’s objective(s) (e.g.: Improvement of the Product/Service Quality, improvement of the Purchase Intent, improvement of the Company’s performance, ...)

Identification of the conceptual dimensions that are linked to those objectives (e.g.: Human resources, Management, Production, Marketing, ...)

Definition of the group of experts that will fully cover all the dimensions (and the different geographical zones), with a small redundancy to allow fruitful debates

Brain Storming Sessions with this group of Experts to manually build the BBN

Plan




written permission

The StructureThe Directed Acyclic Graph

7

For each identified conceptual dimension

Definition of the main variables

Definition of the exclusive states of those variables

Creation of one node per identified variable

Brainstorming to define the direct relationships between the variables, and addition of the corresponding arcs between those dependent variables

The structure elicitation is probably the simplest task of the Brainstorming session

For each root node, i.e. without incoming arc, definition of the marginal probability distribution over the defined states

For each node with incoming arc(s), definition of the conditional probability distribution over the defined states, for each combination of the states of its connected nodes

Each expert gives his/her belief on the distributions

There are various kinds of biases to be aware of

Cognitive (Plausibility, Control, Availability, Anchoring) Emotional (Mood, Motivation) Group (Anchoring, Herding) Facilitator (can be biased toward charismatic experts or toward

the last expressed opinion)

Plan




written permission

The ParametersProbability Distributions

8

Probabilities do not have to be exact to be useful

☛ Use the new BayesiaLab’s Knowledge Elicitation environment to reduce these biases, to improve traceability, to gather all the

useful knowledge, ....

Plan




written permission9


Plan




written permission

The Experts

Definition of the group of Experts

10

- Group of experts can be Imported/Exported- The Open Session button allows opening an Online Brainstorming Session*

- The Generate Tables button allows generating a Bayesian network by using the assessments of the selected experts only

Plan




written permission

The Experts

* Available on subscription only

This Expert Editor allows defining:The Expert’s name, its Credibility (that will be use globally during the consensus computation), her/his Picture, a Comment to describe her/his area of expertise. The last field contains the number of assessments realized by the expert on the

current network

11

Plan




written permission

The Experts’ Assessments

Selecting a cell in the probability table activates the Assessment button for assessing the question corresponding to the selected line, i.e. what is the marginal probability

distribution of Age over the 3 defined states?

12

Plan




written permission

The Experts’ Assessments

Pressing the Assessment button opens the Assessment Editor that allows the Facilitator to manually add, delete and modify Experts’ Assessments.

The Post Assessment button can be used by the Facilitator to Post the question to the BayesiaLab’s secured website for an online assessment

13

Plan




written permission

The Expert Online Assessment Tool

Once logged in, the Expert is waiting for a

question

The secured website

The Expert’s name, case sensitive!

The session name

14

Plan




written permission

The Expert Online Assessment Tool:Example with the 3 states variable Age

Once the Facilitator has posted the question with the Assessment Editor, the question is displayed on the Expert’s webpage.

There is no context (root node). This is then a

marginal probability

Check box for fixing the probability of the

state

The question is relative to the node “Age”, that has 3 states: Young,

Adult and Aged.There are then 3 sliders for the probability

distribution assessment, and another one for the confidence

Pie Chart representing the

probability distribution specified with the sliders

The label corresponds to the

Confidence level the expert has specified with the

Confidence Slider (ranging from “I Do not Know” to “I

am Certain”)

The comment field can be

used for explaining the assessment

15

Plan




written permission

The Expert Online Assessment Tool:Example with the binary variable Cancer

16

The context variables in the BBN

This question is relative to node Cancer, and the specific Context is “Age = Adult” and

“Smoker = Yes”

Hovering over the context variables returns the comment

associated to the corresponding node, if any

Plan




written permission

The Facilitator’s tool

Once the Expert validates her/his assessment, this assessment is sent to the BayesiaLab’s server and the Facilitator’s listener is automatically updated

This listener allows following the status of the

Experts’ assessments

Clicking on OK makes BayesiaLab harvesting the

assessments. Closing the window cleared the question from the webpage of the Experts that do not have

17

Plan




written permission


This gray part corresponds to the Experts’ probability distribution

assessments

This second part contains the Expert’s name, the Assessment’s

Confidence, the associated Comment and the Time (in second) for validating

the assessment

18

Plan




written permission


The content of this editor is sortable by each column just by clicking on the corresponding header

It is sorted here in the ascending order on the probabilities assessed for the

state Young

Selecting the line allows displaying the Expert’s picture

Sorting the assessments by state probabilities can be used for:- detecting Experts’ misunderstanding

- Knowledge sharing, especially by making the 2 “extremes” Experts debate

If some useful knowledge comes out from the debate, the Facilitator can post again the question for a new Expert Assessment. Each Expert will then be

allowed to update her/his assessment online (each Experts’ webpage is initialized with the information she/he set in the previous round)

19

Plan




written permission

The Consensus

A small icon is added at the left of each probability to graphically

represent the consensus degree:from a full transparency when there

all the Experts agree on the probability, to no transparency when the range of the assessments is 1

Once the assessments validated, a Mathematical consensus is computed by using the Experts’ credibility and their assessment’s confidence. This automatic consensus can be manually

modified by the Facilitator to set a Behavioral consensus, i.e. one issued after a fruitful debate

Hovering over this icon returns the minimum and the

maximum assessments, and the number of assessments

20

Plan




written permission

The Consensus

The probability distributions that have a set of assessments are framed

with a green line

An icon is added to the nodes for indicating the nodes that have Experts assessments. The darker the icon is, the lower the global consensus is

Here is the list of assessments corresponding

to the second line of the table

21

Plan




written permission

The Consensus

Pressing the “i” key while hovering over the expert icon allows displaying

the information panel below

This information panel contains:- the number of rows ((Conditional) probability distributions) that comes with Experts

assessments- the total number of assessments that have been set in the probability table

- the number of Experts that have assessed at least one probability distribution in the table - a measure of the global disagreement that takes into account the deviations from the

mathematical consensus- the maximum disagreement corresponding to the greatest difference between two

assessments in the probability table

22

iiiiiii

Plan




written permission

The Assessment Report

Right clicking on the Expert Icon in the lower left corner of the Graph window allows generating the

following HTML report. This report first gives information on the Experts,

then returns a sorted list of the nodes wrt the global disagreements, and another one wrt the

maximal disagreements.Finally, for each node, a summary contains all the

global information on the assessments of the (Conditional) Probability Table

All these informations can be useful for the Model Validation, e.g. by checking first the nodes based

on their associated disagreements (global and maximal), then based on the time for the

assessments (that can reflect a difficulty, or, on the contrary, too prompt assessments)

23

Plan




written permission

Exportation of a Bayesian Network per Expert

This exportation tool allows to create a Bayesian Belief Network for each Expert.

The parameters (probabilities) are those assessed by the Expert. If the Expert has not assessed all the probabilities, the model will use either the

consensual probabilities, or those manually entered by the Facilitator

24

Plan




written permission

Exportation of the Probability Assessments

This exportation tool allows creating a CSV file with all the assessments of the

probabilities. There is one column per variable to describe the context (yellow), one

column to indicate the assessed Node (green), the other columns describing the assessed probability, the confidence level,

the Expert, and the assessment time.Each line describes one assessment of a

(Conditional) Probability Table cell

MitA/TiPo MiAt TiPo Node Probability Confidence Expert Time

Weak Weak Strong MitA/TiPo 0,97 1 Hiro 56

Strong Weak Strong MitA/TiPo 0,03 1 Hiro 56

Weak Weak Strong MitA/TiPo 0,95 1 Haitien 210

Strong Weak Strong MitA/TiPo 0,05 1 Haitien 210

Weak Weak Strong MitA/TiPo 0,8 0,58 Claire 145

Strong Weak Strong MitA/TiPo 0,2 0,58 Claire 145

Weak Weak Strong MitA/TiPo 0,85 0,77 Matt 65

Strong Weak Strong MitA/TiPo 0,15 0,77 Matt 65

Weak Weak Strong MitA/TiPo 0,4 0,8 Mohinder 76

Strong Weak Strong MitA/TiPo 0,6 0,8 Mohinder 76

Weak Weak Strong MitA/TiPo 0,75 0,9 Nathan 50

Strong Weak Strong MitA/TiPo 0,25 0,9 Nathan 50

... ... ... ... ... ... ...

Weak MiAt 0,7 0,2 Noah 76

Strong MiAt 0,3 0,2 Noah 76

Weak MiAt 0,75 1 Matt 90

Strong MiAt 0,25 1 Matt 9025

Plan




written permission

Exportation of the Expert Assessments

This exportation tool generates a CSV file with all the assessments of the Experts.

There is one column per Expert, one column per Expert’s Confidence (yellow), the last column indicating the weight of the line (1/number of states of the assessed variable)

(green).Each line describes the Experts’ assessment

of a(Conditional) Probability Table cell

Hiro Hiro Confidence

Haitien Haitien Confidence

.... Noah Noah Confidence

Weight

0,97 1 0,95 1 .... 0,7 0,8 0,5

0,03 1 0,05 1 .... 0,3 0,8 0,5

0,3 0,81 0,05 1 .... 0,3 0,7 0,5

0,7 0,81 0,05 1 .... 0,7 0,7 0,5

0 1 0 1 .... 0 0,79 0,5

1 1 1 1 .... 1 0,79 0,5

0,65 0,79 0,71 1 .... 0,7 0,2 0,5

0,35 0,79 0,29 1 .... 0,3 0,2 0,5

26

Plan




written permission

Analysis of the Expert Assessments

Each node represents the discretized probabilities

assessed by the Expert

27

We then can used this file to analyze the direct probabilistic relationships that hold between the Experts’ assessments

We then can used this file to analyze the direct probabilistic relationships that hold between the Experts’ assessments

This network has been automatically learned with one of

the BayesiaLab’s Association Discovering algorithms on a set of 120 Experts’ assessments

Plan




written permission

Automatic Segmentation of the Experts

Each color corresponds to a cluster. Three segments of Experts have

been induced in that example. The real experts behind those anonymized

experts have indeed three different profiles (functionally and geographically)

Dendrogram corresponding to that

segmentation

Based on the obtained Expert Segments, one Bayesian network per segment can be generated (by using the Expert Editor). This can be useful for analyzing the sensibility of the model, but also to get specific networks (depending on the geographical localization

for example)

28

Based on the obtained network, Experts can be clustered into homogeneous groups by using the BayesiaLab’s Variable Clustering algorithm

Plan




written permission

Parameter Sensibility Analysis

BayesiaLab also comes with an Assessment Sensitivity Analysis tool that allows measuring the uncertainty associated to the consensus.

The general idea is to generate a set of networks by randomly drawing Experts’ assessments, and then measuring the uncertainty

associated to each probability distribution.

29

Plan




written permission


Three kinds of analysis are available, depending on the Random selection policy that is chosen to generate the set of networks (1000 networks in the above example):

- One Expert per network: each network generated is parametrized by using the selected Expert (or the consensual probability if the selected Expert has not been involved in the

assessment)- One Expert per node: each network generated is parametrized by selecting for each node one Expert. If the selected Expert is not involved, the consensual probability if the selected

- One assessment per Conditional Probability Table’s row (if any)

30

Plan




written permission


States of the analyzed variable

Marginal probability distribution of the Target node computed with

all the consensus

Density function illustrating the uncertainty associated to this node. The Mean over the 1000 networks (one Expert

per network) is 70.62% (versus 70.46% in the monitor), the Standard

Deviation 2.13%. There are 62% of chance of having a

probability comprise between 70 and 72%

31

Tabs for the selection of the variable under analysis

Plan




written permission


One Expert per node

One Expert per Conditional Probability row

32


written permission

Plan



Contact

33

6 rue Léonard de Vinci BP0119

53001 LAVAL CedexFRANCE

Dr. Lionel JOUFFEPresident / CEO

Tel.: +33(0)243 49 75 58Fax: +33(0)243 49 75 83

BayesiaLab Knowledge Elicitation Environment

Business

0knowledge

direct probabilistic

parameter

marginal probability

probability

conditional

marginal probability

probability