AI – CS364 Knowledge Engineering 07 th November 2006 Dr Bogdan L. Vrusias [email protected].

AI – CS364AI – CS364Knowledge EngineeringKnowledge Engineering

Knowledge EngineeringKnowledge Engineering

07th November 2006

Dr Bogdan L. [email protected]

07th November 2006 Bogdan L. Vrusias © 2006 2


ContentsContents• Definitions

• Basic Process of Knowledge Engineering

• Case Studies



DefinitionDefinition• Davis’ law: “For every tool there is a task perfectly suited to it”.

But…

• It would be too optimistic to assume that for every task there is a tool perfectly suited to it.

• The process of building intelligent knowledge-based systems is called knowledge engineering.



Process of Knowledge EngineeringProcess of Knowledge Engineering

Phase 1: Problem assessment

Phase 2: Data and knowledge acquisition

Phase 3: Development of a prototype system

Phase 4: Development of a complete system

Phase 5: Evaluation and revision of the system

Phase 6: Integration and maintenance of the system



Phase 1: Problem assessmentPhase 1: Problem assessment• Determine the problem’s characteristics.

• Identify the main participants in the project.

• Specify the project’s objectives.

• Determine the resources needed for building the system.



Phase 1: Problem assessmentPhase 1: Problem assessment



Phase 2: Data and Knowledge AcquisitionPhase 2: Data and Knowledge Acquisition• Collect and analyse data and knowledge.

• Make key concepts of the system design more explicit.

• Deal with issue of:– Incompatible data

– Inconsistent data

– Missing data



Phase 3: Development of a Prototype SystemPhase 3: Development of a Prototype System

• Choose a tool for building an intelligent system.

• Transform data and represent knowledge.

• Design and implement a prototype system.

• Test the prototype with test cases.



What is a prototype?What is a prototype?• A prototype system is defined as a small version of the final system.

• It is designed to test how well we understand the problem – to make sure that the problem-solving strategy, the tool selected for building a system, and techniques for representing acquired data and knowledge are adequate to the task.

• It also provides us with an opportunity to persuade the sceptics and, in many cases, to actively engage the domain expert in the system’s development.



What is a test case?What is a test case?• A test case is a problem successfully solved in the past for which input

data and an output solution are known.

• During testing, the system is presented with the same input data and its solution is compared with the original solution.



Phase 4: Development of a Complete SystemPhase 4: Development of a Complete System

• Prepare a detailed design for a full-scale system.

• Collect additional data and knowledge.

• Develop the user interface.

• Implement the complete system.



Phase 5: Evaluation and Revision of the SystemPhase 5: Evaluation and Revision of the System

• Evaluate the system against the performance criteria.

• Revise the system as necessary.

• To evaluate an intelligent system is , in fact, to assure that the system performs the intended task to the user’s satisfaction.

• A formal evaluation of the system is normally accomplished with the test cases.

• The system’s performance is compared against the performance criteria that were agreed upon at the end of the prototyping phase.



Phase 6:Phase 6: Integration and MaintenanceIntegration and Maintenance• Make arrangements for technology transfer.

• Establish an effective maintenance program.



Will an Expert System Work for my Problem?Will an Expert System Work for my Problem?

• The Phone Call Rule:

“Any problem that can be solved by your in-house expert in a 10-30 minute phone call can be developed as an expert system”.



Case Study 1: Diagnostic Expert SystemCase Study 1: Diagnostic Expert System• Diagnostic expert systems are relatively easy to develop:

• Most diagnostic problems have a finite list of possible solutions,

• Involve a rather limited amount of well-formalised knowledge, and

• Often take a human expert a short time (say, an hour) to solve.



Case Study 1: Diagnostic Expert SystemCase Study 1: Diagnostic Expert System



Choosing an Expert System Development ToolChoosing an Expert System Development Tool

• Tools range from high-level programming languages such as LISP, PROLOG, OPS, C and Java, to expert system shells.

• High-level programming languages offer a greater flexibility, but they require high-level programming skills.

• Shells provide us with the built-in inference engine, explanation facilities and the user interface. We do not need any programming skills to use a shell – we enter rules in English in the shell’s knowledge base.



Choosing an Expert System ShellChoosing an Expert System Shell• When selecting an expert system shell, we consider:

– how the shell represents knowledge (rules or frames);

– what inference mechanism it uses (forward or backward chaining);

– whether the shell supports inexact reasoning and if so what technique it uses (Bayesian reasoning, certainty factors or fuzzy logic);

– whether the shell has an “open” architecture allowing access to external data files and programs;

– how the user will interact with the expert system (graphical user interface, hypertext).



Case study 2: Classification Expert SystemCase study 2: Classification Expert System• Classification problems can be handled well by both expert systems

and neural networks.

• As an example, we will build an expert system to identify different classes of sail boats. We start with collecting some information about mast structures and sail plans of different sailing vessels. Each boat can be uniquely identified by its sail plans.



Case study 2: Classification Expert SystemCase study 2: Classification Expert System

Jib-headed Cutter Gaff-headed Sloop Staysail Schooner Gaff-headed Schooner

Jib-headed Ketch Gaff-headed KetchJib-headed Yawl Gaff-headed Yawl



Case study 2: Classification Expert SystemCase study 2: Classification Expert System

/* Sailing Vessel Classification Expert System: Mark 1

Rule: 1 if ‘the number of masts’ is oneand ‘the shape of the mainsail’ is triangularthen boat is ‘Jib-headed Cutter’

Rule: 2 if ‘the number of masts’ is oneand ‘the shape of the mainsail’ is quadrilateralthen boat is ‘Gaff-headed Sloop’

Rule: 3 if ‘the number of masts’ is twoand ‘the main mast position’ is ‘forward of the short mast’and ‘the short mast position’ is ‘forward of the helm’and ‘the shape of the mainsail’ is triangularthen boat is ‘Jib-headed Ketch’

Rule: 4 if ‘the number of masts’ is twoand ‘the main mast position’ is ‘forward of the short mast’and ‘the short mast position’ is ‘forward of the helm’and ‘the shape of the mainsail’ is quadrilateralthen boat is ‘Gaff-headed Ketch’



Classification and Certainty FactorsClassification and Certainty Factors• Although solving real-world classification problems often involves

inexact and incomplete data, we still can use the expert system approach.

• However, we need to deal with uncertainties. The certainty factors theory can manage incrementally acquired evidence, as well as information with different degrees of belief.



Classification and Certainty FactorsClassification and Certainty Factors

/* Sailing Vessel Classification Expert System: Mark 2

control cf

Rule: 1 if ‘the number of masts’ is onethen boat is ‘Jib-headed Cutter’ {cf 0.4};

boat is ‘Gaff-headed Sloop’ {cf 0.4}

Rule: 2 if ‘the number of masts’ is oneand ‘the shape of the mainsail’ is triangularthen boat is ‘Jib-headed Cutter’ {cf 1.0}

Rule: 3 if ‘the number of masts’ is oneand ‘the shape of the mainsail’ is quadrilateralthen boat is ‘Gaff-headed Sloop’ {cf 1.0}

Rule: 4 if ‘the number of masts’ is twothen boat is ‘Jib-headed Ketch’ {cf 0.1};

boat is ‘Gaff-headed Ketch’ {cf 0.1};boat is ‘Jib-headed Yawl’ {cf 0.1};boat is ‘Gaff-headed Yawl’ {cf 0.1};boat is ‘Gaff-headed Schooner’ {cf 0.1};boat is ‘Staysail Schooner’ {cf 0.1}



Will a Fuzzy Expert System Work for my Will a Fuzzy Expert System Work for my Problem?Problem?

• If you cannot define a set of exact rules for each possible situation, then use fuzzy logic.

• While certainty factors and Bayesian probabilities are concerned with the imprecision associated with the outcome of a well-defined event, fuzzy logic concentrates on the imprecision of the event itself.

• Inherently imprecise properties of the problem make it a good candidate for fuzzy technology.



Case study 3: Decision-support Fuzzy SystemsCase study 3: Decision-support Fuzzy Systems

• Although, most fuzzy technology applications are still reported in control and engineering, an even larger potential exists in business and finance. Decisions in these areas are often based on human intuition, common sense and experience, rather than on the availability and precision of data.

• Fuzzy technology provides us with a means of coping with the “soft criteria” and “fuzzy data” that are often used in business and finance.




• Mortgage application assessment is a typical problem to which decision-support fuzzy systems can be successfully applied.

• Assessment of a mortgage application is normally based on evaluating the market value and location of the house, the applicant’s assets and income, and the repayment plan, which is decided by the applicant’s income and bank’s interest charges.




Rule Base 1: Home Evaluation

Market value Location

Rule Base 2: Applicant Evaluation

Asset Income

House Interest Applicant

Rule Base 3: Credit Evaluation

Credit

Income



Will a Neural Network Work for my Problem?Will a Neural Network Work for my Problem?

• Neural networks represent a class of very powerful, general-purpose tools that have been successfully applied to prediction, classification and clustering problems.

• They are used in a variety of areas, from speech and character recognition to detecting fraudulent transactions, from medical diagnosis of heart attacks to process control and robotics, from predicting foreign exchange rates to detecting and identifying radar targets.



Case study 4: Character Recognition Neural Case study 4: Character Recognition Neural NetworksNetworks

• Recognition of both printed and handwritten characters is a typical domain where neural networks have been successfully applied.

• Optical character recognition systems were among the first commercial applications of neural networks.

• A multilayer feedforward network could work well for the for a character recognition system.

• For simplicity, we can limit our task to the recognition of digits from 0 to 9. Each digit is represented by a 5 x 9 bit map.

• In commercial applications, where a better resolution is required, at least 16 x 16 bit maps are used.



Case study 4: Character Recognition Neural Case study 4: Character Recognition Neural NetworksNetworks

7 8 9 10

12 13 14 15

17 18 19 20

26 27 28 29

31 32 33 34

36 37 38 39

6

2 3 4 51

16

11

22 23 24 2521

42 43 44 4541

35

40

30



How do we choose the architecture of a neural How do we choose the architecture of a neural network?network?

• The number of neurons in the input layer is decided by the number of pixels in the bit map. The bit map in our example consists of 45 pixels, and thus we need 45 input neurons.

• The output layer has 10 neurons – one neuron for each digit to be recognised.

• Complex patterns cannot be detected by a small number of hidden neurons; however too many of them can dramatically increase the computational burden.

• Another problem is overfitting. The greater the number of hidden neurons, the greater the ability of the network to recognise existing patterns. However, if the number of hidden neurons is too big, the network might simply memorise all training examples.



Character Recognition Neural NetworkCharacter Recognition Neural Network

0

1

1

1

0

1

1

1

1

1

41

42

43

44

45

1

2

3

4

5

1

2

3

4

5

0

1

2

3

4

5

6

7

8

9

10

0

0

0

0

0

0

0

1

0



Testing the character recognition systemTesting the character recognition system• A test set has to be strictly independent from the training examples.

• To test the character recognition network, we present it with examples that include “noise” – the distortion of the input patterns.

• We evaluate the performance of the printed digit recognition networks with 1000 test examples (100 for each digit to be recognised).

0 50 100 150 2001-4

1-3

1-2

1-1

10

101

Epochs

Su

m-S

qua

red

Err

or

250

102

234

1

1 – two hidden neurons;2 – five hidden neurons;3 – ten hidden neurons;4 – twenty hidden neurons

0 0.1 0.2 0.3 0.4 0.50

5

10

15

20

25

Noise Level

Re

cogn

itio

n E

rro

r, %

2 3 4

2 – five hidden neurons;3 – ten hidden neurons;4 – twenty hidden neurons



Can we improve the performance of the character Can we improve the performance of the character recognition neural network?recognition neural network?

• A neural network is as good as the examples used to train it.

• Therefore, we can attempt to improve digit recognition by feeding the network with “noisy” examples of digits from 0 to 9.



Case study 5: Prediction Neural NetworksCase study 5: Prediction Neural Networks• As an example, we consider a problem of predicting the market value

of a given house based on the knowledge of the sales prices of similar houses.

• In this problem, the inputs (the house location, living area, number of bedrooms, number of bathrooms, land size, type of heating system, etc.) are well-defined, and even standardised for sharing the housing market information between different real estate agencies.

• The output is also well-defined – we know what we are trying to predict.

• The features of recently sold houses and their sales prices are examples, which we use for training the neural network.



Case study 5: Prediction Neural NetworksCase study 5: Prediction Neural Networks

1Location rating 7 0.7000House condition 8 0.8000

3

5

7

10

Number of bedrooms 3 0.7500Number of bathrooms 1 0.5000Living area 121.00 m2 0.3550Land size 682.00 m2 0.4682Garage double 1.0000

1.0000Type of heating system 0.0000

0.0000

wood heater

2

4

6

8

9

$120,9200.3546



Validating the ResultsValidating the Results• To validate results, we use a set of examples never seen by the

network.

• Before training, all the available data are randomly divided into a training set and a test set.

• Once the training phase is complete, the network’s ability to generalise is tested against examples of the test set.



Case study 6: Classification Neural Networks Case study 6: Classification Neural Networks with Competitive Learningwith Competitive Learning

• As an example, we will consider an iris plant classification problem.

• Suppose, we are given a data set with several variables but we have no idea how to separate it into different classes because we cannot find any unique or distinctive features in the data.




• Neural networks can discover significant features in input patterns and learn how to separate input data into different classes. A neural network with competitive learning is a suitable tool to accomplish this task.

• The competitive learning rule enables a single-layer neural network to combine similar input data into groups or clusters.

• This process is called clustering. Each cluster is represented by a single output.




• For this case study, we will use a data set of 150 elements that contains three classes of iris plants – setosa, versicolor and virginica.

• Each plant in the data set is represented by four variables: sepal length, sepal width, petal length and petal width. The sepal length ranges between 4.3 and 7.9 cm, sepal width between 2.0 and 4.4 cm, petal length between 1.0 and 6.9 cm, and petal width between 0.1 and 2.5 cm.







• The next step is to generate training and test sets from the available data. The 150-element Iris data is randomly divided into a training set of 100 elements and a test set of 50 elements.

• Now we can train the competitive neural network to divide input vectors into three classes.






ClosingClosing

• Questions???

• Remarks???

• Comments!!!

• Evaluation!

AI – CS364 Knowledge Engineering 07 th November 2006 Dr Bogdan L. Vrusias [email protected].

Documents