4/25/2014 [email protected] Type-2 Fuzzy Logic Advisor for Evaluating Students Cooperative Training Owais Ahmed Malik King Fahd University of Petroleum.

04/10/23 [email protected]

Type-2 Fuzzy Logic Advisor Type-2 Fuzzy Logic Advisor for Evaluating Students’ for Evaluating Students’

Cooperative TrainingCooperative Training

Owais Ahmed MalikOwais Ahmed Malik

King Fahd University of Petroleum & King Fahd University of Petroleum & Minerals Minerals (KFUPM/HBCC)(KFUPM/HBCC)

Saudi ArabiaSaudi Arabia

3rd UK Workshop on AI in Education

[email protected]/10/23 2

OverviewOverview

IntroductionIntroduction

Cooperative Training Assessment Cooperative Training Assessment

Motivation for the Perception-based Motivation for the Perception-based Assessment Assessment

Fuzzy Logic and Fuzzy Logic SystemFuzzy Logic and Fuzzy Logic System

Proposed Model for Cooperative Training Proposed Model for Cooperative Training AssessmentAssessment

Experiments and DiscussionExperiments and Discussion

Conclusions and Future DirectionsConclusions and Future Directions



Students’ learning performance is Students’ learning performance is measured by some measured by some evaluation means.evaluation means.

Students’ EvaluationStudents’ Evaluation Process of collecting students’ workProcess of collecting students’ work Making decision based on collected Making decision based on collected

information information

Methods of EvaluationMethods of Evaluation ObjectiveObjective SubjectiveSubjective



Formal:Formal: HomeworkHomework QuizQuiz Written Exam Written Exam (Majors)(Majors) Lab ExamLab Exam

Informal:Informal: InterviewInterview Class ParticipationClass Participation Team WorkTeam Work Individual ProjectsIndividual Projects

How to evaluate a student?How to evaluate a student?


Cooperative Training Cooperative Training AssessmentAssessment

Cooperative Training/InternshipCooperative Training/Internship An important tool to develop student skillsAn important tool to develop student skills Some real work experience in industrySome real work experience in industry

A typical assessment for Coop A typical assessment for Coop training:training:

Progress reportsProgress reports Final reportFinal report Presenting the workPresenting the work External supervisor remarksExternal supervisor remarks Onsite visit by the internal supervisor Onsite visit by the internal supervisor


Cooperative Training Cooperative Training AssessmentAssessment

Assessment Component

Criteria for Assessment

Final Report(FR)

Format and Structure Literary Quality

Quality of Subject Matter

Progress Report(PR)

Task Description Format and Submission

Final Presentation (FP)

Content and Organization Speaking (Presentation) Skills

Response to Questions

External Evaluation (EE)

Enthusiasm and Interest in Work Ability to Learn and Search for

Information Relations with Co-Workers

Punctuality and Delivering Work on Time


Coop Training Coop Training AssessmentAssessment

Excellent Good Fair

Unsatisfactory

Content

All information related to the

coop training e.g. work place,

time, location, learning etc;

points are clearly presented

with all necessary description

of work done during training

Period

Sufficient information

related to coop training;

points are clearly presented

but description of work is

not thorough

Incomplete information

about coop training;adequate details

about taskscompleted during

training

Inadequate information

about coop training;

incomplete description

about tasks completed

during training

Organization

All information presented in a

logical & interesting sequence;

gives audience very clear

picture of training; goodtransitions; succinct &

clear

Most of the information

presented in logicalsequence; gives

audience anadequate picture oftraining; generally

wellorganized; good

transitions

Lacks some sequence of

information; difficulty in

following for audience;

loosely organized

No sequence of information; nounderstanding foraudience;

presentation is

disjointed

Material (Figures/Visual

Aids,Spelling /

Grammar)

Very effective use of visual aids;

clear figures and charts; no

spelling or grammaticalmistakes

Good use of visual aids;

graphics relate to text

presented; 1 or 2 spelling

/grammar mistakes

Occasional use of visual

aids; not much related to

text; few spelling/ grammar

mistakes

Little/no or ineffective

use of visual aids; many

spelling/grammarmistakes

Speaking Skills

Clear articulation; excellent

delivery with proper volume,

steady pace, good posture and

eye contact; confidence

Clear articulation; good

delivery with good pace,

usually projects voice and

good eye contact

Some mumbling low voice

and uneven pace; little eye

contact

Inaudible or too loud;

pace too slow or too fast;

no eye contact; seems

uninterested

Questions/Answers

Answers questions effectively

and smoothly with fulldescription; satisfy

audience

Answers most of the questions with littleelaboration

Answers only rudimentary

questions; very littleelaboration

Can not answer most of

the questions; no grasp

of subject

Example Rubric for Presentation Assessment:Example Rubric for Presentation Assessment:


Motivation for Perception-based Motivation for Perception-based AssessmentAssessment

Assessment of different components Assessment of different components of Coop training is subjective.of Coop training is subjective.

Communication skills during presentationCommunication skills during presentation Organization of presentation/reportOrganization of presentation/report Literary quality of reportLiterary quality of report Quality of subject matterQuality of subject matter Student’s attitude towards workStudent’s attitude towards work Enthusiasm and interest in workEnthusiasm and interest in work

Difficult to apply the objective Difficult to apply the objective methods to evaluate these student methods to evaluate these student activitiesactivities


Motivation for Perception-based Motivation for Perception-based AssessmentAssessment

Assessment mostly based on Assessment mostly based on perception of an evaluatorperception of an evaluator

Judgment in terms of words (Excellent, Judgment in terms of words (Excellent, Very Good, and Good etc.)Very Good, and Good etc.)

Conventional assessment methods Conventional assessment methods usually do not consider the usually do not consider the uncertainties in usage of wordsuncertainties in usage of words

Motivation for type-2 fuzzy set be used Motivation for type-2 fuzzy set be used to model a wordto model a word


Fuzzy Logic (FL)Fuzzy Logic (FL) Mathematical and Statistical techniques are Mathematical and Statistical techniques are

often unsatisfactory in decision making.often unsatisfactory in decision making. Experts make decisions with imprecise data in an Experts make decisions with imprecise data in an

uncertain world.uncertain world. They work with knowledge that is rarely defined They work with knowledge that is rarely defined

mathematically or algorithmically but uses vague mathematically or algorithmically but uses vague terminology with words.terminology with words.

FL designed to handle imprecision and FL designed to handle imprecision and uncertainty in the measurement processuncertainty in the measurement process

Methodology of computing with words (CW)Methodology of computing with words (CW) Mimics the perception-based decision Mimics the perception-based decision

making done by humansmaking done by humans


Fuzzy LogicFuzzy Logic

Linguistic VariableLinguistic Variable Example : Example : Age of a personAge of a person Term Set: Term Set: Young, Middle-aged, OldYoung, Middle-aged, Old etc. etc.

Each linguistic term is associated with a fuzzy Each linguistic term is associated with a fuzzy setset

Each term has a defined membership function Each term has a defined membership function (MF): (MF):

A fuzzy set A fuzzy set AA in in XX can be expressed as: can be expressed as:

oror



Example Fuzzy Set for Age:



Example Fuzzy Set for Literary Quality of a Report:


Type-2 Fuzzy SetType-2 Fuzzy Set Imprecise perception-based data can be Imprecise perception-based data can be

modelled by using type-2 fuzzy logicmodelled by using type-2 fuzzy logic Type-2 fuzzy set is 3-dimensionalType-2 fuzzy set is 3-dimensional representation representation Type-2 fuzzy sets help us to deal with the Type-2 fuzzy sets help us to deal with the

uncertaintyuncertainty Footprint of Uncertainty (FOU):Footprint of Uncertainty (FOU):

Bounded region in the primary membership function of a Bounded region in the primary membership function of a type-2 fuzzy set type-2 fuzzy set

2-Dimensional depiction of type-2 fuzzy sets2-Dimensional depiction of type-2 fuzzy sets Upper and Lower Membership FunctionsUpper and Lower Membership FunctionsFor more details:For more details: Mendel J. M., Mendel J. M., Uncertain Rule-Based Fuzzy Logic Uncertain Rule-Based Fuzzy Logic SystemsSystems, Prentice-Hall, Upper Saddle River, NJ 07458, (2001), Prentice-Hall, Upper Saddle River, NJ 07458, (2001)


FOUs, Upper and Lower FOUs, Upper and Lower MFsMFs


Type-2 Fuzzy Logic Type-2 Fuzzy Logic SystemSystem

Fuzzifier

Rules

Inference

Defuzzifier

Type Reducer

Output Processing

Fuzzy output setFuzzy input set

Crisp input


Proposed Model for Cooperative Proposed Model for Cooperative Training AssessmentTraining Assessment

Based on knowledge mining (knowledge Based on knowledge mining (knowledge engineering) methodologyengineering) methodology

Information extracted in the form of IF-Information extracted in the form of IF-THEN rules from evaluators (expertsTHEN rules from evaluators (experts))

Rules are modelled using fuzzy logic systemRules are modelled using fuzzy logic system Used as Used as FFuzzy uzzy LLogic ogic AAdvisor (FLA)dvisor (FLA) Two-stage FLA based on interval type-2 Two-stage FLA based on interval type-2

fuzzy logicfuzzy logic Each assessment component is evaluated using Each assessment component is evaluated using

an independent FLAan independent FLA Results of these FLAs are combined to calculate Results of these FLAs are combined to calculate

the final gradethe final grade


Structure of Proposed Structure of Proposed ModelModel

Final ReportFLA

Fuzzy setdefinitions

Fuzzy set definitions

Fuzzy Rules

Fuzzy Rules

AssessmentCriteria

Final Coop Grade

Final Report Grade Progress Report Grade External Evaluation GradeFinal Presentation Grade

Progress ReportFLA

Final PresentationFLA

ExternalEvaluation FLA

Coop EvaluationFLA




Fuzzy Rules Fuzzy Rules Fuzzy Rules

AssessmentCriteria

AssessmentCriteria

AssessmentCriteria


Input/Output Fuzzy Sets for Input/Output Fuzzy Sets for Proposed ModelProposed Model

Input (criteria of assessment) and output Input (criteria of assessment) and output (evaluation) attributes divided into four fuzzy (evaluation) attributes divided into four fuzzy setssets

Type-2 fuzzy sets: Type-2 fuzzy sets: ExcellentExcellent, , GoodGood, , FairFair and and PoorPoor Survey results for labels of fuzzy setsSurvey results for labels of fuzzy sets

Label Mean Std. Deviation

Start End Start End

a b σa σb

Poor 0 4.7389 0 0.4898

Fair 4.7056 6.8778 0.4978 0.4295

Good 6.6556 8.7222 0.4419 0.3153

Excellent 8.4889 10.0000 0.3296 0.0000


Membership Functions for Membership Functions for Proposed ModelProposed Model

FOUs for Literary Quality of a Report:


Rules FormulationRules Formulation All possible combinations of antecedent All possible combinations of antecedent

fuzzy sets are employedfuzzy sets are employed Consequents of rules are provided by the Consequents of rules are provided by the

evaluators (experts)evaluators (experts) Each rule has a histogram of responsesEach rule has a histogram of responses Number of rules depends on the number of Number of rules depends on the number of

inputs and fuzzy sets associated with theminputs and fuzzy sets associated with them Example rule for Example rule for Coop Evaluation FLACoop Evaluation FLA

Rl: Rl: IF IF Final ReportFinal Report is is ExcellentExcellent AND AND Progress ReportProgress Report is is GoodGood AND AND Final PresentationFinal Presentation is is FairFair AND AND External EvaluationExternal Evaluation is is ExcellentExcellent THEN THEN GRADEGRADE is ( is (VERY GOODVERY GOOD))


Type-1 FLA (Individual Type-1 FLA (Individual FLA)FLA)


Partial Histogram of Survey Partial Histogram of Survey Responses for Final Report Responses for Final Report

EvaluationEvaluation

Rule No.

Antecedent 1

Antecedent 2

Antecedent 3

Consequent Type-1 Type-2

Excellent GoodFair Poor

Cavg Clavg Cr

avg

1 Excellent Excellent Excellent 8 0 0 0 9.162 9.077 9.242

2 Excellent Excellent Good 6 2 0 0 8.783 8.688 8.874

3 Excellent Excellent Fair 4 3 1 0 8.17 8.061 8.276

4 Excellent Excellent Poor 0 5 2 1 6.533 6.4 6.666

5 Excellent Good Excellent 6 2 0 0 8.783 8.688 8.874

6 Excellent Good Good 3 4 1 0 7.98 7.866 8.093

7 Excellent Good Fair 0 5 3 0 6.943 6.806 7.079

8 Excellent Good Poor 0 4 3 1 6.298 6.162 6.435

9 Excellent Fair Excellent 2 5 1 0 7.791 7.671 7.909

10 Excellent Fair Good 0 6 2 0 7.178 7.044 7.311

11 Excellent Fair Fair 0 5 3 0 6.943 6.806 7.079

12 Excellent Fair Poor 0 2 5 1 5.829 5.686 5.972

13 Excellent Poor Excellent 0 3 4 1 6.064 5.924 6.204

14 Excellent Poor Good 0 3 4 1 6.064 5.924 6.204

15 Excellent Poor Fair 0 0 6 2 4.95 4.804 5.097


Comparison for Individual and Type-1 Consensus FLAs

Experiments and Experiments and DiscussionDiscussion



Comparison for Individual and Type-2 Consensus FLAs (50% uncertainty)



Comparison for Individual and Type-2 Consensus FLAs (100% uncertainty)


ConclusionsConclusions

Type-2 fuzzy sets model the perception-Type-2 fuzzy sets model the perception-based evaluation based evaluation

Proposed model has the potential to Proposed model has the potential to capture the uncertainties in subjective capture the uncertainties in subjective evaluation evaluation

Successful testing for small group of Successful testing for small group of studentsstudents

Provides more accurate evaluation of a Provides more accurate evaluation of a student as compared to existing methodstudent as compared to existing method


Future DirectionsFuture Directions Testing of the system for large number of Testing of the system for large number of

studentsstudents Investigating the use of the system for other Investigating the use of the system for other

courses/situations e.g. assessing group projects courses/situations e.g. assessing group projects etcetc..

Type-2 fuzzy sets to be tested for representing Type-2 fuzzy sets to be tested for representing final gradesfinal grades

Deciding the optimal number of linguistic Deciding the optimal number of linguistic input/output variables for assessment input/output variables for assessment componentscomponents

Working with non-singleton input from evaluatorsWorking with non-singleton input from evaluators

04/10/23 [email protected]

Thank YouThank You


Question/Question/AnswersAnswers

4/25/2014 [email protected] Type-2 Fuzzy Logic Advisor for Evaluating Students Cooperative Training Owais Ahmed Malik King Fahd University of Petroleum.

Documents

presentation assessment

training organization

coop training points

typical assessment

adequate picture of

coop training adequate

work ability

work place