4/25/2014 mowais@kfupm.edu.sa Type-2 Fuzzy Logic Advisor for Evaluating Students Cooperative Training Owais Ahmed Malik King Fahd University of Petroleum.
Post on 28-Mar-2015
213 Views
Preview:
Transcript
04/10/23 mowais@kfupm.edu.sa
Type-2 Fuzzy Logic Advisor Type-2 Fuzzy Logic Advisor for Evaluating Students’ for Evaluating Students’
Cooperative TrainingCooperative Training
Owais Ahmed MalikOwais Ahmed Malik
King Fahd University of Petroleum & King Fahd University of Petroleum & Minerals Minerals (KFUPM/HBCC)(KFUPM/HBCC)
Saudi ArabiaSaudi Arabia
3rd UK Workshop on AI in Education
mowais@kfupm.edu.sa04/10/23 2
OverviewOverview
IntroductionIntroduction
Cooperative Training Assessment Cooperative Training Assessment
Motivation for the Perception-based Motivation for the Perception-based Assessment Assessment
Fuzzy Logic and Fuzzy Logic SystemFuzzy Logic and Fuzzy Logic System
Proposed Model for Cooperative Training Proposed Model for Cooperative Training AssessmentAssessment
Experiments and DiscussionExperiments and Discussion
Conclusions and Future DirectionsConclusions and Future Directions
mowais@kfupm.edu.sa04/10/23 3
IntroductionIntroduction
Students’ learning performance is Students’ learning performance is measured by some measured by some evaluation means.evaluation means.
Students’ EvaluationStudents’ Evaluation Process of collecting students’ workProcess of collecting students’ work Making decision based on collected Making decision based on collected
information information
Methods of EvaluationMethods of Evaluation ObjectiveObjective SubjectiveSubjective
mowais@kfupm.edu.sa04/10/23 4
IntroductionIntroduction
Formal:Formal: HomeworkHomework QuizQuiz Written Exam Written Exam (Majors)(Majors) Lab ExamLab Exam
Informal:Informal: InterviewInterview Class ParticipationClass Participation Team WorkTeam Work Individual ProjectsIndividual Projects
How to evaluate a student?How to evaluate a student?
mowais@kfupm.edu.sa04/10/23 5
Cooperative Training Cooperative Training AssessmentAssessment
Cooperative Training/InternshipCooperative Training/Internship An important tool to develop student skillsAn important tool to develop student skills Some real work experience in industrySome real work experience in industry
A typical assessment for Coop A typical assessment for Coop training:training:
Progress reportsProgress reports Final reportFinal report Presenting the workPresenting the work External supervisor remarksExternal supervisor remarks Onsite visit by the internal supervisor Onsite visit by the internal supervisor
mowais@kfupm.edu.sa04/10/23 6
Cooperative Training Cooperative Training AssessmentAssessment
Assessment Component
Criteria for Assessment
Final Report(FR)
Format and Structure Literary Quality
Quality of Subject Matter
Progress Report(PR)
Task Description Format and Submission
Final Presentation (FP)
Content and Organization Speaking (Presentation) Skills
Response to Questions
External Evaluation (EE)
Enthusiasm and Interest in Work Ability to Learn and Search for
Information Relations with Co-Workers
Punctuality and Delivering Work on Time
mowais@kfupm.edu.sa04/10/23 7
Coop Training Coop Training AssessmentAssessment
Excellent Good Fair
Unsatisfactory
Content
All information related to the
coop training e.g. work place,
time, location, learning etc;
points are clearly presented
with all necessary description
of work done during training
Period
Sufficient information
related to coop training;
points are clearly presented
but description of work is
not thorough
Incomplete information
about coop training;adequate details
about taskscompleted during
training
Inadequate information
about coop training;
incomplete description
about tasks completed
during training
Organization
All information presented in a
logical & interesting sequence;
gives audience very clear
picture of training; goodtransitions; succinct &
clear
Most of the information
presented in logicalsequence; gives
audience anadequate picture oftraining; generally
wellorganized; good
transitions
Lacks some sequence of
information; difficulty in
following for audience;
loosely organized
No sequence of information; nounderstanding foraudience;
presentation is
disjointed
Material (Figures/Visual
Aids,Spelling /
Grammar)
Very effective use of visual aids;
clear figures and charts; no
spelling or grammaticalmistakes
Good use of visual aids;
graphics relate to text
presented; 1 or 2 spelling
/grammar mistakes
Occasional use of visual
aids; not much related to
text; few spelling/ grammar
mistakes
Little/no or ineffective
use of visual aids; many
spelling/grammarmistakes
Speaking Skills
Clear articulation; excellent
delivery with proper volume,
steady pace, good posture and
eye contact; confidence
Clear articulation; good
delivery with good pace,
usually projects voice and
good eye contact
Some mumbling low voice
and uneven pace; little eye
contact
Inaudible or too loud;
pace too slow or too fast;
no eye contact; seems
uninterested
Questions/Answers
Answers questions effectively
and smoothly with fulldescription; satisfy
audience
Answers most of the questions with littleelaboration
Answers only rudimentary
questions; very littleelaboration
Can not answer most of
the questions; no grasp
of subject
Example Rubric for Presentation Assessment:Example Rubric for Presentation Assessment:
mowais@kfupm.edu.sa04/10/23 8
Motivation for Perception-based Motivation for Perception-based AssessmentAssessment
Assessment of different components Assessment of different components of Coop training is subjective.of Coop training is subjective.
Communication skills during presentationCommunication skills during presentation Organization of presentation/reportOrganization of presentation/report Literary quality of reportLiterary quality of report Quality of subject matterQuality of subject matter Student’s attitude towards workStudent’s attitude towards work Enthusiasm and interest in workEnthusiasm and interest in work
Difficult to apply the objective Difficult to apply the objective methods to evaluate these student methods to evaluate these student activitiesactivities
mowais@kfupm.edu.sa04/10/23 9
Motivation for Perception-based Motivation for Perception-based AssessmentAssessment
Assessment mostly based on Assessment mostly based on perception of an evaluatorperception of an evaluator
Judgment in terms of words (Excellent, Judgment in terms of words (Excellent, Very Good, and Good etc.)Very Good, and Good etc.)
Conventional assessment methods Conventional assessment methods usually do not consider the usually do not consider the uncertainties in usage of wordsuncertainties in usage of words
Motivation for type-2 fuzzy set be used Motivation for type-2 fuzzy set be used to model a wordto model a word
mowais@kfupm.edu.sa04/10/23 10
Fuzzy Logic (FL)Fuzzy Logic (FL) Mathematical and Statistical techniques are Mathematical and Statistical techniques are
often unsatisfactory in decision making.often unsatisfactory in decision making. Experts make decisions with imprecise data in an Experts make decisions with imprecise data in an
uncertain world.uncertain world. They work with knowledge that is rarely defined They work with knowledge that is rarely defined
mathematically or algorithmically but uses vague mathematically or algorithmically but uses vague terminology with words.terminology with words.
FL designed to handle imprecision and FL designed to handle imprecision and uncertainty in the measurement processuncertainty in the measurement process
Methodology of computing with words (CW)Methodology of computing with words (CW) Mimics the perception-based decision Mimics the perception-based decision
making done by humansmaking done by humans
mowais@kfupm.edu.sa04/10/23 11
Fuzzy LogicFuzzy Logic
Linguistic VariableLinguistic Variable Example : Example : Age of a personAge of a person Term Set: Term Set: Young, Middle-aged, OldYoung, Middle-aged, Old etc. etc.
Each linguistic term is associated with a fuzzy Each linguistic term is associated with a fuzzy setset
Each term has a defined membership function Each term has a defined membership function (MF): (MF):
A fuzzy set A fuzzy set AA in in XX can be expressed as: can be expressed as:
oror
mowais@kfupm.edu.sa04/10/23 12
Fuzzy LogicFuzzy Logic
Example Fuzzy Set for Age:
mowais@kfupm.edu.sa04/10/23 13
Fuzzy LogicFuzzy Logic
Example Fuzzy Set for Literary Quality of a Report:
mowais@kfupm.edu.sa04/10/23 14
Type-2 Fuzzy SetType-2 Fuzzy Set Imprecise perception-based data can be Imprecise perception-based data can be
modelled by using type-2 fuzzy logicmodelled by using type-2 fuzzy logic Type-2 fuzzy set is 3-dimensionalType-2 fuzzy set is 3-dimensional representation representation Type-2 fuzzy sets help us to deal with the Type-2 fuzzy sets help us to deal with the
uncertaintyuncertainty Footprint of Uncertainty (FOU):Footprint of Uncertainty (FOU):
Bounded region in the primary membership function of a Bounded region in the primary membership function of a type-2 fuzzy set type-2 fuzzy set
2-Dimensional depiction of type-2 fuzzy sets2-Dimensional depiction of type-2 fuzzy sets Upper and Lower Membership FunctionsUpper and Lower Membership FunctionsFor more details:For more details: Mendel J. M., Mendel J. M., Uncertain Rule-Based Fuzzy Logic Uncertain Rule-Based Fuzzy Logic SystemsSystems, Prentice-Hall, Upper Saddle River, NJ 07458, (2001), Prentice-Hall, Upper Saddle River, NJ 07458, (2001)
mowais@kfupm.edu.sa04/10/23 15
FOUs, Upper and Lower FOUs, Upper and Lower MFsMFs
mowais@kfupm.edu.sa04/10/23 16
Type-2 Fuzzy Logic Type-2 Fuzzy Logic SystemSystem
Fuzzifier
Rules
Inference
Defuzzifier
Type Reducer
Output Processing
Fuzzy output setFuzzy input set
Crisp input
mowais@kfupm.edu.sa04/10/23 17
Proposed Model for Cooperative Proposed Model for Cooperative Training AssessmentTraining Assessment
Based on knowledge mining (knowledge Based on knowledge mining (knowledge engineering) methodologyengineering) methodology
Information extracted in the form of IF-Information extracted in the form of IF-THEN rules from evaluators (expertsTHEN rules from evaluators (experts))
Rules are modelled using fuzzy logic systemRules are modelled using fuzzy logic system Used as Used as FFuzzy uzzy LLogic ogic AAdvisor (FLA)dvisor (FLA) Two-stage FLA based on interval type-2 Two-stage FLA based on interval type-2
fuzzy logicfuzzy logic Each assessment component is evaluated using Each assessment component is evaluated using
an independent FLAan independent FLA Results of these FLAs are combined to calculate Results of these FLAs are combined to calculate
the final gradethe final grade
mowais@kfupm.edu.sa04/10/23 18
Structure of Proposed Structure of Proposed ModelModel
Final ReportFLA
Fuzzy setdefinitions
Fuzzy set definitions
Fuzzy Rules
Fuzzy Rules
AssessmentCriteria
Final Coop Grade
Final Report Grade Progress Report Grade External Evaluation GradeFinal Presentation Grade
Progress ReportFLA
Final PresentationFLA
ExternalEvaluation FLA
Coop EvaluationFLA
Fuzzy setdefinitions
Fuzzy setdefinitions
Fuzzy setdefinitions
Fuzzy Rules Fuzzy Rules Fuzzy Rules
AssessmentCriteria
AssessmentCriteria
AssessmentCriteria
mowais@kfupm.edu.sa04/10/23 19
Input/Output Fuzzy Sets for Input/Output Fuzzy Sets for Proposed ModelProposed Model
Input (criteria of assessment) and output Input (criteria of assessment) and output (evaluation) attributes divided into four fuzzy (evaluation) attributes divided into four fuzzy setssets
Type-2 fuzzy sets: Type-2 fuzzy sets: ExcellentExcellent, , GoodGood, , FairFair and and PoorPoor Survey results for labels of fuzzy setsSurvey results for labels of fuzzy sets
Label Mean Std. Deviation
Start End Start End
a b σa σb
Poor 0 4.7389 0 0.4898
Fair 4.7056 6.8778 0.4978 0.4295
Good 6.6556 8.7222 0.4419 0.3153
Excellent 8.4889 10.0000 0.3296 0.0000
mowais@kfupm.edu.sa04/10/23 20
Membership Functions for Membership Functions for Proposed ModelProposed Model
FOUs for Literary Quality of a Report:
mowais@kfupm.edu.sa04/10/23 21
Rules FormulationRules Formulation All possible combinations of antecedent All possible combinations of antecedent
fuzzy sets are employedfuzzy sets are employed Consequents of rules are provided by the Consequents of rules are provided by the
evaluators (experts)evaluators (experts) Each rule has a histogram of responsesEach rule has a histogram of responses Number of rules depends on the number of Number of rules depends on the number of
inputs and fuzzy sets associated with theminputs and fuzzy sets associated with them Example rule for Example rule for Coop Evaluation FLACoop Evaluation FLA
Rl: Rl: IF IF Final ReportFinal Report is is ExcellentExcellent AND AND Progress ReportProgress Report is is GoodGood AND AND Final PresentationFinal Presentation is is FairFair AND AND External EvaluationExternal Evaluation is is ExcellentExcellent THEN THEN GRADEGRADE is ( is (VERY GOODVERY GOOD))
mowais@kfupm.edu.sa04/10/23 22
Type-1 FLA (Individual Type-1 FLA (Individual FLA)FLA)
mowais@kfupm.edu.sa04/10/23 23
Partial Histogram of Survey Partial Histogram of Survey Responses for Final Report Responses for Final Report
EvaluationEvaluation
Rule No.
Antecedent 1
Antecedent 2
Antecedent 3
Consequent Type-1 Type-2
Excellent GoodFair Poor
Cavg Clavg Cr
avg
1 Excellent Excellent Excellent 8 0 0 0 9.162 9.077 9.242
2 Excellent Excellent Good 6 2 0 0 8.783 8.688 8.874
3 Excellent Excellent Fair 4 3 1 0 8.17 8.061 8.276
4 Excellent Excellent Poor 0 5 2 1 6.533 6.4 6.666
5 Excellent Good Excellent 6 2 0 0 8.783 8.688 8.874
6 Excellent Good Good 3 4 1 0 7.98 7.866 8.093
7 Excellent Good Fair 0 5 3 0 6.943 6.806 7.079
8 Excellent Good Poor 0 4 3 1 6.298 6.162 6.435
9 Excellent Fair Excellent 2 5 1 0 7.791 7.671 7.909
10 Excellent Fair Good 0 6 2 0 7.178 7.044 7.311
11 Excellent Fair Fair 0 5 3 0 6.943 6.806 7.079
12 Excellent Fair Poor 0 2 5 1 5.829 5.686 5.972
13 Excellent Poor Excellent 0 3 4 1 6.064 5.924 6.204
14 Excellent Poor Good 0 3 4 1 6.064 5.924 6.204
15 Excellent Poor Fair 0 0 6 2 4.95 4.804 5.097
mowais@kfupm.edu.sa04/10/23 24
Comparison for Individual and Type-1 Consensus FLAs
Experiments and Experiments and DiscussionDiscussion
mowais@kfupm.edu.sa04/10/23 25
Experiments and Experiments and DiscussionDiscussion
Comparison for Individual and Type-2 Consensus FLAs (50% uncertainty)
mowais@kfupm.edu.sa04/10/23 26
Experiments and Experiments and DiscussionDiscussion
Comparison for Individual and Type-2 Consensus FLAs (100% uncertainty)
mowais@kfupm.edu.sa04/10/23 27
ConclusionsConclusions
Type-2 fuzzy sets model the perception-Type-2 fuzzy sets model the perception-based evaluation based evaluation
Proposed model has the potential to Proposed model has the potential to capture the uncertainties in subjective capture the uncertainties in subjective evaluation evaluation
Successful testing for small group of Successful testing for small group of studentsstudents
Provides more accurate evaluation of a Provides more accurate evaluation of a student as compared to existing methodstudent as compared to existing method
mowais@kfupm.edu.sa04/10/23 28
Future DirectionsFuture Directions Testing of the system for large number of Testing of the system for large number of
studentsstudents Investigating the use of the system for other Investigating the use of the system for other
courses/situations e.g. assessing group projects courses/situations e.g. assessing group projects etcetc..
Type-2 fuzzy sets to be tested for representing Type-2 fuzzy sets to be tested for representing final gradesfinal grades
Deciding the optimal number of linguistic Deciding the optimal number of linguistic input/output variables for assessment input/output variables for assessment componentscomponents
Working with non-singleton input from evaluatorsWorking with non-singleton input from evaluators
04/10/23 mowais@kfupm.edu.sa
Thank YouThank You
mowais@kfupm.edu.sa04/10/23 30
Question/Question/AnswersAnswers
top related