Development of an Assessment System for Engineering Courses Using Petri- Net Tung-Nan Institute of Technology, T wan* Tamkang University , Taiwan** Yu-Hur Chou* Hsin-Yih SHYU**
Dec 26, 2015
Development of an Assessment System for Engineering Courses Using Petri-Net
Tung-Nan Institute of Technology, Taiwan*Tamkang University , Taiwan**
Yu-Hur Chou*Hsin-Yih SHYU**
Outline
1. Objectives
2. System Architecture
3. System Analyses and Design
4. System Demonstrations (Test Demo.)
5. Conclusions
Objectives
1. To specify how to design and develop a partial scoring assessment system.
2. This assessment system can make inference diagnosis in order to investigate examinee’s misconceptions and make the reasonable scoring for engineering courses.
3. Reinforced Concrete Design is the objective course.
System Architecture
Test Generator
TestDatabase
Subject (Z)
Step 1. Select Test Subject
TestDatabase
Subject (M)
TestDatabase
Subject (A)
Active Database
Select Test Concepts
Step 2. Select Test Concepts and style.
Test Items
AnsweringEvaluator
CorrectAnswers
Concept-MappingReply sub-system
Examinees’Answers
DiagnosticEvaluator
DiagnosisResults
Step 3. User Reply
Step 4. Assessment results
Database System
: Database
: Internal Information
: Sub-system
System Analyses and Design (1/7)
1. Defining Subject Concepts.
2. Defining Subject Calculations.
3. Evaluation between Concepts and Calculations.
4. Construct the Logic Structure of Correct Answer.
5. Partial Scoring Function.
6. Misconception Evaluation.
RC Beam Analysis (with Single Layer Bar) as example
Procedures:
System Analyses and Design (2/7)
1. Defining Subject Concepts:Item Concept name Item Concept name
1 Required Resisting Strength 12 Parameter of Equivalent Stress Block
2 Stress-Strain Diagram (Reinforcement)
13 Equivalent Stress Block
3 Modulus of Elasticity (Reinforcement)
14 Concept of Under reinforced Case
4 No. Of Reinforcing Steel 15 Concept of Over reinforced Case
5 Area of Reinforcing Steel 16 Concept of Balanced Case
6 Modulus of Elasticity (Concrete) 17 Concept of Proportional Ratio
7 Ultimate Strain of Concrete 18 Equilibrium of Forces
8 Compressive Strength of Concrete
19 Solution of Linear Equations
9 Capacity Reduction Factor 20 Central of Cross-Section
10 Nominal Resisting Strength 21 Solution of second order equation
11 Designed Resisting Strength
System Analyses and Design (3/7)
2. Defining Subject Calculations:Item Calculation Name Item Calculation Name
1 Balanced Depth of Neutral Axis 12 Depth of Equivalent Stress Block
2 Modulus of Elasticity (Concrete) 13 Depth of Neutral Axis (Balance)
3 Area of Bar 14 Stress of Reinforcement (Balance)
4 Depth of Equivalent Stress Block (Balance)
15 Stress of Reinforcement (Under reinforcement)
5 Parameter of Equivalent Stress Block
16 Nominal Resisting Moment (Tension)
6 Under reinforced Case 17 Nominal Resisting Moment (Compression)
7 Area of Bar (Balance) 18 Total Tensile Force
8 Balanced Reinforcement Ratio 19 Depth of Neutral Axis (Under reinforcement)
9 Over reinforced Case 20 Depth of Neutral Axis (Over reinforcement)
10 Balanced Case 21 Stress of Reinforcement (Over reinforcement)
11 Total Compressive Force 22 Ultimate Resisting Moment
System Analyses and Design (4/7)
3. Evaluation between Concepts and Calculations:
Relationship betweenQi and Cj
Concept Cj
C1 C2 C3 ……. Cn
Calculation
ItemQi
Q1 e11 e12 e13 ……. e1n
Q2 e21 e22 e23 ……. e2n
Q3 e31 e32 e33 ……. e3n
…. …. …. …. ……. ….
Qm em1 em2 em3 ……. emn
eij = 1 (related ) , eij = 0 (irrelated )
System Analyses and Design (5/7)
4. Construct the Logic Structure of Correct Answer:
Total Tensile Force
T
Asfs
ss AfT
Parameter2
Parameter Set
Calculation 1 Calculation 2 Calculation 3 Calculation m
Calculation Set
Input Arcs Output Arcs
Parameter1
Parameter3
Parameter4
ParameterN
An independent unit
The Rule-based Petri-Net
The model of the Rule-based Petri-Net
System Analyses and Design (6/7)
5. Partial Scoring Function:Score = WC x GCI + WS x CRC WhereScore: Final score.GCI: The value of Goldsmith’s Closeness Index (Similarity between correct and examiner's concept map).CRC: The value of Correct Rate of Calculations.WC: Weight of GCI (0~1).WS: Weight of CRC (1-WC).
CRC = Σ( Pi x Di ) / ΣDi ( i = 1 ~ m ) WhereCRC: The value of Correct Rate of Calculations.Pi: The value of each calculation (0 or 1).
Di: The difficulty value of each calculation (1(easy) ~9(hard)).
m: The number of calculations within a test item.
System Analyses and Design (7/7)
6. Misconception Evaluation:
(1) Complete correctness: (2) No reply, but system infers complete correctness: (3) Correct calculation (wrong answer from mistake inheritance): (4) No reply, but system infers correct calculation: (5) Wrong answer: (6) No reply:
Calculation judgment
CRV (i) = WC x C (i) + WS x Pi ( i = 1 ~ m )
WhereCRV (i): The Complete Rating Value of calculation i.C (i): The closeness coefficient of calculation i.
System Demonstrations (1/8)
Entry Screen (Select the Test Subject)
Test Generator
Step 1. Select Test Subject
TestDatabase
Subject (M)Active Database
Select Test Concepts
Step 2. Select Test Concepts and style.
Test Items
AnsweringEvaluator
CorrectAnswers
Concept-MappingReply sub-system
Examinees’Answers
DiagnosticEvaluator
DiagnosisResults
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (2/8)
Select Concepts and Test Style
Test Generator
Step 1. Select Test Subject
TestDatabase
Subject (M)Active Database
Select Test Concepts
Step 2. Select Test Concepts and style.
Test Items
AnsweringEvaluator
CorrectAnswers
Concept-MappingReply sub-system
Examinees’Answers
DiagnosticEvaluator
DiagnosisResults
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (3/8)
Problem’s Description
Test Generator
Step 1. Select Test Subject
TestDatabase
Subject (M)Active Database
Select Test Concepts
Step 2. Select Test Concepts and style.
Test Items
AnsweringEvaluator
CorrectAnswers
Concept-MappingReply sub-system
Examinees’Answers
DiagnosticEvaluator
DiagnosisResults
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (4/8)
Answer Evaluator
Test Generator
Step 1. Select Test Subject
TestDatabase
Subject (M)Active Database
Select Test Concepts
Step 2. Select Test Concepts and style.
Test Items
AnsweringEvaluator
CorrectAnswers
Concept-MappingReply sub-system
Examinees’Answers
DiagnosticEvaluator
DiagnosisResults
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (5/8)
Concept Mapping Reply System
Test Generator
Step 1. Select Test Subject
TestDatabase
Subject (M)Active Database
Select Test Concepts
Step 2. Select Test Concepts and style.
Test Items
AnsweringEvaluator
CorrectAnswers
Concept-MappingReply sub-system
Examinees’Answers
DiagnosticEvaluator
DiagnosisResults
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (6/8)
Assessment Results
Test Generator
Step 1. Select Test Subject
TestDatabase
Subject (M)Active Database
Select Test Concepts
Step 2. Select Test Concepts and style.
Test Items
AnsweringEvaluator
CorrectAnswers
Concept-MappingReply sub-system
Examinees’Answers
DiagnosticEvaluator
DiagnosisResults
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (7/8)
Scoring Instruction
Test Generator
Step 1. Select Test Subject
TestDatabase
Subject (M)Active Database
Select Test Concepts
Step 2. Select Test Concepts and style.
Test Items
AnsweringEvaluator
CorrectAnswers
Concept-MappingReply sub-system
Examinees’Answers
DiagnosticEvaluator
DiagnosisResults
Step 3. User Reply
Step 4. Assessment results
Correct Answer Test Demonstration (3/6)
Step 4: User’s Reply Answer
36.43 cm 0.85
30.97 cm
86.99 cm2
21.85 cm
159.18 T
1(True)
36.42 cm 0.8
29.14 cm
81.87 cm2
23.21 cm
159.21 T
Conclusions
1. This assessment system adopts the dynamic state to make test items. The values of the test variables are generated randomly. Therefore, the same test problem will be shown within different variable values for different tests, to prevent students from memorizing the answers.
2. We also purpose a Rule-based Petri-Net model for building the logic structure of correct answer. This logic structure can match the dynamic pattern and random parameters completely.
3. This system also adopts concept-mapping environment as examiner’s reply system, can acquire enough information to investigate examinee’s misconceptions and measure final score by using partial scoring strategy.