A Streamlined, A Streamlined, Quantitative, Web-based, Quantitative, Web-based, Course-Level Assessment Course-Level Assessment Process Process Jörg Moßbrucker Jörg Moßbrucker Glenn Wrate Glenn Wrate Mike O’Donnell Mike O’Donnell Milwaukee School of Engineering Milwaukee School of Engineering
30
Embed
A Streamlined, Quantitative, Web-based, Course-Level Assessment Process Jörg Moßbrucker Glenn Wrate Mike O’Donnell Milwaukee School of Engineering.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
A Streamlined, A Streamlined, Quantitative, Web-based,Quantitative, Web-based,Course-Level Assessment Course-Level Assessment
Say what you do, do what you Say what you do, do what you saysay
Detailed course descriptionsDetailed course descriptions– Course outlineCourse outline– Course prerequisites by topic and Course prerequisites by topic and
coursecourse– Course outcomesCourse outcomes– Course topicsCourse topics– Day-by-day descriptions of lecture and Day-by-day descriptions of lecture and
Differentiation of algebraic and Differentiation of algebraic and trigonometric functions trigonometric functions
Solution of a system of linear Solution of a system of linear equations using a calculator and/or equations using a calculator and/or computer computer
Example cont: Course Example cont: Course OutcomesOutcomes
Upon successful completion of this course, the student will:Upon successful completion of this course, the student will:
Demonstrate problem solving skills Demonstrate problem solving skills Demonstrate a standard of expertise in the understanding Demonstrate a standard of expertise in the understanding
of circuit laws and in the analysis of electrical circuit of circuit laws and in the analysis of electrical circuit Write and solve KCL and KVL equations using standard Write and solve KCL and KVL equations using standard
methods of circuit analysis for CD circuits methods of circuit analysis for CD circuits Simplify networks using source transformations and Simplify networks using source transformations and
Thevenin's/Norton's theorems Thevenin's/Norton's theorems Use the superposition principle in circuit analysis Use the superposition principle in circuit analysis Demonstrate calculator skills to solve circuit equations Demonstrate calculator skills to solve circuit equations Demonstate the ability to analyze DC circuits using SPICE Demonstate the ability to analyze DC circuits using SPICE Demonstrate circuit laboratory skills and perform DC Demonstrate circuit laboratory skills and perform DC
All the above data plus day-by-day All the above data plus day-by-day description of lecture sessions and description of lecture sessions and suggested laboratory sessions are suggested laboratory sessions are stored in a MS Access databasestored in a MS Access database
Web description is automatically Web description is automatically created from databasecreated from database
Course coordinator is responsible for Course coordinator is responsible for datadata
IN: Assessment of student IN: Assessment of student performance data of prerequisite performance data of prerequisite topicstopics– Prerequ-exam, homework etc.Prerequ-exam, homework etc.
OUT: Assessment of student OUT: Assessment of student performance data of course outcomesperformance data of course outcomes– Final examFinal exam
Assessment of student performance Assessment of student performance data generated out of Prerequ. Exam data generated out of Prerequ. Exam and Final Examand Final Exam
Must generate meaningful dataMust generate meaningful data Must generate only a marginal Must generate only a marginal
increase in workload for facultyincrease in workload for faculty
How can we generate assessment How can we generate assessment data out of already existing (exam) data out of already existing (exam) data?data?– Two-dimensional grading schemeTwo-dimensional grading scheme
How can we generate meaningful How can we generate meaningful data (i.e. is there a difference data (i.e. is there a difference between 80% and 83%)?between 80% and 83%)?– Reduction of input data rangeReduction of input data range
Web-based interfaceWeb-based interface– Allows direct entry of assessment dataAllows direct entry of assessment data– Enters data into MySQL databaseEnters data into MySQL database– Allows also entry of commentsAllows also entry of comments
Implement Changes to Implement Changes to ImproveImprove
What?What? How?How? When?When? Who?Who?
What?What?
– Course outcomes with the lowest Course outcomes with the lowest “Adequate” scores over one academic “Adequate” scores over one academic yearyear
– Collated and weighted results over all Collated and weighted results over all sections over one academic year sections over one academic year reduces noise considerablyreduces noise considerably
– At end of academic year assessment team At end of academic year assessment team assembles results and determines “critical” assembles results and determines “critical” course outcomes requiring actioncourse outcomes requiring action
– Assessment team focuses on “tracks” in EE, Assessment team focuses on “tracks” in EE, such as:such as: Digital, Analog, Circuits, Communications etc.Digital, Analog, Circuits, Communications etc.
– Course coordinators implement changes over Course coordinators implement changes over summersummer
ExampleExampleChange of a 2-course sequence in circuits without lab to a 3-course sequence with labChange of a 2-course sequence in circuits without lab to a 3-course sequence with lab
ConclusionsConclusions Assessment data generated directly from student Assessment data generated directly from student
performance data already availableperformance data already available Data reduction by grouping into three categoriesData reduction by grouping into three categories Instant graphical feedback to teaching faculty on Instant graphical feedback to teaching faculty on
performance of their sectionperformance of their section Simple, web-based interface to input assessment Simple, web-based interface to input assessment
datadata One-click calculation of assessment data of multiple One-click calculation of assessment data of multiple
sections/quarters/yearssections/quarters/years Assessment team determines annually action neededAssessment team determines annually action needed Course coordinators implement changesCourse coordinators implement changes Data provides numerical feedback of changesData provides numerical feedback of changes Program director has overview over course Program director has overview over course
outcomes data for program outcomesoutcomes data for program outcomes
Issues (past and present)Issues (past and present)
No. of prerequisites usually small -> No. of prerequisites usually small -> noise exceptionally largenoise exceptionally large– Common prerequisite exam for all sections Common prerequisite exam for all sections
of a course determined by course of a course determined by course coordinatorcoordinator
Different sections behave very Different sections behave very differentlydifferently– No common final examNo common final exam– Instructor understanding of performanceInstructor understanding of performance– Section/time/student etc. dependentSection/time/student etc. dependent– Adds to overall noiseAdds to overall noise
OverviewOverview Things I need:Things I need: General process (do what you say, measure what you do, improve)General process (do what you say, measure what you do, improve) Do what you sayDo what you say
– Course descriptionsCourse descriptions– Course outcomesCourse outcomes– Course topicsCourse topics– Role of the course coordinatorRole of the course coordinator– Example of course on-lineExample of course on-line– Where is the data storedWhere is the data stored
Measure what you doMeasure what you do– Different methods of assessmentDifferent methods of assessment– In/Out measuresIn/Out measures– Advantages/disadvantagesAdvantages/disadvantages– Numerical/non-numerical dataNumerical/non-numerical data– Common exams yes/noCommon exams yes/no– How can we use data already collected for a class?How can we use data already collected for a class?– Is there a difference between 85% and 80%? If so how muchIs there a difference between 85% and 80%? If so how much– -> reduction of in data (adequate/marginal/inadequate)-> reduction of in data (adequate/marginal/inadequate)– What do these mean?What do these mean?– How to get it out of final pre-requ?How to get it out of final pre-requ?– Multiple sources for a single course outcome from different questionsMultiple sources for a single course outcome from different questions– Computed automatically in Excel (example)Computed automatically in Excel (example)– How do we get the computed data and resultsHow do we get the computed data and results– Databases usedDatabases used– Examples of resultsExamples of results
ImproveImprove– Different possibilities how to improveDifferent possibilities how to improve– When to improveWhen to improve– Who decides when whatWho decides when what– Assessment processAssessment process
IssuesIssues– HonestyHonesty– Integrity of the course coordinatorIntegrity of the course coordinator– NoiseNoise– Still too much dataStill too much data