What Should You Consider When Talking About Technical Assessments? John M. Townsend Tennessee Board of Regents NACTEI 27 th Annual Conference.
Post on 28-Mar-2015
216 Views
Preview:
Transcript
What Should You What Should You Consider When Talking Consider When Talking
About Technical About Technical Assessments?Assessments?
John M. TownsendJohn M. TownsendTennessee Board of RegentsTennessee Board of Regents
NACTEI 27NACTEI 27thth Annual Conference Annual Conference
Why are you assessing?Why are you assessing?
Most important questionMost important question
– For Perkins IV?For Perkins IV?
– For Program Improvement?For Program Improvement?
– For student attainment?For student attainment?
If For Student AttainmentIf For Student Attainment
At what level is appropriate?At what level is appropriate?
– Entry levelEntry level
– Mastery levelMastery level
– Industry certification levelIndustry certification level
When do you assess?When do you assess?
End-of-programEnd-of-program
End-of-courseEnd-of-course
End of PathwayEnd of Pathway
Cluster relatedCluster related
Norm- or Competency-Norm- or Competency-referencedreferenced
Norm-referencedNorm-referenced
– Comparison Comparison among among individualsindividuals
– Demonstrates Demonstrates changes of changes of students as a students as a groupgroup
Competency-Competency-referencedreferenced– Attainment of an Attainment of an
individualindividual– Measurable Measurable
learning learning outcomes based outcomes based upon industry upon industry standardsstandards
Buy or Develop?Buy or Develop?
1.1. Standardized instrumentsStandardized instruments
2.2. Centralized state-developed Centralized state-developed
instrumentsinstruments
3.3. Local assessments recognized by stateLocal assessments recognized by state
Performance or CognatePerformance or Cognate
PerformancePerformance– What the student What the student
should be able to should be able to dodo
– ChecklistChecklist– Limited number Limited number
of areas can be of areas can be assessed in assessed in specified timespecified time
CognateCognate– What the student What the student
should knowshould know
– Test-itemsTest-items– Many areas can Many areas can
be assessed in a be assessed in a specified timespecified time
Building an AssessmentBuilding an Assessment
AdoptAdopt – seek items or assessments that – seek items or assessments that
you can adopt as isyou can adopt as is
AdaptAdapt – items or assessments that can be – items or assessments that can be
modified to fit your standardsmodified to fit your standards
BuildBuild – items or assessments when others – items or assessments when others
cannot be foundcannot be found
Table of Specifications Table of Specifications (blueprint)(blueprint)
Determine competencies to be Determine competencies to be assessedassessed
Determine competency groups (3 – 10)Determine competency groups (3 – 10)
Determine percentage of each Determine percentage of each cognitive typology per groupcognitive typology per group
Development QuestionsDevelopment Questions
Is cost and time worth benefit of Is cost and time worth benefit of
test?test?
Who should develop the assessment?Who should develop the assessment?
Test security issues?Test security issues?
Multiple-choice elements:Multiple-choice elements:
StemStem
Correct responseCorrect response
Foils or distractorsFoils or distractors
Fixed-form Fixed-form vv Variable-form Variable-form
Fixed-formFixed-form
High reliabilityHigh reliability Needs high Needs high
securitysecurity
Limited number of Limited number of test items neededtest items needed
Variable-formVariable-form
Good reliabilityGood reliability Lesser security Lesser security
neededneeded Requires a large Requires a large
number of test number of test itemsitems
Assessment AdministrationAssessment Administration
Decide the Decide the mediamedia for administration for administration
Identify Identify whowho should conduct the should conduct the assessmentassessment
Determine Determine whenwhen the assessment the assessment activities should occuractivities should occur
Reporting issues:Reporting issues:
1.1. Grading of the assessmentGrading of the assessment
2.2. Assessment improvement Assessment improvement needsneeds
3.3. Reports neededReports needed
Assessment ReviewAssessment Review
ValidityValidity – the degree to – the degree to which certain inferences can which certain inferences can be made from test scores.be made from test scores.
ReliabilityReliability – the degree of – the degree of consistency between two or consistency between two or more measures of the same more measures of the same thing.thing.
Item & Assessment AnalysisItem & Assessment Analysis
Find a good statistician, Find a good statistician, psychometrician or psychometrician or educational psychologist educational psychologist !!
Ethical Issues in TestingEthical Issues in Testing
Code of Fair Testing Code of Fair Testing Practices in EducationPractices in Education
by theby theJoint Committee on Testing Practices,Joint Committee on Testing Practices,
American Educational Research Association, American American Educational Research Association, American Psychological Association & Psychological Association &
National Council on Measurement in EducationNational Council on Measurement in Education
Legal IssuesLegal Issues
Accommodations for members of Accommodations for members of special populationsspecial populations
Access to the test-items by Access to the test-items by parents and othersparents and others
Utilization of assessments in Utilization of assessments in relation to passing or graduationrelation to passing or graduation
Development CycleDevelopment Cycle
Competency learning Competency learning outcome developmentoutcome development
Blueprint developmentBlueprint developmentTest-item developmentTest-item developmentField TestingField Testing Item-analysisItem-analysis
Development CycleDevelopment Cycle
Set cut scoresSet cut scoresTest form developmentTest form developmentAssessment administrationAssessment administrationReporting of resultsReporting of results
Contact Information:Contact Information:
Dr. John M. Townsend
Executive Director, Workforce Development
Office of Academic AffairsTennessee Board of Regents
1415 Murfreesboro Pike, Suite 350Nashville, TN 37217
615-366-4428
John.Townsend@tbr.edu
top related