Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data.

Welcome

Workshop Goals:Understand data sources

Learn the basic components of an effective assessment

Analyze and effectively use data

Overview of NOCTI

Over 40 years serving the CTE community Non-profit organization

Owned and operated by the 50 states

Works with both secondary and post-secondary populations

The Whitener Group (a NOCTI subsidiary) provides services to industry

NOCTI Customer Breakdown

Industry Connections

NOCTI Products

Supplemental Products

Over 80 Job Ready tests offered

30 Pathway tests in 10 clusters available

Over 90 private industry employment and/or advancement tests delivered

20 teacher-level assessments

Assessments Study guides for all Job Ready

and Pathway assessments Countdown Kits offered for 40

titles Successful Schools-Successful

Programs Guide

NOCTI Job Ready Assessments

Assess technical skills at the occupation level (e.g., Accounting, Carpentry, Pre-Engineering)

Measure aspects of occupational competence such as factual and theoretical knowledge

Target secondary and post-secondary program completers

Assessment Blueprints

Blueprints include:

Competency List

Content Distribution

Sample Items and Jobs

NOCTI Job Ready Assessment Blueprint

Number of Questions: 222 Areas Covered:

AccountingAccounting

Assessment Blueprints to Prepare

How can I prepare my students for the NOCTI assessment?Assessment Blueprints!!!

Ideas for using blueprints in the classroom:

Use throughout the school year as a syllabus and guide for review

Involve the students: have them calculate the number of questions in each content area

Send a copy home with students to share with parent

Assessment Results

NOCTI’s Standard Score Reporting Package: Individual Score Reports Group Score Reports Comparative Data

Other Reports Available: Task-Link Reports Integrated Academic Reports

NOCTI Standard Score Reports

Individual Score Individual Score Reports Reports

•Provided for each test taker•Scores by duty and total•Pre-/post-test comparisons•Comparative data

Individual Score Individual Score Reports Reports

•Provided for each test taker•Scores by duty and total•Pre-/post-test comparisons•Comparative data

NOCTI Standard Score Reports

Group Score ReportGroup Score Report

•Provided for the testing group•Duty and total scores for each participant

Group Score ReportGroup Score Report

•Provided for the testing group•Duty and total scores for each participant

NOCTI Standard Score Reports

Analysis of Scores Analysis of Scores ReportReport

•Provided for the testing group•Comparative data•Criterion-referenced cut score•Other statistical information

Analysis of Scores Analysis of Scores ReportReport

•Provided for the testing group•Comparative data•Criterion-referenced cut score•Other statistical information

Using Data to Improve Instruction

Using Data to Improve Instruction

Collect Data:Become familiar with

your data

Understanding Your Data

School Program Teacher Class

Student Standards

National State District/System

What the Data Reveal

Achievement gapsDifferences between subgroupsTrends Progress

Assessment

Includes process of collecting, analyzing, and interpreting information

Determines the degree to which students are becoming competent and meeting standards

Occurs at many levels (e.g., classroom and program assessments to state and national assessments)

Good Assessments

Valid Test items measure what they are supposed to

measure: the important content or concepts that were taught and that the student must know

Test is relevant and based on current standards Subject matter experts develop and evaluate

test items

Good Assessments

Reliable Test and its items produce a similar

score over time Scores are consistent, not contradictory Scores are not affected by chance

Student Assessment

On-going with frequent opportunities for feedback Use a variety of different types of assessment (e.g.,

written and performance, formative and summative) during the year

Summative Assessment

Pretests (covering all major content areas) are used to identify entry skills and measure gain

End-of-program or annual test Broader in scope, for comparison of groups

Formative Assessment

Continuous correction from quizzes, homework, projects, participation

Use practice items for instructional revision before summative assessment

Building an Assessment

Test specification matrix or “Blueprint” National Standards Duties and Tasks Written/Performance Weighting Panel of Experts

Step 2

Analyze Data:Draw conclusions from data

Classroom Example

A Carpentry Class

Areas on a NOCTI Carpentry Test

Safety

Tools and Accessories

Blueprint Reading and Estimation

Foundations, Forms and Concrete

Rough Framing

Exterior Finish

Interior Systems Installation

Interior Finish

Carpentry Related Mathematics

Individual Carpentry Pre-Test

Carpentry Pre-test Areas Pre-test % CorrectStudent 1

Post-test % CorrectStudent 1

Safety 54.6 --

Tools and Accessories 30.0 --

Blueprint Reading and Estimation 41.1 --

Foundations, Forms, and Concrete 36.4 --

Rough Framing 53.8 --

Exterior Finish 36.4 --

Interior Systems Installation 40.0 --

Interior Finish 27.3 --

Carpentry Related Mathematics 35.9 --

Total 40.0

Carpentry Class Pre-Test

Carpentry Pre-test Areas

Student 1 % Correct

Student 2 % Correct

Student 3 % Correct

Student 4 % Correct

Student 5 %

Correct

Student 6 %

Correct

Class Average % Correct

Safety 54.6 45.5 36.4 27.3 27.3 54.5 40.9

Tools and Accessories

30.0 20.0 10.0 20.0 10.0 30.0 20.0

Blueprint Reading and Estimation

41.1 17.6 23.5 23.5 29.4 35.3 28.4

Foundations, Forms, and Concrete

36.4 54.5 45.5 45.5 36.4 54.5 45.5

Rough Framing 53.8 38.5 30.8 38.5 30.8 53.8 41.0

Exterior Finish 36.4 36.4 27.3 36.4 36.4 45.5 36.4

Interior Systems Installation

40.0 60.0 40.0 60.0 60.0 80.0 56.7

Interior Finish 27.3 9.1 18.2 27.3 9.1 27.3 19.7

Carpentry Related Mathematics

35.9 45.5 27.3 36.4 36.4 54.5 39.4

Total 40.0 34.0 28.0 33.0 29.0 46.0 35.0

Carpentry Class Post-Test

Carpentry Post-test Areas

Participant Group Site State Nation

Safety 100.0 86.4 79.5 72.5 75.5

Tools and Accessories 91.7 79.2 78.2 70.7 70.6

Blueprint Reading and Estimation

100.0 95.0 83.8 64.3 66.4

Foundations, Forms, and Concrete

92.3 69.2 65.8 51.6 52.9

Rough Framing 97.4 71.8 69.8 63.1 64.1

Exterior Finish 91.7 64.6 67.6 58.1 57.8

Interior Systems Installation 87.5 53.1 61.9 55.6 56.5

Interior Finish 87.5 79.7 75.6 66.6 67.0

Carpentry Related Mathematics

100.0 56.3 68.2 66.1 67.0

Total 94.3 72.0 71.4 62.7 63.4

Thank You!

Questions?

www.nocti.org