Top Banner
Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota
43

Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Mar 27, 2015

Download

Documents

Jessica Bailey
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Michigan Data Quality InstituteJanuary 2006

James R. Stone IIINational Research Center for

Career and Technical EducationUniversity of Minnesota

Page 2: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Today’s agenda

What tech prep was ‘sposed to beWhat the feds think about tech prepLet’s talk dataWhat other states are doing. 

Page 3: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

According to Title II (Section 202), a Tech-Prep program is a program of study that

(A) combines at a minimum two years of secondary education with a minimum of two years of postsecondary education

(B) integrates academic, and vocational and technical, instruction, and utilizes work-based and worksite learning where appropriate and available;

(C) provides technical preparation in a career field (D) builds student competence in mathematics, science,

reading, writing, communications, economics, and workplace skills through applied contextual academics, and integrated instruction, in a coherent sequence of courses;

(E) leads to an associate or baccalaureate degree or a postsecondary certificate in a specific career field;

(F) leads to placement in appropriate employment or to further education.

Page 4: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Elements of a Tech Prep Program Today(USDE) Perkins requires that Tech Prep programs have seven elements:

an articulation agreement between secondary and postsecondary consortium participants;

a 2+2 , 3+2 or a 4+2 design with a common core of proficiency in math, science, communication, and technology;

a specifically developed Tech Prep curriculum; joint in-service training of secondary and postsecondary

teachers to implement the Tech Prep curriculum effectively; training of counselors to recruit students and to ensure program

completion and appropriate employment; equal access of special populations to the full range of Tech

Prep programs; preparatory services such as recruitment,

career and personal counseling, and occupational assessment.

Page 5: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

National Data on Tech Prep

Only 47 percent of all high schools in 2000 reported offering something they call “Tech-Prep.”

The set of activities pointed to as evidence of Tech-Prep are quite modest in many schools, and some preceded federal support for Tech-Prep.

A highly structured form of Tech-Prep is rarely found in U.S.high schools (10%)

Tech-Prep and non-Tech-Prep students attend college at roughly comparable rates.

Page 6: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

NAVE: The good news?

Tech-Prep funds have helped spur community colleges to work with local secondary schools on a variety of issues including student recruitment and articulation agreements

Page 7: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

NAVE: The bad news

Reported participation in Tech-Prep - student increases in student counts - should be viewed cautiously.

Access to and funding of Tech-Prep do not reflect the Perkins Act’s targeting criteria.

Tech-Prep is rarely implemented as a comprehensive program of study; implementation focuses on individual components of Tech-Prep, some of which are becoming more common in vocational education in general.

Page 8: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Defining the Tech Prep Student•Student takes/completes articulated vocational courses. 35%•Student develops an individual student plan (indicating a planned course sequence across the HS & PS lev 27%•Student explicitly elects Tech-Prep as a path, major, • track, or program (e.g.,applies/chooses to be in Tech-Prep) 24% Student takes/completes one or more vocational courses. 19%•Student takes College-Prep-level academic courses. 18%•Student takes/completes one or more applied academic courses (e.g., principles of technology). 11%•Student participates in work/training experience(s) at an employer worksite related to a Tech-Prep course. 8% All vocational students are “in Tech-Prep.” 4%

Tech-Prep participation is most often defined by enrollment in an articulated

course, an increasingly common feature of vocational education in

general.

A student enrolled in a state-approved secondary career and

technical education program that ispart of an articulated Tech Prep

Program that leads to an associate degree or certificate in a

specific career area.

Page 9: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

NAVE Findings on Tech Prep Elements Articulation agreements Work based learning Applied academics

Page 10: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Articulation Agreements: The cornerstone of Tech Prep Course-to-course articulation rather than on

program articulation (Not the vision in legislation) Few Tech-Prep students receive articulated

college credit. However, articulation efforts have

(1) stimulated communication between secondary and postsecondary vocational faculty and

(2) improved the rigor and consistency of some secondary vocational curricula, by encouraging high schools to adopt college curricula and instructional materials as proof of course equivalency

Page 11: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Work Based Learning

Worksite experiences are not a focus of recent Tech-Prep efforts.

Most of the workplace experiences in which Tech-Prep students engaged were available to all students in their schools rather than targeted specifically to Tech-Prep students.

Page 12: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Applied Academics

Early enthusiasm has waned somewhat because:

“Off-the-shelf” applied curriculum packages were expensive

Stigma problem Tech-Prep efforts to encourage academic

teachers to develop and adopt applied approaches has not been widespread

Pressures of state academic standards and high school exit exams have made it more difficult to promote these approaches

Page 13: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

No Progress in PS Settings

Tech-Prep is largely viewed as an effort to upgrade secondary vocational education; it has little impact on postsecondary courses.

Tech-Prep has little impact on college curricula and services.

Relatively few postsecondary institutions have offered new courses for students entering with Tech-Prep experience or modified other courses

No data are available about the proportion of students who complete a two-plus two program or the benefits of doing so.

Page 14: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

NAVE Tech Prep Conclusions

Tech-Prep was a catalyst for certain vocational reform activities but, because few schools implement it as a comprehensive program of study, it is now playing less of a distinctive role

Tech-Prep efforts now overlap substantially with those of regular vocational education.

Eliminate Tech-Prep as a separate title, folding its key activities into postsecondary institutions’ responsibilities.

Page 15: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

The Big Problem: Your Problem . . .

Inadequate data systems limit the tracking of Tech-Prep students into postsecondary

partner institutions.

Page 16: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

In a Perfect World, Tech Prep is a set of Structured Programs

That combine the different elements outlined in the law for those students who choose to participate.

Where students would commit to a program’s career focus That include a well-defined sequence of academic and

vocational courses leading to employment in a specific career field

Where students would be clustered together in many if not most classes.

Where secondary programs would identify the particular program(s) at the postsecondary level in which students are expected to continue, and

Where Articulation agreements really work Where PS institutions would offer a differentiated credential as a

signal to employers that the Tech Prep student was special.

Page 17: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Data

Who Uses It and Why

Page 18: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

General Notions about Data Quality

Reliability and validity Utility Objectivity Integrity

Page 19: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Reliability

Reliability is the extent to which any measuring procedure yields the same result on repeated trials.

Page 20: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Validity

Validity refers to the degree to which a study accurately reflects or assesses the specific concept that the researcher is attempting to measure.

External validity refers to the extent to which the results of a study are generalizable

Internal validity refers to the rigor with which the study was conducted

Page 21: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Data Reporting—US DOE Guidelines

1. Utility:• Usefulness of the information to its intended

users. • It is achieved by staying informed of

information needs and developing new products and services where appropriate.

• Care must be taken in the review stage to ensure that the information can be clearly understood (and reproduction).

Page 22: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Data Reporting—US DOE Guidelines (Ctd)2. Objectivity:

• Accuracy, reliability, and unbiased nature of information.

• It is achieved by using reliable information sources and appropriate techniques to prepare information products.

• Objectivity involves both the content and the presentation of the information.

Page 23: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Data Reporting—US DOE Guidelines (Ctd)

3. Integrity:• Security or protection of information from

unauthorized access or revision. • Integrity ensures that the information is not

compromised through corruption or falsification. • Statutory and administrative guidelines to protect

the integrity of Department information include the following:

Privacy Act; Freedom of Information Act; OMB Circulars A-123, A-127, and A-130; Federal Policy for the Protection of Human Subjects; Family Educational Rights and Privacy Act; Computer Security Act of 1987; Government Information Security Reform Act; and National Education Statistics Act, as amended by the USA Patriot

Act.

Page 24: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Perkins—Tech Prep data: Accountability 3S1 Performance indicator

Placement in, retention in, and completion of, postsecondary education or advanced training, placement in military service, or placement or retention in employment.

Page 25: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tracking the Tech Preppie: Possible Trajectories

PS Year 112th Grade11th Grade PS Year 2

Drop Out

Stop Out

Non Related

PS

+

Out of State

Drop Out

++

Re- medial

Page 26: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Student Trajectories

Page 27: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

What are other states doing for Tech Prep Data Collection (7 States) States with a full data collection process

(secondary and postsecondary cooperation): NY (and FL)

States with data collection process carried out at the college level: OH, NC, CO, CA, CT.

Colleges use different methods (e.g., record matching)

Some states survey the college

Page 28: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection Process Key Elements1. Definition of tech prep program (or student)2. Express commitment3. Clarify the role of the designated officer4. Identification of Tech Prep Student5. Measure Progress

HS completion/program completion Transition to: 2-year college, 4-year college,

job, apprenticeship, military, other6. Postsecondary Enrollment 7. Progress

Completion Post completion: work in related/unrelated area,

other 2-year college degree, 4-year college, military, or other

Page 29: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—DefinitionMichigan

1. Definition of tech prep/tech prep student A student enrolled in a state-approved

secondary career and technical education program that is part of an articulated Tech Prep Program that leads to an associate degree or certificate in a specific career area.

Page 30: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Full Model Practices: NY (and FL)

2. Contract or agreement • Contract signed by student and parent(s) (NY)• Contract signed only if participating school &

school district agrees to it (CO)• Represents the first step towards

identification/progress track (record is flagged)• Allows for:

Use of student identification (SSN,UI, Name) Early data entry (NY & FL) Early intervention of officer in charge (NY) or

information system (flag, in FL)

Page 31: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Full Model Practices: NY (and FL)

3. Role of designated officer• Track progress

Secondary (Meeting standards?) Postsecondary Out of education system

• Ensures data collection process• School official (secondary) (NY)• Tech prep director (postsecondary) (NY,

also CO)• High school officials / School district officials

(FL)

Page 32: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Full Model Practices: NY (and FL)

4. Student identification• Flag (in transcripts or in the information

system, in FL)• Based on contracts (NY)• [SSN, first/last name, birth date (CO)]

Page 33: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Full Model Practices: NY

5. Progress (NY only)• Secondary level (school official):

Meet tech prep/high school standards Grades and courses HS completion

Page 34: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Full Model Practices: NY (and FL)

6. Postsecondary enrollment Based on school or school district reporting

at secondary level (NY). Report sent to PS. Record matching once enrolled in the

postsecondary level, since information is shared and student record has been flagged (FL)

Page 35: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Full Model Practices: NY (and FL)-G

7. Progress (NY only) Postsecondary level (tech prep director):

Next steps after HS graduation: work, the military, apprenticeship, college (also in FL)

If community college: Student progress Graduation Degree obtained Within six months of graduation: Work

(related/unrelated field), further education, military

Page 36: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Computing the 3S1 Ratio in MI

Numerator: The number of 12th grade program completers who graduated the previous year and are in postsecondary education or advanced training, employment, and/or military service.

Denominator: The number of 12th grade program completers who graduated from school the previous year.

State Developed and Locally Administered Survey

Page 37: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Summary of options for improving data quality Incorporate SSN in K-12 records Incorporate K-12 unique identifiers in PS

records Conduct probabillistic matching of HS and PS

student records using FERPA available data Conduct electronic (web based) surveys Automate mail surveys (if used); combine

with web surveys

Page 38: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Obstacles

1. Lack of (or limited) definition for tech prep program and/or tech prep student that adjusts to state needs

2. No commitment with the program (contracts)

3. Non complimentary data system

Page 39: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Obstacles

4. Identification Data privacy Not everybody has the same

designator School districts or families oppose Alternatives:

Flags (integrated system) Contract

Page 40: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Data Quality Issues

Ensure quality of early data entry (FL)

(garbage in; garbage out) Reporting reviews

Matching records: Identifiers (ssn)

Page 41: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Additional Issues

Colleges cannot know, from transcripts, whether the student was in a tech prep program at moment of application to college. (HS transcripts are often issued earlier than completion)

Data collection comes from the college perspective, but some information maybe lost if secondary are not reported as well

Page 42: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Tech Prep Data Collection—Additional Issues

Data are not collected if student is in general education or is not a full time student at the college level

Data are collected for students that have declared to be in tech prep at the secondary level only if they are enrolled in a community college, regardless of continuing or not in a tech prep program

There is no tracking of high school students in tech prep if they go to another school

Page 43: Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota.

Questions / comments

James R. Stone III

National Research Center for Career and Technical Education—UMN

612-624-1795

[email protected]