Michigan Data Quality Institute January 2006 James R. Stone III National Research Center for Career and Technical Education University of Minnesota
Mar 27, 2015
Michigan Data Quality InstituteJanuary 2006
James R. Stone IIINational Research Center for
Career and Technical EducationUniversity of Minnesota
Today’s agenda
What tech prep was ‘sposed to beWhat the feds think about tech prepLet’s talk dataWhat other states are doing.
According to Title II (Section 202), a Tech-Prep program is a program of study that
(A) combines at a minimum two years of secondary education with a minimum of two years of postsecondary education
(B) integrates academic, and vocational and technical, instruction, and utilizes work-based and worksite learning where appropriate and available;
(C) provides technical preparation in a career field (D) builds student competence in mathematics, science,
reading, writing, communications, economics, and workplace skills through applied contextual academics, and integrated instruction, in a coherent sequence of courses;
(E) leads to an associate or baccalaureate degree or a postsecondary certificate in a specific career field;
(F) leads to placement in appropriate employment or to further education.
Elements of a Tech Prep Program Today(USDE) Perkins requires that Tech Prep programs have seven elements:
an articulation agreement between secondary and postsecondary consortium participants;
a 2+2 , 3+2 or a 4+2 design with a common core of proficiency in math, science, communication, and technology;
a specifically developed Tech Prep curriculum; joint in-service training of secondary and postsecondary
teachers to implement the Tech Prep curriculum effectively; training of counselors to recruit students and to ensure program
completion and appropriate employment; equal access of special populations to the full range of Tech
Prep programs; preparatory services such as recruitment,
career and personal counseling, and occupational assessment.
National Data on Tech Prep
Only 47 percent of all high schools in 2000 reported offering something they call “Tech-Prep.”
The set of activities pointed to as evidence of Tech-Prep are quite modest in many schools, and some preceded federal support for Tech-Prep.
A highly structured form of Tech-Prep is rarely found in U.S.high schools (10%)
Tech-Prep and non-Tech-Prep students attend college at roughly comparable rates.
NAVE: The good news?
Tech-Prep funds have helped spur community colleges to work with local secondary schools on a variety of issues including student recruitment and articulation agreements
NAVE: The bad news
Reported participation in Tech-Prep - student increases in student counts - should be viewed cautiously.
Access to and funding of Tech-Prep do not reflect the Perkins Act’s targeting criteria.
Tech-Prep is rarely implemented as a comprehensive program of study; implementation focuses on individual components of Tech-Prep, some of which are becoming more common in vocational education in general.
Defining the Tech Prep Student•Student takes/completes articulated vocational courses. 35%•Student develops an individual student plan (indicating a planned course sequence across the HS & PS lev 27%•Student explicitly elects Tech-Prep as a path, major, • track, or program (e.g.,applies/chooses to be in Tech-Prep) 24% Student takes/completes one or more vocational courses. 19%•Student takes College-Prep-level academic courses. 18%•Student takes/completes one or more applied academic courses (e.g., principles of technology). 11%•Student participates in work/training experience(s) at an employer worksite related to a Tech-Prep course. 8% All vocational students are “in Tech-Prep.” 4%
Tech-Prep participation is most often defined by enrollment in an articulated
course, an increasingly common feature of vocational education in
general.
A student enrolled in a state-approved secondary career and
technical education program that ispart of an articulated Tech Prep
Program that leads to an associate degree or certificate in a
specific career area.
NAVE Findings on Tech Prep Elements Articulation agreements Work based learning Applied academics
Articulation Agreements: The cornerstone of Tech Prep Course-to-course articulation rather than on
program articulation (Not the vision in legislation) Few Tech-Prep students receive articulated
college credit. However, articulation efforts have
(1) stimulated communication between secondary and postsecondary vocational faculty and
(2) improved the rigor and consistency of some secondary vocational curricula, by encouraging high schools to adopt college curricula and instructional materials as proof of course equivalency
Work Based Learning
Worksite experiences are not a focus of recent Tech-Prep efforts.
Most of the workplace experiences in which Tech-Prep students engaged were available to all students in their schools rather than targeted specifically to Tech-Prep students.
Applied Academics
Early enthusiasm has waned somewhat because:
“Off-the-shelf” applied curriculum packages were expensive
Stigma problem Tech-Prep efforts to encourage academic
teachers to develop and adopt applied approaches has not been widespread
Pressures of state academic standards and high school exit exams have made it more difficult to promote these approaches
No Progress in PS Settings
Tech-Prep is largely viewed as an effort to upgrade secondary vocational education; it has little impact on postsecondary courses.
Tech-Prep has little impact on college curricula and services.
Relatively few postsecondary institutions have offered new courses for students entering with Tech-Prep experience or modified other courses
No data are available about the proportion of students who complete a two-plus two program or the benefits of doing so.
NAVE Tech Prep Conclusions
Tech-Prep was a catalyst for certain vocational reform activities but, because few schools implement it as a comprehensive program of study, it is now playing less of a distinctive role
Tech-Prep efforts now overlap substantially with those of regular vocational education.
Eliminate Tech-Prep as a separate title, folding its key activities into postsecondary institutions’ responsibilities.
The Big Problem: Your Problem . . .
Inadequate data systems limit the tracking of Tech-Prep students into postsecondary
partner institutions.
In a Perfect World, Tech Prep is a set of Structured Programs
That combine the different elements outlined in the law for those students who choose to participate.
Where students would commit to a program’s career focus That include a well-defined sequence of academic and
vocational courses leading to employment in a specific career field
Where students would be clustered together in many if not most classes.
Where secondary programs would identify the particular program(s) at the postsecondary level in which students are expected to continue, and
Where Articulation agreements really work Where PS institutions would offer a differentiated credential as a
signal to employers that the Tech Prep student was special.
Data
Who Uses It and Why
General Notions about Data Quality
Reliability and validity Utility Objectivity Integrity
Reliability
Reliability is the extent to which any measuring procedure yields the same result on repeated trials.
Validity
Validity refers to the degree to which a study accurately reflects or assesses the specific concept that the researcher is attempting to measure.
External validity refers to the extent to which the results of a study are generalizable
Internal validity refers to the rigor with which the study was conducted
Data Reporting—US DOE Guidelines
1. Utility:• Usefulness of the information to its intended
users. • It is achieved by staying informed of
information needs and developing new products and services where appropriate.
• Care must be taken in the review stage to ensure that the information can be clearly understood (and reproduction).
Data Reporting—US DOE Guidelines (Ctd)2. Objectivity:
• Accuracy, reliability, and unbiased nature of information.
• It is achieved by using reliable information sources and appropriate techniques to prepare information products.
• Objectivity involves both the content and the presentation of the information.
Data Reporting—US DOE Guidelines (Ctd)
3. Integrity:• Security or protection of information from
unauthorized access or revision. • Integrity ensures that the information is not
compromised through corruption or falsification. • Statutory and administrative guidelines to protect
the integrity of Department information include the following:
Privacy Act; Freedom of Information Act; OMB Circulars A-123, A-127, and A-130; Federal Policy for the Protection of Human Subjects; Family Educational Rights and Privacy Act; Computer Security Act of 1987; Government Information Security Reform Act; and National Education Statistics Act, as amended by the USA Patriot
Act.
Perkins—Tech Prep data: Accountability 3S1 Performance indicator
Placement in, retention in, and completion of, postsecondary education or advanced training, placement in military service, or placement or retention in employment.
Tracking the Tech Preppie: Possible Trajectories
PS Year 112th Grade11th Grade PS Year 2
Drop Out
Stop Out
Non Related
PS
+
Out of State
Drop Out
++
Re- medial
Student Trajectories
What are other states doing for Tech Prep Data Collection (7 States) States with a full data collection process
(secondary and postsecondary cooperation): NY (and FL)
States with data collection process carried out at the college level: OH, NC, CO, CA, CT.
Colleges use different methods (e.g., record matching)
Some states survey the college
Tech Prep Data Collection Process Key Elements1. Definition of tech prep program (or student)2. Express commitment3. Clarify the role of the designated officer4. Identification of Tech Prep Student5. Measure Progress
HS completion/program completion Transition to: 2-year college, 4-year college,
job, apprenticeship, military, other6. Postsecondary Enrollment 7. Progress
Completion Post completion: work in related/unrelated area,
other 2-year college degree, 4-year college, military, or other
Tech Prep Data Collection—DefinitionMichigan
1. Definition of tech prep/tech prep student A student enrolled in a state-approved
secondary career and technical education program that is part of an articulated Tech Prep Program that leads to an associate degree or certificate in a specific career area.
Tech Prep Data Collection—Full Model Practices: NY (and FL)
2. Contract or agreement • Contract signed by student and parent(s) (NY)• Contract signed only if participating school &
school district agrees to it (CO)• Represents the first step towards
identification/progress track (record is flagged)• Allows for:
Use of student identification (SSN,UI, Name) Early data entry (NY & FL) Early intervention of officer in charge (NY) or
information system (flag, in FL)
Tech Prep Data Collection—Full Model Practices: NY (and FL)
3. Role of designated officer• Track progress
Secondary (Meeting standards?) Postsecondary Out of education system
• Ensures data collection process• School official (secondary) (NY)• Tech prep director (postsecondary) (NY,
also CO)• High school officials / School district officials
(FL)
Tech Prep Data Collection—Full Model Practices: NY (and FL)
4. Student identification• Flag (in transcripts or in the information
system, in FL)• Based on contracts (NY)• [SSN, first/last name, birth date (CO)]
Tech Prep Data Collection—Full Model Practices: NY
5. Progress (NY only)• Secondary level (school official):
Meet tech prep/high school standards Grades and courses HS completion
Tech Prep Data Collection—Full Model Practices: NY (and FL)
6. Postsecondary enrollment Based on school or school district reporting
at secondary level (NY). Report sent to PS. Record matching once enrolled in the
postsecondary level, since information is shared and student record has been flagged (FL)
Tech Prep Data Collection—Full Model Practices: NY (and FL)-G
7. Progress (NY only) Postsecondary level (tech prep director):
Next steps after HS graduation: work, the military, apprenticeship, college (also in FL)
If community college: Student progress Graduation Degree obtained Within six months of graduation: Work
(related/unrelated field), further education, military
Computing the 3S1 Ratio in MI
Numerator: The number of 12th grade program completers who graduated the previous year and are in postsecondary education or advanced training, employment, and/or military service.
Denominator: The number of 12th grade program completers who graduated from school the previous year.
State Developed and Locally Administered Survey
Summary of options for improving data quality Incorporate SSN in K-12 records Incorporate K-12 unique identifiers in PS
records Conduct probabillistic matching of HS and PS
student records using FERPA available data Conduct electronic (web based) surveys Automate mail surveys (if used); combine
with web surveys
Tech Prep Data Collection—Obstacles
1. Lack of (or limited) definition for tech prep program and/or tech prep student that adjusts to state needs
2. No commitment with the program (contracts)
3. Non complimentary data system
Tech Prep Data Collection—Obstacles
4. Identification Data privacy Not everybody has the same
designator School districts or families oppose Alternatives:
Flags (integrated system) Contract
Tech Prep Data Collection—Data Quality Issues
Ensure quality of early data entry (FL)
(garbage in; garbage out) Reporting reviews
Matching records: Identifiers (ssn)
Tech Prep Data Collection—Additional Issues
Colleges cannot know, from transcripts, whether the student was in a tech prep program at moment of application to college. (HS transcripts are often issued earlier than completion)
Data collection comes from the college perspective, but some information maybe lost if secondary are not reported as well
Tech Prep Data Collection—Additional Issues
Data are not collected if student is in general education or is not a full time student at the college level
Data are collected for students that have declared to be in tech prep at the secondary level only if they are enrolled in a community college, regardless of continuing or not in a tech prep program
There is no tracking of high school students in tech prep if they go to another school
Questions / comments
James R. Stone III
National Research Center for Career and Technical Education—UMN
612-624-1795