Top Banner
1 Running head: ESTABLISHING EVIDENCE-BASED PRACTICES Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition Valerie L. Mazzotti Western Carolina University Dawn R. Rowe University of Oregon Renee Cameto SRI International David W. Test University of North Carolina at Charlotte Mary Morningstar University of Kansas Final Manuscript as Accepted Final Published as: Mazzotti, V. L., Rowe, D. A., Cameto, R., Test, D. W., & Morningstar, M. E. (2013). Identifying and promoting transition evidence-based practices and predictors of success: A position paper of the division on career development and transition. Career Development and Transition for Exceptional Individuals, 36(3), 140-151.
35

Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

Feb 28, 2023

Download

Documents

DeLys Ostlund
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

1

Running head: ESTABLISHING EVIDENCE-BASED PRACTICES

Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A

Position Paper of the Division on Career Development and Transition

Valerie L. Mazzotti

Western Carolina University

Dawn R. Rowe

University of Oregon

Renee Cameto

SRI International

David W. Test

University of North Carolina at Charlotte

Mary Morningstar

University of Kansas

Final Manuscript as Accepted Final Published as:

Mazzotti, V. L., Rowe, D. A., Cameto, R., Test, D. W., & Morningstar, M. E. (2013). Identifying and promoting transition evidence-based practices and predictors of success: A position paper of the division on career development and transition. Career Development and Transition for Exceptional Individuals, 36(3), 140-151.

Page 2: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 2

Abstract

This position paper describes the Division of Career Development and Transition’s (DCDT)

stance and recommendations for identifying and promoting secondary transition evidence-based

practices and predictors of post-school success for students with disabilities. Recommendations

for experimental research, correlational research, and secondary analysis of NLTS2 data are

provided. Finally, suggestions related to planning for future research in the area of secondary

transition are discussed.

Page 3: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 3

Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A

Position Paper of the Division on Career Development and Transition

Since 1990, and with subsequent amendments in 1997 and 2004, the transition provisions

of the Individuals with Disabilities Education Improvement Act (IDEA) have been a strong

impetus for secondary special education to assume a coordinated approach to planning and

facilitating the transition from school to adulthood for students with disabilities. IDEA contains

language defining transition services, as well as articulating what specifically should be included

in the Individual Education Program (IEP) when planning for transition. Despite such legislative

mandates, students with disabilities continue to leave high school unprepared for the challenges

of adulthood achieving success at a lower rate than their peers without disabilities (Newman,

Wagner, Cameto, & Knokey, 2009).

School programs focusing on post-school outcomes for students with disabilities are not

new phenomena. During the 1980s, federal initiatives to improve outcomes first pronounced the

term “transition” (Will, 1984). However, Rusch, Szymanski, and Chadsey-Rusch (1992) found

"transition services" for youth with disabilities could be traced back to the 1930s for deaf

students, and the 1940s for students with mental retardation. It was not until the 1960s that

models were developed to comprehensively address the dimensions of adult adjustment. These

early career education and vocational models were thought to be precursors to today’s transition

efforts (Halpern, 1992). Kohler (1998) advocated for a transition perspective to undergird all

secondary educational programs and activities using the Taxonomy for Transition Programming

(Kohler, 1996) consisting of five categories (a) Student Focused Planning, (b) Student

Development, (c) Family Involvement, (d) Interagency Collaboration, and (e) Program Structure.

A transition-focused educational approach was articulated that argued for transition to be

Page 4: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 4

considered not as an add-on process only for students with disabilities, but as the primary

perspective guiding secondary educational programs (Kohler, 1998). This approach grew out of

theory and research that emerged as a response to the federal initiatives (Halpern, 1985;

Wehman, 1992). Kohler and Field (2003) noted progress had been made in implementing

effective transition practices with emergent research evaluating the impact of transition services

on student outcomes. However, such efforts were preliminary and scattered among pockets of

excellence, rather than widespread and consistently implemented. Therefore, the purpose of this

paper is to describe the Division of Career Development and Transition’s (DCDT) stance and

recommendations for identifying and promoting secondary transition evidence-based practices

and predictors of post-school success for students with disabilities.

Efforts to systematize the use of effective transition practices has coincided with

educational policies both within general and special education. In particular, the U. S.

Department of Education (USDOE) emphasized a clear focus on improving the quality of

education for all children in policies outlined under the No Child Left Behind Act (NCLB; 2001)

stating "federal support is targeted to those educational programs that have been demonstrated to

be effective through rigorous scientific research." (U. S. Department of Education, n.d., para. 1).

Teachers were now required to use scientifically-based research practices to teach students with

and without disabilities (Odom, Brantlinger, Gersten, Thompson, & Harris, 2005). Secondary

transition education and services for students with disabilities were not excluded from such

authorizations. Indeed, Test, Fowler, et al. (2009) noted that IDEA 2004 echoed the NCLB

definition of scientifically-based research for special education and related/supplemental aids and

services used with children with disabilities.

Page 5: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 5

Since Odom et al. (2005) began calling scientifically-based research practices, evidence-

based practices (EBPs), it appears that EBPs have become more than another education “fad”.

Unfortunately, as the literature to identify, and then disseminate, EBPs has been reviewed, there

has been much confusion about how EBPs are defined. As a result, NSTTAC developed a

suggested set of definitions around EBPs based on the earlier work of Innes-Helsel, Hitchcock,

Miller, Malinow, and Murray (2006) and Twyman and Sota (2008). Table 1 provides a set of

definitions for describing different types of practices. Using these definitions, all practices and

predictors discussed in this paper would be considered, evidence-based.

<Insert Table 1>

Given the history of policies and practices regarding how to prepare and support youth

with disabilities to achieve successful adult outcomes, there is considerable work still to be

accomplished, particularly in ensuring educators know about, and can use, evidence-based

secondary transition practices leading to improved outcomes for students with disabilities. As

described in this position paper, more empirically sound research is needed to possess a full

arsenal of scientifically-based methods addressing the complexity of transition education and

services. Therefore, DCDT is advocating researchers increase efforts to conduct rigorous

experimental research to continue to build on current EBPs.

Additionally, as noted by Berliner (2002), a conceptualization of scientifically-based

educational research has been constructed from the research of the physical sciences where

clinical control is de rigor. Berliner argued educational research maybe the "hardest-to-do

science" (p. 18), given the complexities of educational conditions and multiple levels of

interactions impacting results (e.g., students, teachers, schools, communities). Odom et al. (2005)

furthered this distinction for special education research. Within secondary transition, this

Page 6: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 6

dilemma is manifested in current research in which EBPs may be from controlled studies that do

not take into account contextual or programmatic factors influencing post-school outcomes.

Therefore, in this position paper, we are recommending secondary transition researchers expand

their practices beyond those deemed evidence-based interventions, to also consider research

related to in-school and community predictors (e.g., transition program characteristics, family

influences) for evidence of positive relationships to improved post-school outcomes. To do this,

DCDT is recommending transition researchers focus on experimental and correlational designs.

Establishing Evidence-Based Practices and Predictors of Post-school Success

Over the years, the field of special education has been criticized for the lack of

empirically-based research methods used to teach students with disabilities (Hockenbury,

Kauffman, & Hallahan, 2000; Singer, 2000). When NCLB (2001) and IDEA (2004) required

teachers to use “scientifically-based” research practices, the field of special education responded

in a number of ways. In a Special Exchange Series, “Perspectives on Defining Scientifically

Based Research” (published fall of 2003), the authors discussed several issues, including: (a)

scientifically-based research within the context of federal policy; (b) research with youth with

severe or low incidence disabilities; (c) research methodology (e.g., qualitative research, single-

subject research); and (d) putting research into practice. Attempts were made to better

operationally define the terms “scientific- or research-based” in order to move forward in

establishing a set of practices that schools and districts could confidently use. A major issue

identified was the apparent reliance on randomized controlled trial research in NCLB and IDEA

and whether this should be considered the “gold standard.” In addition, authors of the special

series suggested expanding the repertoire of research designs to include single-subject research

Page 7: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 7

studies to establish EBPs for youth with disabilities (McDonnell & O’Neill, 2003; Spooner &

Browder, 2003).

The debate over whether to include single-subject research methods when establishing

EBPs has continued. Evident from policies, publications, and conversations in the field,

consensus is needed for what constitutes an effective practice. In 2005, a special issue of

Exceptional Children was published responding to the need to determine what constitutes high

quality research. The special issue suggested quality indicators for group experimental (Odom et

al., 2005), single-subject (Horner et al., 2005), correlational (Thompson, Diamond, McWilliam,

Snyder, & Snyder, 2005), and qualitative research designs (Brantlinger, Jimenez, Klingner,

Pugach, & Richardson, 2005). The field of special education now had a set of research standards

to follow. Researchers began using these quality indicators, or variations of these indicators, to

design and implement research. Additionally, researchers have also used these quality indicators,

or variations of, in recent extensive reviews of the literature to identify and establish EBPs (e.g.,

Baker, Chard, Ketterline-Geller, Apichatabutra, & Doabler, 2009; Browder, Ahlgrim-Delzell,

Spooner, Mims, & Baker, 2009; Lane, Kalberg, & Shepcaro, 2009).

As part of the movement to define effective practices in the field, the Council for

Exceptional Children’s (CEC) Professional Standards and Practice Committee is developing

criteria for identifying EBPs that can be used consistently across the field of special education.

This process includes developing quality indicators for group experimental, single-subject, and

correlational research to ensure researchers in the field are conducting high quality research

(Test, Carter, Baer, & Morningstar, 2012). By developing quality indicators researchers can use

consistently in the field, it is more likely researchers will adhere to guidelines that ensure high

quality research is being conducted to build EBPs in special education.

Commented [W1]: So, isn’t this in collaboration with the Division of Research…and Dave, didn’t you guys disseminate these at CEC? May need to update this statement?

Page 8: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 8

Research in Secondary Transition

The use of high quality research is also important in the field of secondary transition to

identify EBPs and in-school predictors of post-school success for secondary students with

disabilities. In 2006, when the National Secondary Transition Technical Assistance Center

(NSTTAC) was funded by the USDOE Office of Special Education Programs (OSEP), one of its

objectives was to identify EBPs in secondary transition for students with disabilities.

To do this, NSTTAC conducted two comprehensive systematic reviews of secondary

transition literature using quality indicators to determine EBPs (Test, Fowler, et al., 2009) and

evidence-based predictors of post-school success for students with disabilities (Test, Mazzotti, et

al., 2009). The reviews examined experimental and correlational research articles published

between 1984 and 2008. Electronic and hand searches have been conducted since that time to

annually update the original findings. Through June 2011, the search yielded 1,562 articles that

met basic inclusion criteria. Of those articles, only 272 (17.4%) group/single-subject research

studies and 100 (6.4%) correlational studies met advanced inclusion criteria to be examined.

Criteria included (a) were published in peer-reviewed journals, (b) included students with

disabilities between the ages of 11 and 22, (c) independent variable or predictor variable related

to a transition practice or program, and (d) dependent variable or outcome variable related to in-

school or post-school outcomes (see Test, Fowler, et al., 2009 [EBPs] and Test, Mazzotti, et al.,

2009 [evidence-based predictors] for detailed lists of inclusion criteria).

Evidence-Based Practices in Secondary Transition

Experimental studies (i.e., group experimental, single subject), that measure the

effectiveness of a particular strategy by examining the effects of the strategy (i.e., independent

variable) on a particular behavior (i.e., dependent variable), are used to establish EBPs. All

Page 9: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 9

experimental studies that met the initial inclusion criteria above were first organized by the broad

areas of Kohler’s Taxonomy for Transition Programming (Kohler, 1996). Articles were then

organized by clusters within each of the broad categories (i.e., IEP Development, Life Skills

Instruction). While an original set of 33 EBPs were published in Test, Fowler, et al. (2009),

NSTTAC has continued to annually update its literature review. The 272 (17.4%) group/single-

subject studies that met advance review criteria were reviewed using quality indicator checklists

for group/single-subject research. Next, articles were analyzed using the quality indicators to

determine quality of the research. NSTTAC developed quality indicator guidelines based on

criteria suggested by Horner et al. (2005; single-subject research) and Gersten et al. (2005; group

experimental) from the Exceptional Children (2005) special issue. For more information, see

NSTTAC’s general coding form instructions found at http://www.nsttac.org/content/literature-

review-process. After this analysis, 102 (37.5%) articles met the criteria for group/single-subject

research quality. Each article was then evaluated to determine the level of evidence. Level of

evidence was determined using criteria adapted from the Institute for Education Science (IES)

standards for levels of evidence (What Works Clearinghouse, n.d.), and EBPs were identified as

having a strong, moderate, or potential level of evidence based on number of studies identified.

Of the 102 (37.5%) articles, 98 (36.0%) contributed to identification of EBPs and practice

descriptions. Table 2 provides the decision rules for determining levels of evidence based on

single-subject and group experimental research designs.

<Insert Table 2>

Currently, NSTTAC has identified 64 EBPs for secondary transition. Of these 64

practices, only 6 have a strong level of evidence based on the adapted IES standards, 34 have a

moderate level of evidence, and 24 have a potential level of evidence. These EBPs fall under

Formatted: Font color: Auto, Not Highlight

Formatted: Font color: Auto, Not Highlight

Page 10: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 10

four areas of Kohler’s Taxonomy, including Student-Focused Planning, Student Development,

Family Involvement, and Program Structure (see http://www.nsttac.org/content/evidence-based-

practices-secondary-transition for a listing of all EBPs in secondary transition).

Secondary Transition Evidence-Based In-School Predictors of Post-School Success

An analysis of correlational research studies was undertaken to identify evidence-based

in-school predictors of post-school success. This effort followed a process similar to that used for

identifying EBPs. As mentioned previously, the search initially yielded 100 (6.4%) correlational

articles related to secondary transition. After excluding articles that did not meet the established

initial inclusion criteria, 28 (28.0%) articles were reviewed using the quality indicator checklist

for correlational research. This checklist was developed using criteria suggested by Thompson et

al. (2005). After the quality indicator review, 22 (78.6%) articles met quality indicator criteria.

These articles were used to identify the evidence-based in-school predictors of post-school

success. Again, the level of evidence was determined using criteria adapted from the IES

standards, and evidence-based predictors were classified as either having a moderate or potential

level of evidence. Because correlational research is not causal, level of evidence can only be

established at the moderate or potential level. Table 1 provides the decision rules for

correlational research levels of evidence. The 22 correlational studies (i.e., 3 exploratory; 19 a

priori) supported the identification of 16 evidence-based in-school predictors of post-school

success (see http://www.nsttac.org/content/literature-review-process for predictor descriptions

and additional information related to predictors). Table 3 includes a list of the 16 evidence-based

predictors by post-school outcome area (i.e., employment, education, independent living).

<Insert Table 3>

Formatted: Font color: Auto, Not Highlight

Formatted: Font color: Auto, Not Highlight

Page 11: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 11

Based on results of the two comprehensive systematic literature reviews, it is clear there

is a paucity of high quality experimental and correlational research in secondary transition that

can be used to identify evidence-based practices or predictors of post-school success. Of all the

studies analyzed for experimental research, only 36.0% were actually rigorous enough to be used

to establish EBPs; and only 34.9% of the correlational studies were of sufficient quality to be

used to establish the evidence base for in-school predictors of post-school success. Although our

field has begun this critical work, there continues to be a lack of high quality research needed to

identify EBPs and programs available for schools and teachers to implement. As a field, it is

necessary that we continue to build on the current evidence-base by conducting high quality

research to support future identification of secondary transition EBPs and predictors. Therefore,

DCDT is calling for the expansion of research that meets the indicators for exhibiting high levels

of evidence; thereby, ensuring that secondary transition EBPs and predictors of post-school

success for students with disabilities are the focus of future high quality research.

Recommendations

The following sections describe DCDT’s recommendations for conducting experimental

research and correlational research, as well as suggestions for planning for future research. A

summary of these recommendations can be found in Table 4.

<Insert Table 4>

DCDT Recommendations for Experimental Research

Clearly, there is a need to continue to identify secondary transition EBPs by conducting

research that meets the quality indicator guidelines established by Gersten et al. (2005) and

Horner et al. (2005). By conducting research that adheres to quality indicators, we can increase

the number of EBPs and predictors available to support teacher and district implementation of

Page 12: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 12

evidence-based practice. As professionals in the field, it is imperative we continue to build on the

evidence base. When conducting group experimental and single-subject research, researchers

should adhere to the following recommendations. First, researchers should build upon

NSTTAC’s levels of evidence. Currently, only six EBPs have achieved a strong level of

evidence (e.g., using mnemonics, self-management, peer-assisted instruction, visual displays).

High quality research is needed to move the remaining EBPs from moderate or potential to

strong. Typically, NSTTAC recommends 1 to 2 high quality group experimental studies; 3 or

more acceptable quality group studies; or 5 or more high quality single-subject studies to move a

practice from potential or moderate levels of evidence to strong levels of evidence.

See http://www.nsttac.org/content/literature-review-process What’s Needed to Increase Level of

Evidence table for specific recommendations based upon currently identified EBPs and

instructional procedures.

Second, researchers should focus on the areas of Kohler’s Taxonomy that have few or no

evidence to support practices. Research is needed to establish evidence of the impact of Program

Structure, Family Involvement, and Interagency Collaboration. Currently, only three EBPs have

been identified under the area of Program Structure. Program Structure is an important area

because it includes program development, planning, policy, evaluation, and resource allocation

(Kohler, 1996; Kohler & Field, 2003). For example, research in this area may focus on programs

that improve the quality of transition programs, personnel, and resources to improve student

outcomes. Another important transition domain is Family Involvement in transition planning.

Although, Family Involvement has been identified as a predictor of positive post-school success,

in addition to increased student participation in the transition planning process (Test, Mazzotti, et

al., 2009; Wagner, Newman, Cameto, Javitz, & Valdes, 2012); currently, only one practice exists

Page 13: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 13

that exhibits a moderate level of evidence (i.e., one group study). Conducting research to

investigate interventions and strategies for facilitating family involvement and promoting strong

parent-school relationships will help break down barriers and enhance family involvement in the

transition planning process. Finally, Interagency Collaboration has been linked to positive post-

school outcomes for youth (Test, Mazzotti, et al., 2009); however, no experimental research or

evidence-base currently exists for practices in this area. Therefore, interagency collaboration is

an additional area of research necessary to identify evidence-based interventions and practices

that support effective interagency collaboration in districts and schools.

Third, researchers must conduct research that includes students from all disability

categories and ethnicities. This recommendation reinforces the DCDT position paper on ensuring

students from diverse populations’ secondary transition needs are met through effective

interventions (Trainor, Lindstrom, Simon-Burroughs, Martin, & Sorrells, 2008). Trainor et al.

(2008) noted the importance of conducting research, which includes students with disabilities

from cultural and linguistically diverse backgrounds. To date, NSTTAC has included disability

and ethnicity in reporting its findings, when available; however, it is necessary for research to

disaggregate and investigate what works for students from various disability groups and diverse

backgrounds.

Finally, longitudinal studies are needed to examine the effects of secondary transition

practices overtime on both in-school and post-school outcomes. The National Longitudinal

Transition Study-2 (NLTS2) has collected up to nine years of outcome data on students with

disabilities after exiting high school, including data related to students with disabilities’

experiences in school (Newman et al., 2009; Newman et al., 2011; Sanford et al., 2011; Wagner,

Newman, Cameto, Garza, & Levine, 2005; Wagner, Newman, Cameto, Levine, & Garza, 2006).

Page 14: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 14

However, these data do not provide information about the long-term effects of specific

interventions and strategies used with students with disabilities. Experimental research typically

occurs over a short amount of time (e.g., 3 months, 6 months, 1 year) with some maintenance

data collected. However, such short snapshots of effectiveness need to be expanded to capture

long-term impacts. Since the ultimate goal of transition is to improve outcomes for students with

disabilities, it is important to determine the long-term impact of specific interventions and EBPs

on outcomes for students with disabilities.

To summarize, DCDT recommends high quality secondary transition research be

conducted to increase the level of evidence for existing practices to “strong,” expand the number

of EBPs in all areas of the Taxonomy for Transition Programming (with emphasis on Program

Structure, Family Involvement, and Interagency Collaboration), investigate what works for

students from culturally and linguistically diverse backgrounds, and examine the longitudinal

effects of practices. Next, we discuss DCDT’s recommendations for correlational research.

DCDT Recommendations for Correlational Research

DCDT recommends researchers extend correlational research by conducting multivariate

studies that build on established levels of evidence (Test, Mazzotti, et al., 2009). Currently, only

one predictor (i.e., inclusion in general education) has established a moderate level of evidence

across all three post-school outcome areas (i.e., employment, education, independent living).

High quality correlational research is needed to move other identified predictors from the

potential to moderate level of evidence. This will require researchers to conduct multivariate

correlational studies that are a priori employing the quality indicators to ensure high quality

correlational research. Thompson et al. (2005) suggested correlational research that is conducted

logically and statistically has the potential to inform causal inference and EBPs. Therefore, a key

Page 15: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 15

role of correlational research is to inform practice (See http://www.nsttac.org/content/predictor-

resources for NSTTAC’s What’s Needed in Correlational Research table).

Second, researchers should conduct high quality correlational research that disaggregates

data by disability category to identify predictors of post-school success for specific disability

groups. The systematic review conducted by Test, Mazzotti, et al. (2009) focused primarily on

in-school predictors related to transition program characteristics and did not disaggregate data by

disability category. Further, many of the studies (e.g., Fabian, Lent, & Willis, 1998; Harvey,

2002; Repetto, Webb, Garven, & Washington, 2002) that adhered to the quality indicator criteria

did not disaggregate or provide data necessary for discriminating between disability groups.

More recently, studies have been published that have disaggregated data by intellectual disability

(Baer, Daviso, Flexer, Queen, & Miendl, 2011; Bouck & Joshi, 2012); however, additional

research is needed to determine if the currently identified predictors are valid across disability

groups. In addition to disability groups, it is essential that correlational research focus on

disaggregating data by race/ethnicity and socio-economic status to determine which predictors

are effective for students with disabilities from diverse groups. Researching predictors of post-

school success by disability category, race/ethnicity, and socio-economic status will provide

information to the field about specific transition program characteristics that lead to post-school

success for students in specific disability groups and from culturally diverse backgrounds.

Further, it is necessary that researchers focus on variables other than demographic variables (e.g.,

post-school goals, student support needs, individual student characteristics) to determine which

practices and predictors are most effective for improving outcomes for students with diverse

characteristics and needs.

Page 16: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 16

Finally, ongoing research is needed to further expand our understanding of how school-

based transition practices predict post-school success for students with disabilities. The current

set of predictor categories defined by NSTTAC were consensus-based and emerged from the

extant data that were analyzed from correlational studies determined to be of high quality.

Therefore, future researchers should consider identifying a more comprehensive analysis of

predictors and definitions when designing correlational studies using these predictor categories.

As researchers begin to investigate each predictor category, consideration should be given to

specifically defining the category and providing in-depth analysis of critical components related

to each predictor. For example, when considering the predictor transition program, researchers

need to investigate which specific components of transition programs result in improved post-

school outcomes. In further defining predictor variables, researchers should conduct research that

determines if currently identified predictor variables hold up over multiple points in time with

students with various disabilities, as well as those from culturally diverse backgrounds.

Clearly, there is a need among the research community to establish “longitudinal research

that convincingly links the impact of specific transition practices and programs to medium- and

long-term student outcomes” (Carter et al., 2013, p. 23). Several large-scale longitudinal datasets

are available from which researchers can examine critical in school programs and practices that

establish strong relationships to post-school success, not the least of which is the NLTS2. The

NLTS2 database provides valuable data researchers can use to conduct high quality correlational

studies related to experiences of youth with disabilities, both while they are in school and

receiving critical transition services, but also as they transition to adulthood. Distinct with its

focus on secondary programs and services as well as post-school outcomes, the NLTS2 is one of

the most compelling datasets offering rich and varied data over the past decade. Because the

Page 17: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 17

NLTS2 project published largely descriptive results across all students with disabilities,

additional analyses are needed to explore in greater depth findings specific to different disability

and demographic categories, as well as particular topical areas. While a number of researchers

are currently conducting secondary analysis of the NLTS2 data (see Table 5 for a sample of

recently published articles), additional research is needed to address the myriad of questions still

to be answered regarding what makes for a successful transition into a particular component of

early post-high school adult life. The most current national transition longitudinal study

supported by the Department of Education is the National Longitudinal Transition Study 2012

(http://www.mathematica-mpr.com/education/nlts.asp), which continues the work of both NLTS

and NLTS2.

<Insert Table 5>

While comprehensive in nature, a single dataset, such as the NLTS2, may not be able to

describe the complexity or breadth of transition practices and outcomes. We recommend that

transition-focused researchers consider other sources available that may offer equally rich and

descriptive variables addressing our unanswered questions. Several large datasets supported by

the National Center for Educational Statistics (NCES) have not been sufficiently tapped by

transition researchers. Of course, one significant difference among some datasets may be

sufficiently representative samples of students and youth with disabilities; however, it is

expected that these large data sets would indeed have samples of students identified as having

disabilities. For example, the Education Longitudinal Study of 2002 follows a cohort of high

school sophomores to understand such issues as home and parental aspirations, course-taking

paths, and how high schools are organized and influence adult success. Similar in scope, the

Career/Technical Education (CTE) Statistics collects data to develop a picture of participation in

Page 18: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 18

CTE. The CTE Statistics system uses data from across other NCES surveys of postsecondary

students and institutions, secondary schools and faculty, and longitudinal data tracking secondary

and postsecondary students into the labor market. Two other datasets, the High School

Longitudinal Study of 2009 and High School and Beyond, maintain equally rich variables for

understanding students’ trajectories from high school into adulthood.

Finally, an initiative of the NCES to align PK-12 data with workforce outcomes has

resulted in a rich source of state-level data through the Statewide Longitudinal Data Systems

(SLDS). The SLDS is intended to enhance the ability of states to efficiently and accurately

manage and use education data, including individual student records. In fact, the National Post-

School Outcomes Center is working with states to support the alignment of disability-specific

outcomes data (i.e., SPP Indicator 14) with SLDS systems (Kellems & Unruh, 2011). Using

existing longitudinal data systems will ensure that researchers, policymakers and other transition

stakeholders make data-based decisions to improve student learning and outcomes and launch

research to increase student achievement and close gaps in post-school outcomes (Morningstar,

Knollman, Semon, & Kleinhammer-Tramill, 2013).

In summary, DCDT strongly encourages transition researchers to consider the expansion

of high quality secondary transition correlational research in order to identify essential predictors

of post-school success, while ensuring increased numbers of “high” level predictors. In addition,

the field has not yet sufficiently examined individual characteristics of students that may impact

their achievement of post-school success, and further research is needed to disaggregate data by

disability category, race/ethnicity, and socio-economic status. Furthermore, a significant need

exists within research to establish comprehensive definitions of currently established predictors

to ensure uniformity across research and programs. Finally, DCDT recommends that researchers

Page 19: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 19

continue to mine extant datasets, particularly those that have been established to examine both

in-school experiences and follow students into adulthood.

Planning for Future Research

The findings from NSTTAC’s literature reviews of secondary transition experimental and

correlational research, combined with emerging research, as well as the increasing number of

secondary analyses of the NLTS2 database, have laid a solid foundation for future research in

secondary transition. By following the recommendations stated earlier for experimental and

correlational research, including secondary analyses of the NLTS2 database, it would be easy to

make a career out of continuing to expand our knowledge of evidence-based secondary transition

practices and predictors. This section describes some additional considerations for planning

future research including quality, complexity, capacity, and implementation.

Quality. As new research is completed, it must be designed to meet the quality indicators

for experimental (Gersten et al., 2005; Horner et al., 2005) and/or correlational (Thompson et al.,

2005) research. This can be achieved by encouraging university research faculty to teach, and

then hold, their students accountable for designing every study to meet the appropriate set of

quality indicators. If journal editors and reviewers also make this an expectation, it will happen.

Then, if every published study is designed to meet quality indicators, the number of EBPs and

predictors available for use in classrooms and schools should rapidly expand.

This position paper has focused on experimental and correlational research because of

IES’ stance that only research that infers causality be used to establish EBPs. However, we

recognize that qualitative research has, and will continue to play, a role in the evidence-based

practice movement. In fact, Brantlinger et al. (2005) suggested qualitative research could “fit

under the National Research Council’s categories of producing descriptive or procedural

Page 20: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 20

knowledge” (p.196). Qualitative methodology is also important to transition research because of

the unique answers it can offer, including (a) understanding an issue or area where little research

exits, (b) making sense of complex phenomena, (c) constructing theory, and (d) learning from

participants about the context as they experience it (Morse & Richards, 2002). Often, qualitative

research serves as the basis for developing further experimental studies once an understanding of

unresolved educational problems has emerged (Test et al., 2012). As a result, qualitative studies

can provide information to inform secondary transition practice and policy, as well providing the

perspectives of individuals involved in implementing and receiving EBPs. This rich detail can

help practitioners select, implement, and adapt EBPs to fit the context of their school or

classroom. Finally, as with other types of research we recommend future qualitative studies in

secondary transition be designed to meet the quality indicators suggested by Brantlinger et al.

Complexity. As Carter et al. (2013) and Madaus et al. (2013) recently suggested based

on their review of Career Development and Transition for Exceptional Individuals’ first 35

years, the field is ready to tackle the more complex issues in the area of secondary transition. As

Carter et al. suggested, future research is needed to compare the effects of different interventions

on both proximal and longitudinal student outcomes. In addition, component analyses that

identify the essential features of an intervention may make them more usable in today’s

classroom. Finally, it would be useful to practitioners to be able to state which practices and

predictors produce the best outcomes for which students in what contexts.

Capacity. If future research in secondary transition is to be of high-quality and address

more complex issues, we need to make sure we have the capacity to do so. This involves both

financial and personnel resources. First, as Carter et al. (2013) noted, transition researchers need

to explore funding options available through the Institute of Education Sciences/National Center

Page 21: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 21

for Special Education Research, National Institute on Disability and Rehabilitation Research, and

the National Institutes of Health, but also from the Department of Justice, Department of Labor,

Department of Health and Human Services, and Substance Abuse and Mental Health Services

Administration. Each has a federal role in improving college and career readiness of students

with disabilities. In addition, future researchers must consider non-federal services of funding,

including foundations (i.e., Gates Foundation), as well as state funding sources (i.e.,

Developmental Disability Councils).

In addition to financial resources, we also need to increase the number of secondary

transition researchers. This can be accomplished by ensuring our doctoral programs prepare

future researchers to conduct high-quality research on the many complex transition issues, as

well as write for funding from the variety of potential funding sources. Besides preparing new

researchers, as Unger and Rumrill (2013) noted, future researchers should be encouraged to

collaborate both across universities, as well with practitioners, families, advocates, and

administrators. Not only could this increase the number of possible secondary transition

researchers, it may help bridge the research-to-practice gap.

Implementation. As the field of secondary transition continues to identify EBPs and

predictors, it will become increasingly important to ensure our science guides practice. To do

this, as Cook and Odom (2013) have stated “implementation, is the next, and arguably most

critical, stage of evidence-based reforms” (p. 142). Therefore, DCDT recommends using the

strategies for dissemination suggested by Cook, Cook, and Landrum (2013), strategies for

implementing practices with fidelity suggested by Harn, Parisi, and Stoolmiller (2013), the

collaborative strategies suggested by Klingner, Boardman, and McMaster (2013), and the

professional development strategies suggested by Odom, Cox, Brock, and National Professional

Page 22: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 22

Development Center on ASD (2013). Finally, and hopefully in the not-too-distant future, as we

move from local to state level dissemination, the lessons learned from Implementation Science

for scaling-up EBPs statewide (Fixsen, Blasé, Metz, & Van Dyke, 2013) can be followed.

In conclusion, the purpose of this paper was to describe DCDT’s stance and

recommendations for identifying and promoting secondary transition EBPs and predictors of

post-school success for students with disabilities. Recommendations were suggested for

conducting future experimental research to increase our knowledge of EBPs and correlational

research to increase our knowledge of evidence-based predictors. To do this, the field will

require researchers to conduct high-quality research investigating the many complex transition

issues, collaborate with others, expand the funding options, and learn from, and contribute to the

emerging field of Implementation Science to ensure our EBPs and predictors are adopted and

implemented with fidelity. After all, if our ultimate goal of improved school services and post-

school outcomes for all students with disabilities is to be realized, our best science must be used

to guide practice.

Page 23: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 23

References

Baer, R. M., Daviso, A. W. III, Flexer, R. W., Queen, R. M., & Meindl, R. S. (2011). Students

with intellectual disabilities: Predictors of transition outcomes. Career Development and

Transition for Exceptional Individuals, 34, 134-141. Doi: 10.1177/0885728811399090

Baker, S. K., Chard, D. J., Ketterline-Geller, L. R., Apichatabutra, C., & Doabler, C. (2009).

Teaching writing to at-risk students: The quality of evidence for self-regulated strategy

development. Exceptional Children, 75, 303-318.

Berliner, D. C. (2002). Educational research: The hardest science of all. Educational

Researchers, 31(8), 18-20.

Bouck, E. C., & Joshi, G. S. (2012). Functional curriculum and students with mild intellectual

disabilities: Exploring postschool outcomes through the NLTS2. Education and Training

in Autism & Developmental Disabilities, 47, 139-153.

Brantlinger, E., Jimenez, R., Klingner, J., Pugach, M., & Richardson, V. (2005). Qualitative

studies in special education. Exceptional Children, 71, 195-207.

Browder, D., Ahlgrim-Delzell, L., Spooner, F., Mims, P. J., & Baker, J. N. (2009). Using time

delay to teach literacy to students with severe developmental disabilities. Exceptional

Children, 75, 343-364.

Carter, E.W., Brock, M.E., Bottema-Beutel, K., Bartholomew, A., Boehm, T.L., & Cease-Cook,

J. (2013). Methodological trends in secondary education and transition research: Looking

backward and moving forward. Career Development and Transition for Exceptional

Individuals, 36, 15-24.

Cook, B.G., Cook, L., & Landrum, T. J. (2013). Moving research into practice: Can we make

dissemination stick? Exceptional Children, 79, 163-180.

Page 24: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 24

Cook, B.G., & Odom, S.L. (2013). Evidence-based practices and implementation science in

special education. Exceptional Children, 79, 135-144.

Fabian, E., Lent, R., & Willis, S. (1998). Predicting work transition outcomes for students with

disabilities: Implications for counselors. Journal of Counseling and Development, 76,

311–316.

Fixsen, D., Blasé, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-

based programs. Exceptional Children, 79, 213-230.

Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005).

Quality indicators for group experimental and quasi-experimental research in special

education. Exceptional Children, 71, 149-164.

Halpern, A. S. (1985). Transition: A look at the foundations. Exceptional Children, 51, 479-486.

Halpern, A. (1992). Transition: New wine in old bottles. Exceptional Children, 58, 202-211.

Harn, B., Parisi, D., & Stoolmiller, M. (2013). Balancing fidelity with flexibility and fit: What do

we really know about fidelity of implementation in schools? Exceptional Children, 79,

181-193.

Harvey, M. W. (2002). Comparison and postsecondary transitional outcomes between students

with and without disabilities by secondary vocational education participation: Findings

from the National Education Longitudinal Study. Career Development for Exceptional

Individuals, 25, 99–122.

Hockenbury, J. C., Kauffman, J. M., & Hallahan, D. P. (2000). What is right about special

education. Exceptionality, 8, 3-11.

Page 25: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 25

Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of

single-subject research to identify evidence-based practice in special education.

Exceptional Children, 71, 165-179.

Individuals with Disabilities Education Improvement Act of 2004, P. L. No. 108-446, 20 U.S.C.

Innes-Helsel, F. K., Hitchcock, J. H., Miller, G., Malinow, A., & Murray, E. (April 7 - 11,

2006). Identifying Evidence-Based, Promising and Emerging Practices That Use Screen-

Based and Calculator Technology to Teach Mathematics in Grades K-12: A Research

Synthesis. Paper prepared for the annual meeting of the American Educational Research

Association, San Francisco.

Kellems, R. & Unruh, D (2011, February). Statewide longitudinal data systems and Indicator 14:

What you need to know. Eugene: University of Oregon, National Post-School Outcomes

Center.

Klingner, J.K., Boardman, A.G., & McMaster, K.L. (2013). What does it take to scale up and

sustain evidence-based practices? Exceptional Children, 79, 195-211.

Kohler, P. D. (1996). A taxonomy for transition programming: Linking research and practice.

Champaign: University of Illinois, Transition Research Institute.

Kohler, P. D. (1998). Implementing a transition perspective of education. In F. Rusch & J.

Chadsey (Eds.), Beyond high school: Transition from school to work (pp. 179-205).

Belmont, CA: Wadsworth Publishing.

Kohler, P. D., & Field, S. (2003). Transition-focused education: Foundation for the future. The

Journal of Special Education, 37, 174-183.

Page 26: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 26

Lane, K. L., Kalberg, J. R., & Shepcaro, J. C. (2009). An examination of the evidence base for

function-based interventions for students with emotional and/or behavioral disorders

attending middle and high schools. Exceptional Children, 75, 321-340.

Madaus, J.W., Gelbar, N.W., Dukes, L.L., Fagella-Luby, M., Lalor, A.R., & Kowitt, J.S. (2013).

Thirty-five years of transition topics: A review of CDTEI issues from 1978-2012. Career

Development and Transition for Exceptional Individuals, 36, 1-8.

McDonnell, J., & O’Neil, R. (2003). A perspective on single/within subject research

methods and scientifically based research. Research and Practice for Persons

with Severe Disabilities, 28, 138-142.

Morningstar, M. E., Knollman, G., Semon, S., & Kleinhammer-Tramill, J. (2012). Accountability

for what matters: Using postschool outcomes to build school and community renewal. In

L. C. Burrello, W. Sailor, & J. Kleinhammer-Tramill (Eds.), Unifying Educational

Systems: Leadership and Policy Perspectives (pp. 158-167). Florence, KY: Routledge,

Taylor & Frances Group.

Morse, J., & Richards, L (2002). Read me first for a user's guide to qualitative

research. Thousand Oaks, CA: Sage Publications.

Newman, L., Wagner, M., Cameto, R., & Knokey, A.-M. (2009). The post-high school

outcomes of youth with disabilities up to 4 years after high school: A report of

findings from the National Longitudinal Transition Study-2 (NLTS2) (NCSER

2009-3017). Menlo Park, CA: SRI International. Retrieved

from www.nlts2.org/reports/2009_04/nlts2_report_2009_04_complete.pdf.

Newman, L., Wagner, M., Knokey, A.-M., Marder, C., Nagle, K., Shaver, D., …

Schwarting, M. (2011). The post-high school outcomes of young adults with

Page 27: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 27

disabilities up to 8 years after high school: A report from the National

Longitudinal Transition Study-2 (NLTS2) (NCSER 2011-3005). Menlo Park, CA:

SRI International. Retrieved from www.nlts2.org/reports/

No Child Left Behind Act of 2001, P. L. 107-110, 115 Stat. 1425 (2002).

Odom, S. L., Brantlinger, E., Gersten, R., Thompson, B., & Harris, K.R. (2005).

Research in special education : Scientific methods and evidence-based practices.

Exceptional Children, 71, 137-148.

Odom, S.L., Cox, A.W., Brock, M.E., & the National Professional Development Center

on Autism Spectrum Disorders. (2013). Implementation science, professional

development, and autism spectrum disorders. Exceptional Children, 79, 233-251.

Rusch, F.R., Szymanski, E., & Chadsey-Rusch, J. (1992). The emerging field of

transition services. In F. Rusch, L. DeStefano, J. Chadsey-Rusch, L.A. Phelps, &

E. Szymanski (Eds.), Transition from school to adult life: Models, linkages, and

policy (pp. 5-17). Sycamore, IL: Sycamore Press.

Repetto, J. B., Webb, K. W., Garvan, C. W., & Washington, T. (2002). Connecting student

outcomes with transition practices in Florida. Career Development for Exceptional

Individuals, 25, 123–139.

Sanford, C., Newman, L., Wagner, M., Cameto, R., Knokey, A.-M., & Shaver, D.

(2011). The post-high school outcomes of young adults with disabilities up to 6

years after high school: Key findings from the National Longitudinal Transition

Study-2 (NLTS2) (NCSER 2011-3004). Menlo Park, CA: SRI International.

Retrieved from www.nlts2.org/reports/

Page 28: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 28

Singer, G.H.S. (2000). Special education research agenda in the United States: Sources of

influence. Exceptionality, 8, 235-246.

Spooner, F., & Browder, D. M. (2003). Scientifically based research in education and students

with low incidence disabilities. Research and Practice for Persons with Severe

Disabilities, 28, 117-125.

Test, D. W., Carter, E. W., Baer, R. M., & Morningstar, M. (2012, April). Research in secondary

transition: Where are we now and where should we be going? Paper presented at the

Council for Exceptional Children 2012 Annual Convention and Expo, Denver, CO.

Test, D. W., Fowler, C. H., Richter, S., White, J. A., Mazzotti, V. L., Walker, A. R., . . .

Kortering, L. (2009). Evidence-based practices in secondary transition. Career

Development for Exceptional Individuals, 32, 115-128.

Test, D. W., Mazzotti, V. L., Mustian, A. L., Fowler, C. H., Kortering, L. J., & Kohler, P. H.

(2009). Evidence-based secondary transition predictors for improving post-school

outcomes for students with disabilities. Career Development for Exceptional Individuals,

32, 160-181.

Thompson B., Diamond, K. E., McWilliam, R., Snyder, P., & Snyder, S. W. (2005). Evaluating

the quality of evidence from correlational research for evidence-based practice.

Exceptional Children, 71, 181-194.

Trainor, A. A., Lindstrom, L., Simon-Burroughs, M., Martin, J. E., & Sorrells. A. M. (2008).

From marginalized to maximized opportunities for diverse youths with disabilities: A

position paper of the Division on Career Development and Transition. Career

Development for Exceptional Individuals, 31, 56-64. doi:10.1177/0885728807313777

Page 29: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 29

Twyman, J. S., & Sota, M. (2008). Identifying research-based practices for response to

intervention: Scientifically based instruction. Journal of Evidence-Based Practices for

Schools, 9, 86-101.

Unger, D. D., & Rumrill, P. D. (2013). An assessment of publication productivity in career

development and transition for exceptional individuals: 1978-2012. Career Development

and Transition for Exceptional Individuals, 36, 25-30.

U. S. Department of Education (n.d.). Proven methods: Questions and answers on No Child Left

Behind. Washington, DC: Author. Retrieved September 16, 2012

from http://www2.ed.gov/nclb/methods/whatworks/doing.html

Wagner, M., Newman, L., Cameto, R., Garza, N., & Levine, P. (2005). After high school: A first

look at the postschool experiences of youth with disabilities. A report from the National

Longitudinal Transition Study-2 (NLTS2). Menlo Park, CA: SRI International. Retrieved

from www.nlts2.org/reports/2005_04/nlts2_report_2005_04_complete.pdf.

Wagner, M., Newman, L., Cameto, R., Javitz, H., & Valdes, K. (2012). A national picture of

parent and youth participation in IEP and transition planning meetings. Journal of

Disability Policy Studies. Advance online publication. doi: 10.1177/1044207311425384

Wagner, M., Newman, L., Cameto, R., Levine, P., & Garza, N. (2006). An overview of findings

from Wave 2 of the National Longitudinal Transition Study-2 (NLTS2). Menlo Park, CA:

SRI International. Retrieved

from www.nlts2.org/reports/2006_08/nlts2_report_2006_08_complete.pdf.

Wehman, P. (1992). Transition for young people with disabilities: Challenges for the 1990s.

Education and Training in Mental Retardation, 27, 112-118.

Page 30: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 30

What Works Clearinghouse (n.d.). WWC study review standards. Washington, DC: Institute of

Education Sciences, U. S. Department of Education. Retrieved from

http://ies.ed.gov/ncee/wwc/documentsum.aspx?sid=19.

Will, M. (1984). OSERS programming for the transition of youth with disabilities: Bridges from

school to working life. Washington, DC: U.S. Department of Education, Office of Special

Education and Rehabilitative Services.

Page 31: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 31

Table 1

Definitions for Identifying Types of Practices

Term Definition

Evidence-Based Practices • Are based on rigorous research designs • Have demonstrated a record of success for improving student

outcomes • Have undergone systematic review process using quality

indicators to evaluate level of evidence Research-Based Practices • Are based on rigorous research designs

• Have demonstrated a record of success for improving student outcomes

Promising Practices • Are based on research • Have demonstrated limited success • Have used a ‘weak’ research design

Unestablished Practices • Are not based on research • Have no data to support effectiveness • Based on anecdotal evidence and/or professional judgment

Page 32: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

32

Table 2

NSTTAC’s Levels of Evidence for Single-Subject Designs

Levels of Evidence of Causal Inference

Single-Subject Designs Group Experimental Designs Correlational Designs

Strong • 5 high quality studies; High quality = meets all QIs

• 3 independent research teams • Must demonstrate a

functional relation • No contradictory evidence

from a study reflecting strong evidence

• 4 acceptable quality or 2 high quality

• High quality = must meet 1,2, 3, 4, 6, 8, 9 & 10 and 5or 7 of EQIs and at least 4 of the DQIs

• Acceptable = must meet 1, 2, 3, 4, 6, 8, 9 & 10 and 5 or 7 of EQIs and at least 1 of the DQIs

• Must calculate ES or report data that allows for calculation

• There is no contradictory evidence from a study reflecting strong evidence

• N/A

Moderate • 3 acceptable studies; Acceptable = meets all QIs except 2, 11, & at least one of 17-20

• 1-2 independent research teams

• Must demonstrate a functional relation

• 2 acceptable quality or 1 high quality

• Must calculate ES or report data that allows for calculation

• 2 a priori (planned, hypothesis stated) studies with consistent significant correlations between predictor and outcome

• Can include exploratory studies only when paired with a priori significant correlations

• Must include ES calculations or the data to calculate ES

Potential (needs additional research)

• 2 high or acceptable studies • 2 independent research teams • Must demonstrate a

functional relation

• 1 acceptable quality • Must calculate ES or report data

that allows for calculation

• 1 a priori (planned, hypothesis stated) study

• Two or more exploratory (no specific hypothesis) studies with significant correlations between predictor and outcome

Note. EQI=Essential Quality Indicators; DQI=Desirable Quality Indicators; QI=Quality Indicators; ES=Effect Size

Page 33: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

33

Table 3

Evidence-Based Predictors by Post-School Outcome Area

Predictor/Outcome Education Employment Independent Living

Career Awareness X X

Community Experiences X

Exit Exam Requirements/High

School Diploma Status

X

Inclusion in General Education X X X

Interagency Collaboration X X

Occupational Courses X X

Paid Employment/Work

Experience

X X X

Parental Involvement X

Program of Study X

Self-Advocacy/Self-Determination X X

Self-Care/

Independent Living

X X X

Social Skills X X

Student Support X X X

Transition Program X X

Vocational Education X X

Work Study X

Page 34: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 34

Table 4 Summary of DCDT’s Recommendations

Recommendations for Experimental Research • Conduct research that meets the quality indicator guidelines established by

Gersten et al. (2005) and Horner et al. (2005) • Focus research efforts on the areas of Kohler’s Taxonomy that have few or no

evidence to support practices (e.g., family involvement, interagency collaboration, program structure)

• Conduct research that includes students from all disability categories and ethnicities

• Conduct longitudinal studies to examine the effects of secondary transition practices overtime on both in-school and post-school outcomes

Recommendations for Correlational Research • Conduct multivariate studies that build on established levels of evidence

(Test, Mazzotti, et al., 2009) • Conduct high quality correlational research that disaggregates data by

disability category to identify predictors of post-school success for specific disability groups

• Conduct research that provides a more comprehensive understanding of in-school predictors of post-school success for students with disabilities

Planning for Future Research • Ensure future qualitative studies in secondary transition be designed to meet

the quality indicators suggested by Brantlinger et al. (2005) • Make high quality research (i.e., using quality indicators suggested by Gersten

et al., 2005; Horner et al., 2005; and Brantlinger et al., 2005) the expectation when reviewing articles for submission in the various journals in the field

• Conduct studies that deal with complex transition issues • Build capacity of future researchers • Collaborate across universities and with stakeholders • Attend to the recommendations from Implementation Science to bridge the

research-to-practice gap

Page 35: Identifying and Promoting Transition Evidence-Based Practices and Predictors of Success: A Position Paper of the Division on Career Development and Transition

ESTABLISHING EVIDENCE-BASED PRACTICES 35

Table 5 A Sample of Published Articles Based on Secondary Analysis of NLTS2 Berry, H. G., Ward, M., & Caplan, L. (2012). Self-determination and access to

postsecondary education in transitioning youths receiving supplemental security income benefits. Career Development and Transition for Exceptional Individuals, 35, 68-75.

Bouck, E. C., & Joshi, G. S. (2012) Functional curriculum and students with mild intellectual disabilities: Exploring postschool outcomes through the NLTS2. Education and Training in Autism & Developmental Disabilities, 47, 139-153.

Carter, E. W., Austin, D., & Trainor, A. A. (2012). Predictors of postschool employment outcomes for young adults with severe disabilities. Journal of Disability Policy Studies, 23, 50-63. doi: 10.1177/1044207311414680

Frazier, T.W., Shattuck, P.T., Narendorf, S.C., Cooper, B., Wagner, M., & Spitznagel,

E.L. (2011) Prevalence and correlates of psychotropic medication use in adolescents with an autism spectrum disorder, with and without ADHD. Journal of Child and Adolescent Psychopharmacology, 21, 571-579.

Grigal M., Hart, D., & Migliore, A. (2011) Comparing the transition planning,

postsecondary education, and employment outcomes of students with intellectual and other disabilities. Career Development for Exceptional Individuals, 34, 4-17.

Joshi, G. S., Bouck, E. C., & Maeda, Y. (2012). Exploring employment preparation

and postschool outcomes for students with mild intellectual disabilities. Career Development and Transition for Exceptional Individuals, 35, 97-107

Kelly, S. M. (2011). The use of assistive technology by high school students with

visual impairments: A second 1ook at the current problem. Journal of Visual Impairment and Blindness, 105, 235-239.

McDonnall, M. C., & O’Mally, J. (2012). Characteristics of early work experiences

and their association with future employment. Journal of Visual Impairment and Blindness, 106, 133-144.

Shattuck, P.T., Wagner, M., Narendorf, S., Sterzing, P., & Hensley, M. (2011). Post

high school service use among young adults with autism. Archives of Pediatrics and Adolescent Medicine, 165, 141-146.

Shogren, K. A., & Plotner, A. J. (2012). Characteristics of transition planning for

students with intellectual disability, autism or other disabilities: Data from the National Longitudinal Transition Study-2. Intellectual and Developmental Disabilities, 50, 16-30.