IES Grant Writing Workshop for Efficacy and Replication Projects Joan E. McLaughlin, Ph.D. Deputy Commissioner National Center for Special Education Research.

IES Grant Writing Workshop for Efficacy and Replication Projects

Joan E. McLaughlin, Ph.D.Deputy Commissioner

National Center for Special Education Research

Overview of Presentation

• Structure of the Institute of Education Sciences• Research Topics within NCSER & NCER• Goals within Research Topics• Research Narrative for Goal 3 applications• Preparing and Submitting an Application• What Happens Next?• Questions

Structure of the Institute of Education Sciences

IES Organizational Structure

Office of the Director

National Board for Education

Sciences

National Center for Education Evaluation

National Center for Education Statistics

National Center for Education Research

National Center for

Special Education Research

FY 2012 Research Grant Topics in NCSER and NCER

Special Education Research Topics (84.324A)• Early Intervention and Early Learning in Special Education• Reading, Writing, and Language Development• Mathematics and Science Education• Social and Behavioral Outcomes to Support Learning• Transition Outcomes for Special Education Secondary Students• Cognition and Student Learning in Special Education• Professional Development for Teachers and Related Service

Providers• Special Education Policy, Finance, and Systems• Autism Spectrum Disorders• Technology for Special Education• Families with Children with Disabilities

Education Research Topics (84.305A)

• Reading and Writing• Mathematics and Science Education• Cognition and Student Learning• Social and Behavioral Context for Academic Learning• Education Technology• Effective Teachers and Effective Teaching

• Improving Education Systems: Policies, Organization, Management, and Leadership

• Postsecondary and Adult Education• Early Learning Programs and Policies• English Learners

Research Goals within Topics

Solution Driven Research

Research intended to contribute to the solution of practical education problems:

• Exploration Goal One• Development and Innovation Goal Two• Efficacy and Replication Goal Three• Scale-up Goal Four• Measurement Goal Five

Exploration• Explore the association between malleable factors and

education outcomes– A malleable factor can be changed by the education

system be it a characteristic of students (e.g., skills, behaviors), teachers (e.g., credentials, practices) or school (e.g., climate, size), or an education program or policy

• Underlying processes that enhance or inhibit learning• Aspects of a school, district, or community associated with

beneficial education outcomes• Education interventions associated with beneficial education

outcomes (e.g., professional development, curricula, policies)

• Explore factors that mediate or moderate the relationship between malleable factors and student outcomes

Development and Innovation

• Develop new interventions (e.g., instructional practices, curricula, teacher professional development)

• Demonstrate the feasibility of the intervention for implementation in an authentic education delivery setting

• Collect pilot data on promise of intervention to achieve intended outcomes

Efficacy and Replication• Causal test of whether or not a fully developed

intervention has a beneficial impact on student outcomes relative to a counterfactual in an authentic educational setting

• Takes place under “ideal” conditions– Homogenous sample of students/schools– Extra assistance to support high implementation

fidelity

Scale-up Evaluation• Independent causal test of whether or not a

fully developed efficacious intervention has a beneficial impact on student outcomes relative to a counterfactual in an authentic educational setting under routine implementation– Independent: evaluation team has no financial

interest in intervention– Efficacious: evidence of intervention’s efficacy– Routine implementation: as implemented by

practitioners with expected level of support if adopted by a school or district

Goal 5: Measurement

• Develop and validate assessments or other measurement tools

Overview of Efficacy and Replication Grant Applications

Efficacy

• Efficacy is the degree to which an intervention has a net positive impact on the outcomes of interest relative to the program/practice to which it is being compared

Replication

• Replication projects determine if an intervention that has produced a positive impact under one set of conditions will produce a positive impact under different conditions (e.g., the same intervention with different populations of students, different educational settings, or different levels of support)

Purpose of Efficacy and Replication Projects

• To estimate the strength or potency of the impact of the intervention

• To inform the degree to which the intervention can be feasibly or practically implemented

• To assess implementation fidelity

Appropriate Interventions for Efficacy and Replication Projects

• Interventions that are widely used but have not been rigorously tested

• Interventions that are fully developed, have evidence of their feasibility and promise, but are not widely used

• Interventions that have a positive effect under one set of conditions

Efficacy and Replication Projects

• Use experimental or quasi-experimental research designs that minimize selection bias or allow it to be modeled

• Employ a limited set of conditions• Often involve more support from the

intervention developer/researcher than is typically available

What percentage of research grants are Efficacy and Replication projects?

• 22 to 23 percent of the research projects funded by IES are Efficacy and Replication projects

Efficacy and Replication Applications

Research Narrative

Components of the Research Narrative

• Significance

• Research Plan

• Personnel

• Resources

Significance

• Describe fully-developed intervention• Provide rationale for why testing the efficacy

of this intervention is important• Present a clear theory of change to guide the

understanding of the intervention and expected outcomes

Research Plan (1)

• Clearly articulate all research questions– Does the intervention lead to a meaningful

improvement in education outcomes?– What are the conditions that support or hinder the

implementation of the intervention?– What processes mediate the relation between the

intervention and student outcomes?

Research Plan (2)

• Define sample (include size requirements, exclusion rules, and discussion of attrition issues)

Research Plan (3)

• Research Design– Use a randomized controlled trial (RCT)

whenever possible– Clearly justify the use a quasi-experimental

design and address internal validity threats

Research Plan (4)

• Describe the power analysis

- Provide rationale for assumptions

- If subgroups are integral to your study, consider whether you have sufficient power to address the relevant research questions

Research Plan (5)

• Provide detailed description of the chosen measures (include information on appropriateness, reliability, validity)– Don’t forget to detail any researcher-developed measures

• Provide clear description of data collection procedures and timing of data collections

Research Plan (6)

• Describe how fidelity will be assessed• Clearly describe the comparison

condition • Identify mediating and moderating

variables and how they will be collected• Detail the plan for analysis of qualitative

and quantitative data

Personnel

• Content domain experts• Methodological expertise• Statistical expert• Staff experienced in conducting research in

chosen setting and with population

Personnel (2)

• Qualifications• Roles and Responsibilities• Percent of time devoted to the project• DON’T FORGET to address the issue of the

objectivity of the evaluation if the intervention developer is part of the team

Resources

• Address institutional resources

• Document the support of those in the research setting (e.g., letters from school districts or schools, preschools)

Efficacy and Replication

• Typical award: $250,000 to $650,000 per year

• Maximum of 4 years and $3,500,000

Efficacy Follow-up Studies

• Examine the sustainability of the impacts of an intervention after the original study on:

- students that received the intervention and have moved on

- a new group of students in the same setting when additional resources are no

longer provided for the intervention

Efficacy Follow-up Studies

• Emphasis in narrative on previous study, including sample and effects

• Typical award: 150,000 to $300,000 per year

• Maximum of 3 years and $1,200,000

Preparing and Submitting an Application

Getting Started

Request for Applications

IES Grants.gov Application Submission Guide

Application Package

Relevant Requests for Applications (RFAs)

• Education Research Grant Program (84.305A)

• Special Education Research Grant Program (84.324A)

Documents on the IES Website

FY 2012 Requests for Applications and the Grant Submission Guide are available on:

http://ies.ed.gov/funding

Sign up for the IES Newsflash:

http://ies.ed.gov/newsflash/

http://ies.ed.gov

Finding Application Packages

• FY 2012 Application Packages are available on www.grants.gov

• June Application Package available on April 21, 2011

• September Application Package available on July 21, 2011

Letters of Intent

• For June competition, due 4/21/2011• For September competition, due 7/21/2011• Non-binding, but strongly encouraged• Submitted electronically • Instructions at iesreview.ed.gov• Receipt via e-mail

IES assistance

• Read the Request for Applications carefully.

• Call or email IES program officers early in the process.

• As they have time, IES program staff will review draft proposals and provide feedback to applicants.

Follow Directions and Attend to Details

Read Program Announcements and Application Instructions carefully

• Eligible Applicants• Special Requirements• Content and Formatting Requirements• Application Submission and Processing • Application Due Dates

Application Due Dates in 2011

• Education and Special Education Research:

June 23 and September 22

At 4:30:00 p.m. (EASTERN)

What Happens Next?

Peer Review

• Proposal is reviewed for compliance.• Compliant proposals are assigned to a review

panel.• For Goal 3, three panel members conduct

primary review of each application.• Most competitive applications are reviewed by

full panel at the panel meeting.

Notification

• All applicants will receive email notification of the status of their application.

• All applicants receive copies of reviewer comments.

• Applicants who do not receive an award should consider revising and resubmitting their application. Applicants are encouraged to contact the program officer about the resubmission.

Questions

Frequently Asked Questions

• Advantage of June vs September submission? No.

• Possible start dates?

For June submission: 3/1/12 – 9/1/12

For September submission: 7/1/12- 9/1/12

Frequently Asked Questions (2)

• Can I plan to resubmit in the next competition if I don’t get funded? There is generally not enough time for you to get the reviews, attend to the comments and resubmit.

• How long does the review process take? About 8 months.

Special Education Research queries:

Joan McLaughlin

[email protected]

Education Research queries:

Elizabeth Albro

[email protected]