Grant Writing Thomas S. Buchanan NIH Review Process Study Sections Review Criteria Summary Statement Responding to a Review.

Grant Writing

Thomas S. Buchanan

NIH Review ProcessStudy Sections

Review Criteria

Summary Statement

Responding to a Review

NIH Study Section Meeting

Each Study Section has 12-23 regular members plus temporary ad hoc members

university, government, industry scientists

“regular” and “ad hoc”

One regular member is chair

Scientific Review Officer (SRO) is NIH’s overseer and works for CSR

Up to 60-100 proposals reviewed in a session

NIH Study Section Meeting

Each proposal is assigned to a primary reviewer

a secondary & usually a tertiary reviewer

can have 1-3 “readers” (do not write full reviews)

Each reviewer has about 10 reviews to write and several proposals to read

Everyone is free to discuss/comment

Everyone scores every proposal

Reviewers

Reviewers are not blinded to the applicantsbecause they must assess their qualifications

The applicants will be told who was on the review panel

Reviewers leave the room during the discussion if they

work at the applicant’s institution

are otherwise close to the applicant

NIH study section meeting

NIH study section meeting

“Streamlining” or triageat start reviewers provide list of proposals they reviewed that were in bottom half

if assigned reviewers agree and no one objects, proposal not scored or discussed

anyone can object, no argument necessary

Usually < half streamlined

Norm is ~10-20 min. per discussed proposal

NIH study section meeting

Initial level of enthusiasmPrimary reviewer presents the proposal

descriptionpositive and negative aspects

Secondary & tertiary reviews followdetail depends on extent of agreement

Readers comment, general discussion1º, 2º, 3º reviewers suggest scoresEveryone writes down their own score

NIH study section meeting

Scores are 1 (best) to 9 (worst)Anything ≥ 5 should be streamlined

Mean score of all study section members x 10 = reported score (i.e., scale = 10-90)

NIH study section meeting

Calculating an R01’s percentile score:

All the applications for the current study section meeting are pooled with those from the previous 2 meetings of the same study section; total = N

The scores are rank-ordered and ith application’s percentile is calculated as

100 x (i - 0.5) / N

Ethics, Etiquette, and Politics

The SRO and chair are ethics watchdogsno conflicts of interest, real or perceived

no discussions of application between reviewer and applicant, before or afterward

all discussions of applications between reviewers must occur in session

The mood of the room is professional

Other NIH administrators usually present

NIH Funding Decisions

Funding is based on 2 levels of reviewstudy section - 90% of the decisionthe institute’s advisory council

The “council” = intramural and extramural scientists and administrators

assess quality of reviews

decide on grant’s budget

factor in legislative mandates

cannot alter the scientific evaluation or score

Program Manager

Note that the Program Manager at the Institute has almost no say in the initial review process

This is very different than at NSF

The Program Manager can help guide you towards particular funding mechanisms (R01 vs R03, etc.)

Once a proposal receives a priority score, the Program Manager has some discretion to “help” borderline proposals.

NIH Review Process

Video clip from CSRhttp://youtu.be/fBDxI6l4dOA

How to Improve your Grant Proposal

Assessment, revisions, etc.

Afterwards: the Summary Statement

Study section, roster

Score, percentile

Budget recommendations

Summary of the discussion

Reviewers’ critiques

The Critique

For R and P grants (e.g., R03, P01), the five scored criteria for research grant applications are Significance, Investigator(s), Innovation, Approach, and Environment.

Other grant types have different scored criteria (e.g., K, F, T and S awards)

The final score for any grant is based on overall impact.

Overall Impact (R & P awards)

“Reviewers will provide an overall impact score to reflect their assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the following five core review criteria, and additional review criteria (as applicable for the project proposed)”

1. Significance

“Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?”

2. Investigators(s)

“Are the PD/PIs, collaborators, and other researchers well suited to the project? If Early Stage Investigators or New Investigators, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project?”

3. Innovation

“Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?”

4. Approach (1 of 2)

“Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed?”

4. Approach (2 of 2)

“If the project involves clinical research, are the plans for 1) protection of human subjects from research risks, and 2) inclusion of minorities and members of both sexes/genders, as well as the inclusion of children, justified in terms of the scientific goals and research strategy proposed?”

5. Enviornment

“Will the scientific environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements?”

Additional Review Criteria

These might not affect the score, but can influence reviewers’ enthusiasm:• Protection of Human Subjects• Inclusion of Women, Minorities & Children• Vertebrate Animals• Biohazards• Budget• Resource Sharing Plan (Data Sharing Plan,

Sharing Model Organisms, & Genome Wide Associate Studies)

These are not

discussed by SS until

after the proposal is

scored.

Responding to a Review

How not to respond to a reviewhttp://youtu.be/H69n3LmwlTI

Afterwards: the Revision

Carefully analyze the critiqueswhat was uniformly disliked

what should be changed vs. re-explained

what additional data could be provided

Are there words of encouragement embedded in the criticisms?

Are significant strengths mentioned?

“... above average enthusiasm…”

Afterwards: the Revision

If the chances for successfully addressing the criticisms seem good, revise

begin with “Introduction” addressing reviewers’ criticisms

be gracious, respond positively

you may or may not get the same reviewers, but your attitude and effort to respond will be appreciated

Afterwards: the Revision

You get 1 chance to revise; after that you have to submit a “different” proposal

If you revise and resubmit promptly, you will have 2 proposals in the “pool”

oddities of scoring and funding occur

if you were close to the funding cutoff, this may increase your odds of success

Summary: the “do’s”

good idea, science, and applicationmechanistic, testable hypothesesconvincing, appropriate preliminary datadetailed research plan, based on statistical planningwrite clearly, state your case as rationally and convincingly as possiblerevise repeatedly before submission

Summary: the “don’ts”

Not too simple, not too ambitiousthe problem must be significant

10 hypotheses is probably too many!

avoid sloppy writinguse spell checker, check your grammar

don't make unsupported statements

don't wait until the last minute; it shows!

And now …a word from our

sponsor

!!! Commercial Break !!!

!!! Commercial Break !!!DRI offers resources to help you with your research!

Grant ReviewsWe will provide pre-reviews for you if you get us your grant proposal to us early

DRI Core Resources for UD facultyResCore• Patient recruitment, scheduling and clinical database • Biostatistics with Barry!

Cytomechanics Core• Confocal µscope and mechanical testing of cells!

Patient Specific Modeling Core• Gait, ultrasound and biomechanical modeling!

Sample Summary Statements

Courtesy of Hank Donahue

Go here!