Cheryl A. Kitt PHD Deputy Director CSR October 30, 2009 Center for Scientific Review and National Institutes of Health U.S. Department of Health and Human.

• Cheryl A. Kitt PHD

• Deputy Director CSR

• October 30, 2009

• Center for Scientific Review and National Institutes of Health

• U.S. Department of Health and Human Services

Cheryl A. Kitt, Ph.D.Cheryl A. Kitt, Ph.D.Deputy Director, CSRDeputy Director, CSR

October 30, 2009October 30, 2009

Nationa-l Institutes of HealthNationa-l Institutes of Health

U.S. Department of Health and HumanU.S. Department of Health and Human ServicesServices

Update on Peer Review

CSR Peer Review: 2008 StatisticsCSR Peer Review: 2008 Statistics

• 77,000 applications received

• 16,000 reviewers

• 1,400 review meetings

• 240 Scientific Review Officers

CSR Peer Review: 2009

• 700,00 applications received

• Was 16,000 reviewers, now 115,000

• Was 1,400 review meetings, now 38,000

• Was 240 Scientific Review Officers, now 1,600

2008: The Year of Peer Review2008: The Year of Peer Review

Enhancing Peer ReviewEnhancing Peer Review

““Fund the best science, by the Fund the best science, by the best scientists, with the least best scientists, with the least

administrative burden…” administrative burden…”

Elias Zerhouni, MD, Former Elias Zerhouni, MD, Former Director, NIHDirector, NIH

Amended Applications:Amended Applications:

To speed the funding of meritorious science To speed the funding of meritorious science and minimize reviewer burden:and minimize reviewer burden:

• As of January 25, 2009, all original new applications (i.e., never submitted) and competing renewal applications are permitted only a single amendment (A1)single amendment (A1).

Balanced and Fair Reviews Across Career Balanced and Fair Reviews Across Career Stages and Scientific FieldsStages and Scientific Fields• New Investigator (NI): New Investigator (NI):

PD/PI who has not yet competed successfully for a substantial NIH research grant

o For multiple PD/PIs-all PD/PIs must meet requirements for NI status

• Early Stage Investigator (ESI):Early Stage Investigator (ESI):

PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent)

• Applies only to Applies only to R01R01 applications applications• New Investigators/Early Stage Investigators are New Investigators/Early Stage Investigators are clusteredclustered

together for reviewtogether for review• TAKE HOME MESSAGE 2: TAKE HOME MESSAGE 2: New Investigators/Early Stage New Investigators/Early Stage

Investigators Investigators Apply Now!!!Apply Now!!!

• New Investigator (NI): New Investigator (NI):

PD/PI who has not yet competed successfully for a substantial NIH research grant

o For multiple PD/PIs-all PD/PIs must meet requirements for NI status

• Early Stage Investigator (ESI):Early Stage Investigator (ESI):

PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent)

• Applies only to R01 applicationsApplies only to R01 applications• New Investigators/Early Stage Investigators clustered New Investigators/Early Stage Investigators clustered

together for reviewtogether for review

Enhanced Review CriteriaEnhanced Review Criteria

• Overall Impact: Overall Impact: Assessment of the likelihood for the project to exert a sustained, powerful influence on the exert a sustained, powerful influence on the research field(s) involvedresearch field(s) involved

• New Core Criteria OrderNew Core Criteria Order::

SignificanceInvestigator(s)InnovationApproachEnvironment

o Review criteria enhanced and expanded Review criteria enhanced and expanded and each scored from 1-9and each scored from 1-9

ScoringScoringTo improve the transparency of the scoring process:To improve the transparency of the scoring process:• Applications scored on five review criteria using a scale of 1-five review criteria using a scale of 1-

99.

• Preliminary overall impact score using 1-9 scaleoverall impact score using 1-9 scale.

• Discussed applications receive an overall impact score from each eligible (i.e., without conflicts of interest) panel member and these scores are averaged to one decimal place, and multiplied by 10. The 81 possible priority scores thus range from 10-90.

• Percentiles reported in whole numbers.• All applications receive scoresAll applications receive scores::

Not DiscussedNot Discussed applications receive initial criterion scores from the three assigned reviewers

Scoring DescriptionsScoring Descriptions

Impact Score Descriptor

High Impact

1 Exceptional

2 Outstanding

3 Excellent

Moderate Impact

4 Very Good

5 Good

6 Satisfactory

Low Impact

7 Fair

8 Marginal

9 Poor

What Happens at the Study Section Meeting?What Happens at the Study Section Meeting?

Discussion of applications Discussion of applications in order in order of of average preliminary score:average preliminary score:

Why:Why:• Concern - variation of scores during different times

of the meeting.

One recommendation was to recalibrate scores at One recommendation was to recalibrate scores at the end of the meeting the end of the meeting

Solution:Solution:• Recalibrate dynamically throughout meeting. Requirement:Requirement:

Reviewers must participate in entire meeting.

CritiquesCritiques

To improve the quality of the critiques and to focus To improve the quality of the critiques and to focus reviewer attention on the review criteria:reviewer attention on the review criteria:

• Electronic template for critiques Electronic template for critiques prompts for strengths and weaknesses for each criterion.

• Summary statements are shortershorter and more focused.

• Discussed applications onlyonly receive a summary of summary of the panel’s discussiondiscussion at the meeting.

• ALL applications are scoredALL applications are scored. and receive critiques

Not discussed Not discussed applications receive criterion scores only

• January 2009 Due Dates (for potential FY2010 funding) 1. Early Stage Investigator (ESI) and New Investigator Policy (NOT-OD-

09-013; http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-013.html)

2. New NIH Policy on Resubmissions (NOT-OD-09-003: http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-003.html)

• May 2009 Review Meetings (for potential FY2010 funding) 1. 9-Point Scoring System 2. Enhanced Review Criteria 3. Formatted Reviewer Critiques 4. Scoring of Individual Review Criteria 5. Clustering of New Investigator Applications During Review • January 2010 Due Dates (for potential FY2011 funding) 1. Shorter Applications for R01s and Other Mechanisms 2. Restructured Applications to Align with Review Criteria

Enhancing Peer Review Implementation TimelineEnhancing Peer Review Implementation TimelineJanuary 2009 Due Dates January 2009 Due Dates (for potential FY2010 (for potential FY2010 funding) funding)

1.1.Early Stage Investigator (ESI) and New Early Stage Investigator (ESI) and New Investigator Policy (NOT-OD-09-013; Investigator Policy (NOT-OD-09-013; http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-013.html) 2.2.New NIH Policy on Resubmissions (NOT-OD-09-New NIH Policy on Resubmissions (NOT-OD-09-003: 003: http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-003.html) May 2009 Review Meetings (for potential May 2009 Review Meetings (for potential

FY2010 funding)FY2010 funding)1.1.9-Point Scoring System 9-Point Scoring System 2.2.Enhanced Review Criteria Enhanced Review Criteria 3.3.Formatted Reviewer Critiques Formatted Reviewer Critiques 4.4.Scoring of Individual Review Criteria Scoring of Individual Review Criteria 5.5.Clustering of New Investigator Clustering of New Investigator Applications During Review Applications During Review

January 2010 Due Dates January 2010 Due Dates (for potential FY2011 (for potential FY2011 funding)funding)

1.1.Shorter Applications for R01s and Other Shorter Applications for R01s and Other Mechanisms Mechanisms 2.2.Restructured Applications to Align with Review Restructured Applications to Align with Review Criteria Criteria

National Institutes of Health

• Home page from NIH web site www.nih.gov

Applications Reviewed by CSR (June-July 2009)Applications Reviewed by CSR (June-July 2009)

• Non ARRA Applications (Normal) 16,312

• Challenges Applications 20,894

• Competitive Revisions 2,077

• GO Grants 2,697 (CSR 500?)

• P 30 561

• TOTAL as of 2/6/09 40,000 plus

• Buliding Sustainable Community-Linked Infrastructure Announcement Number RFA-OD-09-010

• Title: Recovery Act Limited Competition: Building Sustainable Community-linked Infrastructure to Enable Health Science Research (RC4)

• Issuing Org: NIH

• Release Date: 09/18/2009

• Receipt Date: 12/11/2009

http://grants.nih.gov/recovery

Historical Background of the CFS Recurring Special Historical Background of the CFS Recurring Special Emphasis PanelEmphasis Panel

103rd Congress (1993-1995)

H.R.4Title: To amend the Public Health Service Act to revise and extend the programs

of the National Institutes of Health, and for other purposes. [NationalInstitutes of Health Revitalization Act of 1993]

Sponsor: Rep. Waxman, Henry A. [CA-29] (introduced 1/5/1993)

S.1Title: A bill to amend the Public Health Service Act to revise and extend the

programs of the National Institutes of Health, and for other purposes.[National Institutes of Health Revitalization Act of 1993]

Sponsor: Sen. Kennedy, Edward M. [MA] (introduced 1/21/1993)==================================================================

Sec.902. Chronic Fatigue Syndrome(b) EXTRAMURAL STUDY SECTION- Not later than 6 months after the date of

enactment of this Act, the Secretary of Health and Human Services shallestablish an extramural study section for chronic fatigue syndrome research.

==================================================================6/10/1993 Became Public Law No: 103-436/10/1993 Became Public Law No: 103-43

TYPICAL RANGE OF EXPERTISE NEEDED FOR ZRG1 CFS PANEL MEETINGTYPICAL RANGE OF EXPERTISE NEEDED FOR ZRG1 CFS PANEL MEETING

Alternative Medicine Laboratory immunology

Behavioral Medicine Longitudinal animal study design

Biological signal transduction Longitudinal human study design

Biometrics Lymphokines/cytokines

Cardiovascular physiology Magnetic resonance imaging

Clinical Chronic Fatigue Syndrome Neuropharmacology

Clinical Fibromyalgia Neuropsychiatry

Clinical immunology Neurotrophins

Clinical Temporomandibular Disorders Pain control/management

Cognition Pain neurophysiology

Endocrinology Pain perception

Epidemiology Proteomics

Ethicolegal issues Psychology of chronic disease

Exercise physiology/kinesiology Psychometrics

Gene expression Psychoneuroimmunology

Gene regulation Quality of Life assessment

Genomics Receptor biochemistry and genetics

Health Policy Sensory psychology

Health promotion/disease prevention Sleep physiology

Inflammation Sociology of chronic disease

Insomnia/sleep studies

[email protected]

• 301-435-1112• National Institutes of Health (NIH)• U.S. Department of Health and Human Services• Picture of NIH office

[email protected]@csr.nih.gov

301-435-1112301-435-1112