Grantee Meeting Cognition and Student Learning 2010 IES Research Conference Carol O’Donnell Program Officer.

Grantee MeetingCognition and Student Learning 2010 IES Research Conference

Carol O’DonnellProgram Officer

Agenda for Grantee Meeting (1:30 – 1:35)

• 1:35 – 1:45 Introductions (and important note re: 2011 IES conference)• 1:45 – 2:15 State of the CASL Portfolio

– Statistics– Highlights– Trends and Gaps

• 2:15 – 2:45 Discussion Groups: Making Findings Relevant• 2:45 – 3:15 Report Out: Making Research Relevant • 3:15 – 4:00 Research Issues & Solutions• 4:00 – 4:30 Grant Monitoring Tips (Refresher & for New Grantees)

– Grants Administration– Website – New Funding Opportunities

Introductions (1:35 – 1:45)

• Note about 2011 IES Conference – size is increasing, need program committee to conduct peer review of papers, panels, poster; grantee meetings will piggyback off of SREE conference.

• See list of grantees (handout) and posters (pp 105 – 115)• Rapid Introductions—put a face to a name

– Name– University– Goal (Exploration, Develop., Efficacy, Measurement)– Area of study

State of the CASL Portfolio (1:45 – 2:15)

• Purpose of CASL Program• Portfolio Statistics• CASL Highlights • Trends and Gaps in the CASL Portfolio

Purpose of CASL Program

• To establish a scientific foundation for education by building on the theoretical and empirical advances of cognitive science and applying them to education practice with the goal of improving student learning and academic achievement.

Portfolio Statistics

Number of CASL Awards (2002-2010)

0

2

4

6

8

10

12

2002 2003 2004 2005 2006 2007 2008 2009 2010

From FY02-FY10 IES funded 78 CASL research projects, averaging ~ 9 grant awards per year. Funding rate (# awards / # responsive apps) average around 10-12% but will drop as the number of applications increases.

Content CASL Studies (2002-2010)

0

5

10

15

20

25

30

Reading Writing Math Science SocialStudies

StudySkills

Majority of CASL projects focus on math or science, the primary areas of need identified by Bob Bjork in his 2002 address to the IES staff. Fewer focus on reading or writing. Study skills are often contextualized within a domain. Some projects overlap domains.

Age of Participants in CASL (2002-2010)

0

5

10

15

20

25

30

35

Pre-K Elementary Middle High College

# of apps for college students alone dropped to 0% in FY09; many pre-K apply to Early Childhood Program.

Number of Funded CASL Projects Per Goal (2002-2010)

3

57

13

30

10

20

30

40

50

60

ExplorationGoal 1

DevelopmentGoal 2

Efficacy Goal 3

MeasurementGoal 5

CASL Highlights(see highlights handout)

• More than 20 CASL projects were highlighted in the IES Internal Weekly Report and IES Online Newsletter this past year.

• More than 18 articles with positive findings were submitted to the Commissioner for our IES management goals.

• See the attached sheet for some examples.• Thank you to those who send their publications to me

at the time of their publication (not in press).

CASL Highlights

• Findings from 38 separate CASL projects were highlighted in ten accepted symposia presented at the Association for Psychological Science 2009 and 2010 annual conventions. Symposia topics included: – Developing students’ mental models using animated pedagogical agents.– Learning from concrete and abstract representations.– Test-enhanced learning, spacing, and retrieval-feedback-monitoring. – Self-regulated learning.– Fundamental understanding of mathematics.– Conducting RCT’s of attention interventions in school settings.– Argumentation for critical thinking.– Problem solving in schools.– Neurocognitive functions underlying academic achievement.– Perceptual characteristics and concept mastery.

• 12 CASL grantees contributed to a compendium, Handbook of Metacognition in Education (2009), which provides comprehensive coverage of the theoretical basis of metacognition and its applications to educational practice. Contributors include CASL grantees:– John Dunlosky (editor)– Arthur C. Graesser (editor)– Margaret McKeowen– Danielle McNamara– Joe Magliano– Keith Thiede– Thomas Griffin– Jenny Wiley– Joshua Redford– Barry Zimmerman– Adam Moylan– Janet Metcalfe

• National honors bestowed upon two CASL grantees:– David Klahr honored by the 37th Carnegie Symposium on Cognition for his lifetime

of scientific and education contributions to cognitive development, scientific discovery, and reasoning.

– Katherine Rawson received the 2009 Presidential Early Career Award for Scientists and Engineers for her research on improving the comprehension of text and on helping students self-regulate their learning. Nicole McNeil won this prestigious honor last year.

• CASL findings highlighted in national education news outlets and publications:– Education Week reported on three CASL studies focused on argumentation (Britt,

Kuhn, and Anderson) and one focused on test-enhance learning (Roediger). – The Chronicle of Higher Education reported on the findings of CASL PI Henry

Roediger’s work, which confirmed that frequently quizzing students on reading material helps them retain what they have learned.

– Rohrer and Pashler’s review of experimental studies on optimal instructional strategies published in Educational Researcher.

Trends and Gaps in the CASL Portfolio(see chart handout)

Trends and GAPS in CASL Portfolio• Attention addressed in earlier grades (Neville, Steiner, Hooper,

Rabiner), but not in middle or high school. • Perception studied in all grades (Cottrell, Mix, Siegler, Sloutsky,

Uttal, Goldstone, Kellman, Alibali, Whitten, Novick, Puntambekar, Pavlick), but limited to Goal 2 with one exception (Pasnak—efficacy of patterning).

• Memory—including retrieval, learning, and general conception and misconception—and optimal conditions of learning are the most comprehensively addressed in the portfolio; all grades and all Goals (Russell, Ward, Pashler, Rawson, Heckler, Metcalfe, Bjork, Roediger, Delaney-Black, Millis, Anderson, Blair, Aronson, Biswas, Thiede, Wiley, Zimmerman, Dunlosky).

• Language studied predominantly in the early grades (Anthony, Mostow, Connor, Hooper, and Glenberg), with one in middle grades (McCutchen). None in high school.

• Problem solving covers all grades in Math and Science but mostly Goal 2 (Whitten, Swanson, Star, Sternberg, Beal, Bottge, Booth, McNeil, Ward, Ross, Rabinowitz).

• Argumentation and reasoning (including scientific and mathematical) are well-represented (Katz, Klahr, Lorch, Anderson, Kuhn, Gholson, Holyoak, Rips, Britt), but there are no CASL studies of reasoning in the earlier grades.

• Bigger gaps exist within teacher cognition (Alibali) and social cognition / goal orientation (Dweck, Aronson, Mangels, Beilock).

Trends and GAPS in CASL Portfolio

2:15 – 2:45 Discussion Groups: Making Findings Relevant

(suggested division into Groups 1-7)

Classroom Scenario – Group Discussion

• The previous list of Trends and Gaps in the CASL portfolio represents a series of interdependent cognitive processes involved in making sense of new information within the education setting—attention, perception, memory and learning, language, problem solving, argumentation, and reasoning.

• At a practical level, all of the processes and structures of the cognitive system are interdependent and influenced by both the environment and by student characteristics.

• Review the list of select CASL findings highlighted on handout. • Read the scenario, which attempts to demonstrate the complexity

of what a teacher and student might consider if the findings from CASL were implemented into one school day.

• Knowing your own findings and those highlighted, discuss the following: 1. What trends exist in the CASL portfolio that can cohesively inform a teacher’s

instructional practice, development of instructional materials, and students’ study skills?

2. What gaps are missing? For example, while the CASL portfolio has several studies using artificial intelligence (Millis, Ward & Cole, etc) which involve constructing computer systems that produce intelligent outcomes, there are fewer CASL studies of computational modeling, which involves programming computers to model or mimic some aspect of human cognitive functioning (Anderson, Katz). Other gaps include teacher cognition & social cognition.

3. How can CASL researchers continue to contribute to the development and testing of basic theory (e.g., neuro-education) yet make research questions relevant to education settings?

4. How can we ensure CASL exploratory studies are methodological rigorous yet ecologically valid and lead to the development of feasible education interventions?

Classroom Scenario – Group Discussion

2:45 – 3:15 Report Out:

As you report out your group’s discussion of the 4 questions, consider how we might move forward

by making Research relevant.

Problems and Solutions (3:15 – 4:00)Continue our Discussion of Last Year’s Topics• Methodological Issues (measurement challenges, observational

techniques, translating laboratory research to the classroom—effect size reduction from lab to school JREE)

• Logistical Issues (obtaining preK-12 samples; reducing attrition; maintaining school collaboration; ensuring feasibility & fidelity of implementation)

• Dissemination Issues (getting research into the hands of practitioners, sustaining the intervention after the grant ends)

• Future Grant Issues (Goal 1 vs Goal 2; moving from a Goal 2 to 3; applying to Ed Tech vs CASL)

• Others?

4:00 – 4:30 Grant Monitoring Tips (Refresher & for New Grantees)

- Grants Administration- Website - New Funding Opportunities

Grants Administration

Using the “e-grants” system

• Annual Reports – relationship to performance agreement

• Final Reports• Requesting no-cost extensions• Requesting other administrative changes

Annual Reports (see attached Tips Sheet)

• Cover Sheet

• Section A – Project Objectives

• Section B – Budget Information

• Section C – Additional Information

Cover Sheet

• Complete online• Remember to send original signed cover sheet

to: Director, Grants Administration US Department of Education/IES555 New Jersey Ave., Room 508cWashington, D.C. 20208 FAX: (202) 219-2159

Section A – Project Objectives

• Select “PROJ” for “Measure Type.” Ignore all other “Performance Measure” boxes.

• Your Performance Agreement lists your project objectives for each year. List each objective in Section A, then cut and paste text beneath each objective to describe how you have met it (see the next slide).

PROJ

Write your objective from the performance agreement here.

Explanation of progress on this objective goes here. This is often several paragraphs long.

Section A – Project Objectives• Describe status of all steps taken toward

completion of your projects.• Include details of what was done and how it was

done.• Describe what was completed.• Describe any findings to date.• Describe any work planned, but not undertaken.

Section B – Budget Information

• Provide an explanation if you did not expend funds at expected rate during the reporting period.

• Describe any significant changes to your budget resulting from modification of project activities.

• Describe any changes to your budget that affected your ability to achieve your objectives.

• Include a list or table of broad categories (e.g., Personnel, Travel, Subcontract) that compares the Award Amount for the given grant year vs. Actual Expenses.

Section C – Additional Information• Attach a file the contains any additional information that you may want

to include, such as:– Changes you plan to make for next year that are consistent with scope

and objectives.– Anticipated changes to key personnel. – Updated curriculum vitae for proposed personnel.– Information on the measures that will be developed or have been used in

your study– Unanticipated outcomes or benefits from your project. – List of grant-related presentations & publications.– Copies of papers or posters related to the grant - ERIC.– Revised/anticipated timeline for the upcoming year

• There is no limit as to how many pages you can include here, but it has to be one document.

Other Attachments

• Charts – attach the document that contains any figures or charts referenced in Section A.

• Tables – attach the document that contains any tables referenced in Section A.

• Program Specific Requirements – attach your revised IRB Certification here.

Final Reports

• Due 90 days after the end of your grant award.

• Include the same types of information as in the Annual report, described above, but summarize the entire project

• Must answer the final questions (1, 2, 3).

Final Report Question 1

• Utilizing your evaluation results, draw conclusions about the success of the project and its impact.

• Describe any unanticipated outcomes or benefits from your project and any barriers you may have encountered.

Final Report Question 2

• What would you recommend as advice to other educators that are interested in your project?

• How did your original ideas change as a result of conducting this project?

Final Report Question 3

• If applicable, describe your plans for continuing the project (sustainability; capacity building) and/or disseminating the project results.

Administrative Changes (see NCE handout)

• No-cost extensions (why performance agreement updates are important)

• Changes to key personnel

Using e-grants

http://e-grants.ed.gov

No-Cost Extension

(NCE)

Change of Key

Personnel listed on GAN –

upload CV for new person

IES Website:National Center for Education Research

(NCER)

Go to:

http://ies.ed.gov/ncer

To learn about other CASL grantees’ work, pick a Year, Goal, or PI, then click on the project URL.

ERIC

Education Resources Information Center

http://ies.ed.gov/ncee/projects/eric.asp

Acknowledgement/Disclaimer

• The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant <insert grant number> to <insert name of university or institution>. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

New Funding Opportunities

http://ies.ed.gov/ncer/funding

Applications for Additional Funding

• Education Research – 84.305A (June 24th, Sept 16th deadlines)• Training Grants – 84.305B (June 24th only)

– Postdoctoral Fellowships Only

• New R&D Centers – 84.305C (Sept 16th only)– National Center on Cognition and Adult Literacy– National Center on State and Local Policy– National Center on Postsecondary Education and Employment

• New R&D Center – 84.324C (Sept 16th only)– National Center on Working Memory Interventions for Students with

Disabilities

Cognition and Student Learning

R&D Center on Cognition and Adult Literacy

R&D Center on Working Memory Interventions for Students with Disabilities

Wrap-up and Final Q&A (4:25 – 4:30 pm)

Carol O’Donnell

Cognition & Student Learning

[email protected]

202-208-3749