Top Banner
FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 1 Gaylen Moore Program Evaluation Services 304 West 89 Street, New York, New York 10024 Telephone / Fax (212) 724-8812 gmoore1@nyc.rr.com Evaluation of the Wildlife Conservation Society, The New York Botanical Garden, Lehman College Department of Early Childhood and Elementary Education Fund for Improvement of Secondary Education (FIPSE) Outside-the-Box Professional Development Program Year 3 2013-14 Submitted by Gaylen Moore
40

Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

Mar 26, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 1

Gaylen Moore Program Evaluation Services 304 West 89 Street, New York, New York 10024

Telephone / Fax (212) 724-8812 [email protected]

Evaluation of the Wildlife Conservation Society, The New York Botanical Garden,

Lehman College Department of Early Childhood and Elementary Education

Fund for Improvement of Secondary Education (FIPSE) Outside-the-Box Professional Development Program

Year 3 2013-14

Submitted by

Gaylen Moore

Page 2: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 2

Gaylen Moore Program Evaluation Services 304 West 89 Street, New York, New York 10024

Telephone / Fax (212) 724-8812 [email protected]

Wildlife Conservation Society, The New York Botanical Garden,

Lehman College Department of Early Childhood and Elementary Education Fund for Improvement of Secondary Education (FIPSE)

Outside-the-Box Professional Development Program

EVALUATION REPORT Year 3

2013-14

I. INTRODUCTION PROGRAM DESCRIPTION The Wildlife Conservation Society/Bronx Zoo (WCS) concluded the third year of the grant awarded by the Fund for Improvement of Secondary Education (FIPSE) in June 2013. The Outside-the-Box (OTB) program was designed to introduce science to pre-service elementary teachers in the Lehman College Department of Early Childhood and Elementary Education and model the use of outdoor settings to teach inquiry science and integrate science into all subject areas. Project staff educators from the Wildlife Conservation Society/Bronx Zoo (WCS) and The New York Botanical Garden (NYBG) delivered one full-credit science methods course called Learning and Teaching Science in Childhood Settings in June 2013 to conclude the third year of implementation. The course was delivered at both the New York Botanical Garden and WCS sites. To help students complete their assignments and utilize the resources of these institutions to facilitate planning outdoor learning activities, the Outside-the-Box program provided semester-long free passes for students to visit both sites and an online website which provided access to a rich variety of content and instructional resources. II. EVALUATION METHODOLOGY EVALUATION GOALS AND OBJECTIVES Program Goals

Page 3: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 3

The FIPSE Outside-the-Box program was designed to answer the following questions: 1) What science attitudes and disciplinary understandings do participants hold before beginning the program? 2) What changes in science attitudes and disciplinary understandings do participants demonstrate by the end of their experiences? 3) What teaching and learning practices do Lehman teachers engage in to support participants’ science learning after conclusion of the

program? 4) How do participants’ new practices influence their students’ engagement and understanding of science? 5) How can the project most effectively disseminate the lessons learned so that others can replicate all or parts of the model?

Program Objectives Between Year 1 and Year 2, project staff revisited the goals for achieving program effectiveness that were established in Year 1, and determined that not all the proposed objectives were being addressed by program activities. Objectives 3 and 4 cold not be addressed by the program or the evaluator due to a lack of response from participants after they graduated from Lehman. Consequently, the staff identified four areas in which program activities were aligned with program goals. The following goals are those that will be addressed in this report. It was expected that students completing the entire program of Outside-the-Box courses would be able to:

• demonstrate positive attitudes and knowledge about life science • design science lessons that utilized outdoor settings to facilitate student learning • demonstrate knowledge and ability to integrate science into other subject areas • increase confidence, comfort, and preparation to apply the skills and knowledge acquired during the program

In Year 3, the evaluator conducted the following evaluation activities:

• Pre/Post Knowledge Assessment administered in the Learning and Teaching Science in Childhood Settings course to assess changes in science content knowledge.

• Pre/Post Participant Survey to assess interest and attitudes toward science, the integration of science into other subject areas, and use of outdoor settings to teach science.

• Observations of Outside-the-Box science course sessions to assess quality of project implementation • Course Evaluation Survey to assess participants’ perceptions of the quality and impact of project implementation

Page 4: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4

SCOPE OF REPORT This report includes an analysis of the implementation of the Teaching and Learning Science in Childhood Settings science course, which concluded the grant in June 2013. The analysis includes: 1) an analysis of Year 3 implementation; 2) a three-year cohort comparison of outcomes; and 3) an aggregated analysis of outcomes for all three cohorts combined. The evaluator was unable to answer research questions 3 and 4 to assess how participants utilized content and practices learned in the Out-of-the-Box program in their own classrooms. Repeated efforts to schedule focus groups, interviews, and visits to participants’ classrooms after they graduated from Lehman College resulted in no responses. Another factor that limited collection of data to answer these questions was that not all Lehman Out-of-the-Box graduates obtained teaching positions when they graduated. III. FINDINGS YEAR 3 ANALYSIS IMPLEMENTATION The Learning and Teaching Science in Childhood Settings course was taught at the New York Botanical Garden and the Bronx Zoo in nine four-hour classes in June 2013, five sessions at the Garden and four sessions at the Zoo. The curriculum addressed the nature of science (making observations, asking questions, collecting and interpreting data) and highlighted basic concepts of life science (defining living things, identifying biotic and abiotic elements – and their interactions -- in an ecosystem, and exploring biodiversity). The course activities included: exploring habitats, adaptations and survival strategies; identifying relationships and changes in the ecosystem; recognizing the diversity of life; and examining human impact on the environment. The course also touched on components of physical, planetary, and earth science. During this course, students conducted their own plant growth experiments; visited exhibits at NYBG and the Bronx Zoo; engaged in scientific investigations and interactive activities outdoors at these sites that were designed to facilitate their understanding of science, and provide opportunities for them to practice the skills and processes of scientific inquiry. A total of 15 students enrolled in the Lehman College Department of Early Childhood and Elementary Education participated in the final year of the Out-of-the-Box grant activities. QUALITY OF IMPLEMENTATION

Page 5: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 5

In Year 3, the Outside-the-Box program continued to demonstrate overall high quality implementation, incorporating the evaluator’s recommendations from previous years to improve the course. The course modifications reflected a more simplified and streamlined approach to the plant experiment as the focus of the course, based on the course time constraints and the previous assessment of participants’ limited prior knowledge related to controlled experiments. Consequently, participants’ acquired greater understanding of the scientific skills and processes involved in a controlled experiment, and were able to build their knowledge related to plant growth and apply their skills to the process of science investigation in general. Participants’ Perceptions of Implementation: Year 3 At the conclusion of the course in June 2013, the evaluator administered a course evaluation survey with a five-point scale (1=Very Low to 5= Very High) to assess participants’ perceptions of the quality of implementation. Findings indicate that participants found the Learning and Teaching Science in Childhood Settings course to be effective in achieving the program’s objectives to improve participants’ capacity to:

• demonstrate positive attitudes and knowledge about life science • design science lessons that utilized outdoor settings to facilitate student learning • demonstrate knowledge and ability to integrate science into other subject areas • increase confidence, comfort, and preparation to apply the skills and knowledge acquired during the program

Participants rated the science course highly overall by participants for the quality of implementation; 96 percent gave positive ratings to their overall engagement in the course, and 86 percent rated the quality of the course as “High” to “Very High.”

Page 6: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 6

Science Course Evaluation, Year 3 N= 14

93% 79%

100% 100% 93% 93% 86%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

Your overall self-engagement in

the module

Your knowledge of the module

content

Your increased knowledge of wildlife/animal

conservation and WCS

Your increased knowledge of plants and the

natural environment and

NYBG

Your increased awareness of

outdoor sites to support teaching

and learning

Overall likelihood you will use or

apply information obtained from this course in

your classroom

Overall quality of course

experience

% ra

ted

high

/ver

y hi

gh

Learning and Teaching Science in Childhood Settings Science Course Evaluation

Page 7: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 7

• 96 to 100 percent of participants reported the science course contributed to increasing: knowledge about wildlife and animal conservation and plants and the natural environment; awareness of outdoor sites to support teaching and learning; and the likelihood that they would use what they learned in the course in their future classrooms.

• The majority of participants reported the amount of information and the range of topics covered in the course were “just right.” They also believed the assumed level of prior knowledge and the amount of time that instructors devoted to each topic was just right. Fewer participants reported that the overall length of the science course was just right. Those who were dissatisfied with the length of the course felt it was too long. Some felt the nine sessions were too much.

79% 86% 79% 86% 57%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

Assumed level of your prior knowledge

Amount of information presented

Amount of time devoted to each topic

or learning experience

Range of topics Overall length of module

% ra

ted

just

righ

t

General Impressions of Science Course Year 3

Page 8: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 8

• All participants were satisfied that the science course introduced them to new and innovative ideas (100 percent) and prepared them to teach science in other subject areas using the facilities and recourses of WCS and NYBG (100 percent). Almost all participants (93 percent) reported that the science knowledge they gained enhanced their ability to teach science and integrate it into other subject areas. Fewer participants were satisfied with their ability to implement the pedagogical practices modeled in the course to teach science.

100% 93% 79%

93% 100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

I was introduced to new or innovative

ideas

The science knowledge gained from this module enhanced

my ability to teach science to elementary

children

The pedagogical practices and

instructional strategies modeled in this module enhanced my ability to

teach science effectively to

elementary children

The module facilitated my ability to integrate wildlife and the natural environment into my instruction in other

subject areas

The module prepared me to teach science in

other subject areas using the Wildlife

Conservation Society and the New York Botanical Garden

% ra

ted

agre

e/st

rong

ly a

gree

Impact of Science Course

Year 3

Page 9: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 9

• Participants believed the WCS and NYBG websites were clear and easy to navigate for the purposes of the course assignments and submissions. Further, they reported the online resources would be relevant to their future role as elementary teachers, and would facilitate their ability to incorporate wildlife and the natural environment into lesson planning and instruction.

86% 93% 93% 93%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

The WCS and NYBG websites were clear and easy to navigate for my

purposes

The resources were easy to access

The resources were relevant to my role as a

future elementary teachers

The resources facilitated my ability to incorporate wildlife and the natural

environment into lesson planning and instruction.

% ra

ted

agre

e/st

rong

ly a

gree

Rating of Online Component Year 3

Page 10: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 10

• Participants found the interactive activities to be the most effective strategy in facilitating their learning, followed by the discussions, then pictures and other visuals used by instructors to teach the course.

86%

57%

100%

71% 93%

79%

50%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

Pictures and other visuals

Lecture presentation

Interactive Activities

Suggested resources

Discussions Assignments Readings

% ra

ted

influ

entia

l/mos

t inf

luen

tial

Contributions of Program Components to Learning in the Science Course Year 3

Page 11: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 11

An analysis of participants’ use of NYBG and WCS free passes provided by these institutions to do their assignments for the course shows that a majority of participants used their passes to visit the sites.

• On average, participants visited the Bronx Zoo two times and the New York Botanical Garden once • Participants visited both institutions to prepare coursework, and plan lessons or field trips to meet the course requirements.

Participants Open-Ended Responses to Science Course In response to questions about how they planned to use what they learned in the science course in their own classrooms, when they became teachers, participants cited the following activities and practices:

• The adaptation game with the candy. • The lima bean experiment. • I will use the outdoor resources to teach all subjects. • I would teach students about biotic, abiotic, ecosystems and the environment in creative ways, such as taking them on a leaf

identification field trip. • Using outdoor resources to further ELA, math lessons with science.

79% 71%

0%

20%

40%

60%

80%

100%

% u

sed

Use of passes to visit zoo or garden Year 3

Bronx Zoo Passes New York Botanical Garden Passes

Page 12: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 12

• I learned being in this course that there are many different ways to teach science. Science can be done in the outdoors, which makes students more excited to learn.

• The use of follow-up sessions as a means for differentiation to make sure students understand objectives. When asked how they would integrate science into other disciplines, participants reported they would adopt the following activities or practices:

• Nature walk counting living and non-living thing. • Incorporate ELA & math in my outdoor lesson plans such as measuring trees and writing a leaf story. • Graphing ,science research, math, reading, science literature-ELA. • When I visited the classroom at the Bronx Zoo where a lesson was being conducted I saw how science could be incorporated with

literacy. For example, the teacher had the students write poems about the animal they were observing. • Students can take their observations and claims to research more background on a topic and unite informational text on the topic. • Use of Venn diagrams for comparing and contrasting ,then students can write a passage on observation from the diagram,

incorporating literacy. When asked how they would use NYBG and WCS with their students, participants reported:

• Use the Bronx zoo to study animals & habitats. • I plan to focus on specific exhibits and teach about them prior to visiting. • Children can research a specific animal then take a field trip to the zoo and observe the animal they researched. • I saw how the Bronx Zoo and NYBG can be used to teach many different lessons about plants and animals.

When asked to cite the greatest benefit of the course, participants reported:

• It was helpful in using outdoor environments. • Overall my experience with the course was pleasant. • The course was very informative and I found it to be engaging. In a 4+ hour class, it was good to be able to move around and explore

while learning. • This program made science less frightening and fun. • This program strengthened my knowledge about science and I was surprised at my knowledge from the pre test to the post test. • Very exciting, interesting and informational class. Enjoyed the class since I was about to learn new things on how I can teach science

in my future class. • I loved this class. It wasn't what I expected. I now aspire to be that awesome science teacher I never had. I am soooo excited and

ready to learn more. • This program models great and useful methods of using outdoor spaces in all school subjects.

Page 13: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 13

IMPACT ON PARTICIPANTS: YEAR 3 To assess the impact of the Outside-the-Box program on the Lehman College pre-service elementary education students, and the program’s effectiveness in achieving its revised objectives, the evaluator used two instruments that were used in the evaluation of each of the previous years of the program. The instruments included: 1); a pre/post knowledge assessment administered to students in the Learning and Teaching Science in Childhood Settings science course, and 2) a pre/post attitude survey administered to students before and after participation in the science course to assess their personal beliefs and attitudes toward science, knowledge of life science, attitudes toward inquiry practice, and attitudes about using inquiry in outdoor and out-of-the classroom settings to facilitate learning. Impact on Participant Knowledge: Year 3 The pre-post knowledge assessment was administered to participants before and after their participation in the Learning and Teaching Science in Childhood Settings science course. The instrument had 22 items, representing a combination of multiple choice and open-ended questions related to key concepts of life science. Findings indicate that the program had a positive impact on increasing participants’ knowledge of the content covered in the course.

• Participants showed tremendous gains in their knowledge assessments from pre to post test. On average participants had a percentage gain of 20 percent. A paired t-test showed that the gains were significant.

62.93% 82.00%

0.00%

50.00%

100.00%

% c

orre

ct

Learning and Teaching Science in Childhood Settings Science Course

Pre/Post Knowledge Assessment Year 3

Pre-Know N=15 Post Know N=14

Page 14: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 14

Impact On Participants’ Attitudes: Year 3 Analysis of survey data collected in the science course indicate the program achieved its objective to promote a positive impact on participants’ attitudes about using outdoor settings to facilitate student learning in science and other disciplines.

• By the end of the Year 3 science course all participants believed science was important for all children to know. The course also developed more positive attitudes and beliefs among participants about their personal interest in science and confidence in understanding of science. The experience did not result in a greater interest in learning more science, however.

60%

20%

73%

33%

80% 69%

15%

69% 77%

100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

I have a personal interest in science

I read science's books and articles for pleasure and

learning

Interest in learning more science

Confident in my understanding of

science

The importance for children to know

science

% ra

ted

agre

e/ag

ree

to a

gre

at e

xten

t

Science Course Participants' Personal Attitudes/Beliefs: Year 3

Pre N=15

Post N=13

Page 15: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 15

• Participants overall showed great gains in their confidence about their content knowledge concerning these key concepts of life science and topics about the environment. The gains were significant.

My current knowledge

of life science is adequate

for elementary

students

Confident about

classify objects as living or

non-living

Confident about

describing the role of structure

and function within

plants and animals

Confident about

recognizing that traits of living things

are inherited,

acquired or learned

Confident explaining how the

adaptations of species

depends on their

environment

Confident identifying life cycles

and processes of growth

and reproductio

n

Confident knowing about the

abiotic environmen

t

Confident about

relationships between biotic and

abiotic component

s of the ecosystem

Confident knowing

how humans

have changed

environment, positively

& negatively

and its effects

Pre N=15 20% 47% 33% 40% 47% 27% 33% 20% 47% Post N=13 77% 100% 85% 85% 92% 85% 100% 100% 100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

% a

gree

/agr

ee to

a g

reat

ext

ent

Science Course Participants' Confidence in Content Knowledge: Year 3

Page 16: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 16

• Participants overall showed gains in their beliefs about their ability to use what they learned in the course to facilitate student learning. The program appeared to be most effective in teaching participants how to use the NYBG and Bronx Zoo to generate interest and excitement about science.

Use essential

questions to structure science lessons

Use observations

of specimens

or phenomena to stimulate

students

Provide reading

opportunities to develop students'

understanding of science

integrate science into

math, literacy, ss, and other disciplines

use questioning strategies to

elicit students

ideas

use collaborative small group

activities

use the NY botanical

garden and bronx zoo to

generate interest and excitement

about science

engage students in

observations or hands-on investigation

s in the classroom

use outdoor sites, parks, museums, and sites

outside the classroom to

conduct observations

or investigation

s Pre N=15 73% 73% 87% 73% 80% 87% 67% 87% 93% Post N=13 92% 92% 85% 100% 92% 100% 100% 100% 100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

% ra

ted

agre

e/st

rong

ly a

gree

Science Course Pedagogy: Year 3

To facilitate students' science learning it's important to:

Page 17: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 17

• Participants significantly increased confidence on all parameters measured in their ability to teach science, achieving the program’s objectives in this sphere. They reported increased confidence in their ability to identify science inquiry, design and teach developmentally appropriate lessons using inquiry; address the science content standards; integrate the principles of environmental education; integrate science teaching and learning into science learning and other subject areas; and use outdoor settings to facilitate student learning.

THREE-YEAR COHORT COMPARISON Three-Year Knowledge Comparisoon In this final year of the project, the evaluator compared findings for the three years of the Learning and Teaching Science in Childhood Settings course to assess the program’s long-term effectiveness. The evaluator examined the pre/post knowledge and pre/post attitudes of each cohort participating in the science course in May/June 2011, 2102, and 2013.

identify the elements of an effective

science inquiry lesson

designing a developmentally appropriate

science inquiry lesson

designing a developmentally appropriate

science inquiry in

connection with nature

teach a developmentally appropriate

inquiry science lesson

address science content

standards in inquiry lesson

incorporating the principles

of environmental education into inquiry lesson

intergrate science

teaching and learning into other subject

areas

use outdoor settings to facilitate students' science learning

Pre N=15 33% 47% 53% 40% 40% 47% 47% 67% Post N=13 92% 77% 92% 77% 92% 92% 100% 100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

% ra

ted

agre

e/ag

ree

to a

gre

at e

xten

t Science Inquiry Practice: Year 3

Participants' confidence in ability to:

Page 18: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 18

Findings show that the program was increasingly effective in achieving its objective to build participants’ knowledge of science concepts related to plant and animal life.

*only post scores shown

• A paired-sampled t-test was conducted to assess the significance of the outcomes of the pre/post knowledge assessment. There was a

significant difference in the scores for the pre and post-knowledge assessment across all Cohorts (p<. 000). Cohort 3 demonstrated the highest average scores of any cohort. This finding provides further evidence of the program’s effectiveness in achieving its goal to increase participants’ knowledge of life science.

Three-Year Attitude Comparison To assess the program’s effectiveness over the three years in developing more positive attitudes among participants about learning science and teaching science, the evaluator examined pre/post attitude survey findings from participants in all three cohorts of the science course in June

Avg.  Knowledge  Assessment  Score  Cohort  1  (n=32)   68%  Cohort  2  (n=25)   79%  Cohort  3  (n=15)   82%  

68%  79%   82%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

Science  Course  Knowledge  Domain:  Cohort  Comparison  of  Knowledge  Assessment  Scores  

Page 19: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 19

2011, 2012, 2013. On a Likert scale of 1 (Strongly Disagree) to 5 (Strongly Agree), participants rated their attitudes about science content knowledge, teaching science, and implementing science. The attitude findings have been organized into three domains: 1) the Knowledge Domain which addresses the science content knowledge objective; 2) the Teaching Domain, which addresses the objectives in using outdoor settings and integrating science into other subject areas, and 3) the Application Domain, which addresses the objective related to students’ confidence and comfort in using newly acquired knowledge and skills in their future classrooms. Analysis of the data in all three domains combined shows that participants reported a total positive increase in attitudes. The increase in positive attitudes across the three cohorts of participants was statistically significant (p < .05). These findings indicate the program achieved its objectives to develop and promote more positive attitudes about participants’ interest in science and confidence in their ability to incorporate knowledge, strategies, and practices they learned in the course when they got their own classrooms. Knowledge Domain: Three-Year Cohort Comparison

*only post scores shown

I  have  a  personal  interest  in  science    

I  read  science's  books  and  articles  for  pleasure  

and  learning          

Interest  in  learning  more  science      

ConLident  in  my  understanding  of  science  

The  importance  for  children  to  know  science    

Cohort  1   3.60   3.17   3.80   3.43   4.31  Cohort  2   3.43   2.61   3.52   3.57   4.30  Cohort  3   3.69   2.85   3.92   3.85   4.77  

0.00  0.50  1.00  1.50  2.00  2.50  3.00  3.50  4.00  4.50  5.00  

Science  Course  Knowledge  Domain:  Cohort  Comparison  of  Attitudes  About  Science,        

by  Question    

Page 20: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 20

• There was a significant difference in scores for the overall pre attitude beliefs and post attitude beliefs for all cohorts individually and aggregated (p<. 05). These findings provide evidence that the science course contributed to achieving the program’s objective to increase participants’ interest and attitudes about increasing science knowledge for themselves and their students.

*only post scores shown

• When assessing participants confidence in science knowledge, findings show there was a significant difference in participants’ pre and

post confidence in their knowledge of key concepts of life science for Cohorts 2 and 3 (p<.05) after participating in the science course. This finding provides evidence of the increasing effectiveness of the WCS and NYBG educators in teaching these concepts. The

My  current  knowledge  of  life  science  is  adequate  for  elementary  students      

ConLident  about  classify  objects  as  living  or  non-­‐

living    

ConLident  about  describing  the  

role  of  structure  and  function  within  plants  and  animals    

ConLident  about  recognizing  that  traits  of  

living  things  are  inherited,  acquired  or  learned          

ConLident  explaining  how  the  adaptations  of  species  depends  on  

their  environment      

ConLident  identifying  life  cycles  and  processes  of  growth  and  reproduction    

ConLident  knowing  about  the  abiotic  environment    

ConLident  about  relationships  between  biotic  and  abiotic  

components  of  the  ecosystem        

ConLident  knowing  how  humans  have  changed  

environment,  positively  &  negatively  and  its  effects        

Cohort  1   3.35   3.78   3.72   3.75   3.77   3.49   3.66   3.42   3.82  Cohort  2   3.51   3.85   3.69   3.76   3.65   3.51   3.61   3.35   3.78  Cohort  3   3.85   4.31   4.08   4.15   4.23   4.08   4.31   4.23   4.38  

0.00  

1.00  

2.00  

3.00  

4.00  

5.00  

Science  Course  Knowledge  Domain:  Cohort  Comparison  of  Con>idence  in  Science  Knowledge        

Page 21: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 21

program’s effectiveness in contributing to this outcome grew each year, as is evident in the smaller increases in Cohort 1 findings compared to Cohort 2 and 3 findings.

Teaching Domain: Three-Year Cohort Comparison

*only post scores shown

• There was a significant difference across all participants in pre and post confidence in their ability to teach life science across all cohorts

(p<.000) after participating in the Out-of-the-Box science course.

The  importance  of  integrating  science  into  math,  literacy,  social  studies  and  other  disciplines    

The  importance  of  using  New  York  

Botanical  Garden  and  Bronx  Zoo  

The  importance  of  using  outdoor  sites,  parks,  museums      

ConLident  in  using  outdoor  settings  to  facilitate  students'  science  learning          

ConLident  in  integrating  science  teaching  and  learning  into  other  subject  

areas          Cohort  1   4.29   4.40   4.38   4.03   3.97  Cohort  2   4.77   4.85   4.92   4.23   4.23  Cohort  3   4.15   4.31   4.38   4.54   4.38  

0.00  

1.00  

2.00  

3.00  

4.00  

5.00  

Science  Course  Teaching  Domain:  Cohort  Comparison  of  A7tudes  About  Science  Teaching        

Cohort  1  

Cohort  2  

Cohort  3  

Page 22: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 22

Attitude Domain: Three-Year Cohort Comparison

*only post scores shown

• A comparison of the pre/post outcomes for all application items combined indicates that there was a significant difference in Cohorts 2

and 3 pre and post confidence about using the inquiry practices they had learned as participants in the science course (p<.05). Cohort 1 did not show any significant differences. These results demonstrate that both cohorts 2 and 3 improved their confidence in using inquiry in and out of the classroom.

ConLident  in  identify  the  elements  of  an  effective  science  inquiry  lesson      

ConLident  in  designing  a  

developmentally  appropriate  

science  inquiry  lesson    

ConLident  in  designing  a  

developmentally  appropriate  

science  inquiry  in  connection  with  

nature    

ConLident  in  teach/facilitate  a  developmentally  appropriate  

inquiry  science  lesson    

ConLident  in  addressing  

science  content  standards  in  inquiry  lesson    

ConLident  in  incorporating  the  principles  of  environmental  education  into  inquiry  lesson    

Cohort  1   3.82   3.77   3.78   3.72   3.88   3.77  Cohort  2   4.00   4.00   4.00   4.15   4.23   4.00  Cohort  3   4.15   3.92   4.08   4.00   4.23   4.23  

0.00  

1.00  

2.00  

3.00  

4.00  

5.00  

Science  Course  Application  Domain:  Cohort  Comparison  of  Con>idence  in  Attitudes  About  

Implementing  Science  Instruction      

Cohort  1  

Cohort  2  

Cohort  3  

Page 23: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 23

Science Course Evaluation: Three-Year Cohort Comparison

At the conclusion of the Teaching and Learning Science in Childhood Settings course each year, participants were asked to rate the quality of implementation and the impact of the experience on their knowledge and attitudes about outdoor learning. The extent to which participants reported they were engaged overall in the course was rated highly across all three cohorts.

• In general, the findings show that the program achieved its goals. Participants in Cohorts 1 and 2 believed they increased their knowledge of course content (91 and 93 percent), while Cohort 3 did not rate their increase in knowledge as highly as the previous cohorts (79 percent). All cohorts believed they increased their knowledge of the course content of wildlife/animal conservation and the Bronx Zoo. Cohorts 2 and 3 rated the science course very highly for increasing their knowledge of plants and the natural environment and NYBG. Participants overwhelmingly stated the course increased their awareness of outdoor sites to support teaching and learning,

Overall self-engagement in the

course

Knowledge of the course content

Course increased knowledge of

wildlife/animal conservation and

WCS

Course increased knowledge of plants and the

natural environment and

NYBG

Course increased awareness of

outdoor sites to support teaching

and learning

Overall likelihood you will use or

apply information obtained from this

course in your classroom

Overall quality of course experience

Cohort 1 (N=21) 91% 91% 91% 87% 96% 96% 96%Cohort 2 (N=24) 93% 93% 93% 100% 100% 100% 93%Cohort 3 (N=14) 93% 79% 100% 100% 93% 93% 86%

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

Cohort  Comparison  of  Science  Course  Evaluation  

Page 24: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 24

as well as their aspirations to apply this knowledge when they had their own classrooms. Finally, Cohorts 1 and 2 rated the overall quality of the course as “excellent”, while Cohort 3 believed the course was “very good”.

• Participants’ general impressions about the science course were rated “just right” for all three cohorts, when it came to assumed level of knowledge and the amount of information presented during the course, the time devoted to wildlife and natural environment topics. There were mixed feelings across cohorts when it came to rating the overall length of course. Cohort 1, generally believed the course length was “just right”, while Cohort 2 deemed they would like to see the course be extended to a full-semester course, rather than this condensed version. Finally Cohort 3, believed the course was too long.

Assumed  level  of  your  prior  knowledge  

Amount  of  information  presented  

Amount  of  time  devoted  to  each  topic  or  learning  experience  

Range  of  topics   Overall  length  of  course  

Cohort  1  (N=21)   83%   96%   87%   78%   83%  Cohort  2  (N=24)   80%   93%   93%   100%   73%  Cohort  3  (N=14)   79%   86%   79%   86%   57%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

 Cohort  Comparison:  General  Impressions  of  Science  Course    

Page 25: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 25

• The findings show that the high quality implementation of the science course contributed significantly to achieving the program’s key objectives. Participants across all three cohorts overwhelmingly agreed the science course increased their knowledge of life science and their ability to teach it to elementary students. There were mixed reactions to the course’s facilitation of the ability to integrate life science into other subjects in Cohorts 1 and 3, but Cohort 1 strongly believed the course increased their pedagogical practices and instructional strategies “to a great extent”. Cohort 3 believed this statement to a moderate extent. Both Cohorts 2 and 3 believed the course facilitated participant’s ability to integrate wildlife and the natural environment into their instruction in other subject areas. All cohorts believed the course prepared them to use outdoor settings at WCS and NYBG to do it.

* NOTE: Due to changes in the course evaluation instrument, data was not collected for some items.

I  was  introduced  to  new  or  innovative  

ideas  

The  science  knowledge  gained  from  this  

course  enhanced  my  ability  to  teach  science  to  elementary  children  

The  pedagogical  practices  and  

instructional  strategies  modeled  in  this  course  enhanced  my  ability  to  

teach  science  effectively  to  

elementary  children  

The  course  facilitated  my  ability  to  integrate  

wildlife  and  the  natural  environment  into  my  instruction  in  other  subject  areas  

The  course  prepared  me  to  teach  science  in  other  subject  areas  using  the  Wildlife  

Conservation  Society  and  the  New  York  Botanical  Garden  

Cohort  1  (N=21)   100%   91%   91%   91%  Cohort  2  (N=24)   100%   100%   100%   100%  Cohort  3  (N=14)   100%   93%   79%   93%   100%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

Cohort  Comparison:    Impact  of  Science  Course*  

Page 26: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 26

• The online components of the science course were generally rated highly across Cohorts 2 and 3 because the websites were easy to access and navigate. Participants found the resources to be relevant to their future work as teachers, and contributed to their learning through the organization of the materials. The online components included engaging content; facilitated opportunities to share thoughts and ideas; and clearly defined the course assignments which were posted and submitted online. Cohort 1 did not believe the WCS and NYBG websites were as easy to navigate or access, but did believe the resources facilitated their ability to incorporate information into their own lesson planning. This difference may be due in part to improvements made to both the WCS and NYBG websites over the period of this grant, and the adoption of a new platform (moodle) in Year 2.

The  WCS  and  NYBG  websites  were  clear  and  easy  to  navigate  for  my  purposes  

The  resources  were  easy  to  access  

The  resources  were  relevant  to  my  role  as  a  future  elementary  teachers  

The  resources  facilitated  my  ability  to  incorporate  wildlife  and  the  natural  environment  into  lesson  planning  and  

instruction.    Cohort  1  (N=21)   74%   78%   91%   91%  Cohort  2  (N=24)   93%   93%   100%   100%  Cohort  3  (N=14)   86%   93%   93%   93%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

Cohort  Comparison:  Rating  of  Online  Component  

Page 27: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 27

• In rating the pedagogical elements of the course that contributed most to their learning, Cohort 2 and 3 participants showed consensus in their ratings of the interactive activities and discussions as having the most positive impact on their learning. Cohort 2 further believed pictures and other visuals, lecture presentation, online resources and readings were also factors contributing to their learning, but slightly less so. Cohort 3, in comparison only believed pictures and visuals, assignments, and suggested resources were influential. Cohort 1 was most critical, rating only the interactive activities as influential, while rating all other components as ‘not helpful ‘or ‘useless.’ This finding suggests the program improved the quality of the other instructional strategies in Years 2 and 3. Both Cohorts 1 and 3 believed the reading activities had the least influence on their learning in the science course.

Pictures  and  other  visuals  

Lecture  presentation  

Interactive  Activities  

Suggested  resources   Discussions   Assignments   Readings  

Cohort  1  (N=21)   57%   39%   78%   52%   48%   39%   35%  Cohort  2  (N=24)   87%   87%   100%   80%   100%   93%   87%  Cohort  3  (N=14)   86%   57%   100%   71%   93%   79%   50%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

Cohort  Comparison:    Contributions  of  Components  to  Learning  in  the  Science  Course  

Page 28: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 28

AGGREGATED THREE-YEAR ANALYSIS To assess the performance of the program overall at the end of the grant period, the evaluator prepared an aggregated analysis of the three-year comparison data that were discussed above. The findings for the three-year aggregated analysis are presented below.

N=62

• A paired-sampled t-test was conducted to compare pre and post knowledge assessment scores for the Learning and Teaching Science in Early Childhood Settings course across all cohorts. There was a significant difference in the scores for the pre knowledge assessment (M=59.19) and post-knowledge assessment (M=75.77). The two-tailed P value is less than 0.0001. By conventional criteria, this difference is considered to be extremely statistically significant. This finding suggests the program overall was effective in achieving its objective to increase participants’ knowledge of important life science concepts.

59%  76%  

0%  

20%  

40%  

60%  

80%  

100%  

Learning  and  Teaching  Science  in  Childhood  Settings  Science  Course  

Pre/Post  Knowledge  Assessment:  Aggregated  

Pre_Know   Post_Know  

Page 29: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 29

• Participants’ attitudes about the value of science education changed over the course of the three-year grant period. By the end of the course nearly all participants believed science was important for all children to know. Participant’s attitudes about reading science books for pleasure, interest in learning more about science, and their confidence in their understanding of science nearly doubled. Participant’s personal interest in science increased slightly, but not as dramatically as the other attitudes and beliefs measured.

50%  

19%  

48%   54%   60%  59%  

31%  

76%  90%   84%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

I  have  a  personal  interest  in  science    

I  read  science's  books  and  articles  for  pleasure  and  

learning          

Interest  in  learning  more  science      

ConLident  in  my  understanding  of  

science  

The  importance  for  children  to  know  

science    

Science  Course  Participants'  Personal  Attitudes/Beliefs:    Aggregated  

Pre  (n=48)   Post  (n=49)  

Page 30: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 30

• Participants overall showed extremely significant (p<.0001) gains in confidence in their content knowledge concerning life science and the environment.

My current knowledge of life science is adequate for elementary

students

Confident about

classify objects as

living or non-living

Confident about

describing the role of

structure and function

within plants and animals

Confident about

recognizing that traits of living things

are inherited, acquired or

learned

Confident explaining how the

adaptations of species

depends on their

environment

Confident identifying life cycles

and processes of growth and

reproduction

Confident knowing about the

abiotic environment

Confident about

relationships between biotic and

abiotic components

of the ecosystem

Confident knowing how humans have

changed environment, positively & negatively

and its effects

Pre (n=48) 44% 40% 52% 48% 35% 27% 25% 21% 38% Post (n=49) 84% 78% 80% 88% 84% 82% 92% 82% 84%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

Science Course Participants' Confidence in Content Knowledge: Aggregated

Page 31: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 31

• Participants overall showed extremely significant (p<.00) gains in their beliefs about facilitating student learning. Using the NYBG and Bronx Zoo to generate interest and excitement about science showed the greatest gains. This outcome was a key objective of the program, and achieving it was a key accomplishment.

Use essential

questions to structure science lessons

Use observations

of specimens

or phenomena to stimulate

students

Provide reading

opportunities to develop

students' understandi

ng of science

integrate science into

math, literacy, ss, and other disciplines

use questioning strategies to

elicit students

ideas

use collaborative small group

activities

use the NY botanical

garden and bronx zoo to

generate interest and excitement

about science

engage students in

observations or hands-on investigation

s in the classroom

Pre (n=48) 50% 83% 75% 83% 81% 83% 83% 79% 90% Post (n=49) 86% 92% 82% 98% 92% 98% 98% 92% 98%

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

Science  Course  Pedagogy    

To  facilitate  students'  science  learning  it's  important  to:    Aggregated    

Page 32: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 32

• Participants demonstrated extremely significant (p<.000) increases in their confidence and ability to identify science inquiry, design and teach developmentally appropriate lessons, and integrate the principles of environmental education into science learning and other subject areas. The outcomes indicate that the program was effective in achieving the objectives to instill understanding of inquiry pedagogy and interdisciplinary learning.

identify the

elements of an

effective science inquiry lesson

designing a

developmentally

appropriate science

inquiry lesson

designing a

developmentally

appropriate science inquiry in connectio

n with nature

teach a developm

entally appropriate inquiry science lesson

address science content

standards in inquiry lesson

incorporating the

principles of

environmental

education into

inquiry lesson

intergrate science teaching

and learning

into other subject areas

use outdoor settings

to facilitate students' science learning

Pre (n=48) 58% 67% 63% 60% 65% 65% 58% 69% Post (n=49) 92% 86% 90% 88% 92% 84% 86% 90%

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

Science  Inquiry  Practice    Participants'  con>idence  in  ability  to:  Aggregated  

Page 33: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 33

(N=59)

• In  course  evaluations  administered  at  the  end  of  each  session,  participants  were  highly  positive  about  their  experience  in  the  course.    On  six  of  the  seven  items  measured,  91  to  96  percent  rated  them  positively,  indicated  they  were  very  favorable  (ratings  of  4  or  5)  about  the  overall  quality  of  the  course  experience;  overall  engagement  in  the  course;  the  impact  of  the  course  on  increasing  their  knowledge  of  animal  life  and  plants;  and  the  ability  to  use  outdoor  resources  to  facilitate  science  learning  in  science  and  other  subject  areas.    When  asked  if  they  would  use  their  newly  acquired  learning  when  they  had  their  own  classrooms,  96  percent  of  participants  reported  they  would.  This  findings  supports  the  participants’  committment    to  applying  the  program’s  knowledge  and  skills,  even  though  no  data  was  collected  to  provide  evidence  of  participants’  actual  use.  of  what  they  learned  in  the  course.  

92%   88%   95%   96%   96%   96%   91%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

%  rated  high/very  high  

 Science  Course  Evaluation:  Aggregated  

Page 34: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 34

• Findings from the aggregated course evaluation data show that the majority of participants were satisfied that the amount of information and the range of topics covered in the course, as well as the amount of time devoted to each topic over the three years were “just right”. They also believed the course was directed to a level of prior knowledge that was “just right.” Most participants believed the overall length of module was appropriate but more than a fourth of participants felt it could have been longer or shorter. Both opinions were expressed by participants in the course evaluation data.

80%   91%   86%   88%  71%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

Assumed  level  of  your  prior  knowledge  

Amount  of  information  presented  

Amount  of  time  devoted  to  each  topic  

or  learning  experience  

Range  of  topics   Overall  length  of  course  

%  rated  just  right  

 General  Impressions  of  Science  Course:  Aggregated  

Page 35: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 35

• All cohorts of participants combined were unanimous in reporting that the science course introduced them to new and innovative ideas. Most reported that the course prepared them to teach science and to teach science in other subject areas using WCS and NYBG. These outcomes suggest that the program achieved another of its key objectives.

100%   95%   97%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

I was introduced to new or innovative ideas

The science knowledge gained from this module

enhanced my ability to teach science to elementary

children

The course prepared me to teach science in other subject

areas using the Wildlife Conservation Society and the New York Botanical Garden

%  rated  agree/strongly  agree  

Impact  of  Science    Course:  Aggregated  

Page 36: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 36

• Three cohorts of participants believed the WCS and NYBG websites and online support component were clear and easy to navigate. Further, they deemed the online components to be relevant to their future role as elementary teachers because they provided lesson planning guidance and other resources that could be used in their future classrooms. They also believed the online resources and facilitated their ability to incorporate wildlife and the natural environment into lesson planning and instruction.

84%   88%   95%   95%  

0%  10%  20%  30%  40%  50%  60%  70%  80%  90%  100%  

The  WCS  and  NYBG  websites  were  clear  and  easy  to  navigate  for  my  

purposes  

The  resources  were  easy  to  access  

The  resources  were  relevant  to  my  role  as  a  future  elementary  

teachers  

The  resources  facilitated  my  ability  to  incorporate  wildlife  and  the  natural  environment  into  lesson  planning  and  instruction.    

%  rated  agree/strongly  agree  

Ratings  of  Online  Component:  Aggregated  

Page 37: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 37

• Participants found the interactive activities to be the most effective instructional practices of the course in instilling the desired learning,

followed by the class discussions and educators’ use of pictures and other visuals. DISSEMINATION AND REPLICATION To meet the program’s final objective to disseminate the accomplishments of the Out-of-the-Box program and identify the effective elements of the program for possible replication, project staff and educators made presentations at national conferences throughout the three-year grant period. Presentations were made by both WCS and NYBG program staff in 2011, 2012, and 2013 at conferences of the following organizations: the North American Conference for Environmental Education (NAAEE), American Public Gardens Association (APGA), Association of Zoos and Aquariums (AZA), and the National Science Teachers Association (NSTA). The key findings presented at each conference are summarized below. The major overall theme of the presentations was to provide evidence from the evaluation findings to disseminate the idea that informal science institutions can play a key role in developing inquiry science teaching at the elementary school level.

76%  61%  

93%  

68%  80%  

70%  57%  

0%  

10%  

20%  

30%  

40%  

50%  

60%  

70%  

80%  

90%  

100%  

Pictures  and  other  visuals  

Lecture  presentation  

Interactive  Activities  

Suggested  resources  

Discussions   Assignments   Readings  

Ax%  rated  in>luential/m

ost  in>luential  

Contributions  of  Components  to  Learning  in  the  Science  Course;  Aggregated  

Page 38: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 38

NSTA Presentation Evaluation results after the first year (May 2011 through May 2012), suggest that Informal Science Institutions (ISIs) can play a significant role in promoting science literacy through teacher development. A pre/post comparison of the elementary pre-service teachers’ personal attitudes about science, content knowledge, attitudes toward inquiry, and use of inquiry in/out of the classroom shows that they reported a positive increase in attitudes after the program’s coursework. The surveys demonstrate that students were committed to using their newly acquired knowledge and skills when in their own classrooms -- a testament to the positive impact of this program on introducing science and outdoor settings in the context of other disciplines. Students’ confidence in their ability to implement science inquiry in and out of the classroom improved on all parameters. AZA Presentation The Outside-the-Box program is a demonstration of one way in which a partnership between ISIs and higher education institutions can create a unique opportunity for the preparation of future elementary teachers, with an emphasis on science and nature education. Findings show that the semester-long science course made significant gains in pre-service teachers’ attitudes about science and science education and contributed to increasing their knowledge of life science. The experience also increased their confidence and ability to apply the knowledge and skills acquired in the program. As a result of participating in the Out-of-the-Box Program, Pre-Service Teachers:

• Were better able to use informal science institutions as valuable educational resources • Demonstrated enthusiasm about science and science teaching • Increased their appreciation for the role of wildlife and animals in the environment • Developed skills in pedagogy and science instructional strategies • Demonstrated confidence in teaching science • Were able to incorporate science into other disciplines • Were able to recognize inquiry in science lessons

AGPA Presentation Informal science institutions represent a tremendous resource for teacher education. Science-rich institutions can play a leading role in providing opportunities that support teachers’ professional development and offer experiences beyond what most traditional teacher preparation programs provide. However, few informal science institutions provide programs for pre-service teachers and the effectiveness of these programs has not been extensively documented or broadly disseminated. The Out-of-the-Box program demonstrated one way in which partnership among gardens, zoos and higher education institutions can be formed to support teacher education.

Page 39: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 39

The Out-of-the-Box program model was designed to contribute to the preparation of pre-service elementary teachers in collaboration with their formal graduate studies. Conference participants will be able to replicate elements of the Out-side=the-Box program through partnerships involved APGA institutions and universities. Students: • demonstrate greater confidence in using informal science institutions as valuable educational resources

• demonstrate enthusiasm about science and science teaching • Increasd appreciation for the role of wildlife and animals in the environment • Develop skills in pedagogy and science instructional strategies • Demonstrate confidence in teaching science • Are able to incorporate science into other disciplines

IV. CONCLUSION CONCLUSION The FIPSE Outside-the-Box program completed its third year of implementation in June 2014 with 15 students enrolled in the Lehman College Early Childhood and Elementary Education Program. Year 3 concluded with the third iteration of the Learning and Teaching Science in Childhood Settings science course in May/June 2013. The course continued to be implemented at a high level, incorporating the evaluator’s recommendations from previous years. Project staff and educators from WCS and NYBG delivered a curriculum that targeted the program’s objectives to prepare participants to:

• increase knowledge and develop positive attitudes about life science; • use the New York Botanical Garden and Bronx Zoo, and other outdoor settings to teach science • integrate inquiry and use of outdoor settings to each science in any subject area • utilize inquiry practices (questioning, observing, exploring phenomena) to facilitate science learning in and out of the classroom

Project staff and educators’ decision to modify the science course in Year 2 to focus more intensely on both modeling and explicitly addressing the practice of inquiry in the context of outdoor education, using the WCS and NYBG sites, paid off in Year 3. Findings showed that the program achieved its objectives to increase participants’ knowledge of life science, develop more positive attitudes about the importance of science, develop a greater personal interest in science, build confidence in their life science knowledge and use of inquiry strategies to teach science and incorporate science with other subject areas in outdoors settings.

Page 40: Gaylen Moore Program Evaluation Servicesfscdn.wcs.org/2015/10/05/6oihteahy5_FIPSE_3yr_Report_11...FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 4 SCOPE

FIPSE Out of the Box Program 13-14 Gaylen Moore Program Evaluation Services 40

A three-year cohort analysis of the data demonstrated the program was increasingly more effective with each new cohort in achieving the program’s objectives. A further analysis of the data, aggregating outcomes from all three years of the program provide largely significant evidence that the program was effective in developing participants’ ability to teach science using inquiry practices in outdoor settings. The three-year aggregated findings show that the pre-service elementary teachers:

• demonstrated positive attitudes and knowledge about life science • designed science lessons that utilized outdoor settings to facilitate student learning • demonstrated knowledge and ability to integrate science into other subject areas • increased confidence, comfort, and preparation to apply the skills and knowledge acquired during the program

One of the project’s other accomplishments was building participants’ confidence and a positive commitment to applying their newly acquired content knowledge and practices when they became teachers. Participants reported they planned to integrate life science content into their instruction across subject areas, adopt the practices of inquiry, and use outdoor settings to teach science. The successful components of the Out-of-the-Box program and the program’s overall successes in introducing pre-service teachers to the value of outdoor learning were disseminated by project staff and educators in several national conferences throughout the grant period. Overall, the key contribution of this program to the field was providing evidence that informal science institutions can play a key role in developing inquiry science teaching content and practice at the elementary school level.