THE EFFECTVENESS OF THE TORONTO DISTRICT …edugains.ca/...EffectivenessofBlendedProfessionalLearning_June2013… · Diane Bailey, ... classroom, school, ... TEACHER PROFESSIONAL

THE EFFECTVENESS OF THE TORONTO DISTRICT

SCHOOL BOARD’S TEACHER PROFESSIONAL

LEARNING IN THE 2012-2013 BLENDED

PROFESSIONAL LEARNING COMMUNITY

Organizational Development/Research & Information Services Toronto District School Board

12/13-11

R10(PL_TakingItGlobal\EffectivenessOfTDSBTeacherPL2012-13BlendedPLC)sz.3457

The Effectiveness of the Toronto District School Board’s Teacher Professional Learning in the 2012-2013 Blended Professional Learning Community

Copyright © (July 2013) Toronto District School Board

Reproduction of this document for use in the schools of the Toronto District School Board is encouraged.

For any other purpose, permission must be requested and obtained in writing from: Organizational Development/Research & Information Services Toronto District School Board 1 Civic Centre Court, Lower Level Etobicoke, ON M9C 2B3

Tel.: 416-394-4929 Fax: 416-394-4946 Every reasonable precaution has been taken to trace the owners of copyrighted material and to make due acknowledgement. Any omission will gladly be rectified in future printings.


TABLE OF CONTENTS EXECUTIVE SUMMARY .................................................................................................. Pg. 3 BACKGROUND ................................................................................................................ Pg. 4 LITERATURE REVIEW ON EFFECTIVE TEACHER PROFESSIONAL LEARNING ........ Pg. 5 TDSB’S FRAMEWORK FOR EVALUATING TEACHER PROFESSIONAL LEARNING .......................................................................................... Pg. 7 RESEARCH QUESTIONS AND METHODOLOGY........................................................... Pg. 10 DEMOGRAPHICS AND ACHIEVEMENT OF PARTICIAPTING SCHOOLS ..................... Pg. 12 BACKGROUND OF PARTICIPANTS ............................................................................... Pg. 14 Participants by Focus Area of Inquiry .................................................................... Pg. 14 Gender .................................................................................................................. Pg. 14 Role with the Board ............................................................................................... Pg. 15 Teaching Experience ............................................................................................. Pg. 15 Teaching Qualifications ......................................................................................... Pg. 16 TEACHER PERCEPTIONS ON THE BLENDED PROFESSIONAL LEARNING .............. Pg. 16 New Knowledge and Skills ..................................................................................... Pg. 16 Teaching Practice .................................................................................................. Pg. 19 Attitudes and Beliefs .............................................................................................. Pg. 22 Organizational Supporting Structures .................................................................... Pg. 23 Overall Experience ................................................................................................ Pg. 24 General Comments about the Professional Learning ............................................. Pg. 26 SUMMARY OF FINDINGS ................................................................................................ Pg. 29 REFERENCES .................................................................................................................. Pg. 33


TABLES Table 1: Demographics of Participating WR3 Schools ........................................... Pg. 12 Table 2: Number of Participants by Focus Areas of Inquiry .................................... Pg. 14 Table 3: Teaching Qualifications of Participants .................................................... Pg. 16 FIGURES Figure 1: The Ontario Ministry of Education’s Effective Professional Learning ....... Pg. 6 Figure 2: Desimone’s Core Conceptual Framework for Studying the Effects for Studying the Effects of Professional Development on Teachers and Students .......................................................................... Pg. 7 Figure 3: Logic Model for Evaluating Professional Learning in the TDSB ............... Pg. 9 Figure 4: Percentage of All Grade 3 Students at Levels 3 & 4 on EQAO Assessments ........................................................................................... Pg. 13 Figure 5: Percentage of All Grade 6 Students at Levels 3 & 4 on EQAO Assessments ........................................................................................... Pg. 13 Figure 6: Percentage of Grade 9-10 Students Meeting Expectations on EQAO Assessments ............................................................................... Pg. 13 Figure 7: Percentage of Grade 9-10 Students Meeting Expectations in Credit Accumulation ............................................................................... Pg. 13 Figure 8: Participants’ Roles with the Board........................................................... Pg. 15 Figure 9: Participants’ Teaching Experience .......................................................... Pg. 15 Figure 10: Generic Knowledge/Skills for Classroom Teaching: Percentage of Participants’ with “Expert” or “Competent” Response ....................... Pg. 17 Figure 11: Generic Knowledge/Skills for Inquiry-based Teaching and Learning: Percentage of Participants’ with “Expert” or “Competent” Response .... Pg. 17 Figure 12: Specific Knowledge/Skills for Inquiry-based Teaching and Learning: Percentage of Participants’ with “Expert” or “Competent” Response .... Pg. 18 Figure 13: Generic Practice for Classroom Teachers: Percentage of Participants with “Proficient” or “Routine” Response ................................................ Pg. 20 Figure 14: Specific Practice for Inquiry-based Teaching and Learning: Percentage of Participants with “Proficient” or “Routine” Response ...... Pg. 21


Figure 15: Attitudes/Beliefs about Generic Classroom Teaching and Student Learning: Percentage of Participants with “Strongly Agree” or “Agree” Response ................................................................................. Pg. 22 Figure 16: Attitudes/Beliefs about Inquiry-based Teaching and Learning: Percentage of Participants with “Strongly Agree” or “Agree” Response ................................................................................. Pg. 22 Figure 17: Attitudes/Beliefs in the Three Specific Focus Areas of Inquiry: Percentage with “Strongly Agree” or “Agree” Response ........................ Pg. 23 Figure 18: Organizational Supporting Structures: Percentage of Participants with “Great Extent” or “Moderate Extent” Response .................................... Pg. 23 Figure 19: Overall Experience: Percentage of Participants with “Strongly Agreed” or “Agree” Response ............................................................................ Pg. 25 Figure 20: Overall Summary of the TDSB 2012-13 Blended Professional Learning: Percentage of Participants Meeting Expectations ................. Pg. 30 APPENDICES Appendix A: Professional Learning with E-Technologies 2012-13: Teacher Questionnaire ..................................................................................... Pg. 35 Appendix B: Professional Learning with E-Technologies 2012-13: Teacher Questionnaire ..................................................................................... Pg. 39

This publication contains pages that have been left intentionally blank for proper pagination when printing.

1 | P a g e THE EFFECTIVENESS OF THE TORONTO DISTRICT SCHOOL BOARD’S

TEACHER PROFESSIONAL LEARNING IN THE 2012-2013 BLENDED PROFESSIONAL LEARNING COMMUNITY

ACKNOWLEDGEMENTS

The funding for this 21st Century Pilot for System Learning initiative was provided by the

Curriculum and Assessment Policy Branch, Ministry of Education and the Council of Directors of

Education.

Special thanks to Samuel Zheng, Research Coordinator for this project and author of this report.

This research would have been impossible without the following dedicated people:

TDSB Steering Committee

Gen Ling Chang, Associate Director (A)

Jeff Hainbuch, Superintendent of Education Family of Schools WR3

Peter Paputsis, Principal Northview Heights SS

Diane Bailey, Principal Wilmington

Richard Stein, Principal Flemington

Roula Anastasakos, Executive Superintendent, Organization Development/Research & Information Services

Samuel Zheng, Research Coordinator

Frank Liu, Central Coordinating Principal Teaching & Learning

Marguerite Campbell, Central Coordinating Principal Teaching & Learning

John Tancredi, Central Coordinating Principal Teaching & Learning

Mary Jane McNamara, Central Coordinating Principal Teaching & Learning – Secondary

Tracy Hayhurst, Central Coordinating Principal Teaching & Learning – Elementary

Kien Luu, Central Coordinating Vice-Principal Student Success/Learning to 18

Diane Sharpe, Central Coordinating Vice-Principal Student Success/Learning to 18

Stephan Bibla, Program Coordinator Science & Technology

Vanessa Barnett, Program Coordinator The Arts

Nancy Britton, Program Coordinator Experiential Learning

Rob Mewhinney, Program Coordinator Social & World Studies & Humanities

Alix Harte, Program Coordinator English & Literacy

Ruth Hall, Program Coordinator Library & Learning Resources & Interdisciplinary Studies

Peter Singh, Chief Technology Officer

Kevin Bradbeer, Program Coordinator Teaching & Learning with Technology



Partnership with TakingitGlobal

Michael Furdyk, TakingItGlobal (Founder)

Kate Gatto, TakingIt Global

Sara Hassan, Taking It Global

We would like to express our sincere thanks to our colleagues in the Research and Information

Services department for their professional support in this research project:

Sally Erling, Research Coordinator, for co-developing the TDSB’s Evaluation Framework

for Teacher Professional Learning;

Gerty Chiau, Research and Information Analyst, for cleaning, processing, and reporting

all the survey data;

Karen Kozovski, Research and Information Analyst, for preparing the scannable survey

forms;

Matthew Botts, Research/Data Technician, for scanning the surveys and verifying the

survey data; and

Tammy Denike, Office Manager, for editing and formatting this report.

Finally, we would like to thank all the teachers and students who participated in this research

project and the principals for their support.



EXECUTIVE SUMMARY

Guided by the Toronto District School Board’s (TDSB) generic framework for evaluating the

effectiveness of teacher professional learning, this research study examined the 2012-13

blended professional learning with technologies, and attempted to assess the impact of this

professional learning on the 107 participating teachers and administrators from 18 TDSB

schools and the Board’s Teaching and Learning department. The study employed pre and post

teacher questionnaires to evaluate the effectiveness of the professional learning in the areas of

teacher knowledge/skills, teaching practice, attitudes/beliefs, as well as organizational

supporting structures.

The study found that the vast majority of participants had “expert” or “competent” levels for

generic knowledge and skills for generic classroom teaching but not for inquiry-based teaching

and learning before the initiative. After the professional learning, the proportion of participants

considered themselves as having “expert” or “competent” knowledge/skills for inquiry-based

teaching and learning increased significantly. Teachers and administrators who did not

participate in one particular focus area of inquiry also showed moderate gains in their

knowledge/skills in the subject area which they did not participate, possibly due to the

collaboration and sharing among participants and the presence of online learning environment.

After the professional learning, participants improved the level of proficiency in their teaching

practice for generic classroom teaching and significantly for inquiry-based classroom teaching

and student learning. They had very positive attitudes and beliefs about classroom teaching and

student learning, and their attitudes and beliefs did not change much after the professional

learning as they are already very positive.

Overall, the majority of participants were satisfied with the level of support they received from

their schools or from the TDSB. When comparing their perceptions before and after the

professional learning, the most positive change occurred at teacher collaboration within the

same school and with their colleagues from other schools.

Overall, the majority of participants felt this professional learning activity was a successful

learning experience. They also intended to transfer and apply newly acquired knowledge and

skills to daily practice in the classroom to improve the impact on students.



BACKGROUND

Funded by the Ontario Ministry of Education, the Toronto District School Board’s Developing a

Model of e-Learning and Innovative Pedagogy: Teachers as Designers of Problem–based

Learning with Technology initiative is a professional learning initiative for teachers and school

administrators from 13 elementary and 5 secondary schools in the Board’s West Region 3

(WR3) Family of Schools, and instructional leaders from the Board’s Teaching and Learning

Department. The scope of the project is three fold:

To develop a blended learning environment that consists of synchronous, asynchronous,

and job-embedded learning as the architecture within which educators from three levels

of the system – classroom, school, and district learn together as professional learning

communities engaged in one of three areas of inquiry;

To gain pedagogical understanding of how technology changes teaching practice when

educators use technology as tools to connect students’ inside and outside school

learning as part of challenging and compelling problem-based learning; and

To understand how technology can be strategically used as a key lever in scaling

improvement by intentionally piloting this project with elementary and secondary school

educators in collaboration with internal and external partners. Of note is the question

whether there are differences between elementary and secondary school practices when

connecting technology to instructional strategies and pedagogy.

The initiative was grounded in the belief that learning is the work. What was unique and

innovative about this project is while educators were engaged as a professional learning

community in designing rich problem-based learning for students using technology as the key to

increasing student engagement, they were also at the same time learning to use technology as

tools for learning, thinking, and working among themselves.

In addition to a blended professional learning environment, participating educators chose one of

three areas of inquiry that current thinking in education and pedagogy considers as essential for

learning and teaching in a global context. Through the TDSB’s partnership with an external

partner, TakingITGlobal, who provided instructors online learning platforms, and virtual

classrooms for participants, the three areas of inquiry in this initiative were:



Global Education: The goal is to develop understanding of and competencies in technology-

enabled global education. The participants will explore why global education is so vital currently,

and how to develop authentic and transformative global experiences for students.

Environmental Stewardship: This area of study provides educators with ideas and concepts

for bringing principles of environmental literacy into the classroom. Using online and offline

resources and tools, topics include the critical need for meaningful environmental education,

and how to foster a sense of environmental stewardship for students through a project-based,

practical approach.

Student Voice: This area of study provides participants practical support to transform teacher

and student learning partnerships both inside and outside of the classroom. Through this

course, participants will critically examine a set of agreed principles which underpin the

development of all student voice work and seek to design a project that enables and enhances

student voice in their own schools. It also evaluates the impact of personalized learning meeting

the needs of more students to a fuller extent than ever before and how this can be embedded in

a school’s curriculum to reach effective and sustainable results.

While undertaking inquiry in the previously mentioned areas as a virtual professional learning

community, TDSB participants had opportunities to design and implement problem-based

learning for their students. Technology was an essential element to this leaning, which included

virtual classrooms, online learning sessions, and teachers working as school teams to design

problem-based learning involving students using digital tools and applications. Some teams

were engaged in collaboration between their students and students in another part of the world.

LITERATURE REVIEW ON EFFECTIVE TEACHER PROFESSIONAL LEARNING

The quality of teaching is the largest, single variable in a school’s impact on student learning

(Ontario Ministry of Education, 2007). Among elements such as a well-articulated curriculum

and a safe and orderly environment, the one factor that surfaced as the single most influential

component of an effective school is the individual teachers within that school (Marzano, 2007).

Therefore, it is vital that teachers engage in effective, ongoing professional learning to

continuously develop their practice.



Professional learning in schools has traditionally consisted of activities such as attending

conferences or working on curriculum during teacher workshop days. However, these strategies

have proved to be inadequate in a number of ways (Kelleher, 2003). Over the past decade a

broader view of teacher professional learning has emerged treating teacher learning as

integrative, social, formal, and informal (Desimone, 2011). The range of opportunities reflects

the complexity of the teaching profession and identifies ways that educators remain current in

their practice (Ontario College of Teachers, n.d.). Webster-Wright (2009) explains the

importance of reframing professional development to focus on learning rather than

development. “A focus on learning with a shift of emphasis from passive development to active

learning implies a different conceptualization of knowledge” (Webster-Wright, 2009, p. 713).

In 2007, the Ontario Ministry of Education’s task force, The Working Table on Teacher

Development, recommended the implementation of the five characteristics of teacher

professional learning: coherent, attentive to adult learning styles, goal orientated, sustainable,

and evidence informed. Figure 1 outlines the Ministry’s cycle of considerations for effective

professional learning.

Figure 1: The Ontario Ministry of Education’s Effective Professional Learning

Despite the extensive literature on effective professional learning, there seems to be little

agreement among professional development researchers or practitioners about the criteria for

effectiveness of professional learning. The characteristics that influence the effectiveness of

professional learning are multiple and complex. Therefore, it is not feasible to assume that a

single list of characteristics leading to policies and guidelines for effective professional



development will ever emerge. “Still, by agreeing on the criteria for “effectiveness” and providing

clear descriptions of important contextual elements, we can guarantee sure and steady

progress in our efforts to improve the quality of professional development endeavors” (Guskey,

2003, p. 750). Killion (2008) also agrees that the most useful evaluations result from a desire to

improve both the program and its results. Haslam (2010) argues that although the ultimate goal

of teacher professional development is to improve student learning, the more immediate goal

(as reflected in the outcomes and indicators) is improved teacher knowledge, skills, and

practice.

To provide evidence-based decision making in developing effective teacher professional

learning in the TDSB, an attempt has been made to design a comprehensive and generic

framework and associated research instrument(s) for evaluating professional learning in the

Board. The evaluation framework will focus not only on outcomes of professional learning but

also on each phase of the implementation of teacher professional learning.

TDSB’S FRAMEWORK FOR EVALUATING TEACHER PROFESSIONAL LEARNING

The design of this comprehensive and generic framework for evaluating teacher professional

learning in the TDSB is intended to reflect effective evaluation practices for professional

learning as identified in the literature. It incorporates many of the same elements/critical

features found within the current literature on professional learning evaluations and expands on

a conceptual framework developed by Desimone, Porter, Garet, Yoon, and Birman (2002,

2009) that represents a “theory of change” from professional development (PD) activities to

changes in knowledge, practice, and outcomes (see Figure 2).

Figure 2: Desimone’s Core Conceptual Framework for Studying the

Effects of Professional Development on Teachers and Students

Core Features of

Professional development:

~ Content focus

~ Active learning

~ Coherence

~ Duration

~ Collective participation

Increased teacher

knowledge & skills

Change in attitudes

& beliefs

Change in

instruction

Improved student

learning

Context such as teacher and student characteristics, curriculum, school leadership, policy environment



As described by Killion (2008), a logic model for program evaluation outlines the program

resources or inputs and the actions or strategies used to produce results. A logic model serves

to guide the evaluation design, particularly in the formative stages, because it identifies both the

initial and intermediate outcomes that serve as benchmarks of the program’s progress towards

its goals.

With the consideration of the Board’s context, the logic model for evaluating all phases of the

implementation of teacher professional learning in the TDSB was developed primarily based on

Guskey’s (2000) compelling five-level evaluation process for determining the best fit of a

teacher professional development program for one's local context (Participants' Reactions,

Participants' Learning, Organization Support and Change, Participants' Use of New Knowledge

and Skills, and Student Learning Outcomes), which emerged from Kirkpatrick’s (1994, 2006)

original business training model which incorporated four levels of training outcomes (Reaction,

Learning, Behaviour, and Results). Both models also reference the commonly used KAS or

KASAB descriptions/levels of learning:

K (Knowledge) = factual information, conceptual understanding

A (Attitudes) = beliefs

S (Skills) = strategies

A (Aspirations) = motivation, desire

B (Behaviours) = transfer/application of knowledge

Figure 3 shows the key components of the TDSB’s logic model for evaluating teacher

professional learning (Erling & Zheng, 2011). It should be noted that this logic model has been

designed as a generic model that can be customized to reflect specific professional learning

programs, and that each initiative will likely require its own unique variation of the logic model

(Haslam, 2010).



This generic TDSB logic model itemizes a suggested list of professional learning alternatives

and delivery models, again referencing identified best practices. A final evaluation tool could

provide a similar checklist of multiple options, expanded or customized, to reflect the particular

initiative. From a theory of change perspective, the initial planning process for a professional

learning program would have already identified each of the key components and activities of the

program, and outlined the expected sequence of steps. Each of these steps create a link

between the professional learning event(s) and the intended outcome(s) and effectively design

the roadmap or pathway leading to the expected changes, thus representing an intentional

program of change rather than an isolated learning event (Killion, 2003).

The TDSB’s logic model identifies three stages of evaluation – initial evaluation, formative

evaluation, summative evaluation – occurring at different points in time with associated

indicators (baseline, interim, and final) to be developed in the research tool(s). The assumption

here is that the professional learning program is an ongoing process and that the activities,

indicators, and evaluation measures will vary at each stage.



Where relevant, contextual factors need to be considered and explained when they have an

impact/influence on or interfere with the delivery, implementation, or effectiveness of the

professional learning initiative. These factors might include:

Board policy and priorities (coherence, consistency with board priorities, directions, etc.)

School leadership support and advocacy

Teacher characteristics (teacher profile – education, experience, skills, grade level,

subject area expertise, beliefs, etc.)

Student characteristics (student demographic/performance profiles; disadvantage, etc.)

Other initiatives occurring simultaneously, competing priorities

Specific evaluation measures would be determined based on the logic model and would

eventually include a description of both the data sources (who) and the data collection methods

(how). A variety of measurement tools/options could be considered (e.g., rating checklists, pre

and post assessments, self-reporting surveys, focus groups, classroom observations, etc.) as

appropriate, cost-effective, or feasible for each evaluation, although not all will necessarily be

used.

Erling and Zheng (2011) also developed an associated research instrument (a generic question

item bank) to measure the various indicators in the three stages of the evaluation in the TDSB’s

logic model (initial, formative, and summative evaluations). With adequate inputs of the

professional learning, such as evidence-based needs assessment and well-defined and

research-based content, the effectiveness of teacher professional learning can be assessed by

measuring teachers’ perceptions, new knowledge and skills, changes in teaching practice,

changes in organizational supporting structure, and improved student academic achievement

and school engagement.

RESEARCH QUESTIONS AND METHODOLOGY

Mixed research methods, which include both qualitative and quantitative research, have been

used in this study. Creswell (2003) defines a mixed methods research as a research design with

philosophical assumptions as well as methods of inquiry. As a methodology, it involves

philosophical assumptions that guide the direction of the collection and analysis of data and the

mixture of qualitative and quantitative approaches in many phases in the research process. As a

method, it focuses on collecting, analyzing, and mixing both quantitative and qualitative data in

a single study or series of studies. Its central premise is that the use of quantitative and



qualitative approaches in combination provides a better understanding of research problems

than either approach alone.

Several aspects of this teacher professional learning were investigated in this research using

the mixed research methods: participating teachers’ perception towards their knowledge and

skills, particularly in using technologies and online learning, their teaching practice, as well as

the impact of the support from their school administrators and the Teaching and Learning staff.

The three centred research questions are:

1. How effective is this teacher professional learning? Have teachers changed their

attitudes and beliefs towards teaching? Have teachers gained the new knowledge and

skills required to teach? Have they applied the new knowledge and skills in their

teaching practice?

2. What is the impact of this teacher professional learning on students’ academic

achievement and school engagement, when controlling other factors such as students’

socio-economic status and prior attainment?

3. What is the impact of the support provided by TDSB administrators and Teaching and

Learning staff on participating teachers’ teaching and learning?

To answer these research questions, the study employed pre and post teacher questionnaires

(see Appendix A and B) according to the TDSB’s framework for evaluating teacher professional

learning described in the previous section. The questionnaires used the same items to measure

participants’ perception on their knowledge and skills, teaching practice, attitude and beliefs,

and on the support they have received from their schools and from the TDSB.

As there are three focus areas of inquiry, the teacher questionnaires were purposely designed

to have common questions for all participants, and to have specific questions for each focus

area of inquiry, and all participants were asked to complete the same questionnaires. In this

way, teachers and administrators who did not participate in a particular focus area would serve

as the control group for those who participated. The hypothesis is that after the professional

learning participants involved in one focus area of inquiry will have better outcomes in their

knowledge/skills, teaching practice, and attitudes/beliefs, than those who served as the control

group.



DEMOGRAPHICS AND ACHIEVEMENT OF PARTICIPATING SCHOOLS

The TDSB’s Learning Opportunity Index (LOI) ranks each school based on measures of

external challenges affecting student success. The school with the greatest level of external

challenges is ranked number one and is described as highest on the index. The LOI was

derived from variables such as median family income, percent of families whose income is

below the Low Income Measure (before tax), percent of families receiving social assistance,

adults with low education (without a high school diploma), adults with university degrees or

above, and percent of lone-parent families. These variables were calculated from the 2006

Federal Census and annual income tax data from Revenue Canada and were for families with

school-age children only.

In its 2011 version, the lowest on the index (i.e., with the least external challenges) is 479 for

elementary schools and 109 for secondary schools, as they were ranked separately.

Table 1 shows the 18 participating schools’ demographic characteristics (the overall LOI 2011

rank and actual values for each LOI variables).

Table 1: Demographics of Participating WR3 Schools

LOI 2011

Rank

Adults with

High

Education

(with

university

degree or

above)

Adults with

Low

Education

(without

high

school

diploma)

Single-

Parent

Families

Median

Family

Income

Families

with

Income

below Low

Income

Measure

Families

Receiving

Social

Assistance

Allenby Junior Public School JK-Gr. 6 791 473 71% 2% 19% 184,908$ 11% 4%

Ancaster Public School JK-Gr. 5 102 184 14% 20% 36% 44,170$ 33% 17%

Charles H Best Middle School Gr. 5-8 304 212 39% 11% 29% 41,763$ 42% 23%

Dublin Heights Elementary and Middle School JK-Gr. 8 552 347 44% 8% 26% 47,183$ 32% 15%

Fairbank Middle School JK-Gr. 8 264 105 18% 24% 37% 40,821$ 40% 25%

Faywood Arts-Based Curriculum School JK-Gr. 8 342 374 39% 10% 26% 63,953$ 28% 12%

Flemington Public School JK-Gr. 5 306 7 26% 17% 61% 27,770$ 55% 38%

Glen Park Public School JK-Gr. 6 459 208 35% 13% 37% 47,163$ 36% 20%

Glenview Senior Public School Gr. 7-8 679 465 68% 3% 19% 176,488$ 16% 6%

Ledbury Park Elementary and Middle School JK-Gr. 8 468 440 52% 6% 21% 135,525$ 25% 10%

North Preparatory Junior Public School JK-Gr. 6 205 383 54% 6% 31% 49,865$ 26% 11%

Rockford Public School JK-Gr. 6 564 270 48% 5% 29% 34,618$ 40% 17%

Wilmington Elementary School JK-Gr. 4 299 274 38% 11% 27% 45,545$ 37% 18%

John Polanyi Collegiate Institute Gr. 9-12 568 10 29% 16% 46% 33,813$ 45% 28%

Lawrence Park Collegiate Institute Gr. 9-12 1170 109 64% 4% 20% 169,568$ 21% 10%

Northview Heights Secondary School Gr. 9-12 1391 75 41% 9% 29% 42,386$ 38% 18%

Vaughan Road Academy Gr. 9-12 550 68 35% 15% 33% 51,838$ 35% 19%

William Lyon Mackenzie Collegiate Institute Gr. 9-12 1274 82 39% 11% 27% 53,438$ 34% 17%

School NameGrade

Range

Learning Opportunities Index (LOI) 2011: Rank and Variables

Enrolment

(October

31, 2012)



One can see that the socio-economic status of school communities varies from very low [e.g.,

Flemington PS (ranked 7 among elementary schools), John Polanyi CI (ranked 10 among

secondary schools)] to very high [e.g., Allenby Junior PS (ranked 473 among the 479

elementary schools), Lawrence Park CI (ranked 109 among the 109 secondary schools)], and in

between. Student enrolment (as of October 2012) varies from 102 (Ancaster PS) to 791

students (Allenby Junior PS) for elementary schools, and from 550 (Vaughan Road Academy)

to 1,391 students (Northview Heights SS) for secondary schools. Therefore, it can be concluded

that participants of this professional learning initiative came from a wide range of TDSB schools

with different sizes and socio-economic status.

The Ontario’s standard Grade 3 and 6 Assessments of Reading, Writing, and Mathematics, the

Grade 9 Assessment of Mathematics, and the Ontario Secondary School Literacy Test (OSSLT)

are administered annually by the province’s Education Quality and Accountability Office

(EQAO). For the Grade 3, 6, and 9 assessments, Level 3 is the provincial standard. Passing the

OSSLT (or passing the Ontario Secondary Literacy Course) is one of the Ontario Secondary

School Diploma (OSSD) requirements.

Figures 4 to 6 show the results for all the schools in the West Region 3 from these EQAO

assessments for the past two school years, and Figure 7 shows the Grades 9 to 12 credit

accumulation information for the five secondary schools.



Overall, schools in the West Region 3 performed well on these provincial assessments: they

achieved above the Board or the Province averages in all the measures but one (the 2011-12

Grade 9 Applied Mathematics). It should be noted that the proportion of students taking the

2011-12 Grade 9 Applied-level Mathematics Assessment in this Family of Schools (21%) is

lower than the Board (29%) or the Province (30%). In credit accumulation, the WR3 secondary

school students achieved better than the Board averages in all grades.

BACKGROUNDS OF PARTICIPANTS

Participants of this teacher professional learning were asked about their background information

such as gender, years of teaching experience, positions with the Board, and teaching

qualifications. This section summarizes these background information based on participants’

responses to the survey questions administered to them during the first learning session.

Participants by Focus Area of Inquiry

Table 2 shows the number of participants in the three focus areas of inquiry. In total 107

educators (89 teachers and 18 administrators) participated in the three focus areas of this

teacher professional learning initiative.

Table 2: Number of Participants by Focus Areas of Inquiry

Focus Area of Inquiry School

Teachers School

Administrators Teaching and Learning Staff

Total

Project-based Learning for Global Citizenship 25 2 12 39

Empowering Student Voice in Education 25 7 14 46

Education for Environmental Stewardship 9 2 11 22

Total 59 11 37 107

Gender

Sixty-five percent (65%) of participants are female and 35% are male.



Role with the Board

Educators across the Board are

well-represented: 29% are from

the Board’s Teaching and

Learning Department, 12% are

school administrators, and 59%

are school teachers [11%, 10%,

9% and 24% teach in the Primary,

Junior, Intermediate and Senior

Divisions, and 5% are school

resource teachers (teaching

English Language Learners,

Special Education, and teacher librarians etc.)]

Teaching Experience

Educators from all age groups are

well-represented: 12% have five or

fewer years of teaching experience,

22% have 6-10 years’ teaching

experience, 26% have 11-15 years’

teaching experience, 15% have 16-20

years’ teaching experience, and 25%

have more than 20 years’ teaching

experience.



Teaching Qualifications Table 3: Teaching Qualifications of Participants

As shown in Table 3, the wide

range of participants’ teaching

qualifications indicates that they

represent TDSB teachers well in

terms of teaching qualifications

and subject areas. It should be

noted that teachers can have

multiple teaching qualifications.

TEACHER PERCEPTIONS ON THE BLENDED PROFESSIONAL LEARNING

As mentioned earlier, pre and post teacher questionnaires were administered to

teachers/administrators participating in this professional learning initiative during a three-month

period from March to May 2013. The teacher questionnaires (see Appendix A and B) were

designed according to the TDSB’s Teacher Professional Learning Evaluation Framework and

the associated research instrument (Erling & Zheng, 2011), measuring the following four

aspects of the teacher professional learning:

New Knowledge/Skills

Change in Teaching Practice

Change in Attitudes/Beliefs

Change in Organizational Supporting Structure

The following results are based on the 102 and 75 returned pre and post questionnaires from

participants of this professional learning.

New Knowledge and Skills

Both the pre and post questionnaires had 22 items asking all participants about their knowledge

and skills for teaching and learning. By design, the items can be classified into three categories:

1) Generic Knowledge/Skills for Classroom Teaching (Items 1 to 8)

2) Generic Knowledge/Skills for Inquiry-based Teaching and Learning (Items 9 to 13)

3) Specific Knowledge/Skills for the Three Focus Areas of Inquiry:

Global Education (Items 14, 17, and 20)

Environmental Stewardship (Items 15, 18, and 21)

Student Voice (Items 16, 19, and 22)

Teaching Qualifications Count Percent

Primary 49 48%

Junior 69 68%

Intermediate 77 75%

Senior 54 53%

Special Education 35 34%

Principal's Qualification Program (PQP) 28 27%

Literacy related subjects (Intermediate/Senior) 23 23%

Mathematics (Intermediate/Senior) 15 15%

Science and Technology (Intermediate/Senior) 19 19%

Arts (Intermediate/Senior) 12 12%



Figures 10 to 12 show the pre and post questionnaire results in these three categories.

Not surprising participants’ generic knowledge and skills for classroom teaching did not change

dramatically after the professional learning as this category of knowledge/skills was not the

focus areas of inquiry, and the vast majority already considered themselves as having ‘expert’

or ‘competent’ levels of knowledge/skills before the professional learning (see Figure 10).

1) Knowing how to determine the different learning needs of

students

2) Asking higher-order questions to all students

3) Using a continuum to plan trajectory of lessons

4) Communicating learning goals and/or targets to students prior

to the task

5) Using a variety of assessment practices to plan for effective

instruction

6) Providing activities that identify similarities and differences

7) Using task specific rubrics for assessing students’ learning

goals when appropriate

8) Devoting equitable time to all students in their learning (e.g.,

opportunities to explain and understand concepts)

Figure 10: Generic Knowledge/Skills for Classroom Teaching: % of Participants with “Expert” or

“Competent” Response

100%

98%

97%

88%

96%

91%

97%

94%

97%

92%

93%

89%

93%

92%

97%

95%

Pre Post

9) Applying inquiry-based learning principles to engage students

in creating change

10) Using interdisciplinary approaches to enrich students’

learning experiences

11) Involving students in goal setting and project

design/assessment

12) Util izing technologies to empower students as active learners

and leaders for the 21st Century

13) Cultivating students' leadership skil ls by having student-led

initiatives and direct action opportunities

Figure 11: Generic Knowledge/Skills for Inquiry-based Teaching and Learning: % of Participants with

“Expert” or “Competent” Response

96%87%

89%

75%

83%

67%

88%

71%

89%

75%

Pre Post

Paradisu

Highlight



After this professional learning, 9-17% more teachers considered them having “expert” or

“competent” knowledge/skills for inquiry-based teaching and learning. The most noticeable

difference occurred at “cultivating students' leadership skills by having student-led initiatives and

direct action opportunities” (17% more) and “utilizing technologies to empower students as

active learners and leaders for the 21st Century” (16% more) (see Figure 11).

Significant gains in participants’ knowledge/skills were observed in all three focus areas of

inquiry: in Global Education 28-37% more participants rated themselves as “expert” or

“competent” in the subject area after the learning initiative. The increases in proportion of

participants with “expert” or “competent” levels of knowledge/skills are 13-20% for

Environmental Stewardship and 16-21% for Student Voice (see Figure 12).

Interestingly, teachers/administrators who did not participate in one particular focus area of

inquiry also showed moderate gains in their knowledge and skills in the subject area (they

completed the same pre and post questionnaires). For example, teachers who participated in

the Global Education course gained 0.5 point (from 2.5 to 3.0) in average score (the highest

14) Integrating global educational concepts into the curriculum

17) Enriching students’ collaborative and cross-cultural learning

experiences from global partnerships

20) Creating a culture of global leadership, in which students are

empowered to engage in controversial global issues and create

positive change

15) Understanding the pedagogies that drive environmental

education

18) Knowing how to ensure longevity and recognition of

environmental projects

21) Fitting environmental principles into various disciplines and

subject areas

16) Using student voice in informing your professional

development

19) Knowing how to embed student voice in your

classroom/school

22) Util izing student voice as a pathway for student success to

meet curriculum goals

Figure 12: Specific Knowledge/Skills for Inquiry-based Teaching and Learning: % of Participants with

“Expert” or “Competent” Response

70%

42%

84%

47%

75%

55%

58%

45%

92%

73%

90%

71%

90%

69%

90%

74%

91%

61%

Pre Post

Paradisu

Highlight

Paradisu

Highlight

Paradisu

Highlight

Paradisu

Highlight

Paradisu

Highlight



score is 4 points and the lowest score is 1 point), while teachers who participated in the other

two courses gained 0.3 point (from 2.4 to 2.7) in average score for items related to Global

Education. This phenomenon might be explained that during the three months of learning,

participants from different courses had plenty of opportunities to share their knowledge and

learning experience, especially with the presence of online learning platforms and virtual

classrooms.

Teaching Practice

There are 21 items in the pre and post questionnaires asking participants their levels of teaching

practice. Similarly, these items have been designed to fit into two categories:

1) Generic Teaching Practice (Items 1 to 10); and

2) Specific Teaching Practice on the Three Focus Areas of Inquiry:

Global Education (Items 11, 14, 17, and 19)

Environmental Stewardship (Items 12, 15, and 20)

Student Voice (Items 13, 16, 18, and 21)

Figures 13 and 14 show participants’ self-rating of their levels of teaching practice in these two

categories at the beginning and the end of the professional learning.



After the professional learning, up to 14% more participants indicated that they had “proficient”

or “routine” levels of practice in applying these generic yet proven to be effective teaching

strategies (see Figure 13). The most significant increases occurred at:

Item 9 - Celebrating student achievement throughout the unit (e.g., work displayed, letter

to home): 14% increase (from 74% to 88%)

Item 5 - Providing just in time learning for every student: 13% increase (from 75% to

88%); and

Item 8 - Providing explicit ongoing feedback for students’ progress towards their learning

goals throughout the unit: 10% increase (from 86% to 96%)

1) Planning lessons based on information you have gathered

about students' thinking of the topic

2) Providing opportunities for group learning

3) Having students use concrete materials and models to

represent their thinking

4) Anticipating student responses to questions

5) Providing just in time learning for every student

6) Helping students to identify and correct errors

7) Providing students opportunities to activate prior knowledge

and make connections to new content

8) Providing explicit ongoing feedback for students’ progress

towards their learning goals throughout the unit

9) Celebrating student achievement throughout the unit (e.g., work

displayed, letter to home)

10) Ensuring all homework has a clear purpose, with relevance to

identified learning goals

Figure 13: Generic Practice for Classroom Teaching: % of Participants with “Proficient” or “Routine”

Response

99%

98%

89%

86%

97%

90%

88%

75%

96%

90%

99%

95%

96%

86%

88%

74%

93%

87%

91%

85%

Pre Post



For specific practice for inquiry-based teaching and learning, significant increases in proportion

of participants with “proficient” or “routine” levels of teaching practice was observed in all the

items directly related to the three focus areas of inquiry: 13-27% in Global Education, 9-11% in

Environmental Stewardship, and 23-38% in Student Voice. The most significant improvement

occurred in these areas:

Global Education - Working with colleagues to develop authentic global education

projects (Item 14): 27% increase (from 32% to 59%);

Environmental Stewardship - Developing a sense of environmental literacy in students

(Items 12); Connecting with other teachers to boost the presence of environmental

issues in your school community (Item 15): 11% increase (from 64% to 75%);

Student Voice (Item 21) - Designing student voice projects to meet curriculum goals and

school policies s: 38% increase (from 52% to 90%).

11) Employing the concept of theme-based education to facil itate

global education activities

14) Working with colleagues to develop authentic global

education projects

17) Util izing the strengths of school community for planning

global education projects

19) Planning student-driven global learning and leadership

development projects using all available e-technologies

12) Developing a sense of environmental l iteracy in students

15) Connecting with other teachers to boost the presence of

environmental issues in your school community

20) Developing environmental education competencies using

online and offline tools and platforms

13) Ensuring student voice projects are effective and sustainable

16) Using student voice to create learning partnerships between

students and teachers

18) Using social networking and media to empower student voice

at a local and global level

21) Designing student voice projects to meet curriculum goals

and school policies

Figure 14: Specific Practice for Inquiry-based Teaching and Learning: % of Participants with “Proficient”

or “Routine” Response

59%

32%

53%

32%

47%

34%

75%

64%

75%

64%

50%

41%

83%

60%

83%

48%

55%

31%

90%

52%

75%

55%

Pre Post



Attitudes and Beliefs

In both the pre and post questionnaires participants were asked three types of questions about

their attitudes and beliefs about classroom teaching and student learning:

1) Attitudes/Beliefs about Generic Classroom Teaching and Student Learning (Items 1 to 3)

2) Attitudes/Beliefs about Inquiry-based Teaching and Learning (Items 4 to 6), and

3) Attitudes/Beliefs in the Three Specific Focus Areas of Inquiry in Global Education,

Environmental Stewardship, and Student Voice (Items 7 to 9)

Figures 15 to 17 show the results in these three categories.

Not surprisingly almost all participants (93-100%) had very positive attitudes and beliefs for the

three generic questions about classroom teaching and student learning (see Figure 15).

Almost all participants (97-99%) were confident in their abilities to implement inquiry-based

teaching and learning, but not all of them were confident that they can use technologies to

assist their teaching (pre: 82%; post: 88%) (see Figure 16).

1) I believe all students can learn.

2) I believe that I can provide intervention in a whole class

setting.

3) I can differentiate my instruction for different needs of my

students.

Figure 15: Attitudes/Beliefs about Generic Classroom Teaching and Student Learning: % of Participants

with “Strongly Agree” or “Agree” Response

93%

94%

99%

98%

99%

100%

Pre Post

4) I am confident in my ability to use e-technologies to assist my

teaching.

5) I am confident in my ability to implement student-centered

approaches in my teaching.

6) I am confident in my ability to implement interdisciplinary

studies approaches in my teaching.

Figure 16: Attitudes/Beliefsabout Inquiry-based Teaching and Learning: % of Participants with

“Strongly Agree” or “Agree” Response

99%

97%

99%

97%

88%

82%

Pre Post

Figure 16: Attitudes/Beliefs about Inquiry-based Teaching and Learning: % of Participants with “Strongly Agreed” or “Agreed” Response



The vast majority of participants (91-100%) had confidence in creating a positive school culture

in the three focus areas of inquiry, indicating they are a highly aspirated group of educators in

these aspects (see Figure 17).

Organizational Supporting Structures

In the pre and post questionnaires participants were asked the same set of questions about how

they view their school environment for collaborated learning, and how much supports they have

received from their school administrators and colleagues. The results are shown in Figure 18.

7) I am confident in my ability to help my school create a culture

of global leadership focused on positive global change.


of environmental stewardship.


that facilitates student voice.

Figure 17: Attitudes/Beliefs in the Three Specific Focus Areas of Inquiry: % of Participants with

“Strongly Agree” or “Agree” Response

91%

100%

100%

98%

94%

92%

Pre Post

1) New ideas and improvement efforts are encouraged and

supported by the school administrative team.

2) New ideas and teaching strategies are regularly shared at staff

meetings.

3) Adequate time is provided for teachers to implement new ideas

and teaching strategies.

4) Regular follow-up support is provided for new initiatives.

5) My colleagues collaborate and share a commitment to improve

teaching and student learning.

6) There are opportunities for sharing with colleagues from other

schools.

7) There is a commitment to provide adequate planning,

preparation and self-reflection time for staff.

8) Successes are acknowledged and shared.

Figure 18: Organizational Supporting Structures: % of Participants with “Great Extent” or “Moderate

Extent” Response

58%

61%

55%

53%

49%

55%

84%

67%

46%

37%

52%

45%

66%

72%

88%

89%

Pre Post



Participants’ perceptions on the level of organizational support they received varied widely from

low [There are opportunities for sharing with colleagues from other schools (pre: 37%)] to high

[New ideas and improvement efforts are encouraged and supported by the school

administrative team (pre: 89%; post: 88%)], and in between (see Figure 18).

When comparing their perceptions before and after the professional learning, the most positive

change occurred at teacher collaboration within the same school (Item 5: My colleagues

collaborate and share a commitment to improve teaching and student learning), with 84% of

participants agreeing with the statement after the professional learning. This is a 17% increase

from their opinions at the beginning. Interestingly the next largest perception change (9% more)

came from teacher collaboration from different schools (Item 6: There are opportunities for

sharing with colleagues from other schools), from 37% of participants agreeing in the beginning

to 46% agreeing in the end. This is also the same question that had the lowest score from

participants at the beginning (see Figure 18).

Overall Experience

In the post teacher questionnaire participants were asked 17 questions about their overall

learning experience and satisfaction with this professional learning. Figure 19 shows their

responses to these questions.



The majority (59%) of participants felt this professional learning activity was a successful

learning experience, which was good news for the TDSB; however, the most exciting findings

are that the vast majority of participants planned to apply/integrate the new knowledge/skills

they acquired from this professional learning to their daily teaching practice:

Item 14 - I am able to modify the use of these new knowledge/skills within my own

context to improve the impact on students: 85% strongly agreed or agreed;

1) The objectives and goals for the initiative were clearly

articulated.

2) The objectives for the initiative were based on identified

student learning needs.

3) The content was relevant to our school’s improvement plan and

goals.

4) The content was relevant to my own professional needs.

5) The content was focussed on quality, evidence-based best

practices.

6) The difficulty level of the content was about right for me.

7) The activities were well planned and organized.

8) My overall understanding of the content area was enhanced.

9) I gained new knowledge and skil ls that has direct and practical

value for doing my job.

10) I feel more competent and confident about my abilities in

teaching this topic.

11) I need more information and training about this topic in order

for me to use the strategies effectively.

12) I need more time to explore and reflect on what I learned in

order for me to use the strategies effectively.

13) I intend to transfer and apply newly acquired knowledge and

skil ls to my daily practice in the classroom right away.

14) I am able to modify the use of these new knowledge/skills

within my own context to improve the impact on students.

15) There is a collective commitment among my colleagues to

implement the vision of this professional learning initiative.

16) Overall I feel this professional learning activity was a

successful learning experience for me.

17) I would recommend this activity to my colleagues.

Figure 19: Overall Experience: % of Participants with “Strongly Agree” or “Agree” Response

53%

49%

56%

62%

58%

66%

45%

59%

63%

63%

55%

80%

74%

85%

46%

59%

53%



Item 12 - I need more time to explore and reflect on what I learned in order for me to use

the strategies effectively: 80% strongly agreed or agreed; and

Item 13 - I intend to transfer and apply newly acquired knowledge and skills to my daily

practice in the classroom right away: 74% strongly agreed or agreed

The least agreed item (45% of participants strongly agreed or agreed) is about the

planning/organizing of the event (time between sessions was too short, not given enough time

to finish the course project, etc.).

General Comments about the Professional Learning

Participants were given three opportunities in the post questionnaire to write down their general

comments about what they have liked the most, what their biggest challenges were in this

professional learning, and what suggestions they can provide to the TDSB for future

professional learning. Following is a summary of these comments.

1) What specific elements or activities of this professional learning initiative do you think were

the most useful for you?

In general, teachers and administrators attending the professional learning enjoyed being able

to work together with their colleagues and engage in discussion and share ideas. Another trend

among participants was being able to collaborate on projects. They enjoyed the resources and

articles that were provided and believed that it helped to expand their knowledge and technical

vocabulary on the subject matter. Additionally, many participants appreciated having access to

experts in this field of online learning. They enjoyed having the opportunity to engage in

professional dialogues with their colleagues as well as the Teaching and Learning staff.

Participants enjoyed being able to practice things hands on with the various technologies and

learn new ideas of incorporating global issues into their curriculum and teaching students. They

felt that the use of online communication tools played an integral role while being able to reflect

on their learning process through various reflection papers. They also felt that it was important

to connect with different schools and they enjoyed this aspect of the course.

Paradisu

Highlight



2) What do you think was (or will be) the most significant/beneficial change in your teaching

practices as a result of this professional learning initiative?

Teachers and administrators felt more confidence, access, and awareness in the subject area of

e-learning as their knowledge and skills were increased through attending this professional

learning initiative. They noted that their lessons were given more focus on global issues.

Another common trend among participants was a shift in focus to student voice in the classroom

and understanding the importance of student voice in the classroom. Participants believed that it

is important to build global citizenship skills with students in the classroom through giving more

emphasis to student voice and empathy to students as well as making learning relevant to

students. Some participants felt that their learning and pedagogy was not impacted from the

course due to previous knowledge and research that they had or time constraints in the

classroom which has impacted the implementation of their new knowledge and skills with their

students.

Participants felt that their new knowledge and skills could provide stimulating learning

experiences to students and cater to various learning styles. They understood the important

impact technology has in the classroom and learning environment, and felt that the new

resources they were provided with could increase student engagement and help students reach

realistic outcomes. Participants also felt that they could make their students more aware of

global issues and current events. Additionally, some teachers mentioned being pleased to have

the opportunity to use the Problem Based Learning (PBL) model to develop learning activities

and assessment pieces for their classes.

3) Do you have any suggestions or comments?

Many participants felt that a collaborative effort was missing from their learning experience and

they expressed that they wish they would have had more time and access in collaborating with

their colleagues from other schools. Additionally, some felt that the content was not applicable

to the diverse learning population of the TDSB and the lack of Canadian content did not

promote an informative learning experience.

Paradisu

Highlight



Participants would have liked to have more hands on examples and step-by-step explanation of

the various interfaces, applications, and problem based learning. Some participants did not like

having to miss multiple days from their classrooms and felt that it was redundant to travel to the

course location if they were going to sit in a room and all watch one screen, they felt that this

was not the purpose of an online learning course. Some participants noted a lack of

engagement among participants as there was too much listening and lecturing and not enough

discussion promoted among themselves. Some participants noted that they were not given

enough information about the course prior to registering and felt that they were misled and that

the course did not meet their expectations. There was opposing viewpoints on the course

duration, some felt that the course should have been shortened and condensed while others

believed that it should have taken place throughout the whole year spread out with the ability to

start from their own school or homes.

Participants noted the technical issues (e.g., wireless connections) impacted the learning

environment although they felt the speaker was knowledgeable. They felt that the speaker could

not provide them with information current to Canadian content and the TDSB. Some participants

felt that some of the assignments were repetitive, time consuming, and the workload was a bit

too much at times. They also felt that the feedback to their assignments was not adequate or

constructive. Another common trend among participants was that they felt that the course was a

bit disorganized at times and that aspects of the course should not be changed mid-way through

the course. They hoped that they would have had more time to work on projects during class

time and have more collaboration time outside the classroom. Some mentioned that the pace

was too slow and that the course readings and lectures did not correlate to each other well.

Paradisu

Highlight



SUMMARY OF FINDINGS

Guided by the TDSB’s generic framework for evaluating the effectiveness of teacher

professional learning, this research study examined the 2012-13 blended professional learning

with technologies, and attempted to assess the impact of this professional learning on the 107

participating teachers and administrators from 18 TDSB schools and the Board’s Teaching and

Learning department. The study employed pre and post teacher questionnaires to evaluate the

effectiveness of the professional learning in the areas of teacher knowledge/skills, teaching

practice, attitudes/beliefs, as well as organizational supporting structures.

Figure 20 shows the overall summary of the participants’ perceptions on the professional

learning in these categories and their satisfaction with this professional learning initiative.

The vast majority of participants (93%) indicated that they had “expert” or “competent” levels for

generic knowledge and skills for classroom teaching, and their perceptions did not change

dramatically after the professional learning (96%). However, 14% more participants considered

themselves as having “expert” or “competent” knowledge/skills for inquiry-based teaching and

learning, and 24% more for the three specific focus areas of inquiry (32% more in Global

Education, 17% more in Environmental Stewardship, and 18% more in Student Voice). Overall,

11% more (from 81% to 92%) participants considered themselves as having “expert” or

“competent” knowledge/skills after this professional learning.

Teachers and administrators who did not participate in one particular focus area of inquiry also

showed moderate gains in their knowledge and skills in the subject area (they completed the

same pre and post questionnaires). For example, participants in the Global Education course

gained 0.5 point in average score, while participants in the other two courses gained 0.3 point in

average score for items related to teaching Global Education. This phenomenon might be

explained that during the three months of learning participants from different courses had plenty

of opportunities to share their knowledge and learning experience, especially with the presence

of online learning platforms and virtual classrooms.



After the professional learning, 7% more participants indicated that they had “proficient” or

“routine” levels of practice in generic yet proven to be effective teaching strategies, and 22%

more participants considered themselves as having “proficient” or “routine” levels in inquiry-

based classroom teaching and student learning (20% more in Global Education, 11% in

Environmental Stewardship, and 30% in Student Voice). Overall, 11% more participants (from

75% to 86%) considered themselves as having “proficient” or “routine” levels of teaching

practice after this professional learning.

Generic Knowledge/Skills for Classroom Teaching

Generic Knowledge/Skills for Inquiry-based Teaching and

Learning

Specific Knowledge/Skills for Inquiry-based Teaching and

Learning

Knowledge/Skills: Overall

Generic Practice for Classroom Teaching

Specific Practice for Inquiry-based Teaching and Learning

Teaching Practice: Overall

Attitudes/Beliefs about Generic Classroom Teaching and Student

Learning

Attitudes/Beliefs about Inquiry-based Teaching and Learning

Attitudes/Beliefs in the Three Specific Focus Areas of Inquiry

Attitudes/Beliefs: Overall

Organizational Supporting Structures

Overall Experience

Figure 20: Overall Summary of the TDSB 2012-13 Blended Professional Learning: % of Participants

Meeting Expectations

89%

75%

84%

60%

92%

81%

93%

87%

68%

45%

86%

75%

97%

97%

95%

92%

96%

96%

96%

95%

62%

60%

60%

96%

93%

Pre Post



Almost all participants (97%) had very positive attitudes and beliefs about generic classroom

teaching and student learning, and they had the same attitudes and beliefs in these areas after

the professional learning. They were also confident in their abilities to implement inquiry-based

teaching and learning (pre: 92%; post: 95%). The vast majority of participants (96%) had

confidence in creating a positive school culture in the three focus areas of inquiry, indicating

they are a highly aspirated group of educators in their attitudes and beliefs overall (pre: 95%;

post: 96%).

Overall, the majority of participants (pre: 60%; post: 62%) were satisfied with the level of support

they received from their schools or from the TDSB. When comparing their perceptions before

and after the professional learning, the most positive change occurred at teacher collaboration

within the same school, with 84% of participants agreeing after the professional learning. This is

a 17% increase from their opinions at the beginning. The next largest perception change (9%

more) came from teacher collaboration from different schools, from 37% of participants agreeing

in the beginning to 46% agreeing in the end.

In general, teachers and administrators attending the professional learning enjoyed being able

to work together with their colleagues and the Teaching and Learning staff, engage in

discussion and share ideas. They appreciated the resources and articles that were provided and

believed that it helped to expand their knowledge and technical vocabulary on the subject

matter. Additionally, many participants enjoyed being able to practice things hands on with the

various technologies and learn new ideas of incorporating global issues into their curriculum and

teaching students. They felt that the use of online communication tools played an integral role

while being able to reflect on their learning process through various reflection papers. Overall

the majority of participants (60%) felt this professional learning activity was a successful

learning experience.

For next steps, 74% indicated that they intended to transfer and apply newly acquired

knowledge/skills to daily practice in the classroom right away, 85% indicated that they were able

to modify the use of these new knowledge/skills within their own context to improve the impact

on students, and 80% felt they need more time to explore and reflect on what they had learned

in order for them to use the strategies effectively.



The least agreed item (45% of participants strongly agreed or agreed) is about the

planning/organizing of the event (time between sessions was too short, not given enough time

to finish the course project, etc.).

Although the short-term results listed in this report are positive and encouraging, further studies

are necessary to track the progress of the teachers and school administrators involved in the

professional learning initiative, especially their teaching practice in the three focus areas of

inquiry. A further analysis is recommended on the differentiated effects of participating teachers’

and their students’ background characteristics and school demographics on students’ learning

and school engagement.



REFERENCES

Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed method

approaches, 2nd Edition. Thousand Oaks, CA: Sage Publications.

Desimone, L. M. (2009). Improving impact studies of teachers' professional development:

Toward better conceptualizations and measures. Educational Researcher, 38, 181.

Desimone, L. M. (2011). A primer on effective professional development. Kappan, 92(6), 68-71.

Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K., S., & Birman, B. F. (2002). Effects of

professional development on teachers' instruction: Results from a three-year longitudinal

study. Educational Evaluation and Policy Analysis, 24, 81.

Erling, S., & Zheng, S. (2011). Logic model for evaluating professional learning in the Toronto

District School Board. Toronto, Ontario, Canada: Toronto District School Board.

Erling, S., & Zheng, S. (2011). Assessing effectiveness of professional learning in the TDSB:

Framework and samples for a generic question item bank. Toronto, Ontario, Canada:

Toronto District School Board.

Kelleher, J. (2003). A model for assessment-driven professional learning. Phi Delta Kappan,

June, 751-756.

Killion, J. (2003). Solid footwork makes evaluation of staff development programs a song. JSD

Fall 24(4).

Killion, J. (2008). Assessing impact: Evaluating staff development: 2nd edition. Thousand Oaks,

CA: Corwin.

Kirkpatrick, D. (2006). Evaluating training programs: 3rd edition. San Francisco, CA: Berrett-

Koehler Publishers Inc.

Guskey, T. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin.

Guskey, T. (2003). What makes professional development effective. Phi Delta Kappan, June,

748-750.



Haslam, M. (2010). Teacher professional development evaluation guide. Prepared for the

National Staff Development Council (NSDC).

Marzano, R. (2007). The art and science of teaching. Alexandra, VA: Association for

Supervision and Curriculum Development.

Ontario College of Teachers. (n.d.). The professional learning framework for the teaching

profession. Retrieved from http://www.oct.ca/publications/PDF/framework_e.pdf

Ontario Ministry of Education. (2007). Report to the partnership table on teacher professional

learning: Recommendations of the working table on teacher development. Retrieved

from http://www.edu.gov.on.ca/eng/teacher/pdfs/partnerReport.pdf

Presley, A., Zheng, S., & Erling, S. (2011). Effective professional learning: Review of the

literature (Research Brief). Toronto, Ontario, Canada: Toronto District School Board.

Toronto District School Board. (2011). Mathematics junior intervention initiative (Research

Report). Toronto, Ontario, Canada: Toronto District School Board.

Webster-Wright, A. (2009). Reframing professional development through understanding

authentic professional learning. Review of Educational Research, 79(2), 702-739.

http://www.oct.ca/publications/PDF/framework_e.pdf

http://www.edu.gov.on.ca/eng/teacher/pdfs/partnerReport.pdf

Thank you for participating in this teacher professional learning survey. Your responses will help the Boardunderstand current classroom practices, and identify professional development needs in continuous efforts toimprove student learning. While your participation is voluntary, your assistance is essential for the survey results to beaccurate. This survey is anonymous and confidential. The findings of the survey will not be used to evaluate you oryour school.

Please create your unique identification by using the first letters of your mother’s first name and last name, and thetwo digits of your birth month and day (for example, if your mother’s name is Mary Smith, and you were born on April25, your identification for this study would be ‘MS0425’). This same identification will be used again in a later survey inorder for the Board to link and analyze the responses of both surveys while not identifying you or your school:

Instructions for filling out form: For optimum accuracy,please use a black pen or dark pencil.

A. Your Background Information

0 - 3 4 - 5 6 - 10 11 - 15 16 - 20 More than 20

5) The eLearning course you are taking is:

3) How many years have you been teaching?

4) What are your teaching qualifications (select all that apply):

Primary Junior Intermediate

Senior Special Education PQP

Literacy related subjects (Intermediate/Secondary)

Mathematics (Intermediate/Senior)

Science and Technology (Intermediate/Senior)

Arts (Intermediate/Senior)

Page 1 of 4

Professional Learning with E-Technologies 2012-13: Teacher Questionnaire

1) You are: Male Female

2) Please describe your role with the Board (please choose only one):

Teaching and Learning Staff

School Administrator

School Resource Teacher (ELL / SERT / Teacher Librarian, etc.)

Primary Division Teacher

Junior Division Teacher

Intermediate Division Teacher

Secondary School Teacher: Subject Area

Project-based Learning for Global Citizenship

Education for Environmental Stewardship

Empowering Student Voice in Education

13485

088271

Typewritten Text

088271

Typewritten Text

088271

Typewritten Text

APPENDIX A

088271

Typewritten Text

1) Knowing how to determine the different learning needs of students



4) Communicating learning goals and/or targets to students prior tothe task

5) Using a variety of assessment practices to plan for effectiveinstruction


11) Involving students in goal setting and project design/assessment

12) Utilizing technologies to empower students as active learnersand leaders for the 21st Century

13) Cultivating students' leadership skills by having student-ledinitiatives and direct action opportunities


B. Please describe your current level of knowledge/skills in the following areas:

8) Devoting equitable time to all students in their learning (e.g.,opportunities to explain and understand concepts)

7) Using task specific rubrics for assessing students’ learning goalswhen appropriate

9) Applying inquiry-based learning principles to engage students increating change

10) Using interdisciplinary approaches to enrich students’ learningexperiences

15) Understanding the pedagogies that drive environmental education

16) Using student voice in informing your professional development

17) Enriching students’ collaborative and cross-cultural learningexperiences from global partnerships

19) Knowing how to embed student voice in your classroom/school

20) Creating a culture of global leadership, in which students areempowered to engage in controversial global issues and createpositive change

18) Knowing how to ensure longevity and recognition of environmentalprojects

21) Fitting environmental principles into various disciplines andsubject areas

22) Utilizing student voice as a pathway for student success to meetcurriculum goals

Page 2 of 4

Expert Competent Novice Limited

13485

1) Planning lessons based on information you have gathered aboutstudents' thinking of the topic


3) Having students use concrete materials and models to representtheir thinking




9) Celebrating student achievement throughout the unit (e.g., workdisplayed, letter to home)

10) Ensuring all homework has a clear purpose, with relevance toidentified learning goals

11) Employing the concept of theme-based education to facilitateglobal education activities

C. Please describe your current level of practice in the following areas:

7) Providing students opportunities to activate prior knowledge andmake connections to new content

8) Providing explicit ongoing feedback for students’ progresstowards their learning goals throughout the unit (e.g., writtenfeedback, reflection time)

15) Connecting with other teachers to boost the presence ofenvironmental issues in your school community

12) Developing a sense of environmental literacy in students


14) Working with colleagues to develop authentic global educationprojects

17) Utilizing the strengths of school community for planning globaleducation projects

18) Using social networking and media to empower student voice at alocal and global level

16) Using student voice to create learning partnerships betweenstudents and teachers

Page 3 of 4

19) Planning student-driven global learning and leadershipdevelopment projects using all available e-technologies

20) Developing environmental education competencies using onlineand offline tools and platforms

21) Designing student voice projects to meet curriculum goals andschool policies

Proficient Routine Beginner Non-use

13485


2) I believe that I can provide intervention in a whole class setting.

3) I can differentiate my instruction for different needs of mystudents.

4) I am confident in my ability to use e-technologies to assist myteaching.

5) I am confident in my ability to implement student-centeredapproaches in my teaching.

6) I am confident in my ability to implement interdisciplinary studiesapproaches in my teaching.

7) I am confident in my ability to help my school create a culture ofglobal leadership focused on positive global change.

1) New ideas and improvement efforts are encouraged and supported by the school administrative team.

2) New ideas and teaching strategies are regularly shared at staff meetings.

3) Adequate time is provided for teachers to implement new ideas and teaching strategies.


5) My colleagues collaborate and share a commitment toimprove teaching and student learning.

6) There are opportunities for sharing with colleagues from other schools.

7) There is a commitment to provide adequate planning, preparation and self-reflection time for staff.


D. Please describe how much you agree or disagree with each statement. Choose only one response for each statement.

StronglyAgree Agree Disagree

StronglyDisagree

E. To what extent are the following structures currently in place at your school?

GreatExtent

ModerateExtent

SlightExtent

Not atAll

Suggestions/Comments:

Thank you for your participation!

Page 4 of 4

8) I am confident in my ability to help my school create a culture ofenvironmental stewardship.

9) I am confident in my ability to help my school create a culturethat facilitates student voice.

13485

Thank you for participating again in this teacher professional learning survey. Your responses will help the Boardunderstand current classroom practices, and identify professional development needs in continuous efforts to improvestudent learning. While your participation is voluntary, your assistance is essential for the survey results to be accurate.This survey is anonymous and confidential. The findings of the survey will not be used to evaluate you or your school.

Please enter your unique identification by using the first letters of your mother's first name and last name, and the twodigits of your birth month and day (for example, if your mother’s name is Mary Smith, and you were born on April 25,your identification for this study would be ‘MS0425’). This should be the same unique ID you created in a previoussurvey in March 2013:

Instructions for filling out form: For optimum accuracy,please use a black pen or dark pencil.

Page 1 of 4

Professional Learning with E-Technologies 2012-13: Teacher Questionnaire

The eLearning course you are taking is:

Project-based Learning for Global Citizenship

Education for Environmental Stewardship

Empowering Student Voice in Education

1) Knowing how to determine the different learning needs of students



4) Communicating learning goals and/or targets to students prior tothe task

5) Using a variety of assessment practices to plan for effectiveinstruction


A. Please describe your current level of knowledge/skills in thefollowing areas:

8) Devoting equitable time to all students in their learning (e.g.,opportunities to explain and understand concepts)

7) Using task specific rubrics for assessing students’ learning goalswhen appropriate

9) Applying inquiry-based learning principles to engage students increating change

10) Using interdisciplinary approaches to enrich students’ learningexperiences

11) Involving students in goal setting and project design/assessment

12) Utilizing technologies to empower students as active learnersand leaders for the 21st Century

13) Cultivating students' leadership skills by having student-ledinitiatives and direct action opportunities


15) Understanding the pedagogies that drive environmental education

16) Using student voice in informing your professional development

17) Enriching students’ collaborative and cross-cultural learningexperiences from global partnerships


54561

088271

Typewritten Text

Appendix B

A. Please describe your current level of knowledge/skills in thefollowing areas: (con't)

20) Creating a culture of global leadership, in which students areempowered to engage in controversial global issues and createpositive change

18) Knowing how to ensure longevity and recognition of environmentalprojects

21) Fitting environmental principles into various disciplines andsubject areas

22) Utilizing student voice as a pathway for student success to meetcurriculum goals

Page 2 of 4

1) Planning lessons based on information you have gathered aboutstudents' thinking of the topic


3) Having students use concrete materials and models to representtheir thinking




9) Celebrating student achievement throughout the unit (e.g., workdisplayed, letter to home)

10) Ensuring all homework has a clear purpose, with relevance toidentified learning goals

11) Employing the concept of theme-based education to facilitateglobal education activities

B. Please describe your current level of practice in the followingareas:

7) Providing students opportunities to activate prior knowledge andmake connections to new content

8) Providing explicit ongoing feedback for students’ progresstowards their learning goals throughout the unit (e.g., writtenfeedback, reflection time)

15) Connecting with other teachers to boost the presence ofenvironmental issues in your school community

12) Developing a sense of environmental literacy in students


14) Working with colleagues to develop authentic global educationprojects

17) Utilizing the strengths of school community for planning globaleducation projects

18) Using social networking and media to empower student voice ata local and global level

16) Using student voice to create learning partnerships betweenstudents and teachers

19) Planning student-driven global learning and leadershipdevelopment projects using all available e-technologies

20) Developing environmental education competencies using onlineand offline tools and platforms

21) Designing student voice projects to meet curriculum goals andschool policies

Proficient Routine Beginner Non-use

19) Knowing how to embed student voice in your classroom/school


54561

Page 3 of 4


2) I believe that I can provide intervention in a whole class setting.

3) I can differentiate my instruction for different needs of mystudents.

4) I am confident in my ability to use e-technologies to assist myteaching.

5) I am confident in my ability to implement student-centeredapproaches in my teaching.

6) I am confident in my ability to implement interdisciplinary studiesapproaches in my teaching.

7) I am confident in my ability to help my school create a culture ofglobal leadership focused on positive global change.

1) New ideas and improvement efforts are encouraged and supported by the school administrative team.

2) New ideas and teaching strategies are regularly shared atstaff meetings.

3) Adequate time is provided for teachers to implement new ideas and teaching strategies.


5) My colleagues collaborate and share a commitment toimprove teaching and student learning.

6) There are opportunities for sharing with colleagues from other schools.

7) There is a commitment to provide adequate planning, preparation and self-reflection time for staff.


C. Please describe how much you agree or disagree with eachstatement. Choose only one response for each statement.

D. To what extent are the following structures currently in placeat your school?

8) I am confident in my ability to help my school create a culture ofenvironmental stewardship.

9) I am confident in my ability to help my school create a culturethat facilitates student voice.


StronglyDisagree

1) The objectives and goals for the initiative were clearly articulated.

2) The objectives for the initiative were based on identifiedstudent learning needs.

3) The content was relevant to our school’s improvement planand goals.

4) The content was relevant to my own professional needs.

5) The content was focussed on quality, evidence-based bestpractices.

6) The difficulty level of the content was about right for me.

7) The activities were well planned and organized.

8) My overall understanding of the content area was enhanced.

E. To what degree do the following statements reflect yourexperience with this professional learning Initiative?

GreatExtent

ModerateExtent

SlightExtent

Not atAll

StronglyAgree

Agree DisagreeStronglyDisagree

54561

H. Do you have any suggestions or comments?

Thank you for your participation!

Page 4 of 4

9) I gained new knowledge and skills that has direct and practicalvalue for doing my job.

10) I feel more competent and confident about my abilities inteaching this topic.

11) I need more information and training about this topic in orderfor me to use the strategies effectively.

12) I need more time to explore and reflect on what I learned inorder for me to use the strategies effectively.

14) I am able to modify the use of these new knowledge/skillswithin my own context to improve the impact on students.

E. To what degree do the following statements reflect yourexperience with this professional learning Initiative? (con't)

13) I intend to transfer and apply newly acquired knowledge andskills to my daily practice in the classroom right away.

15) There is a collective commitment among my colleagues toimplement the vision of this professional learning initiative.

16) Overall I feel this professional learning activity was a successfullearninging experience for me.

17) I would recommend this activity to my colleagues.

F. What specific elements or activities of this professional learning initiative do you think were the mostuseful for you?

G. What do you think was (or will be) the most significant/ beneficial change in your teaching practices as aresult of this professional learning initiative?


StronglyDisagree

54561

THE EFFECTVENESS OF THE TORONTO DISTRICT …edugains.ca/...EffectivenessofBlendedProfessionalLearning_June2013… · Diane Bailey, ... classroom, school, ... TEACHER PROFESSIONAL

Documents