Evaluation of the Disciplinary Literacy-Professional ... · Disciplinary Literacy-Professional Learning Community ... Our final evaluation report, to be delivered in Summer ... in

Center for Research on the Context of Teaching Stanford University

Evaluation of the Disciplinary Literacy-Professional Learning

Community (DL-PLC) Initiative In Austin Independent School District

Interim Report

Joan E.Talbert Jane L. David

August 2007

Table of Contents Introduction 1

Context 1 What is DL-PLC? 2 Evaluation Approach 4

Early Implementation and Outcomes 6

IFL Professional Development and Support: Ratings and Impact 6 DL Professional Learning Communities: Early Implementation 11 Measuring Progress in PLC Development 13 Benefits of DL-PCL 15 School and District Context and Support 17

Summary and Implications 21

Signs of Teacher Learning, Improved Instruction, and PLC Development 21 Emergent Issues 22

Appendices Appendix A. Student Demographics, Teacher Characteristics, and TAKS

Results in AISD Appendix B. AISD Survey Instruments and Data

Figures Figure 1. Logic Model of DL-PLC 4 Figure 2. Value Ratings of IFL-DL Professional Development and

Support 7 Figure 3. Ratings of Preparedness to Enact Leadership Practices,

Before and After DL Professional Development 9 Figure 4. Self-ratings of Teaching Skills and Knowledge, Before and

After DL Professional Development 10 Figure 5. Nature and Monthly Frequency of DL Lead Teachers’ Work 11 Figure 6. Focus of PLC Work 13 Figure 7. Professional Learning Community: Survey Indicators 14 Figure 8. Benefits of DL-PLC for Instruction: Lead Teacher Ratings 16 Figure 9. Reform Coherence: How Do High School Redesign (SLCs)

and DL-PLC Work Together in Your School? 19

- 1 -

I. Introduction

At the request of the William and Flora Hewlett Foundation and Austin Independent School District (AISD), Stanford University’s Center for Research on the Context of Teaching (CRC) undertook an 18-month evaluation in January 2007 that focuses on Disciplinary Literacy-Professional Learning Communities (DL-PLC) and its role in AISD’s efforts to strengthen instruction in their high schools. This report, six months after the study’s inception, provides a snapshot of DL-PLC at the end of its first year of implementation.1 Our final evaluation report, to be delivered in Summer 2008, will assess the initiative’s implementation and outcomes after two years. Context The Austin Independent School District (AISD) has launched an ambitious district-wide initiative to redesign its eleven comprehensive high schools to increase completion rates and to ensure that all students are ready for college and careers. To accomplish these goals, the initiative is guided by a vision of rigorous standards-based instruction for all students and a vision of professional learning for all adults in the system: teachers, principals, and central office staff. In every high school, AISD aims to develop teacher learning communities whose work is grounded in deep knowledge of their core subject discipline and who collaborate to ensure that their instruction is both rigorous and engaging to their students. AISD envisions a portfolio of high school redesign efforts that balances individual school autonomy and district direction. During 2005-06 every high school went through an iterative district-led Request for Design process which specified certain elements to be included in all the design plans and provided each school the flexibility to create programs and structures appropriate to its particular context. The district prescribed three structures: an advisory, small learning communities (SLCs), and professional learning communities (PLCs).2 The way the day is scheduled, how teachers are organized into SLCs and PLCs, choices about specific programs, and approaches to curriculum and instruction are decided by each school within the bounds of district guidelines and the requirements of the instructional design selected. For example, some schools chose to implement First Things First (FTF) which requires a block schedule and use of a specific literacy program. Others opted to participate in DL-PLC which provides training and support for teachers in four subject areas to design and implement rigorous lessons and, ultimately, to increase the rigor of both their lessons and the quality of teaching and learning in their school. 1 The 2006-07 school year was the first year of implementation of DL-PLC in six schools. However, some central office staff and lead teachers had participated in DL training in Pittsburg, the Institute for Learning’s home base, during the prior year which was considered a “set-up” year. 2 In AISD’s high school redesign initiative, an advisory is a group of students that meets regularly with one teacher for personal, academic, and post-secondary guidance; an SLC (small learning community) is a thematic educational program that involves teachers across disciplines in providing personalized instruction to a few hundred students; and a PLC (professional learning community) is a group of teachers who share a discipline or course or who share students in an SLC who collaborate to strengthen teaching and learning.

- 2 -

At this time, five of the eleven comprehensive high schools are implementing DL-PLC3, three are implementing FTF, and three are using models of their own design. Notably, the five comprehensive high schools involved in DL-PLC are not among the four district schools with the highest poverty and lowest performance (see Appendix A for demographic data for DL-PLC schools and other district high schools).

In addition, all regular high schools participate in AP Strategies which provides professional development and, for a subset of schools, incentives for teachers and students to expand the number of 9th and 10th grade students who will be ready to take Advanced Placement courses and exams. Following two years of planning, the 2006-07 school year marked each high school’s first steps in implementing their redesign plans. What is DL-PLC? DL-PLC is not a program in the usual sense; it does not specify a particular curriculum or instructional routines to be implemented. Rather, it promotes the use of a set of principles for teaching and learning – including the Principles of Learning, as well as principles for lesson design and instruction – that engender rigor and engagement for students.4 DL-PLC provides lead teachers with training in how to plan rigorous lessons in each of four disciplines and how to teach the lessons in ways that embody the rigor.5 In turn, the lead teachers6 work with their subject area colleagues in PLCs to understand what rigor means both in lessons and in instruction. The ultimate goal is for teachers to create rigorous lessons that encourage students to think and apply knowledge and to instruct in ways that elicit these behaviors in students. DL-PLC promotes a process of continuous improvement in which PLCs use the study of student work and structured observations in each others’ classrooms as evidence to inform lesson design and improve instruction. Developed by the Institute for Learning (IFL) at the University of Pittsburgh, DL-PLC builds on a seven-year partnership between AISD and IFL. Throughout this period, IFL provided professional development and tools around their Principles of Learning. Over time, the language of these principles took hold, particularly around the ideas of 3 These are Akins, Anderson, Austin, Crockett, and McCallum. International, an alternative high school serving English Language Learners, also is implementing DL-PLC. 4 The concept of rigor encompasses both the cognitive demand of the task and the instructional approaches that elicit student reasoning and application of knowledge, such as asking thought-provoking questions and having students discuss problems or issues with each other and present to the class. The core content knowledge and specific teaching strategies vary across content areas; for example, in Language Arts, rigor includes reading with a purpose, time and routines for students to talk to each other about what they are learning, and opportunities to write. 5 Research indicates that high-quality curriculum materials are not sufficient to ensure rigorous instruction, because teachers who do not understand the discipline content well unintentionally weaken the rigor of the lesson. Therefore, DL-PLC invests in training teachers to use the materials in ways that create rigor for students. 6 DL-PLC’s Lead Teachers are also referred to as Teacher Leaders or Coaches.

- 3 -

effort-based learning and accountable talk. As the superintendent described the partnership:

We teamed with the Institute for Learning at the University of Pittsburgh to develop a professional development program for all our administrators and teachers so that we would have a common language of teaching and learning. In this way, we became an effort-based, standards-based school system. We believe that effort creates ability. You become smarter by working harder in a system set up for high achievement. This is our responsibility as a public school system.7

DL-PLC benefits from much of this history.8 In particular, district teachers and administrators are familiar with the language of Principles of Learning. They understand key concepts such as “academic rigor”--the idea that instruction includes core content knowledge, reasoning, and active use of knowledge; and “accountable talk”--the expectation that students will discuss ideas in ways that further their understanding. The process by which teachers are expected to increase the rigor of their lessons and their instruction – DL-PLC’s logic model or theory of action – begins with intensive training of a cadre of lead teachers representing mathematics, English, science, and social studies in the participating high schools. Lead teachers, as well as school principals, and central office curriculum staff and leaders participate in the training sessions. Over four sessions, the content of the professional development includes the characteristics of disciplinary literacy and the principles for designing powerful lessons and instruction in each core subject. Lead teachers also learn ways of analyzing student work, instructional materials, lesson plans as well as videos of teaching and professional articles. Leading Learning Walks and facilitating PLCs are also central elements of the training. Participants learn the content through engaging as learners in the lesson exemplars, which facilitates their reflection, understanding, and ownership of the ideas. IFL support then moves to the school site to help lead teachers carry out their new roles.9 Figure 1 on the next page depicts this model. At each school site, lead teachers with the support of the principal are expected to implement more rigorous teaching practices in their own classrooms and, at the same time, to help their colleagues learn what they have just learned from IFL. Initially the expectation is that teachers will meet in subject area PLCs, facilitated by the lead teacher, to learn the features of rigorous lessons in a particular content area and to implement model lessons or begin to develop their own.. Their learning will be enhanced by structured observations of the lead teacher and/or peers deemed to exemplify strong instruction.

7 Dr. Pat Forgione, AERA 2006 8 Not all was positive. The original Learning Walks brought in by IFL were not received well. They have since been significantly redesigned. 9 This is the core design. Some schools have chosen to purchase additional IFL services to assist in implementing DL practices.

- 4 -

District staff participating in the IFL professional development are then in a position to support the work of principals and lead teachers and, through feedback from them, adjust district policies to support the work. Together these efforts are intended to build professional communities in core content areas who continually develop knowledge and skills to improve instructional rigor and, as a result, improve learning outcomes for all students. Evaluation Approach The first phase of our evaluation focuses on the shaded boxes of the logic model. Six months into the initiative, IFL and AISD realistically expect to see that lead teachers who participated in DL-PLC training are able to implement more rigorous and engaging

Figure 1. Logic Model of DL-PLC

DL-PLC PD and support - Lead Teachers - Principals - District Staff

School PLCs collaborate on rigorous standards-based lessons and instruction in core content areas

Lead Teacher and principal knowledge and skill to lead DL-PLC

System leader knowledge and skill to support DL-PLC work in schools

System leadership for curriculum alignment, program coherence, and policies in support of DL-PLC

Improved instructional rigor in core content areas for all students

Improved learning outcomes in core subjects for all students

School leadership to develop PLCs focused on DL principles and lessons

- 5 -

lessons and instruction in their own classrooms and work with their colleagues to introduce them to these ideas through meetings, at least once a month, and through classroom observations.

To track progress, we looked for evidence that the DL-PLC design was being implemented as intended in the schools. We interviewed central office leaders, principals in four DL schools, and lead teachers in four DL schools. We also administered an online survey of all lead teachers in DL-PLC schools and of department chairs in the four core content areas in all other AISD regular high schools.10 The interviews and surveys were designed to elicit evidence about:

• the quality of IFL professional development and support • the impact of training on lead teachers’ leadership and instructional knowledge

and skills • implementation of lead teachers’ roles in developing PLCs, supporting

colleagues’ learning, and guiding instructional improvement • PLC practices to improve instruction and initial evidence of progress • school and district leadership and support

The survey was also designed to gather baseline data (as of the end of the 2006-07 school year) on lead teachers’ perceptions of instructional practices and student practices across all classrooms in their content area.

10 Survey data for DL-PLC schools are based on responses from 41 Lead Teachers in the 6 DL-PLC schools (including 7 teachers from International). Responses ranged from 5-13 teachers across these schools, with an overall response rate of 80 percent.

- 6 -

II. Early Implementation and Outcomes

Following the logic model in Figure 1, we begin by describing findings on the quality and impact of the professional development and support that IFL provides. We then turn to early implementation of PLCs, which includes findings on what lead teachers do and the structure and activities of PLCs. The next section reports on reported benefits of DL-PLCs for instruction. The last section looks at school and district supports for DL-PLC and how it fits into the district’s broader high school redesign initiative. IFL Professional Development and Support: Ratings and Impact Lead teachers are positive about DL training and support Teacher leaders give high ratings to DL training sessions in the district. Lead teachers participated regularly in the training sessions on DL-PLC that IFL provided. On average, roughly 60 teachers and their principals from the current six DL-PLC schools11 attended each of four district training session. They were joined by approximately 15 central office staff on average. Of the 41 lead teachers we surveyed, 86 percent attended at least three of the four multi-day sessions on DL-PLC from July 2006 to January 2007. In the survey and in interviews teachers praised the IFL-run training sessions, in terms of both quality and value. Roughly three-quarters of the lead teachers gave these training sessions value ratings of 4 or 5 on a 5-point scale. See Figure 2. Respondents pointed out in interviews that the DL training deepened their understanding of the Principals of Learning [POLs], which they had learned in theory through prior AISD professional development. One lead teacher said: “POL came from a directive. Never really showed how it worked in the classroom. Now we have examples of how you can use instructional techniques with accountable talk. But when it was introduced, there was no reason to do it.” Similarly, a principal noted: “The DL model lessons have really helped the teachers see exactly what the POLs look like.” Lead teachers had differential access to on-site supports from IFL consultants. Although most DL lead teachers reported receiving support in the form of co-facilitation and debriefing of PLC meetings (76%), only about half of them were involved in IFL-led classroom observations or demonstration lessons (see percentages in left column of Figure 2). Differences in the prevalence of particular forms of professional development reflect consultants’ and lead teachers’ preferences, as well as differences in the developmental stage of PLCs within and across schools.

The intensity of lead teachers’ involvement in the range of professional development and support provided by IFL varied somewhat by school. Overall, 43% of the lead teachers indicated that they were involved in at least 4 of the 5 kinds of activities shown in Figure 2, while one-fourth participated in 2 or fewer. However, in one school

11 Two other schools participated in DL-PLC initially but decided not to continue.

- 7 -

all lead teachers were involved in 4 or 5 of the activities while in another their involvement ranged widely from 0 to 5 activities. In the four other DL-PLC schools, most lead teachers reported doing 2-3 of the activities.

Figure 2. Value Ratings and Experience of IFL DL

Professional Development and Support

0 20 40 60 80 100

Training sessions on DL-PLC inthe district (97.4%)

Co-facilitation and/or debriefingof PLC meeting (76.3%)

IFL-led classroom observation inmy classroom (55.3%)

Demonstration or co-taughtlessons (51.3%)

IFL-led classroom observation inanother teacher’s classroom

(48.7%)

Percent Value RatingsModerately valuable Quite valuable Extremely valuable

Note: The numbers in paratheses are percent respondents who had received the training or support. Value ratings point to areas for strengthening IFL supports. The strongest ratings across types of IFL supports went to demonstration or co-taught lessons (79% rated 4 or 5), yet only about half the leaders received this support. This kind of on-site support provides lead teachers and their colleagues with concrete examples of how key facets of DL lessons are enacted, e.g., how guiding questions involve students in discussing core concepts. The data suggest that teachers’ access to such opportunities to see DL instruction in action may be important to their understandings of, and ability to enact, the underlying principles.

IFL-led classroom observations received highly mixed ratings from the lead teachers: half found observations in their classroom to be “extremely valuable” (‘5’ rating); yet another 28% rated this form of support as only somewhat or not at all valuable (‘1’ or ‘2’ rating). Reasons for these divergent experiences might include teacher readiness differences, consultant skill differences, or school context conditions. District and IFL leaders may want to investigate the sources in order to refine this facet of the DL-PLC support for improved teaching.

- 8 -

Lead teachers value many DL-PLC tools. In an open-ended survey question inviting comments on the DL-PLC tools that are “most valuable in your work as a Teacher Leaders”, those mentioned most frequently were: the Studying Student Work protocol and observation protocols (9 teachers mentioned each). Others frequently mentioned were: The Architecture, “Thinking through a Lesson”, model lessons, Revisiting Texts, and DL Charts, each rated as most valuable by at least 3 lead teachers. In our interviews, lead teachers were quite knowledgeable and articulate about the content of DL work and the tools to support it. Not only were they able to call up particularly valuable tools in the survey, but several teachers described in interviews the ways in which they had used or adapted DL lessons and tools. Lead teachers strengthened their leadership and instructional knowledge and skills

The DL lead teacher survey asked individuals to rate how well prepared they were before and after DL training to do two kinds of things required by their new role: a) lead colleagues in work toward DL instruction, and b) implement DL instruction in their own classroom. While these ratings may be susceptible to inflation in individuals’ sense of change, they track with individuals’ comments about their learning during interviews. For example, one lead teacher said: “It [DL-PLC] has made me a better teacher and instructional leader.” This teacher went on to describe how the use of primary and secondary documents increased the rigor of instruction, providing opportunities to ask thought-provoking questions. She also credited DL’s “overarching question” and graphic organizers as strategies for organizing her lessons and helping students see the big picture. Lead teachers gain confidence in their instructional leadership skills. Data summarized in Figure 3 show changes in teachers’ self-ratings on particular leadership skills and point to areas where they currently rate themselves strongest and weakest. The self-ratings show substantial growth in each facet of instructional leadership. Leadership roles for which teachers felt least prepared at the beginning of the year – facilitating instructionally-focused group meetings and conferencing with individuals – showed substantial gains (from 18% to 63% well prepared and from 18% to 50% well prepared, respectively). However, even with these gains, teachers still feel less-well prepared for these two roles relative to the others. Lead teachers currently feel most confident in roles of planning instruction with colleagues (84% rate themselves as well-prepared) and working with colleagues to develop more rigorous questions for class instruction (84%).

These data suggest that DL training is preparing teachers to take the kind of

leadership roles with their colleagues that have traditionally been most difficult –those that focus on classroom instruction and challenge teacher autonomy. Lead teachers’ ability to enact and refine these leadership skills varies according to the resources for

- 9 -

PLC development. Some lead teachers reported having insufficient time to work with colleagues, especially in departments where the chair was not involved in DL professional development and did not support collaboration on instruction. In other words, lead teachers’ use of skills developed through DL training depends upon the extent to which their department and the school context supports their work, as described later.

Figure 3. Ratings of Preparedness to Enact Leadership Practices, Before and After DL Professional Development

0 20 40 60 80 100

Plan instruction together

Develop more rigorous questionsfor class discussion

Examine instructional practices tomake sure they promote academic

rigor and student learning

Look closely at student work

Facilitate instructionally focusedgroup meetings

Conference with individuals

Percent Well and Very Well PreparedBefore After Lead teachers report major gains in their instructional knowledge and skills. Similar magnitudes of change are shown for teachers’ survey reports on their classroom teaching skills and knowledge before and after their DL professional development (see Figure 4). Teachers’ self-described competence grew especially in the particular kinds of classroom practices that DL training promotes – skills in using DL lessons and applying DL principles to classroom instruction. DL teachers also reported growth of skills in engaging students in classroom discussion, challenging students to think deeply about content, developing or selecting rigorous lessons.

Teachers’ comments reflect their growing confidence in providing rigorous instruction. One teacher wrote on the survey: “DL has made me a better teacher, and

- 10 -

though I am eligible to retire, I want to continue working on DL lessons and strategies with my students. The results have been wonderful.” Another teacher said in our interview: “IFL has helped me incredibly... [especially in] questioning skills. I just wish I knew as much as the IFL people.”

Figure 4. Self-ratings of Teaching Skills and Knowledge, Before and After DL Professional Development

0 20 40 60 80 100

Organize students to work in pairsor groups

Challenge students to think deeplyabout content

Develop or select my own rigorouslessons

Engage students in classroomdiscussion

Link lessons to assessments

Apply DL principles to classroominstruction

Teach DL lessons

Percent Well and Very Well PreparedBefore After Lead teachers implement and develop their own DL lessons. The level of implementation of DL instruction among the lead teachers appears to be quite high. On average, DL teacher leaders reported teaching four DL lessons in their classroom this year, and 83% said that they had developed other lessons using DL ideas. Interviews with DL lead teachers validate their use of DL principles and model lessons to craft new lessons for their students. One lead teacher described IFL folks observing her class and how much she learned from their feedback. “I created the lesson using the DL strategies. I realized I needed more guiding questions.” Her comments reflect many lead teachers’ experience of continually working to improve their instruction by using the DL lessons and refining their implementation through feedback from IFL consultants, colleagues, and/or students. Although we know that in takes time for teachers to shift their practice to high-quality DL instruction, evidence shows that most of the lead teachers are making strides in this direction.

- 11 -

DL Professional Learning Communities: Early Implementation Professional growth among the DL lead teachers is just part of the theory of action for DL/PLC school change. These teachers are expected to lead the content colleagues in their school toward DL teaching and learning, together advancing their individual and collective knowledge of DL lessons and principles and their skills in implementing them in their classrooms. When asked how well the DL-PLC program prepared them for the role of lead teacher, overall, their responses were quite diverse – with 15% saying “extremely well” and 28% “quite well” to 18% “somewhat or not at all well”. These differences reflect not only differences in the intensity of their professional development experiences, as described earlier, but also differences in the challenges they faced in implementing the role in their particular school and department contexts. Lead teachers fulfill their key responsibilities and focus on DL lessons In addition to implementing DL lessons and principles in their own classrooms, lead teachers are expected to lead PLC meetings with their subject area colleagues, confer with individual teachers, observe or be observed by other teachers, as well as plan DL-PLC work in their school. Lead teachers report that they engage in all of these activities; in fact, more than two-thirds do each activity at least once a month (see Figure 5). The most common are conferring with teachers and observing classrooms; over 80 percent report doing these at least once a month and most at least twice a month.

Figure 5. Nature and Monthly Frequency of DL Lead Teachers’ Work

0 20 40 60 80 100

Confer with teachers in mycontent area

Observe other teachers

Lead study groups

Work with principal and/orcore DL team coach to plan

DL-PLC work

Am observed by otherteachers

% RespondentsOnce per monthTwice or more per month

- 12 -

Across all six DL schools, PLCs are meeting and doing DL-PLC work together. Three-quarters of the lead teachers reported that their PLCs have a regular time committed to meet. Eighty-four percent said that their PLCs meet at least monthly, and one-quarter reported meeting weekly. These figures vary between schools and between departments within schools. PLCs vary in structure and DL activities across schools and departments Teachers are attending PLC meetings and focusing on DL lessons. On average across departments and schools, two-thirds of the teachers regularly attend DL-PLC meetings. School averages range from 37 to 84 percent. Within a school, rates of regular attendance across departments vary as much as 20 to 92 percent. What constitutes the DL-PLC also differs across departments and schools. Some are entire departments; others are course groups within departments, or combinations of both. As a lead teacher described: “We consider our department our PLC but it is in the course groups where the lesson planning and discussion happens.” However they are structured, PLCs provide a forum in which teachers discuss their work, most frequently through discussing and developing DL lessons. One teacher described a two-hour department meeting in which the lead teacher led the group through a DL lesson. They then split into course groups, developed lessons, and provided feedback to each other. Another described how one teacher brought a lesson she had used for several years to work with colleagues to make it higher level and more rigorous. As shown in Figure 6, lead teachers report that the major focus of their work with their PLCs has been on DL lessons and units – discussing their features (68%) and selecting or developing them (45%). On average across schools, lead teachers report that slightly over a third of the content colleagues in their school have taught at least one DL lesson this year (roughly 5 of 15 teachers on average). However, this average masks a broad range, from a 17 percent average across departments in one school to 84 percent in another school. Similarly, within schools, the percent of teachers in a department who have taught at least one DL lesson ranges as much as from 12 to 100 percent. PLCs also focus on DL tools, both those that are content-specific and those that cut across subjects. The fact that various designs for DL-PLC work are being implemented does not imply, however, that the teacher groups have become strong professional learning communities that collaborate to improve instruction in their classrooms. These tools and activities may support PLC development, as intended, but the nature and quality of their use depends considerably upon the group’s prior experience with collaboration. A key question for the evaluation is the extent to which, how, and under what conditions schools’ participation in DL-PLC results in the development of professional learning communities that improve teaching and learning in the core subjects.

- 13 -

Figure 6. Focus of PLC Work

0 20 40 60 80 100

Discussing features of DL lessonsand/or units

Developing or selecting DLlessons and/or units

Learning to use content-specificDL tools (e.g., ELA Design

Features)

Examining studenttasks/assignments

Learning to use the DLObservation Protocol

Learning to use the Protocol forStudying Student Work

Organizing Learning Walks

% RespondentsModerate focus Major focus

Measuring Progress in PLC Development Leading indicators capture PLC development An initial reading of PLC development across all schools during the first year comes from a survey question asking lead teachers to indicate proportions of teachers in their content area who collaborated in various ways. Leading indicators of PLC development are shown in Figure 7. Each refers to a particular kind of teacher collaboration that would be expected in a mature professional learning community and that should contribute to their ability to learn and implement DL instruction to improve student learning. They are “leading indicators” in the sense that they help to track a teacher community’s move toward inclusive and effective collaboration on instructional improvement. A mature PLC would involve most or all teachers in the full range of collaborative work tapped by this set of indicators. Taken together the indicators could be used to define stages of PLC development over time and, for this evaluation, to track the implementation path for DL-PLCs. Ultimately, we want to assess the extent to which and how this initiative brought about significant shifts in the character and effectiveness of content PLCs.

- 14 -

Figure 7. Professional Learning Community: Survey Indicators

0 20 40 60 80 100

Share ideas on teaching

Discuss what you/they learned at a workshop or conference

Explore new teaching approaches to increase studentengagement and understanding

Discuss beliefs about teaching and learning

Share and discuss student work

Plan lessons and units together

Discuss particular lessons that were not very successful

Share and discuss research on effective teaching methods

Share and discuss research on effective instructional practicesfor English language learners

Observe each other’s classroom instruction

% RespondentsMost teachers All teachers Lead teachers’ overall responses to these survey items reveal the most and least

common kinds of interactions among teachers across the DL-PLC schools after their first year in the initiative. The most prevalent collaborative practices are sharing ideas on teaching (78% reporting that most or all of their content colleagues do this), discussing what was learned in professional development (66%), exploring new teaching approaches to increase student engagement and understanding (63%), and discussing beliefs about teaching and learning (59%). Notable among the low-prevalence forms of teacher collaboration is sharing and discussing research on effective instructional practices for English language learners (34% reporting that most or all of their colleagues do this).12 In general, sharing and discussing research on teaching is not prevalent in these teacher communities (39%); we might expect this form of ‘evidence-based practice’ to increase over time in the PLCs.13 12 Percent LEP students in DL-PLC schools in 2005-06 ranged from 2.9% in Anderson to 9.5% in Akins. 13 Surveys of teachers in 8 high schools participating in the Bay Area School Reform Collaborative (BASRC) during 2002-2005 provide some basis for comparison. Compared to BASRC teachers in the four core subjects, the DL-PLC teacher leaders report substantially higher rates of teacher involvement in discussing what was learned at a workshop or conference (66% versus 19%), exploring new teaching approaches (63% versus 32%), and discussing research on effective ELL instruction (34% versus 10%). They are somewhat lower than Bay Area counterparts on discussing beliefs about teaching and learning (59% versus 87%) and discussing particular lessons that were not very successful (41% versus 58%). BASRC data provide only crude norms for locating the DL-PLC baseline data for two reasons. The survey response categories differed and had to be interpolated, and the AISD data were reports from the lead teachers, rather than aggregated teacher responses.

- 15 -

Least common is observing each others’ classroom instruction (20%), a rare

practice in all but very strong PLCs. Prior research on high school teaching documents teachers’ resistance to opening their classrooms to colleague scrutiny, as well as the view that their scarce prep time is better spent on the classes they teach. Still, in the DL-PLC schools we encountered examples of observations which were valued by teachers. One teacher provided this example: “I was asked to do a DL lesson. There was a 20 minute walk through with teachers from different content areas. I told them what I wanted them to focus on. I really liked that. The post conference was too rushed but they brought up a couple of good points. The lead worked on writing the comments in a way that was not threatening.” This suggests that IFL’s protocol for teacher walk-throughs may increase teachers’ readiness for cross-class observations with colleagues, but the outside facilitator seems pivotal to supporting such cultural change.

After next spring’s follow-up lead teacher survey, we will assess change on these

PLC indicators for each school and separately for content areas within each school. We also will assess the expected effects of PLC strength on changes toward DL teaching and learning outcomes using the longitudinal data. Benefits of DL-PLC

Survey data capture DL lead teachers’ perceptions that they are making a

difference in teacher collaboration and change toward desired instructional outcomes. They were asked to rate how much difference DL-PLC has made for several facets of instruction:

• Selection and development of rigorous lessons • Teacher collaboration on instruction • Teachers asking more of their students • Student engagement • Student test performance.

The data summarized in Figure 8 suggest benefits in all of these areas, though

they are quite variable across the six schools. Significantly, lead teachers in all DL-PLC schools report some benefits in each area, while those in schools in which progress has been greatest report consistently “major benefits” for instruction and student engagement and “considerable benefits” for student test performance.14 As we elaborate below, the schools and subject departments within them varied considerably in their readiness to enact DL-PLC’s vision.

14 The graphic summary shows overall ratings of lead teachers on benefits of DL-PLC for instruction in their content area in their school. Bars on the graph show the highest and lowest ratings across the six schools.

- 16 -

Figure 8. Benefits of DL-PLC for Instruction: Lead Teacher Ratings

Selection ordevelopment ofrigorous lessons

Teachercollaboration on

instruction

Teachers askingmore of their

students

Student engagement Student testperformance

Mean of all DL-PLC Schools Lowest School Mean Highest School Mean

Major benefits

Considerable benefits

Minor benefits

Nobenefits

Teachers we interviewed illustrated ways in which DL-PLC deepened their understanding of rigorous instruction and influenced their own teaching. For example, one lead teacher captured the differences between a DL lesson and a non-DL lesson: “Everything. The non-DL activities are very traditional: Here is a problem. Lots of worksheets. Some are exploratory but the richness isn’t there. Opportunities for kids to discuss isn’t there. Opportunities to get into deep thinking, active engagement, pull from lots of resources—that’s what is missing.” Several lead teachers commented on the value of PLC work. As one put it: “It has really changed me this year, sharing lessons. The importance of teaming and working with your colleagues is invaluable: sharing your work and seeing how you can improve.” Teachers also highlighted specific benefits to their instruction. One said: “I have always done hands-on but with DL it isn’t that so much as it is whether you are asking the right question—questions that lead [students] to do and think.” Another pointed to her recognition of the need to provide student groups with tasks that are worthy of groupwork. “We noticed that sometimes we have students work in groups on things that are not complex enough. We have to develop our skills on what is beginning level and more complex.”

- 17 -

One lead teacher spoke of the potential she sees in DL: “I have looked at the three other cores and the thing that is constant is inquiry and reading with a rigorous purpose, always time and routine for students to talk to each other about what they are learning, and always asked to write. So it would seem to me if DL gets off the ground, it would have the potential to really accelerate learning.” Further assessment of DL-PLC outcomes awaits follow-up survey data and student outcome data beyond this year. Given differences in professional context and PLC strength between subject areas within schools, we will track progress on DL-PLC outcomes for departments across the six schools. School and District Context and Support Schools, and departments, began with different levels of readiness DL-PLC schools – and content areas within them – began this initiative with differing levels of “readiness.” As would be expected, they varied in the extent to which teachers in a given content area were accustomed and oriented to collaboration. Principals varied in the knowledge and skills they brought to supporting PLC development and DL instruction. Some had been immersed in IFL’s principles and philosophy; others were not as familiar. Principals also varied on their ability and resources to schedule significant blocks of time for teachers to meet regularly. Further, the schools and subject departments had more or less established relationships for seeking support from district content specialists, by virtue of their reform histories and teacher networks that had formed around particular grants.15

We created a scale composed of several items designed to tap aspects of “instructional improvement culture” in schools.16 Significantly, differences on this scale appear to be greater among content areas within schools than among the schools.17 Interviews with principals and with DL lead teachers suggest that one or more content area in each high school struggled with DL-PLC work because teachers lacked experience with, and commitment to, collaboration. In particular, tensions arise in situations in which the department chair is not a strong leader or does not embrace DL-PLC. Roughly a third of the lead teachers are not department heads. In many situations, when the chair supports PLC work on DL instruction, this works smoothly. In some it

15 Recent research on district reform, and research on mathematics reform, point to the important role that subject instructional improvement grants play in developing district staff capacity to support instruction in the schools. In addition, differences in schools’ participations in particular grants result in unequal school capacity to take advantage of central office instructional supports. 16 See Appendix B.2 for scale definitions. 17 We estimated variance at school and department levels using dummy-variable regression analysis, in which variance explained (R2) by the set of 6 dummy variables representing DL-PLC schools was compared with R2 for the set of 24 dummy variables representing content areas within the schools. For the “instructional Reform Culture” scale, 44% of the variance in teacher scores was linked to between-department differences, compared to just 11% between schools. Other context measures showed significant between-school differences, rather than department differences within schools. These findings generally replicate results from our earlier research in high schools.

- 18 -

does not. Principals acknowledged that choices for lead teachers are critical. In fact, some initial choices made under time pressure are now being changed. Lead teachers report widely varying levels of support How lead teachers carry out their roles depends in part on the supports they receive from school and district leaders. In the survey, they were asked to rate the level of support they got from their principal, SLC leaders, district office staff, and consultants. On average, they reported moderate levels of support from their principal and their DL-PLC consultants (from IFL) and lower levels of support from SLC leaders in the school and district office staff.

School differences in support for DL-PLC work are considerable and derive in part from pre-existing differences in professional culture. But they also come from tensions in leadership and focus between DL-PLC work on content instruction and demands of SLC designs for cross-discipline instruction. Notably, one-fourth of the lead teachers report that they do not have a regular time committed to meeting with their content PLCs. Principals play a key role in establishing such key resources for DL-PLC work and in creating coherence across various strands of the districts’ high school redesign initiative. For example, the principal of one DL-PLC school has used site-based professional development days as well as paid after-school time to further this work; this principal has also invested fiscal resources in enhanced IFL support and in a design for instructional coaching to advance DL teaching and learning across the school. Obviously, commitment to regular and sufficient time for PLCs to meet is prerequisite to expectations for progress.

Despite school and department differences in reports of administrative support,

lead teachers were in agreement that time to work with their content colleagues was too little. In response to an open-ended survey question asking lead teachers to write in barriers they have faced in their role as lead teacher, over three-quarters of the 28 teachers who listed barriers named time as the biggest problem. Especially they reported insufficient time for their PLCs to meet regularly in a long-enough block of time to accomplish their goals. District leaders work to create coherence around DL-PLC Central office administrators and staff are involved in learning and supporting DL-PLC. Through their participation in IFL meetings in Pittsburgh over recent years and in IFL-led professional development on Disciplinary Literacy in the district this year, several top administrators have become quite knowledgeable about DL instruction and the design for implementing DL-PLCs in the schools. Administrators of Curriculum and Instruction and the acting Associate Superintendent of High Schools have taken leadership roles in rolling out DL-PLC in the six district schools, and its core ideas have also been part of principal professional development and learning community sessions for all high school principals.

- 19 -

Content supervisors also have participated in DL trainings and are knowledgeable about the principles of learning that under gird DL lessons. They are familiar with DL units and lessons, and in some cases are working on mapping the lessons into the district’s Instructional Pacing Guides (IPGs) for their content area. Interviews suggest that they share a view of DL instruction as high-quality teaching and learning and are working to incorporate it into district instructional supports for the core high school subjects. Given inevitable turnover of central office personnel (and school principals), the district will want to find ways to ensure that newcomers to these positions become knowledgeable about DL principles. Schools struggle to forge coherence between DL-PLC and high school redesign. DL-PLC high schools have experienced tensions between the many strands of high school reform in the district this past year. In particular, as indicated by the survey responses shown in Figure 9, lead teachers see DL-PLC and high school redesign (SLCs in particular) as supporting one another in theory, but not in practice. This stems in part from lead teachers’ experience of competing time demands from multiple reform strands. As one teacher put it:

There are not enough hours in my day to complete all the things being mandated. I believe in DL principles, and I believe in strong AP strategies, and I believe in PLCs – but I cannot [do all this] at the same time. The district knows how to ruin a good thing! My teachers don’t want to be heroes –they just want to be able to teach and teach well… and they want the time to process information at a deep level without being bombarded with ‘the next thing.’ Figure 9. Reform Coherence: How Do High School Redesign (SLCs)

and DL-PLC Work Together in Your School?

0 20 40 60 80 100

Support each other in theory

Support each in practice

% Respondents

Agree Strongly agree

- 20 -

This lead teacher attributed the incoherence problem to a lack of coordination among central administration units involved in high school redesign. “They mandate programs without coordinating their efforts; they isolate programs rather than integrating their efforts; they mismanage implementation of programs. Any good intentions that programs may have are lost in bureaucratic mismanagement.”

Although this individual’s statements are harsher and more elaborate than most, lead teachers from several DL-PLC schools expressed the view that central office “mandates” interfered with their teachers’ efforts to improve teaching and learning in their content area.18 Several respondents explained in some detail the challenges of maintaining cohesive PLCs while reorganizing into SLCs. For example, since departments and course teachers (e.g., Algebra 1 teachers) are split up across SLCs, it will be difficult to maintain similar schedules and physical proximity, both of which encourage sharing and planning together.

The success of the “late start” pilot this year and School Board approval to expand

to 11 days for all schools next year will alleviate this situation somewhat. Still, multiple high school reform agendas threaten to compete for that time, and even if all late start days were devoted to PLC-DL, the available time would fall short of the expected two sessions per month.

Next year’s evaluation research will continue to tap teachers’ perceptions of

coherence in the district’s high school reform efforts. Our data collection will attend to district and school policies and structures that advance, and that inhibit, coherent and consistent improvement efforts in schools involved in the DL-PLC initiative.

18 These concerns are not limited to DL-PLC schools or lead teachers. Department chairs in non-DL schools expressed concerns with the district’s “cookie cutter approach” and tensions between site-based reform leadership and standardized district models.

- 21 -

III. Summary and Implications DL-PLC aims to prepare lead teachers (1) to create rigorous lessons that encourage students to think and apply knowledge, (2) to instruct in ways that elicit these behaviors in students, and (3) to lead their peers in acquiring this knowledge, engaging them in a process of continuous improvement through PLCs. After DL-PLC’s first year of implementation in six AISD high schools, we see signs of progress on these desired outcomes, as well as emergent issues and challenges that district and school leaders will want to address as the initiative moves forward. Signs of Teacher Learning, Improved Instruction, and PLC Development

Our interviews with DL-PLC school principals and lead teachers, coupled with a survey of lead teachers, provide evidence of progress and suggest the initiative’s potential to increase the academic rigor of teaching and learning in AISD high schools. Notable indicators of progress include:

• IFL-led professional development and support gets high ratings from

participants. Participation in IFL-run professional development sessions during 2006-07 was consistently high across the six schools, as were participants’ ratings of the sessions’ quality and of a range of DL-PLC tools and on-site supports.

• Lead teachers report substantial growth in their instructional knowledge and

leadership skills. Over the course of the year, these teachers developed confidence in their ability to implement DL teaching in their classroom and to guide the learning and improvement efforts of colleagues in their PLCs.

• Lead teachers embrace core DL ideas. Individuals expected to lead instructional

improvements in their schools are enthusiastic and articulate about DL lessons and the principles that guide their development and use.

• Principals and lead teachers report important benefits of DL-PLC. Across the six

schools, lead teachers report positive impacts of DL-PLC on teacher collaboration, use of rigorous lessons, expectations for student learning, and student engagement.

In general, IFL’s DL lessons in each of the core content areas have been the vehicle

for teacher learning and efforts to provide more rigorous instruction for their students. Although evidence of progress comes from all DL-PLC schools, some schools have achieved broader and deeper teacher engagement than others. Depending upon school and department conditions, teachers have become more or less involved in co-developing new DL-type lessons and observing each others’ classrooms for evidence of student engagement in rigorous content learning and ideas for improvement. The most active PLCs had principals and department chairs who were engaged in and committed to supporting DL professional development. They provided critical time resources for PLCs

- 22 -

to work on DL instruction and placed priority on this component of their high school’s reform.

Our evaluation research thus far, as well the literature on PLC development, suggests

that DL-PLC provides an effective vehicle for developing teacher collaboration on instruction, as well as for increasing the academic rigor of teaching and learning. This is because a) the DL lessons provide focus for teacher work in newly formed PLCs, b) the task of implementing and creating DL lessons is sufficiently complex and challenging to warrant collaboration and knowledge sharing among colleagues, and c) as a result teachers see benefits to their own instructional practices. Studies of PLC development document that they often fizzle because teachers perceive that required collaboration with colleagues is not worth their time. Absent joint work that is worthy of their collective effort and pays off in the classroom, teachers see time spent in PLC meetings in compliance terms and develop rituals to nominally satisfy the requirement. Serious collaboration in PLCs grows around authentic instructional challenges and tasks, and it appears that DL lessons and tools are well-designed to foster the development of teacher learning communities. Emergent Issues In studying DL-PLC we identified three key issues that merit district attention. One concerns the need for greater coherence among the multiple district initiatives in the eyes of principals and teachers. A second centers on strategies for instructional improvement and perceived conflicts between goals of depth and coverage for content teaching and learning. The third pertains to the importance of clear indicators and benchmarks to track DL-PLC progress The coherence issue stems from the fact that AISD embraced several initiatives at once as part of high school reform during 2006-07. Although schools had some choice among the initiatives, others were mandated by the district. DL-PLC high schools juggle time and attention demands from the multiple initiatives underway. While they work to improve academic teaching and learning through subject area PLCs engaged in DL instruction, they also are trying to establish new, SLC organization units and advisory designs and to provide time for teachers to participate in AP Strategies. Although DL lead teachers see the key high school reform initiatives as coherent in theory, they see them as incoherent in practice. Not only do the initiatives compete for teachers’ time and effort, but they create potential for tension among school leaders responsible for one or another of the initiatives. Absent support from SLC heads and involvement of department chairs, for example, the lead teachers’ hands are tied when it comes to engaging colleagues in sustained work on content instruction. Strong progress on DL-PLC depended this year on school and district authorities’ granting it high priority. A key issue for district and school leaders to address at this juncture is how the various district initiatives and their leadership fit together in terms of their guiding principles and implementation capacity.

- 23 -

The second issue that emerged is lead teachers’ perceived tensions between DL lessons’ emphasis on depth of content knowledge and discipline skills versus state and local pressures toward topical coverage of subject matter. DL-PLC’s emphasis on rigorous instruction to promote depth of students’ content knowledge responds directly to criticisms of the lack of cognitive demand and rigor in the nation’s public schools. Yet this runs counter to emphasis in the high-stakes state assessment (TAKS) on breadth of factual knowledge, pacing guides that interfere with extended lessons, and test prep conceptions of teaching and learning. In this respect, DL-PLC is counter-normative and therefore calls for explicit support and guidance from district authorities for how to navigate the competing strategies for instructional improvement. The third issue—indicators and benchmarks for assessing progress on the DL-PLC work—surfaced in our study as a question of reasonable expectations for change across schools and their subject departments. The dual goals of increasing the rigor of teaching and learning and developing PLCs that collaborate on instruction are major undertakings for high school reform. How quickly and through what developmental stages can DL-PLC schools implement this ambitious reform model? What are valid indicators of progress and what are markers or benchmarks for assessing reasonable progress on both outcomes for the initiative? We have observed that under positive implementation conditions, lead teachers strive to provide rigorous and engaging content instruction for their students and collaborate with their subject area colleagues to develop and continually refine DL-type lessons. What, then, does it take to provide these conditions at the school and district level, and what are reasonable expectations for and indicators of progress?

In our view, an important step in this direction would be for AISD to create a system of leading indicators to capture DL-PLC progress and to guide practice at all levels of the system. This is no small task. Because DL-PLC is a set of principles and tools for instruction and collaborative teacher learning, rather than a program to be implemented, it does not lend itself to traditional measures of program fidelity. Despite considerable challenges in developing a valid and effective indicator system, the importance of doing so seems clear. For one, the district’s use of indicators that are valid with respect to DL-PLC goals would help to legitimize its distinctive approach to high school instructional reform. (Conversely, indicators not perceived as valid by teachers risk undermining the entire endeavor.) Second, a district DL-PLC indicator system would enable leaders to track progress and improve their supports for change at the district, school, and PLC levels and to adapt them to developmental differences. Third, including indicators of implementation infrastructure and leadership conditions would help to clarify expectations among principals for their roles to support teachers’ movement toward the reform vision, which also speaks to the coherence issue.

Together, these emergent issues point in the same direction—a direction well-

documented in research on district improvement efforts. For any initiative to succeed, those carrying out the work in schools need clear signals about its importance, clear expectations for what they are to do, useful feedback on how they are doing, and supports necessary to do the work well.

Appendix A

Student Demographics, Teacher Characteristics, and TAKS Results in AISD

- A1 -

Table A.1. 2005-06 Student Demographics in AISD High Schools

School #

students

% Economically Disadvantaged % LEP

% Special

Ed.

% African

American%

Hispanic

% Asian / Pacific Islander % White

DL-PLC Schools Anderson 2088 17.9 2.9 10.2 8.1 21.5 6.8 63 Austin 2150 26.8 3.8 14.0 6.2 36.3 1.1 56.1 Crockett 2001 50.5 7.8 18.6 10.2 56 1.2 32.2 Akins 2351 53.8 9.5 15.1 12.3 64.7 2.4 20.3 International * 280 88.2 98.6 0 2.9 92.5 3.2 1.1 McCallum 1671 34.0 3.9 14.2 22.0 26.5 0.8 50.5

Other High Schools Bowie 2673 7.7 1.1 9.2 4.2 24.2 4.2 66.9 Johnson - LBJ 1666 48.9 8.2 8.9 29.2 35.1 6.5 28.9 Lanier 1589 78.5 28.3 14.3 14.7 73.1 3.1 8.9 Travis 1555 78.9 22.1 17.7 13.0 79.3 0.8 6.7 Reagan 1009 79.7 22.8 16.2 33.9 63.4 0.7 2 Johnston 735 82.6 19.2 21.8 17.7 80.7 0 1.6

N 11 11 11 11 11 11 11 11 Mean 1771.6 50.9 11.8 14.6 15.6 51.0 2.5 30.7

All regular high schools sd 565.8 26.9 9.53 4.02 9.4 22.8 2.4 25.0 District mean 729.5 63.7 26.1 14.2 14.8 57.5 2.5 24.9 Texas State Mean 566.3 55.6 15.8 11.1 13.7 35.3 3.5 47.2

* International High School is an alternative school.

- A2 -

Table A.2. 2005-06 Teacher Characteristics in AISD High Schools

School

FTE # All

Staff

FTE # Professional

Staff FTE #

Teachers

% Beginning Teachers

% with 1-5 Years

Experience

% with 6-10 Years

Experience

% with 11-20 Years

Experience

Average Years of Teaching

Average Years in District

DL-PLC Schools Anderson 148.6 135.6 119.1 6.2 27.2 15.1 17.3 14.3 10.9 Austin 172.8 148.8 132.2 1.5 29.2 26.0 19.6 12.7 9.4 Crockett 171.3 152.3 139.1 8.2 30.2 17.4 24.2 10.9 7.4 Akins 183.3 167.3 147.8 5.7 37.3 21.0 17.3 10.9 5.7 International * 30.6 28.6 22.5 32.1 31.8 13.4 1.0 7.9 3.4 McCallum 124.5 112.6 99.2 2.6 24.2 21.7 18.0 14.9 11.6

Other High Schools Bowie 184.5 170.5 151.1 5.9 22.9 17.2 24.2 14.3 10.2 Johnson - LBJ 137.0 128.0 106.7 10.6 33.0 18.8 15.3 11.3 7.4 Lanier 144.4 128.0 109.6 9.9 27.8 21.6 19.6 11.3 8.0 Travis 153.4 133.4 109.1 7.1 37.4 15.8 18.4 11.1 6.3 Reagan 101.9 91.9 74.0 17.3 33.9 20.3 16.2 8.6 6.5 Johnston 89.8 82.8 60.6 17.1 26.0 17.6 16.1 10.7 6.2

N 11 11 11 11 11 11 11 11 11 Mean 146.5 131.9 113.5 8.4 29.9 19.3 18.7 11.9 8.2

All regular high schools sd 31.5 28.1 28.8 5.2 5.0 3.2 3.0 1.9 2.0 District mean 65.2 4.3 50.5 8.4 30.1 18.9 21.8 11.2 8.2 Texas State Mean 53.4 4.2 39.2 7.5 29.0 19.4 24.2 11.5 7.6


- A3 -

Table A.3. 2005-06 TAKS results in AISD High Schools (results of all tests for all students)

9th Grade 10th Grade 11th Grade

School

# Students Tested

% Met Standards

# Students Tested

% Met Standards

# Students Tested

% Met Standards

DL-PLC Schools Akins 656 44 531 32 465 51 Anderson 519 75 488 74 419 87 Austin 583 67 525 61 436 77 Crockett 549 44 449 38 395 55 International * 5 20 2 -- McCallum 457 57 378 58 292 79

Other High Schools Bowie 681 83 691 73 632 81 Johnson - LBJ 483 61 409 63 323 69 Johnston 226 21 147 22 121 42 Lanier 502 31 246 30 256 45 Reagan 311 28 198 21 165 34 Travis 413 38 354 27 216 48

District mean 5486 53 4484 51 3818 66 Texas state mean 345288 56 291725 49 243457 64


Appendix B

AISD Survey Scale Definitions

B.1. Survey Response Distributions

B.2. Survey Scales Developed for Tracking Changes

Center for Research on the Context of Teaching Stanford University

Austin Independent School District On-line Survey

DL-PLC School

Department Chairs and

Teacher Leaders

Spring 2007

Response Summary (data from 41 respondents in 6 schools)

CRC: School of Education, CERAS Building, Stanford University, Stanford, CA 94305-3084 (650) 723-4972

DL-PLC Department Chairs and Teacher Leaders Spring 2007 Response Summary -1

ABOUT THE SURVEY

This survey is being conducted by the Center for Research on the Context of Teaching (CRC) at Stanford University as part of an evaluation of high school improvement efforts in Austin Independent School District (AISD). The survey includes department chairs and Lead Teachers in DL-PLC schools. Questions focus on school and district leadership and instructional support. Time needed to complete the survey is 30-40 minutes. If you cannot complete the survey at once, you can come back and start from the page you left by clicking on the link in the email we sent you. Responses are entirely confidential. The survey is governed by stringent Stanford University regulations designed to ensure the privacy of individuals’ responses. All survey results will be reported only in statistical summaries so that no individuals can be identified. Thank you for contributing your time and thoughtful responses to this survey.

* * * * * * * * * * 1. Please indicate your primary area of teaching. 2. Which of the following positions do you hold?

% Department Chair and DL Participant 65.9 Department Chair (not in DL) .0 DL teacher leader 34.1

% English / Language Arts 31.7 Math 17.1 Science 22.0 Social Studies 29.3


INSTRUCTIONAL CLIMATE AND LEADERSHIP

3. How well does each of the following statements describe your school’s instructional climate? Teachers in my school … Strongly

disagree Disagree Neutral Agree Strongly

agree

a. Have a clear vision of instruction linked to standards for student learning and growth .0 2.4 4.9 65.9 26.8

b. Openly examine and acknowledge progress towards an instructional vision .0 4.9 12.2 56.1 26.8

c. Have made changes designed to better meet the needs of the school’s diverse student body .0 4.9 17.1 39.0 39.0

d. Are engaged in systematic analysis of student performance data .0 9.8 9.8 61.0 19.5

e. Have well-defined plans for instructional improvement .0 9.8 24.4 46.3 19.5 f. Review student learning and understanding in order to

adjust their practices 2.4 12.2 17.1 53.7 14.6 4 This question concerns how teachers in your content area regularly interact with one another. How

many teachers do each of the following?

Teachers in my content area … No teachers

Few teachers

About half the teachers

Most teachers

All teachers

a. Share ideas on teaching .0 2.4 19.5 43.9 34.1

b. Discuss what you/they learned at a workshop or conference .0 12.2 22.0 46.3 19.5

c. Share and discuss student work .0 26.8 22.0 31.7 19.5

d. Discuss particular lessons that were not very successful 4.9 24.4 29.3 26.8 14.6

e. Discuss beliefs about teaching and learning .0 7.3 34.1 34.1 24.4

f. Share and discuss research on effective teaching methods 2.4 31.7 26.8 29.3 9.8

g. Share and discuss research on effective instructional practices for English language learners 4.9 48.8 12.2 22.0 12.2

h. Explore new teaching approaches to increase student engagement and understanding .0 9.8 26.8 39.0 24.4

i. Observe each other’s classroom instruction .0 56.1 24.4 9.8 9.8

j. Plan lessons and units together .0 17.1 31.7 26.8 24.4


5 Now consider instruction in your content area. To what extent does each statement describe

teaching practices? Teachers in my content area … Strongly


agree

a. Communicate clear expectations to students .0 4.9 2.4 63.4 29.3

b. Create learning environments in which efforts pays off .0 2.4 9.8 51.2 36.6

c. Recognize student accomplishments that meet standards or benchmarks for progress .0 .0 12.2 56.1 31.7

d. Provide an academically rigorous curriculum .0 2.4 9.8 63.4 24.4

e. Promote accountable talk in their classrooms .0 2.4 22.0 48.8 26.8

f. Help students self-manage and self-monitor their learning .0 7.3 17.1 61.0 14.6

6. Considering all teachers’ classes in your content area, in your estimation, in about how many

classes do students do each of the following?

Students in the class… No

classes Few

classes

About half the classes

Most classes

All classes

a. Learn core concepts and habits of thinking .0 2.4 39.0 48.8 9.8

b. Learn by “doing” the discipline, i.e., doing tasks that mirror the work of the discipline .0 9.8 39.0 41.5 9.8

c. Can explain how and why their class activities help them understand a concept .0 17.1 51.2 26.8 4.9

d. Ask questions and insist on understanding the answers .0 22.0 41.5 29.3 7.3

e. Learn from one another through classroom discussion .0 12.2 31.7 48.8 7.3

f. Persist in working on challenging tasks .0 19.5 56.1 19.5 4.9

g. Work in pairs or groups during most class periods .0 14.6 41.5 34.1 9.8

h. Engage in self-assessment that develops their ability to effectively manage their own learning .0 34.1 41.5 22.0 2.4


7. This question concerns the professional climate of your school. Please indicate how strongly you agree or disagree with each of the statements below.

The principal … Strongly


agree

a. Inspires the very best in the job performance of all teachers 2.4 4.9 9.8 51.2 31.7 b. Supports regular use of student assessment data 2.4 4.9 7.3 51.2 34.1 c. Supports our school’s redesign effort 2.5 5.0 12.5 37.5 42.5 d. Promotes teachers’ ongoing professional development 2.5 .0 7.5 35.0 55.0 e. Ensures that student learning is the “bottom line” in our

school 2.4 2.4 4.9 46.3 43.9 f. Is increasing support for teaching and learning 2.4 2.4 7.3 39.0 48.8 g. Supports the development of teacher professional learning

communities in our school 2.4 .0 7.3 31.7 58.5 h. Provides time and money for teachers to collaborate and plan

together .0 7.3 14.6 31.7 46.3 8. This question concerns the professional climate of your district’s central office. Please indicate how

strongly you agree or disagree with each of the statements below.

The central office … Strongly disagree Disagree Neutral Agree

Strongly agree

a. Helps my school focus on teaching and learning 7.3 12.2 29.3 43.9 7.3 b. Is committed to high standards for every student 2.4 9.8 29.3 43.9 14.6 c. Supports school innovation 5.0 12.5 15.0 57.5 10.0 d. Inspires the very best in the job performance of its teachers 10.0 12.5 35.0 35.0 7.5 e. Holds high expectations for our school 2.5 10.0 25.0 40.0 22.5 f. Supports my school’s redesign effort .0 10.0 15.0 57.5 17.5 g. Promotes the professional development of teachers 4.9 14.6 26.8 41.5 12.2 h. Ensures that student learning is the “bottom line” in our

school 9.8 12.2 36.6 29.3 12.2 i. Provides support for teaching and learning 9.8 12.2 34.1 31.7 12.2 j. Supports the development of teacher professional learning

communities in our school 2.5 10.0 22.5 47.5 17.5 k. Allows high schools the flexibility to choose and adapt new

programs and practices 7.3 24.4 22.0 39.0 7.3 l. Seeks input from teachers and listens to their ideas and

concerns 17.5 15.0 40.0 25.0 2.5


Please write any comments you wish to make about the organization and leadership of your school and/or district, in particular how it supports or inhibits improved teaching and learning in your content area.

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________


DL-PLC INITIATIVE

9. When did the DL Professional Learning Community in your core content area first meet?

(Month and year) _______________

10. Do you have regular time committed to meet?

Yes 75.6 No 24.4

a) If yes, how often do you meet?

Daily .0 Weekly 25.8 Monthly 58.1 Other (specify): 16.1

11. a) How many teachers are in your department? _Mean = 15.5 (std. dev. = 6.3)_

b) On average, how many teachers regularly attend DL-PLC meetings? _Mean = 11.0 (sd = 6.2)

c) How many teachers have taught at least one DL lesson this year? _Mean = 5.9 (sd = 4.6)

12. Please indicate the extent to which each of the following has been a focus of your PLC meetings this

year. Our PLC has focused on … Not a

focus

Minor focus

Moderate focus

Major focus

a. Learning to use the Protocol for Studying Student Work 23.7 26.3 47.4 2.6 b. Examining student tasks/assignments 12.5 25.0 47.5 15.0 c. Discussing features of DL lessons and/or units 2.5 5.0 25.0 67.5 d. Developing or selecting DL lessons and/or units 2.5 27.5 25.0 45.0 e. Organizing Learning Walks 20.0 35.0 35.0 10.0 f. Learning to use the DL Observation Protocol 17.5 30.0 42.5 10.0 g. Learning to use content-specific DL tools (e.g., ELA

Design Features) 10.0 25.0 30.0 35.0 h. Other: _________________________________________


13. Which DL-PLC tools do you find to be most valuable in your role as a Teacher Leader? (e.g.,

Observation protocols, Studying Student Work protocol) Tools # Study Student Work 9 Observation protocols 9 The Architecture 5 “Thinking Through a Lesson” 4 Model Lessons 4 Revisiting Texts 3 DL charts 3

14. To what extent does your principal support your work as a DL Lead Teacher? My principal … Strongly

disagree Strongly

agree

a. Is informed about the content and process of the DL-PLC model 2.5 2.5 7.5 57.5 30.0

b. Works in a collaborative partnership with me 2.5 7.5 20.0 50.0 20.0 c. Provides time for teachers in my PLC to work together 2.5 5.0 12.5 37.5 42.5 d. Follows up with me to see how the DL lessons are

working for teachers and students 7.5 7.5 32.5 32.5 20.0 e. Encourages teachers to implement DL lessons and

practices 2.5 10.0 17.5 20.0 50.0 15. Now consider how the district-wide high school redesign work (SLCs) and the DL-PLC work together

in your school. AISD’s high school redesign and DL-PLC…. Strongly

disagree Strongly

agree

a. Support each other in theory 2.5 12.5 5.0 62.5 17.5

b. Support each in practice 2.6 25.6 33.3 33.3 5.1

c. Compete for teachers’ limited time available for PD .0 7.5 25.0 47.5 20.0

d. Compete for teachers’ limited time to meet together .0 5.0 20.0 55.0 20.0


16. In your role as a DL Lead Teacher, how frequently do you do each of the following in a typical

month?

As DL Lead Teacher, I… Have not done this

yet Once 2-3

times Weekly

2-3 times a week Daily

a. Work with principal and/or core DL team coach to plan DL-PLC work 32.5 22.5 30.0 10.0 5.0 .0

b. Lead study groups 30.0 25.0 37.5 7.5 .0 .0

c. Confer with teachers in my content area 5.0 7.5 32.5 25.0 12.5 17.5

d. Reflect on my learning 7.7 5.1 17.9 17.9 7.7 43.6

e. Observe other teachers 15.0 30.0 32.5 15.0 2.5 5.0

f. Am observed by other teachers 32.5 30.0 32.5 2.5 .0 2.5

g. Other (please specify):

______________________________________ 17. Please indicate which of the following kinds of professional development and support you have

received from Institute for Learning (IFL) consultants over the past year. For any that you have experienced, please indicate how valuable it has been for you.

I have

received this

Not at all valuable

Somewhat valuable

Moderately valuable

Quite valuable

Extremely valuable

a. Training sessions on DL-PLC in the district 97.4 5.4 2.7 18.9 45.9 27.0

b. IFL-led classroom observation in my classroom 55.3 7.1 21.4 14.3 7.1 50.0

c. IFL-led classroom observation in another teacher’s classroom 48.7 5.6 22.2 33.3 11.1 27.8

d. Demonstration or co-taught lessons 51.3 .0 .0 21.1 42.1 36.8

e. Co-facilitation and/or debriefing of PLC meeting 76.3 3.4 3.4 20.7 37.9 34.5

f. Other (write in: __________________________)

DL-PLC Department Chairs and Teacher Leaders Spring 2007 Response Summary - 9-

18. Various DL-PLC teacher leadership practices are listed below. For each, please a) indicate how prepared you felt to work with teachers in the

following ways before beginning DL Professional Development last summer, and b) how prepared you feel now to do this. A. BEFORE I felt … B. NOW I feel … to work with teachers in the following ways … Not well

prepared Somewhat prepared

Well prepared

Very well prepared

Not well prepared

Somewhat prepared

Well prepared

Very well prepared

a. Facilitate instructionally focused group meetings 39.5 42.1 18.4 .0 2.6 34.2 47.4 15.8 b. Conference with individuals 36.8 44.7 15.8 2.6 2.6 47.4 36.8 13.2 c. Look closely at student work 21.1 42.1 31.6 5.3 5.3 23.7 47.4 23.7 d. Plan instruction together 15.8 42.1 34.2 7.9 2.6 13.2 52.6 31.6 e. Examine instructional practices to make sure they

promote academic rigor and student learning 23.7 39.5 28.9 7.9 2.6 23.7 47.4 26.3 f. Develop more rigorous questions for class

discussion 15.8 50.0 28.9 5.3 2.6 13.2 60.5 23.7 g. Other (write in):_____________________________


19. Overall, how well has the DL-PLC program prepared you for the role of a Lead Teacher?

Not well at all

Somewhat well

Moderately well

Quite well

Extremely well

7.7 10.3 38.5 28.2 15.4 20. How has the DL-PLC program enhanced your own skills and knowledge as a classroom teacher? For each, please indicate a) how well

prepared you felt to do this before beginning DL Professional Development last summer, and b) how prepared you feel now. A. BEFORE I felt … B. NOW I feel … To do the following in my classroom … Not well

prepared Somewhat prepared

Well prepared

Very well prepared

Not well prepared

Somewhat prepared

Well prepared

Very well prepared

a. Teach DL lessons 56.4 33.3 10.3 .0 2.6 43.6 23.1 30.8 b. Engage students in classroom discussion 2.6 39.5 50.0 7.9 2.6 12.8 48.7 35.9 c. Challenge students to think deeply about content 5.1 43.6 41.0 10.3 2.6 10.3 51.3 35.9 d. Develop or select my own rigorous lessons 10.3 43.6 35.9 10.3 2.6 10.3 56.4 30.8 e. Link lessons to assessments 7.7 46.2 23.1 23.1 2.6 12.8 51.3 33.3 f. Organize students to work in pairs or groups 5.1 38.5 35.9 20.5 2.6 7.7 51.3 38.5 g. Apply DL principles to classroom instruction 47.4 28.9 21.1 2.6 2.6 23.7 39.5 34.2 h. Other (write in):_____________________________

21. a) How many DL lessons have you taught in your classroom this year? _Mean = 4.1 (sd = 4.1)

b) Have you developed other lessons using DL ideas? Yes 82.5 No 17.5


22. Please indicate how much support of your DL-PLC role you receive from each of the following people.

Extent of support I receive from … None Minimal Moderate Substantial

a. Teachers in my content area 5.0 17.5 52.5 25.0 b. Teacher Leaders in other content areas 22.5 17.5 35.0 25.0 c. Department Chair (if you are not chair) 10.0 16.7 40.0 33.3 d. SLC leader 19.4 19.4 47.2 13.9 e. Principal 10.3 7.7 43.6 38.5 f. District office staff 20.0 22.5 35.0 22.5 g. DL-PLC consultants 7.5 17.5 37.5 37.5 h. Other (write in): ______________________________

23. What kinds of additional help or support do you feel that you need to succeed in your role as a DL-

PLC lead teacher?

Additional Support Needed # Training; better understand of role and DL work 10 Time (for planning or collaboration) 9 Presentation to Dept. or other teacher buy-in 4

24. Please write in what, if any, barriers you have faced in carrying out your lead teacher role at your

school.

Barriers # Time 18 Support and involvement of department leaders 6 Other teacher buy in 2

25. Now consider how DL-PLC has made a difference for teaching and learning in your content

area in your school. For each of the following outcomes, please indicate how much difference DL-PLC has made.

No

benefits Minor

benefits Considerable

benefits Major

benefits

a. Selection or development of rigorous lessons 7.7 12.8 59.0 20.5 b. Teacher collaboration on instruction .0 20.5 48.7 30.8 c. Teachers asking more of their students 2.5 15.0 55.0 27.5 d. Student engagement 5.0 12.5 52.5 30.0 e. Student test performance 5.0 32.5 50.0 12.5 f. Other (write in):

_________________________________________


Please write any comments you wish to make about your experiences with the DL-PLC Program.

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

AISD Survey Scale Definitions - 1

BACKGROUND AND CAREER

These questions pertain to your professional background. All answers will be used strictly for statistical summaries of the survey data. 26. a) How many years have you worked full-time as a teacher in an elementary or secondary school?

(Include this school year.) Mean = 15.1 (sd = 12.2) years

b) How many years have you taught in this school? Mean = 9.0 (sd = 9.5) years c) How many years have you taught in the Austin Independent School District?

Mean = 11.3 (sd = 10.6) years 27. If you could go back to your college days and start over again, would you become a teacher?

Certainly would become a teacher 48.8 Probably would become a teacher 34.1 Chances about even for and against 7.3 Probably would not become a teacher 7.3 Certainly would not become a teacher 2.4

28. How long do you plan to remain in teaching?

As long as I am able 51.2 Until I am eligible for retirement 29.3 Will probably continue unless something better comes along 12.2 Definitely plan to leave teaching as soon as I can .0 Undecided at this time 7.3

29. What, if any, school or district administrative positions do you plan to pursue in the next 5 years?

Please mark the first column if you currently hold that position. Administrative position Current

position No plans to pursue

May not pursue

May pursue

Definitely will pursue

a. Department Chair 33.3 35.9 2.6 20.5 7.7 b. SLC Head 5.4 67.6 2.7 13.5 10.8 c. Assistant Principal .0 67.6 5.4 13.5 13.5 d. Principal .0 78.4 2.7 8.1 10.8 e. District Administrator .0 75.7 5.4 .0 18.9 f. Other (please write in):

______________________________________


30. a) What is your gender?

Male 25.0 Female 75.0

b) What is your ethnicity/race?

American Indian or Alaska Native .0 Asian .0 Hispanic 18.4 Native Hawaiian or Other Pacific Islander .0 Non-Hispanic Black or African American 2.6 Non-Hispanic White or Caucasian 78.9 Multiracial (write in)__________________________________ .0

Please add any comments you would like to make about your career in education and your plans or interests for the future.

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

*** END OF SURVEY *** THANK YOU FOR THE TIME AND THOUGHT YOU CONTRIBUTED TO THIS SURVEY!


ASID Survey Scale Definitions These survey scales measure school and district context conditions, DL-PLC school support, professional outcomes for DL-PLC lead teachers, and DL-PLC outcomes (professional learning community and instruction). Context scales were used to assess variation in baseline conditions across the DL-PLC schools and their subject departments. Other scales represent leading indicators for evaluating progress on DL-PLC in terms of: the nature and extent of school support, knowledge and skills of lead teachers, and desired PLC and instructional outcomes. Follow-up surveys will assess change in school and subject department scores on these measures. The scales were developed with data from the baseline survey of 47 department chairs and Lead Teachers in the six DL-PLC schools and a survey of 21 department chairs in other AISD high schools. Principal components analysis was used to identify survey items that loaded on a common factor; the alpha coefficient indicates internal consistency of the scale. CONTEXT CONDITIONS

Instructional Improvement Culture (5-point frequency scale, 6 items. Alpha = .88)

3. How well does each of the following statements describe your school’s instructional climate?

Teachers in my school … a. Have a clear vision of instruction linked to standards for student learning and

growth b. Openly examine and acknowledge progress towards an instructional vision c. Have made changes designed to better meet the needs of the school’s diverse

student body d. Are engaged in systematic analysis of student performance data e. Have well-defined plans for instructional improvement f. Review student learning and understanding in order to adjust their practices


Principal Leadership (5-point frequency scale, 8 items. Alpha = .92)

7. This question concerns the professional climate of your school. Please indicate how strongly you agree or disagree with each of the statements below.

The principal … a. Inspires the very best in the job performance of all teachers b. Supports regular use of student assessment data c. Supports our school’s redesign effort d. Promotes teachers’ ongoing professional development e. Ensures that student learning is the “bottom line” in our school f. Is increasing support for teaching and learning g. Supports the development of teacher professional learning communities in our

school h. Provides time and money for teachers to collaborate and plan together

District Professionalism (5-point frequency scale, 5 items. Alpha = .92)

8. This question concerns the professional climate of your district’s central office. Please indicate how strongly you agree or disagree with each of the statements below.

The central office … a. Helps my school focus on teaching and learning d. Inspires the very best in the job performance of its teachers g. Promotes the professional development of teachers i. Provides support for teaching and learning l. Seeks input from teachers and listens to their ideas and concerns

District Reform Support (5-point frequency scale, 6 items. Alpha = .89)

8. This question concerns the professional climate of your district’s central office. Please indicate how strongly you agree or disagree with each of the statements below.

The central office … b. Is committed to high standards for every student c. Supports school innovation f. Supports my school’s redesign effort h. Ensures that student learning is the “bottom line” in our school j. Supports the development of teacher professional learning communities in

our school k. Allows high schools the flexibility to choose and adapt new programs and

practices


SCHOOL DL-PLC SUPPORT

Principal DL-PLC Support (5-point frequency scale, 5 items. Alpha = .91)

14. To what extent does your principal support your work as a DL Lead Teacher?

My principal … a. Is informed about the content and process of the DL-PLC model b. Works in a collaborative partnership with me c. Provides time for teachers in my PLC to work together d. Follows up with me to see how the DL lessons are working for teachers and

students e. Encourages teachers to implement DL lessons and practices

Reform Coherence (5-point frequency scale, 4 items. Alpha = .74)

15. Now consider how the district-wide high school redesign work (SLCs) and the DL-PLC work together in your school.

AISD’s high school redesign and DL-PLC…. a. Support each other in theory b. Support each in practice c. Compete for teachers’ limited time available for PD d. Compete for teachers’ limited time to meet together

PROFESSIONAL OUTCOMES FOR DL-PLC LEAD TEACHERS

Preparedness: DL-PLC Teacher Leadership Practices (5-point frequency scale, 6 items. Alpha = .91)

18. Various DL-PLC teacher leadership practices are listed below. For each, please a) indicate

how prepared you felt to work with teachers in the following ways before beginning DL Professional Development last summer, and b) how prepared you feel now to do this. (The scale was based on the ratings of preparedness now.)

NOW, I feel prepared to … a. Facilitate instructionally focused group meetings b. Conference with individuals c. Look closely at student work d. Plan instruction together e. Examine instructional practices to make sure they promote academic rigor

and student learning f. Develop more rigorous questions for class discussion


Preparedness: Teacher Skills and Knowledge (5-point frequency scale, 7 items. Alpha = .95)

20. How has the DL-PLC program enhanced your own skills and knowledge as a classroom

teacher? For each, please indicate a) how well prepared you felt to do this before beginning DL Professional Development last summer, and b) how prepared you feel now. (The scale was based on the ratings of preparedness now.)

NOW I feel prepared to … a. Teach DL lessons b. Engage students in classroom discussion c. Challenge students to think deeply about content d. Develop or select my own rigorous lessons e. Link lessons to assessments f. Organize students to work in pairs or groups g. Apply DL principles to classroom instruction


DL-PLC OUTCOMES

Professional Learning Community in Content Area (5-point frequency scale, 10 items. Alpha = .92)

4 This question concerns how teachers in your content area regularly interact with one

another. How many teachers do each of the following?

Teachers in my content area … a. Share ideas on teaching b. Discuss what you/they learned at a workshop or conference c. Share and discuss student work d. Discuss particular lessons that were not very successful e. Discuss beliefs about teaching and learning f. Share and discuss research on effective teaching methods g. Share and discuss research on effective instructional practices for English

language learners h. Explore new teaching approaches to increase student engagement and

understanding i. Observe each other’s classroom instruction j. Plan lessons and units together

DL Instructional Practices (5-point frequency scale, 6 items. Alpha = .87)

5 Now consider instruction in your content area. To what extent does each statement describe teaching practices?

Teachers in my content area … a. Communicate clear expectations to students b. Create learning environments in which efforts pays off c. Recognize student accomplishments that meet standards or benchmarks for

progress d. Provide an academically rigorous curriculum e. Promote accountable talk in their classroom f. Help students self-manage and self-monitor their learning


DL Student Practices (5-point frequency scale, 8 items. Alpha = .0-)

7. Considering all teachers’ classes in your content area, in your estimation, in about how

many classes do students do each of the following?

Students in the class… a. Learn core concepts and habits of thinking b. Learn by “doing” the discipline, i.e., doing tasks that mirror the work of the

discipline c. Can explain how and why their class activities help them understand a conceptd. Ask questions and insist on understanding the answers e. Learn from one another through classroom discussion f. Persist in working on challenging tasks g. Work in pairs or groups during most class periods h. Engage in self-assessment that develops their ability to effectively manage

their own learning

Evaluation of the Disciplinary Literacy-Professional ... · Disciplinary Literacy-Professional Learning Community ... Our final evaluation report, to be delivered in Summer ... in

Documents