How districts prepare site administrators for data-driven decision making

HOW DISTRICTS PREPARE SITE ADMINISTRATORS

FOR DATA-DRIVEN DECISION MAKING

by

Miriam Deborah Ezzani

A Dissertation Presented to the FACULTY OF THE ROSSIER SCHOOL OF EDUCATION

UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the

Requirements for the Degree DOCTOR OF EDUCATION

May 2009

Copyright 2009 Miriam Deborah Ezzani

ii

DEDICATION

To Baba, Mohamed Ezzani, for his unrelenting faith and stamina and Mama, Dabia

Ezzani, for her enduring love and passion, all of which are the cornerstone of my

spirit and will to continually pursue knowledge for the purpose of serving others.

To my children, Jamila and Ali, who taught me patience and acceptance -- patience

as I gauge my work with the rhythm of the world and acceptance of that which I am

unable to control. To my sister Huda, brother-in-law Khalid, nieces and nephew,

Heba, Sophia, and Yousef who are all a source of inspiration in that they are all in

my sphere of influence. To Hasmik Danielian, an extraordinary human being, who

has become my soul sister as a result of this once in a lifetime endeavor.

iii

ACKNOWLEDGMENTS

I would like to acknowledge the district administrators and principals at Buck and

D’Angelo Unified School Districts for their openness during interviews and

observations to evaluate professional development for school site administrators.

Certainly, I am grateful to the Superintendents for their ability to value, appreciate,

and concede to this research being done in their districts. I am truly appreciative of

the support from the Rossier School of Education faculty at the University of

Southern California. I am also grateful to my dissertation committee: Dr. Edmond

Heatley, fellow Trojan, Superintendent of Chino Valley USD, and mentor, who

providing the administrative opportunity which enhanced my professional and

educational experiences; Dr. Kathy Stowe, the Executive Director of the Doctor of

Education program for her generous sharing of knowledge, support, and feedback

regarding my ability to lead; and, my utmost appreciation to my Committee Chair,

Dr. Amanda Datnow, for her unrelenting commitment, impeccable knowledge in her

craft, and timely support and feedback from the first course in my program to the end

of this dissertation.

iv

TABLE OF CONTENTS

DEDICATION ii ACKNOWLEDGEMENTS iii LIST OF TABLES vi ABSTRACT vii CHAPTER ONE: 1 Overview of Study 1 Introduction 1 Professional Development 3 Research Questions 10 Significance of the Study 11

CHAPTER TWO: 13 Literature Review 13 Introduction 13 Accountability in K-12 Education 14 The Role of Data-Driven Decision Making in K-12 Education Reform 20 Professional Development to Build Capacity of School Leaders 26 Summary of the Literature Review 43 Figure 1: Data-Informed Leadership 46

CHAPTER THREE: 48 Methodology 48 Research Design 48 Sample 51 Overview of the Districts 53 Buck Unified School District 53 D’Angelo Unified School District 54 Data Collection Procedures 56 Data Analysis 59 Ethical Considerations 62 Limitations of the Study 63 Summary 64

v

CHAPTER FOUR: 65 Data Analysis and Data Interpretation 65 Introduction 65 Framework 67

Need for Professional Development 69 Theme 1: Methods of Needs Assessment 70 Theme 2: Leadership Capacity Building 85 Training and Support for School Site Leaders 99 Theme 3: Accessibility to Tools 99 Theme 4: Process of Data-Driven Decision Making 104 Evidence that Leaders Have Put New Practices into Place 117 Theme 5: Effectiveness of Professional Development 117 Theme 6: Culture of Data Use 131 Existing Knowledge Gaps 143 Theme 7: District Expectations 145 Conclusion 154

CHAPTER FIVE: 159 Summary and Implications of Findings 159 Introduction 159 Connections to Prior Research 162 Accountability in K-12 Education 162 The Role of DDDM in K-12 Educational Reform 166 Professional Development to Build Capacity of School Site Leaders in DDDM 172 Implications for Future Research 180 Implications for Policy and Practice 182 Mastery of Leadership Strategies in Data Informed Leadership 182 Conclusion 186

REFERENCES 187 APPENDIX A: District Administrator Interview Protocol 192 APPENDIX B: Administrator Interview Protocol 195 APPENDIX C: Observation Protocol 198 APPENDIX D: Preliminary Codes 199 APPENDIX E: Salient Codes 200

vi

LIST OF TABLES Table 1: Interview Participants in Buck and D’Angelo USDs 57 Table 2: Organizational Process for Coding, Themes,

and Integration with Sub-questions 61 Table 3: Organizational Chart for Integration of Sub-questions, Themes,

and Elements of Data-Informed Leadership 62 Table 4: California Department of Education, Accountability Progress

Report for Buck Unified School District (2008) 84 Table 5: California Department of Education, Accountability Progress

Report for D’Angelo Unified School District (2008) 84 Table 6: Accountability Progress Report for Buck and D’Angelo USD 145

Table 7: Effective Professional Development Practices in DDDM for School Administrators 179 Table 8: Mastery of Leadership Strategies in Data Informed Leadership 184

vii

ABSTRACT

The federal government’s No Child Left Behind Act and California’s Public

School Accountability Act have mandated California to establish a system of

accountability for their districts. Thus, California’s districts have understood the

need for data-driven decision making (DDDM) to facilitate the monitoring and

improvement of teaching and learning practices to increase student achievement.

The intent of this qualitative study is to investigate the types of professional

development that districts offer school leaders to increase capacity in DDDM. Five

research questions were formulated to guide this study: (1) How do districts build

the capacity of school leaders to engage in data-driven decision making? (2) How is

the need for professional development assessed? (3) What types of training and

support are provided to school site leaders? (4) What evidence is there that leaders

put new data-driven practices into place? (5) What knowledge gaps still exist?

Two districts were selected based on their focus to provide principal

professional development in DDDM. The districts attributed some of their increase

in student achievement to the utilization of data. Interviews of superintendents,

district administrators, and principals were the primary source of information.

Observations and document analysis were also utilized. The research was conducted

through the lens of Data-Informed Leadership, a framework by Knapp et al. (2006).

Professional development for principals at Buck and D’Angelo Unified

School Districts lent to models that allowed for breadth and depth. According to

viii

participants, principal professional development at these districts directly improved

classroom instruction and student outcomes. The superintendents in both districts

possessed strategies which made a difference in the preparation of principals in

DDDM and in practices that gave way to Data-Informed Leadership. It was the

mastery of leadership strategies that moved their organizations into a culture and

cycle of inquiry based on data.

While this study provides relevant information and a roadmap into the

process by which districts provide school leaders with knowledge in DDDM, further

research is necessary in the mastery of leadership strategies in Data-Informed

Leadership. Mastery of leadership strategies will ensure an established culture and

cycle of inquiry in the use of data.

1

CHAPTER ONE

Overview of the Study

If education is always to be conceived along the same antiquated lines of a mere transmission of knowledge, there is little to be hoped from it in the bettering of man's future. For what is the use of transmitting knowledge if the individual's total

development lags behind? -- Maria Montessori (1870-1952)

Introduction

Background of the Problem

The State of California, in response to a growing concern in the quality of

education, passed the Public School Accountability Act (PSAA) in April 1999 –

three years prior to the implementation of the federal government’s No Child Left

Behind (NCLB) Act of 2001. California’s PSAA is a set of programs to assist the

state, local education agencies (LEAs) or districts, school leaders, and the public to

monitor and evaluate the growth of California public schools (Ed Code, 52050-

52050.5). The Education Code states,

. . . pupils in California are not now, generally, progressing at a satisfactory rate to achieve a high quality education. To remedy this, the state is in need of . . . comprehensive accountability system to hold each of the state's public schools accountable for the academic progress and achievement of its pupils . . . (52050.5, b, c, and d)

California later modified its accountability system in August 2003 in order to

meet NCLB funding requirements, which was put into place in January 2002 to

assure that all students by 2013-2014 can read, write, and perform mathematics in

order to graduate from high school (U.S. Department of Education, n.d.). NCLB is

2

founded on four pillars: (1) stronger accountability for results, (2) more freedom for

states and communities, (3) proven education methods and, (4) more choices for

parents.

Both NCLB and PSAA accountability systems are all-inclusive efforts that

hold all stakeholders accountable for raising the level of education of the nation’s

children and this requires “. . . support to devise an acceptable way to balance public,

academic, and market concerns” (Burke, 2004, p. 24). At the local level are

instructional leaders that set the pace of accountability at their school sites. But do

the instructional leaders possess the knowledge and skills to implement, monitor, and

evaluate an accountability system that produces student achievement? Have states

and districts provided adequate professional development to school site leaders in

order to meet the accountability demands as stated above?

As a consequence of NCLB, district and school site leaders are expected to

have the knowledge and skills to recognize and implement all the modalities

necessary to meet the pressures of performance-based accountability to increase the

quality of public education for all students. Moreover, it has become crucial for

school site leaders to be proficient in data use and analysis to gauge academic

progress (Elmore, 2002).

3

Professional Development

Numerous scholars suggest that capacity must be built at the school site for

data-driven decision-making to be an integral part of the organization (Datnow, Park

& Wohlstetter, 2007; Knapp, Swinnerton & Copland, 2006; DuFour, DuFour, Eaker

& Many, 2006). This, however, requires professional development and ongoing

training of school site administrators. Elmore (2002) states, “Accountability must be

a reciprocal process. For every increment of performance I demand from you, I have

an equal responsibility to provide you with the capacity to meet that expectation” (p.

5).

The State of California, in an attempt to build capacity in school site

administrators to meet the federal and state accountability requirements, offers The

Administrative Training Program (AB 430), authorized by Assembly Bill 430, and

previously known as AB 75. AB 430 was introduced through the California

legislature in 2005-2006. AB 430 provides funding to districts to train principals and

assistant principals in the knowledge and skills needed for their convoluted roles as

instructional leaders. This two-year program is divided into three modules and can

only be delivered by a provider that is state-approved. This is the only state of

California initiative that focuses on principal professional development. It consists

of: Module 1, Leadership and Support - Student Instructional Program (five days),

which reviews the core academic standards, curriculum frameworks and instructional

materials aligned to the state standards, use of assessments, ways to master use of

4

Standardized Testing and Reporting (STAR) assessment data, and school

management technology to improve student performance. Module 2, Leadership and

Management for Instructional Improvement (two days), which includes school

financial and personnel management, and knowledge and skills necessary to serve all

students. Lastly, Module 3, Instructional Technology to Improve Pupil Performance

(three days), which consists of strategies for instructional leadership and

management utilizing instructional technology to improve pupil achievement. To

fund this program, each district pays $1000 and receives $3000 from the State of

California for each participant. An estimated 10,000 administrators participated

during the first five years. The crux of this training is a didactic ‘one-size-fits-all’

training, which familiarizes administrators with curriculum and instruction as it

relates to state-adopted texts and standards (CDE, 2006).

The effects of organizational leadership that are not addressed in the Principal

Training program are highlighted in an in-depth study by Policy Analysis for

California Education (PACE), which investigates how California principals acquire

and organize resources in comparison to other states (Fuller, 2006). It reveals that

principals spend a considerable amount of time interacting with parents, attending to

student discipline, interfacing with the district office, working on compliance

requirements, and attending to teacher concerns. Less time is spent evaluating

and/or coaching teachers, assisting teachers in aligning pedagogy, and attending to

student work or classroom performance. An overall comparison of California

5

principals with principals of other states reveals that they spend more time on

activities that are not related to instruction. Even though California principals

reported that student achievement data are a valuable resource, only one–third

reported using this information at least once a month, and rarely do they use this

information to assess teachers’ strengths and weaknesses (Fuller et al., 2006).

Responses to the PACE survey are indicative of the didactic type training

California K-12 principals have received through the state and their districts. When

California principals were surveyed regarding their prime educational goals as well

as that of their superintendent, they ranked basic literacy skills among the top;

although, they believed they were less focused on test preparation in relation to their

superintendent. Four out of five principals surveyed also reported that they have

shifted instructional time to place more emphasis on reading and language arts; two-

fifths and three-fifths of elementary and high school principals, respectively, have

increased the time spent on test prep. Principals also espoused concerns regarding

teacher dismissal, stating that barriers are greater than their peers in other states

regarding firing ineffective teachers, with principals in Title I schools reporting

lower levels of satisfaction in teacher quality. However, when answering the

question regarding the number of teachers they would like to dismiss from their

schools, 75% stated they would dismiss two or less. Although some of the concerns

mentioned above are valid with regard to priorities and tools needed for standard-

based reforms, only a small percentage of K-12 principals, according to the PACE

6

survey, appear to be utilizing data to drive decision-making to improve student

achievement (Fuller et al., 2006).

With respect to new principals, the accountability and compliance toward

meeting the Interstate School Leadership Licensure Consortium Standards by

university preparation programs has been hampered by the elimination of the

accountability piece due to budget constraints. Accreditation visitations were

discontinued and there is little oversight of program to guarantee standards and

program alignment (Darling-Hammond & Orphanos, 2006). Training provided by

university preparation and state programs is thought to be theoretical and unrelated

to the everyday demands that school site administrators face. Furthermore, learning

is arduous in that information received is ill-sequenced, thus making scaffolded

learning extremely difficult (Hale & Moorman, 2003). So broad is the concern that

the National Commission for the Advancement of Educational Leadership

Preparation (NCAELP) was formed in 2001. This organization is composed of

prominent scholars and leaders in the area of educational leadership. Their task is to

evaluate and improve the caliber of educational leadership in the United States.

There are some school districts, however, that have taken vital steps to

establish system-wide plans for the improvement of instruction by: (1) public

acknowledgement of unacceptable student achievement, (2) taking responsibility for

the crisis, (3) stating that the solution will be the responsibility of all stakeholders,

and (4) making long term commitments and providing novel supports (Togneri &

7

Anderson, 2003). Case studies of districts focused on reform, such as New York

City District #2, found the “cognitive apprenticeship” professional development

approach to include peer-learning and formal mentoring, study groups, support

groups to build leadership skills, and monthly one day conferences and institutes,

visitation to other school sites, and one-on-one coaching from the district office,

which includes walkthroughs of school sites. The combination of all of the above

was said to be more valuable than any other training these principals had experienced

(Mac Iver & Farley, 2003, pp. 16-17).

One foundational piece of professional development is that school site

administrators accept and believe that data use is necessary and should be a part of

the school culture (Mac Iver & Farley, 2003). Although research shows that district

leaders accept and promote evidence-based school improvement, it is also known

that obstacles such as lack of technology, lack of access to the right data, as well as

engrained cultural norms about teaching and learning continues to exist at school

sites (Colbert & Talbert, 2006; Ingram et al., 2004; Supovitz & Klein, 2003; Young,

2006). There is, however, a developing body of research that shows that school

leadership is a needed element for successful implementation of evidence based

decision making. Furthermore, those school site leaders who are successful possess

the knowledge and skills, the commitment, and moreover the vision that is necessary

(Kerr et al., 2006).

8

According to conclusions in the study of Mac Iver and Farley (2003),

important content for capacity building in school leaders, by their school districts,

are: advising on curriculum and instruction; recruiting and providing effective

principals and teachers; assisting in the analysis and evaluation of data for the

purpose of instructional changes; and, lastly providing support to administrators so

that data-driven decision making dictates instructional needs.

The accountability piece, or the hallmark, that school site leaders are using

data-driven decision making to lead their schools in student achievement, is the

Single School Plan for Student Achievement (SSP). This document is an annual

requirement that school leaders formulate in conjunction with their School Site

Council (composed of administration, teachers, parents, and community) and submit

to their school districts. The SSP links resources to learning, in other words, goals

are established based on student achievement data, then a budget which is directly

linked to the goals is formulated to support student achievement and ultimately

increase a school’s API. Four precise leadership domains were found to produce a

higher Academic Performance Index (API), and they are as follows: making student

achievement a priority, implementation of a coherent standards-based curriculum,

utilization of assessment data, and ensuring availability of instructional resources

(EdSource, et al 2007). A principal’s behavior in terms of making student

achievement a priority is espoused through a clear vision in setting high expectations

for teaching and learning. As a consequence, implementation of a coherent

9

standards-based curriculum is essential. The school leader must assure that there is

curriculum coherence through school-wide consistency, both horizontally and

vertically. Assessments must be timely and should correlate with curriculum and

state standards. A district that supports their school sites allots resources to support

principals in site level planning, and site level planning is constructed through the

SSP (CDE, 2006).

Formulating a SSP requires specific knowledge and skills by the school

leader, namely: develop avenues to collect and interpret data in an effort to meet

NCLB’s requirements, which includes meeting Academic Yearly Progress (AYP).

This complex process requires disaggregating data, for the purpose of determining

that all subgroups are meeting AYP. In addition to gathering and analyzing data, and

developing an action plan for implementation and monitoring, school leaders must

also be adept in gleaning the bigger picture. This means knowing the current data

and statistics that are available through the state website and being aware of future

developments. The State of California publishes data and statistics collected from

schools in California so that trends and needs can be measured and identified. The

state has a number of items under Data and Statistics, specifically: (1) DataQuest

which provides API, AYP, test data, enrollment, graduates, dropouts, course

enrollment, staffing, and data on English Language Learners; (2) Ed-Data, which

provides school, district, county, and state reports addressing students, staffing,

finances, and performance ranking; (3) pending (2009-2010) will be the Longitudinal

10

Education Data Systems which will allow maintenance of longitudinal data based on

student demographics, program participation, enrollment, discipline, state assessment

results, teacher assignment, and other data necessary to meet state and federal

reporting requirements (CDE, 2007).

With the growing demands and requirements of the federal and state

governments, school districts and school site leaders are expected to have the

knowledge and skills to recognize and implement all the modalities necessary to

meet the pressures of performance-based accountability to increase the quality of

public education for all students. Moreover, it has become crucial for school site

leaders to be proficient in data use and analysis to gauge academic progress (Elmore,

2002). Although research efforts have been made in the area of professional

development for school site administrators in data-driven decision making, the

empirical research is less than exhaustive. Hence, the aim of this research study is to

enhance the existing body of knowledge, with a particular focus on promising

practices of district provided professional development to school site administrators

in data-driven decision making.

Research Questions

The proposed qualitative multiple case study endeavors to extend the

understanding of effective district professional development in data driven decision

making for school site administrators. The study will focus on two districts that are

11

using promising practices in professional development for their school site

administrators in evidence-based instructional improvement.

This qualitative study will speak to the following overarching question:

How do districts build the capacity of school leaders to engage in data-

driven decision making?

In conducting the study, the below-noted sub-questions will also be

addressed:

a) How is the need for professional development assessed?

b) What types of training and support are provided school site leaders?

c) What evidence is there that leaders put new data-driven practices into

place?

d) What knowledge gaps still exist?

Significance of the Study

Professional development of school site administrators in school-wide data-

driven decision making, couched in the current federal and state accountability

systems, is a new area of study; therefore, research in this area is yet to be fully

developed into an exhaustive body of research. This qualitative study proposes to

capture and contribute to this new body of literature, in an effort to assist districts in

providing their school site administrators with appropriate and effective professional

development. The intent is to offer rituals and routines in effective professional

12

development that is not necessarily a mere transmission of knowledge but a base of

knowledge and skills that can be implemented to establish and sustain schools to

incorporate evidence based practices to improve student performance. It also may be

used to provide districts with a road map to support their school site leaders to

confidently acquire the knowledge and skills to implement an evidence-based culture

at their school sites.

13

CHAPTER TWO

Literature Review

There's nothing wrong with setting goals, but it doesn't mean a thing if you don't pay attention to the day-to-day details. --Don Shula, Coach, Miami Dolphins

Introduction

It is certain that the establishment of NCLB and PSAA dictates that an

accountability system is clearly in place to improve student achievement and to hold

those within the system, states, districts, and schools, to certain expectations. The

federal government and the State of California’s publication of Adequate Yearly

Progress (AYP) scores and Academic Performance Index (API) have induced

educators to focus and pay attention to data (O’Day, 2004). As districts establish a

system of accountability for their schools, the building or establishment of social

capital amongst the site administrators and teachers is necessary to form a cohesive

system (Spillane & Thompson, 1997). In essence, this translates into all schools

within the system believing that their work is geared toward incremental gains from

year to year, and that evaluation and monitoring is a matter of policy and procedures.

At the core of the accountability system are data and the ability of individuals

within the system to use them. With the district leaders at the helm and the school

site administrators as the leaders at the school sites, it is incumbent upon the district

to assure that the school site administrators have the knowledge and skills in data-

driven decision making (DDDM) to meet the accountability expectations. The

14

essence of this study is to examine promising practices in professional development

for school site administrators in the area of DDDM. Thus, an exploration of issues

related to DDDM professional development for school site administrators will be

addressed in the following literature review. Three crucial areas will be discussed:

1. An examination of accountability in K-12 education as defined by United

States and California policy and the implication for data use at the district

and school levels.

2. An evaluation of the role of data-driven decision making in K-12

educational reform.

3. An investigation of professional development to build the capacity of

school site administrators in DDDM.

Accountability in K-12 Education

Historically, the nation’s public schools have been held accountable for

complying with federal and state regulations as well as financial management;

however, around the year 2000 the United States government reframed

accountability for public schools. In 1999, some states as well as the State of

California began holding schools accountable for student performance by way of

scores on standardized tests. It is important to note that the term “accountability”

falls under the umbrella of “standards-based” reform, wherein each state is required

to develop standards for each grade level that delineate what a student should know

15

and be able to do. Hence, in a standards-based system, the following policies are

set: each state must have standards for each grade level in the various content areas,

assessments should be developed to measure mastery of standards at year’s end, and

rewards or sanctions based on success or failure are meted out. For the first time in

the history of the United States, the federal government has outlined specific

guidelines for states to follow in formulating their own accountability systems.

States, however, vary in the implementation of the aforementioned reforms via their

state testing, textbook adoptions, and professional development (EdSource, 2007).

Federal Accountability System (ESEA & NCLB)

The current system, NCLB, implemented in 2002, is the most recent version

of the Elementary and Secondary Education Act (ESEA) of 1965. It requires all

states that accept ESEA funding to put into place an accountability system that holds

states and their respective districts and schools responsible for the academic

performance of their students. The federal government expects each state to

demonstrate adequate yearly progress with upward movement in order to meet the

goal of proficiency for all students by the year 2013-2014.

The aforementioned goals of NCLB are quite similar to the accountability

system that California established in 1999; however, NCLB required that the State of

California make more changes to its system and, subsequently, revisions were made

in 2003. The consistencies between NCLB and California’s PSAA are that student

performance is based on standardized test results, which includes specific line items

16

that show performance of numerically significant subgroups. Another commonality

is that interventions are mandated to help those schools that are low-performing or

program improvement (PI) schools. The federal government, however, is raising the

bar for 2008. From the inception of NCLB in 2002, targets have been leveled twice.

In 2008 and for each year thereafter until 2014, there will be a target increase in AYP

goals. It is assumed that states and their districts have had ample opportunity to raise

the capacity at the state, district, and school levels (CDE, 2007).

California’s Accountability System

As noted earlier, prior to NCLB, many states including California

implemented their own performance-based accountability system. In 1999, the

legislature in California approved the Public Schools Accountability Act (PSAA),

which included three areas designed to hold schools accountable for improving

student learning. Shortly thereafter, in June 2003, American Institute for Research

(AIR) completed the first legislatively mandated study of the PSAA, with support

from Policy Analysis for California Education (PACE) and EdSource. In 2004, the

California Department of Education (CDE) contracted AIR to pursue a continuation

study to resolve differences between the state and federal accountability systems

(AIR, n.d.). Some of the respondents in the survey stated that having two

accountability systems (NCLB and PSAA) was overwhelming district and school

staff. Furthermore, it was generally stated that AYP linked to NCLB was a more

challenging target (requiring schools to meet proficiency targets) versus California’s

17

API target wherein the focus is on growth. The report revealed that factors hindering

school improvement were: “. . . the district role in school improvement, the

influence of instructional coherence, and internal school capacity” (AIR, p. I-5).

Other noteworthy findings in the study were:

. . . differences emerged between the data use practices of growth schools and schools that made low levels of growth (including two schools that were also SAIT schools) . . . main difference . . . was the extent of data use to inform instruction. Growth schools reported using data frequently and extensively to guide instruction, while the other schools, more often than not, reported using it to generally identify students not performing well (p. IV-13). While it is true that districts must adhere to federal and state

expectations, a focus on the interdependence between district and school sites’

is crucial for growth within a school system to occur. Hence, the onus to

create a comprehensive school system falls on the district to build the capacity

of those involved in DDDM (Leithwood, Louis, Anderson & Wahlstrom,

2004).

Implications of Data Use at District and School Level

Data have become the avenue by which accountability is ensured on the

federal, state, and district level. Accountability is multi-tiered with various

assessment systems, establishment of benchmarks of effectiveness and targets, and

systems for monitoring and evaluating (Earl & Katz, 2002). Furthermore, numerous

studies have noted significant factors eliciting growth in student achievement, as

evidenced in studies that identified schools that have made consistent or high growth,

18

which include but are not limited to: capacity at the school level, instructional

consistency, regular assessment and data-driven decision making (AIR n.d.; Coburn

& Talbert, 2006; Mandinach, Honey & Light, 2006; Marsh, Pane, & Hamilton, 2006;

Knapp, Swinnerton, Copland & Monpas-Huber, 2006).

In 2004, the RAND Corporation began a study of data-driven decision

making in the Southwestern Pennsylvania school districts. Their purpose was to

study the districts in this particular area for challenges in data use in an effort to

improve school performance to meet state and federal requirements. This study

looked at the steps taken by the districts to develop capacity for DDDM for the

purpose of identifying the process of improving capacity in developing its use. The

study states that DDDM is not always successful, and key factors toward success are:

powerful leadership, sufficient resources, and planning for data collection and its

utilization. An emphasis is placed on the finding that empirical studies on DDDM

reveals school leadership is a required element for implementation to be eventful in

districts (Marsh, Pane, & Hamilton, 2006).

RAND researchers noted that specific types of data are preferred. For

example, administrators find summative tests as great tools for school improvement,

whereas teachers believe classroom assessments are more useful for planning

instruction than are summative assessments. Principals noted that data from state

testing assisted in identifying areas of improvement. Moreover, due to high stakes

accountability, test data are also used to identify “bubble kids,” those students who

19

are shy of reaching proficient on state testing (Marsh et al, 2006). The RAND work

proposes that the majority of educators believe data are useful for informing

improvements in teaching and learning. It is further suggested that DDDM does not

lead to effective decision making; however, the following is suggested:

• promotion of various data at various times

• triangulation of data to find a balance

• analysis of data and actions based on the same

• provide training on data analysis, identifying, and implementing solutions

• allocation of time

• partner with institutions that support data use

• assigning people to filter data

• user-friendly technology and data system

Lastly, a word of caution is made about the negative consequences of high-stakes

state testing and data use, i.e. using data analysis to target bubble kids who are near

proficient (Marsh et al, 2006). It is noteworthy at this point to state that a school

district’s espoused priorities and emphasis play a huge role in the implications and

role of data use at the district and school level. It has also been emphasized in the

majority of the above studies that knowledge and training of leadership is important.

20

The Role of Data-Driven Decision Making in K-12 Educational Reform

Data-driven decision making (DDDM) in education as defined by Marsh et

al, 2006) is: “. . . teachers, principals, and administrators systematically collecting

and analyzing various types of data, including input, process, outcome and

satisfaction data, to guide a range of decisions to help improve the success of

students and schools” (p.1). The inclusion of DDDM in the current K-12 education

reform is a result of previously failed educational reforms (Fullan, 2000). However,

Fullan (2002) also states, “Only principals who are equipped to handle a complex,

rapidly changing environment can implement the reforms that lead to sustained

improvement in student achievement (p.1).” One might then ask about the various

perspectives that have hampered the process of reform.

Data-driven decision making (DDDM) can be an integral part of district and

school accountability, though it is also seen by many as simply the “latest fad.” It

refers to policies and practices regarding student achievement and other data (such as

demographics, attendance, and graduation rate) that guide school improvement at the

state, district, and school levels. Studies have shown that DDDM is seen as having

substantial potential to advance efforts in order to improve learning outcomes

(Datnow et al, 2007; Darling-Hammond & Orphanos, 2006; Elmore, 2002). The

purpose behind DDDM is that analysis of student data will enable schools, districts,

and states to target areas where progress is needed. Furthermore, DDDM serves as a

gauge to evaluate and monitor success or failure of the action plans put into place to

21

improve a given system whether at the district or school level (Elmore & Rothman,

1999). Datnow et al (2007), in a study of leaders in the schools and systems, argued

that DDDM should be a norm via “. . . stating explicitly that data use is non-

negotiable” (p. 71).

From an organizational perspective, the superintendent, as the head of the

organization, can promote a culture of data use through professional learning

communities. Such data discussions at the administrative level build capacity for

school site administrators to continue the process at their respective school sites.

This may entail school leaders forming distributive leadership at their school sites

and providing the foundation for collaborative data discussions where grade level

chairs lead their grade level in activities. Such activities include conducting common

assessments and analyzing data in order to formulate action plans that allow for

adjustments in instruction (Schmoker, 2006). “A principal who is data-driven or

technically savvy can exert substantial influence on the faculty, communicating the

importance and thereby stimulating use (Mandinach et al, 2006, p. 13).” A

principal’s capacity to implement reforms, however, is a direct result of district best-

practices in building the capacity of its school site administrators. Best practices to

build capacity within a school district include, but are not limited to, the

implementation and training in data tools for effective DDDM curriculum and

instructional practice, the effective recruitment practices of principals and teachers,

supporting schools in analyzing data and developing action plans through the

22

offering of administrative support. One must also be cognizant of the fact that the use

of DDDM is a tiered process to be used at the district, school, and classroom levels

(Mac Iver & Farley, 2003). Consistency at all levels is necessary for the

development of a comprehensive school system in DDDM.

Some of the barriers to establishing a comprehensive system are espoused by

Coburn and Talbert (2006). In a two-year study wherein research data were

collected from a large K-12 urban school district, it was found that individuals

respond differently to NCLB and PSAA policy pressures. The response is fashioned

by beliefs related to evidence and research as well as previous reform policies.

Furthermore, responses to accountability policy differ at the various levels of a

school system creating different views among district leaders, school administrators,

and teachers. In other words, the issue at hand is the coexistence of conflicting or

different views at the various levels that lend to or hamper the use of data to make

decisions that will ultimately improve the quality of education for students (Coburn

& Talbert, 2006).

A symbiotic relationship between the various levels within the system is

discussed in a study, entitled How Leadership Influences Student Learning

(Leithwood et al, 2004). It is stated that a comprehensive and coherent educational

system is possible via: establishing directions by charting a course that is understood

by all; establishing high expectations and utilizing data to track performance and

progress; developing people in the system to succeed through support and training;

23

and, ensuring that all conditions and incentives throughout the system support rather

than inhibit teaching and learning.

Well-informed school site leadership in data use can be powerful. Supovitz

and Klein (2003) examined how innovative schools systematically used student

performance data to guide school improvement. Upon doing so, they discovered that

discussions on student outcome centers and connects to issues of curriculum,

teaching, equity, professional development, school organization, use of time, and

role of parents/community. When deciphering what influences student gains, one

must look at all the indicators. Furthermore, it is up to the instructional leader to

bring forth the totality of the organization through the espoused vision. With the

school at the heart of instructional improvement, it is safe to say that principals are

those educational leaders who make the difference; however, it is not without

distributive leadership within the school, such as lead teachers, coaches, assistant

principals and department chairs that this effort can come to the fore (DuFour, 2007).

Supovitz and Klein, however, conclude that the American system of education does

not allow for the structures and avenues to engage in the in-depth inquiry necessary

to improve student learning outcomes.

In another study related to leadership and building support structures

conducted by Springboard Schools (2007), entitled Bringing the State and Locals

Together: Developing Effective Data Systems in California School Districts the

following policy were noted: data are crucial to improving teaching and learning;

24

the State of California’s data system is fragmented; and, the state does not provide

appropriate support and resources for its school districts to develop comprehensive

data systems. Specific recommendations for California include, but are not limited

to:

• Invest the resources to develop an integrated state data system.

• Provide funding for districts to develop or maintain local data systems.

• Develop web-based guidelines for best practices, and provide liaisons to

work with clusters of individual school districts.

Springboard made some of the following recommendations for districts:

• Monitor progress in data-based improvement goals through appraisal of key

district leaders and principals.

• Set goals for development of district data system by including all

stakeholders.

• Include in district’s data plan - linking data from student information system

to assessment data.

• Invest in resources to attain technological capacity to collect, organize,

analyze and report data.

• Encourage data accessibility, ownership, and ease of use at multiple levels of

the system (district, school leader, and teacher level).

• Invest in human resources (teachers and administrators) who can help in

developing capacity of effective data use.

25

In summation, the Springboard report stresses the importance in investing in

human resources to build capacity and that a high quality data system is necessary to

support school improvement. Hence, a recommendation is made for policymakers,

at the local and state levels, to invest in policies that help to build such

infrastructures (Oberman, Hollis & Dailey, 2007).

With student learning as the pinnacle of school improvement, the American

system, thru states and policymakers, have responded by targeting efforts that have

an exclusive connection to student learning, such as: the recruitment, training, and

credentialing of teachers; curriculum and instruction aligned to standards and

textbooks; class size reduction; and, of course, testing. Consequently, these efforts

have triggered a demand by the state and district for principals to implement as well

as monitor the above without considering the capacity of principals’ knowledge and

skills to manage such reforms. The role of the principal in these regards could

become quite daunting and was essentially overlooked in the 80s and 90s (Darling-

Hammond & Orphanos, 2006).

Based on the aforementioned, school site administrators are the liaisons

between the district and schools in forming a comprehensive school system. Vital to

this study is an exploration of issues related to professional development, by the

school districts, for their school site administrators. In the following sections I will

review literature on professional development for school site administrators based on

the need for them to have a firm understanding of their school’s data, be able to

26

analyze them, and then reveal the evidence and produce the action plan necessary for

school improvement. This includes the school site administrator’s ability to create a

school culture and increase the capacity of teachers to use data in making decisions

about learning and instruction.

Professional Development to Build Capacity of School Leaders

We are on a long journey, and the trip has just begun. --Charles Savage, Leading Thinker

The impetus of the NCLB and PSAA accountability systems has placed new

demands on school site administrators. In essence they must lead organizations that

effectively teach a spectrum of students (numerically significant subgroups), who

have a variety of needs. Furthermore, they must simultaneously meet the targets set

by federal and state government. Coburn, Honig, and Stein (in press) note that

policy makers pressure districts to use evidence in decision making. They also note

that in a number of studies, administrators have been noted to make most decisions

without the use of data. In other words, data use has no role in decision making. In

another study done by Corcoran, Fuhrman, and Belcher (2001) it was noted that none

of the districts that were evaluated utilized data to make decisions about professional

development. It should be noted that the routine of data use is not simply to analyze,

interpret, and utilize data to formulate an action plan for the SSP, but to create a

culture of data use at the school site (Datnow et al, 2007). Thus, the most crucial and

27

most difficult investment is the development of human capital, which is necessary

for the development of data literacy and analysis skills. This requires school site

administrators not to simply be managers but to be organizational leaders with highly

developed skills to understand and fashion an organization and lead it in

organizational change (Darling-Hammond & Orphanos, 2006).

Opportunities to gain new knowledge, skills, and tools to develop the

repertoire of a school site administrator’s effectiveness in instructional leadership are

dependent on effective professional development. Butler (1992) states that effective

professional development is dependent on three areas: (1) characteristics and needs

of the learner, (2) characteristics of professional development vis-à-vis purpose,

structure, content, process, and follow-up, and (3) characteristics of the organization

in terms of support (p. 1). Furthermore, for professional development to be effective,

six conditions must be met:

• Individuals must feel the need to learn.

• Learner must be able to make a connection to past and present experiences to

see the value of new learning.

• Learning must relate to the individual’s stage of development.

• Autonomy is calculated based on method or mode of new learning.

• Climate minimizes anxiety and encourages experimental freedom.

• Learning styles are considered.

28

In essence, school site administrators must be able to visualize the professional

development as part and parcel of their professional life and not as a remediation

with implications of inadequacy (Butler, 1992).

School site administrators’ professional development, however, has for the

most part taken the form of transmitting procedural knowledge which often times

does not pertain to practical application. This type of professional development

leaves recipients in a vulnerable position, one that insinuates dependence on external

sources rather than empowerment and ownership (Schmoker, 2006). Schmoker goes

on to state that professional development is “. . . often bad beyond hope” (p. 100). In

a review of literature, wherein he summarizes several studies on school districts, he

notes that successful districts are those which spend heavily in human, financial, and

technological resources. The states’ funding priorities may affect districts’ resource

allocations.

The State Role in Professional Development

Since the district is the major intermediary between the school and the state,

the state must ensure that the district has the necessary resources (Blasé & Blasé,

2004). When considering the number of California schools that are

underperforming, the state should provide the district the necessary support and

financial wherewithal. This would enable districts to provide the knowledge and

skills in state and benchmark assessment data use to determine the needs of students

and ultimately to provide appropriate instruction. Resources should also be provided

29

to allow districts to supply its schools with the appropriate technology and user-

friendly tools to decipher student results and specifically target areas of need. The

state can also provide incentive structures for those low-performing schools that

require experienced and motivated administrators, teachers, and coaches (O’Day, J.,

Bitter, C., Kirst, M., Carnoy, M., Woody, E., & Buttles, M., et al., 2004).

The State of California has decided not to allocate all professional

development resources to its districts. It has taken some responsibility for

professional development of school site administrators by offering AB 430. It is the

only state initiative focused at principal professional development with modules 1, 2,

and 3 (a total of two weeks), but can be taken over a two year span. This didactic

one-size fits all training costs the district $1000 and for each participant the district

receives $3000. The bulk of the training acquaints administrators with curriculum

and instruction as it relates to state-adopted texts and standards (CDE, 2006). Such

learning is learning out of the environment where situations take place; hence, it is

not the type of application learning that makes a true effect (Fullan, 2002). Also,

given that the state is detached from the school’s situation, it is incumbent upon the

state to directly provide monetary resources to the districts so they may provide its

school site leaders with contextual organizational leadership knowledge and skills.

In an attempt to study issues of leadership development in the State of

California in relation to the nation, Darling-Hammond & Orphanos (2006), utilized

data mostly from a national study that included a national survey of approximately

30

1000 principals, with an oversampling in eight of the states including the state of

California. The study consisted of a review of policy documents, literature, and

interviews of a wide array of stakeholders from policymakers to principals and

superintendents, and individuals in professional associations, university preparation,

and professional development programs. They also studied eight leadership

development programs, which they deem exemplary, documenting their strategies

and results. Findings revealed that in order for the State of California to reach its

“ambitious aspirations for raising student achievement” that major systemic changes

and investment in knowledge and skills must be made. Specifically the

recommendations are: knowing the research-based practices in teaching and

learning as well as clinical experience; appropriate allocation of resources -- time,

human resource, and money; skill-building of leaders, teachers, and other staff

through effective evaluation, feedback, and professional development; and,

management of a successful school improvement process through the use of data

(Darling-Hammond & Orphanos, 2006).

Other principal professional development programs through universities,

professional associations, not-for-profit organizations, and government agencies

were analyzed for focus of program, purpose, curriculum, instructional strategies,

coherence, and technology by Peterson (2002). Peterson states that at the very

minimum a conceptual frame such as the state standards should serve as a template

and a closer link should be made with preparation program in order to enhance

31

learning. Also the length and structure plays a critical role, wherein most

professional development is a one-time workshop – all day and sequential meetings

over a period of a year is a more productive approach. Many programs are described,

but two in particular are reported to have a comprehensive approach with a well-

designed structure for principal professional development: (1) California School

Leadership Academy (CSLA), a statewide program and, (2) CLASS, a program

which serves Chicago principals.

CSLA currently offers three primary programs: Foundation 3.0, School

Leadership Team, and Ventures. The Ventures program deals more with

experienced principals and focuses on skills and knowledge associated with

transformational leadership. Through an ethnographic research method, principals

study their own school setting for a three year period (Peterson, 2002). Through the

direction of CSLA facilitators, principals document transformation in school culture

and their roles as leader. Program design is in three phases:

Phase 1, participants define their “theory of action” and their “field of action” and the focus of their change efforts. In Phase 2, participants begin implementing a transformation strategy in their schools based on their analysis of multiple sets of data. In Phase 3, participants present an exhibition of their achievement to colleagues. This includes a narrative analysis of the change process in their school and data that tracks (Peterson, 2002, p. 224). CLASS, the other program deemed comprehensive, is a joint effort between

the Chicago Principals and Administrators Association (CPAA) and the Chicago

32

public schools. The three programs that are offered are sequential and provide

breadth and depth. They are: (1) LAUNCH, for aspiring principals; (2) LIFT, for

1st-year principals; and, (3) Chicago Academy for School Leaders (CASL) for

experienced principals and other administrators. The thrust of these programs are to

assist principals to develop skills and knowledge that coincide with governmental

reforms of the 1990s. Thus, it is standards-based and related to the ISLLC standards.

CSLA and CLASS are comprehensive long term professional development,

which allow for breadth and depth. These programs are unlike on-the-job in service

trainings which are offered to administrators as a quick look at the requirements for

the position. Also, on the job trainings do not offer ample time to learn everything

about instructional leadership as it relates to DDDM and accountability. Only when

school site administrators feel capable and secure in DDDM will they then ensure

that this filters down to the classroom (Heritage & Yeagley, 2005).

Districts along with professional educational organizations have been

intermediaries for the state by collaborating to form long term professional

development programs that would allow administrators to acquire the sustained

learning necessary to conduct data-driven decision making. A smooth transition

between the federal mandate of NCLB, the states, and the districts to effect

significant change in the classroom has been a slow and arduous process. States have

attempted, with great difficulty, to be the key players in this process. The districts,

33

however, must play a major role in the systemic change that would ultimately effect

improvement in student outcome (Darling-Hammond & Orphanos, 2006).

District’s Role in Professional Development

If professional development is to have relevance, then it should be done

tandem with activities that reflect instructional improvement and should not be

detached from its organizational system – the district. Fullan (1990) confirms that

the learning should not be in isolation, but rather to produce individual and

organizational habits and structures which encourage learning on a continuum. The

relationship between the district office and its schools with relationship to student

achievement was the focus of a literature review by Mac Iver and Farley (2003). It

was noted that professional development for teachers and principals is one of four

important district roles. They found that there was consensus in the literature

regarding professional development for principals and teachers in the interpretation

of data in order to make sound instructional decisions. They further note that more

longitudinal research is needed in the area of professional development strategies

provided by the district to its school site administrators.

The importance of the district role in developing principal capacity is also

noted in an analysis of district funding applications by Glass (2002). In a descriptive

analysis of what districts identified as specific needs, principal improvement was

noted. Districts that included principal improvement as a need stated that

professional development should be provided in the areas of: instructional strategies

34

(24%), team leadership (24%), data driven decision making (16%), and in alignment

of curriculum (16%). Of note, however, is that districts must be aware of the need to

build organizational culture simultaneously with professional development to assure

a meeting of the minds.

In order for district leaders to evaluate the culture of the organization, they

must ask, Are school site administrators actions based on intrinsic reasons or because

they must conform to external pressure? Furthermore, if districts expect data to be

utilized in the improvement of service to students, then pressure for use must be

accompanied by external mandates that are couched in a mind-shift that DDDM is

part of systemic or organizational improvement that empowers educational leaders to

have a collection of tools to plan and make decisions (Darling-Hammond, 2002).

The immediate implications are that insights and developments are unveiled by the

evidence of new knowledge via data, wherein it is shared, meaning is constructed,

and school site leaders set the pace and cycle for data literacy, inquiry, and action.

Also, an injection of reality is much needed – the reality that we live in an age of

information technology and with this comes not only data but data that must be

formed into knowledge (Earl & Katz, 2002). Creating this type of understanding

requires skills of transformational leadership.

Enhancing the quality of schools via school-based principal professional

development is the focus of Leaders and Leadership by the National Staff

Development Council (n.d). The following recommendations for district offices are:

35

• Encourage principals to utilize distributive leadership and evaluate principals

on their ability to establish a culture of collaboration;

• Rethink approach for determination and development of principals, such as

leadership certification through district academies that entail an academic

base as well as superior learning experiences wherein districts and

universities partner.

• Apprenticeship model, such as The Aspiring Principals Program run by

Dennis Littky of Rhode Island's Big Picture Company. Program trains

aspiring principals through a one-year apprenticeship with a celebrated

principal simultaneously with an academic component through a university.

• Coaching to assist principals to remain focused on instructional goals while at

the same time achieving balance in life.

• Superintendents design and lead professional development that requires

principals to spend more time in classrooms as well as demonstrate

knowledge and skills needed to use with teachers.

• Develop programs to help teachers become instructional leaders.

• Districts reconfigure time of school day to allow for professional

development that will provide opportunities for team meetings focused on

instructional improvement, and time for teacher leaders to facilitate, plan,

mentor, and coach other teachers.

36

The above is a collaborative model that forms interdependence between

district and school site administrators – a model that is conducive to building a

comprehensive system (Bloom, 2003). Eliciting and obtaining feedback from school

site administrators are key in strengthening the relationship.

Building the capacity of principals to be instructional leaders was studied by

Fink and Resnick (2001) who examined districts' efforts to create leaders who could

accomplish sweeping turns in language arts and mathematics. Essential capacity

building strategies described in their study were: nested learning communities,

principal institutes, leadership for instruction, peer learning, and individual coaching.

Datnow (2005) suggests that institutionalization which leads to sustainability

involves a layered process wherein structural routines and rituals are built into the

organization. It was further noted that the quality of a site leader should be that of “a

savvy political leader,” in an already well-institutionalized setting. Datnow also

emphasizes that reform is a result of state, district, school, and classroom forces all

acting together to shape long term sustainability.

Sustainability ought to begin as part of the district plan. Elmore and Burney

(2000) suggest that a district’s requirement for instructional leadership falls under its

strategic plan for instructional improvement. In a study conducted of six new

principals in Community School District #2 in New York City, Elmore states that the

district’s plan for massive instructional improvement consisted of professional

development that focused on instruction for teachers and principals. It is further

37

noted that a focus was placed on the need of furthering knowledge, skills, and

professional development for principals. This district’s practices included:

• Principal Conferences - monthly day long conferences wherein agendas were

established to provide principals with direction in which to lead their school

sites.

• School-Based Staff Developers (direct work with teachers) that address one

content area at a time.

• Principal Site Visits – one to four times per year district personnel visit

school sites to monitor progress and provide guidance.

• Principal Study Groups – topics of discussion include conducting effective

staff meetings, long term planning, intervention practices, etc.

• New Principal Support Groups – one type is a mentor/mentee relationship

which falls under the guidance of district personnel and the other is to

procure direct support or assistance from district personnel to assist with

instructional and administrative issues.

Elmore concludes from this study that despite the background and experiences of the

new principals, they all have the foundational pieces to carry out the agenda of the

district on improvement of instruction. Strong organizational leadership with a

vision that entails continuous learning through a comprehensive professional

development plan will set the path toward improvement efforts.

38

Leadership and Learning Organizations

In the Leaders New Work Building Learning Organizations by Peter M.

Senge (1999), he states at a conference sponsored by MIT entitled, “Transforming

Organizations,” that there were two reoccurring questions: “How can we build

organizations in which continuous learning occurs?” and “What kind of a person can

best lead the learning organization?” These questions are extremely relevant in a

public education system that demands accountability wherein data analysis is

continuous, requires constant evaluation, planning, and action. Senge notes that the

leader of a learning organization requires new skills and tools.

A leadership development model approach, fashioned after that of the

business world, is showcased repeatedly in Senge’s work. He describes three types

of leaders: (1) the local line leader, (2) the executive leader, and (3) the internal

networkers. These levels of leadership are analogous to distributive leadership,

wherein the executive leader supports the local line leaders, formulates the learning

infrastructure, and leads by example in the steady process of developing the customs

and behaviors of a learning culture. Senge (2001) has coined the term ‘learning

organization’ to mean "a group of people continually enhancing their capacity to

create what they want to create” (Senge, 2001, p. 125). Simply put, it is an avenue to

an endless process of increasing improvement capacity – instructional improvement

if you will in the case of public education institutions. However, one may wonder if

this is at all possible in public education where districts and schools are constantly

39

reacting to external pressures? Is it possible to somehow disengage from these

pressures and develop thinking patterns and collaborative interaction to advance an

organization that is always learning?

Leadership Development Models

Fullan (2002) contends that “learning at work — learning in context occurs,

for example, when principals are members of a district's inter-visitation study team

for which they examine real problems — and the solutions they have devised — in

their own systems” (p. 19). This type of situational learning fosters shared

knowledge and commitment, improving the social context of an organization. It also

allows selective preservation of ideas and a social milieu for performance

monitoring. This type of support system within an organization via problem-solving

and strategizing allows for interdependency between district leaders and school site

leaders – it is contrary to the familiar district and school culture wherein individuals

are isolated (Fink & Resnick, 2001). This would require relationship-building to

foster trust amongst colleagues, an endeavor which requires time and effort.

In an effort to place an emphasis on relationship-building and trust, but more

importantly as a means to build around specific needs of school leaders, some

districts have taken on the coaching model of professional development for their

school site administrators. This is not to be misconstrued with mentoring, which is a

senior coaching a novice. CLASS (Coaching Leaders to Attain Student Success),

developed by the New Teacher Center at University of Santa Cruz collaboratively

40

with the Association of California School Administrators is a professional

development specifically for coaching (Bloom, Castagna, Warren, 2003). It is based

on the following:

• Coach brings a different perspective -- sees circumstances and possibilities.

• Relationship in coaching is based on trust and consent.

• Based on assessment of coachee’s needs, coach moves between instruction

and facilitation coaching strategies.

• Coach commitment is to student success and will push coachee with that in

mind.

• Goal-setting and ongoing assessment is based on professional administrative

standards (ISLLC and CaPSELs).1

Bloom et al (2003) further emphasize that coaching could be cognitive,

transformational, facilitative, or instructional (for instance when a principal needs

assistance in gathering, analyzing, and evaluating data). Coaching in this case can be

one way of assisting principals with gaining the knowledge and skills necessary to

make evidence based decisions to improve school quality.

California principals’ perspectives on effective professional development are

noted in a 2006 EdSource survey wherein California principals were asked to rank

training needs. They indicated that utilization of assessment data is their priority for

1 ISLLC (Interstate School Leadership Licensure Consortium) and CaPSELs (California Professional Standards for Educational Leaders).

41

professional development. Other items that followed, particularly of principals in

high priority schools, were evaluation in teacher instruction and a focus on the needs

of English language learners. When surveyed and asked to rank the types of

professional development experiences they’ve had to date, based on their

effectiveness of their day to day practice, they responded:

• Workshops/conferences related to principalship, 76% reported great or

moderate influence.

• Individual or collaborative research on a topic of appeal 70%.

• Partaking in a principal group 62%.

• Mentoring or peer observation/coaching 59%.

Other items on the bottom of the list were district training, other school visits,

university course, and AB 75 training, wherein 47% said it had a great or moderate

influence on their practice (Darling-Hammond & Orphanos, 2006). These results

corroborate previous findings of some of the unsuccessful state and district trainings

(Darling-Hammond & Orphanos, 2006; Fullan, 2002; Schmoker, 2006).

To understand the effects of a school site administrator’s leadership it is

important to gain the teachers’ perspectives as well. Teachers’ perspectives on

effective instructional leadership in a study by Blasé & Blasé (1999) reflect

similarities to the above principals’ perspectives. Teachers note six strategies that

principals use to promote teaching and learning in schools and to increase teachers’

professional development:

42

(1) Emphasizing the study of teaching and learning.

(2) Supporting collaboration efforts among educators.

(3) Developing coaching relationships among educators.

(4) Encouraging and supporting redesign of programs.

(5) Applying the principles of adult learning, growth, and development to all

phases of staff development.

(6) Implementing action research to inform instructional decision making.

Of particular note is the “implementing action research to inform instructional

decision making” (Blasé & Blasé, 1999, p. 135). Principals conducted professional

development through action research projects with a focus on class and school-based

data. This type of professional development takes place with the understanding that

proper determination of the effects of teaching and learning need to be centered on

careful collection of data in order to properly diagnose challenges. This research

further states that the process continues with an exploration for different solutions,

consensus to act, and the careful monitoring of what worked and what didn’t. The

process and cycle continues on either the same challenge or addressing a new one.

In conclusion, it was noted that such efforts by principals is in the early

developmental stage and data did not reflect a strong effect on teachers (Blasé &

Blasé, 1999). Hence, there is a need for districts to acquire comprehensive

professional development for their school site administrators in organizational

leadership that entails evidence-based decision making.

43

Professional development for school site administrators, based on the above

research, is fragmented in the State of California with the State and the districts

providing what they believe is best for administrators. Also cited in the research are

some long term professional development models that allow for breadth and depth

based on sequencing and continuity for a prolonged period of time. It is yet to be

known whether such efforts directly affect classroom instruction and improvement in

student outcome. Thus, there is a need for continued research in effective

professional development for school site administrators for the purpose of increasing

student outcomes. Security in the practice of evidence-based leadership includes

many of the elements mentioned above, such as: leadership focus, values, action

theories, and the availability of data and their attendant literacy (Knapp et al., 2006).

Summary of the Literature Review

The public school system requires special kinds of leaders – organizational

leaders – who possess the knowledge, skills, and motivation that are fundamental in

carrying out the task of instructional leadership in today’s high-stakes accountability

era. The district, although confined by funding allocations from the state, is

responsible for implementing a strategic plan which is directly aligned with student

performance and priorities, central to language arts and mathematics instruction. A

district superintendent’s resolve to accomplish a strategic plan that increases student

achievement requires a team effort, necessitating the support of district and school

44

site personnel. Further, the philosophy that rigorous achievement standards,

curriculum, instruction, assessments, capacity development, management and

organizational strategies are vital to raising achievement must be pervasive

throughout the schools (Spillane & Thompson, 1997).

Such ambitious endeavors will require school districts to take a

comprehensive and systematic approach to developing habits of mind wherein

everyone shares the belief that “. . . to make responsible decisions, workers in any

organization must have a steady flow of information about their work and its

outcomes and continual opportunities to build their knowledge” (Darling-Hammond,

2002).

As cited in the literature review, a systemic, comprehensive, and well

structured approach to professional development for school site administrators is

necessary to meet today’s challenges (Peterson, 2002). There also must be an

understanding that accountability happens at all levels ranging from the federal

government all the way to the school site; hence, an approach that includes the

attitude of “What gets monitored, gets done” can make a significant difference in

evaluating what works and what doesn’t (DuFour & Burnette, 2002).

In a search and review of the literature to find how school districts prepare

their school site leaders to lead via data-driven decision making, it was discovered

that various methods and approaches to professional development have been

attempted. The bulk of the cited research concludes that indeed professional

45

development in data-driven decision making is necessary for school site leaders to

effectively improve student outcomes.

The federal government, through NCLB, will have a target increase in AYP

goals beginning in 2008 and increasing each year thereafter until 2014 (CDE, 2007).

School districts must not only raise the standards for principal performance, but

assure that principals receive a highly skilled professional development program that

is comprehensive and continuous (NSDC, 2008). Based on the above, I intend to

study two school districts in the State of California who have comprehensive

professional development programs in data-driven decision making for their school

site administrators.

I will view the professional development programs in these districts through

the framework of Knapp et al. (2006), Element of Data-Informed Leadership, which

offers ways of recognizing and understanding what is or what is not happening in a

particular setting. The following is a visual of the framework:

46

Figure 1. Data-Informed Leadership, Knapp, Swinnerton, Copland, Monpas-Huber (2006)

Furthermore, the districts being examined will attribute their success of

evidence-based decision making practices at their school sites to their efforts in

professional development. I anticipate finding strong leadership at the district and

school site levels that value data use and its attendant effect on student achievement

with the belief that “. . . schools need accurate and actionable information about what

students know and can do so that they can plan effectively for student learning”

(Heritage & Yeagley, 2005, p. 320).

The intent of this study is to add to the literature as well as to provide a

roadmap to understanding the comprehensive and systematic process in which

47

districts conduct professional development training for their school site

administrators in data-driven decision making. The aforementioned literature review

provided a broad perspective in such areas as models of professional development as

well as some longitudinal studies in data-driven decision making or leadership

training. This study aims to answer the “What works?” and “How does it work?”

questions through in depth multiple case study of two districts, wherein an effective

professional development plan in DDDM for the purpose of increasing student

achievement has been carried out.

48

CHAPTER THREE

Methodology

This chapter explains the research design, which includes the proposed

qualitative methodology, sampling, instrumentation, data collection plan, and

proposed data analysis.

The intent of the researcher is to spotlight districts that affirm promising

practices in professional development for their school site administrators in data-

driven decision making (DDDM) for the purpose of increasing student achievement.

Two districts in the State of California were studied in an effort to find answers to

the following research questions:

How do districts build the capacity of school leaders to engage in data-driven

decision making?

• How is the need for professional development assessed?

• What type of training and support is provided school site leaders?

• What evidence is there that leaders put new data-driven practices into

place?

• What knowledge gaps still exist?

Research Design

The intent of this study is to employ qualitative research methods to provide

an in-depth description of the policies and procedures deployed in carrying out

49

effective professional development for school site administrators in DDDM.

Through a qualitative design, it is hoped that the, ‘What is effective?’ as well as the

‘How does it work?’ questions are answered. The strength of the study comes from

district and school administrator interviews, professional development observations,

and document analysis (Patton, 2002).

The above forms of data collection and analysis procedures are for the

purpose of: understanding the type of professional development program offered by

the districts to their school site administrators; discovering specific district actions

and strategies to promote data use at the school sites; ascertaining the outcomes of

the trainings; and, to determine the type of monitoring and follow-up on the part of

district leaders.

The multiple case study design, one of various forms of qualitative research,

is specifically used when studying educational innovations (Merriam, 1998). This

type of approach is valuable when questions of “how” and “why” need to be

answered and the recommended method is via contextual conditions which are

specific to a phenomenon of study. Furthermore, the multiple case study design is

conducive to answering research questions that require multiple data sources. This

influenced the choice of multiple case study design as the best qualitative research

method for this inquiry (Merriam, 1998). The choice of a small select sample to

research, that of two districts, resulted from the opportunity to obtain sensitive and

50

descriptive data rather than that which would have been collected in a large-scale

study (Merriam, 1998).

Furthermore, the qualitative descriptive-analytic multiple case study research

method encompasses the use of interviews, observations, and document analysis to

collect data. Merriam (1998) describes the case study method to be well suited for

inquiry in research that intends to examine a process in its actual context. The intent

of this multiple case study is to provide an in-depth description of the processes

involved in carrying out professional development to school site administrators in

data-driven decision making from the perspective of select districts and school site

administrators. Merriam (1998) further notes that case study is conducive to

understanding a phenomenon from the perspective of the participants. My decision

to examine two school districts and their selected school site administrators is based

on my wish to acquire insight into the districts’ and school site administrators’

perspectives on the professional development instruction as well as the process

followed to increase capacity in DDDM. Merriam (1998) states that case studies are

the primary designs to gain insight, discovery, and analysis, rather than hypothesis

testing. Thus, an acquisition of insight through discovery and analysis of two school

districts and their respective schools in school site administrators’ professional

development in data-driven decision making has been conducted. A case study

approach has allowed me to discover the process (Merriam, 1998). Process includes

a description of the context and the people in the studies, but most importantly, to

51

describe the development and practices by which data-driven decision making

professional development began and continues to evolve within the districts and their

respective school sites.

Sample

This study focuses on two California school districts and selected school sites

within the districts. Intensity sampling was sought in order to choose units of

analysis that would yield the most information and have the greatest impact on the

development of knowledge and skills of professional development for school site

administrators in DDDM. A smaller number of districts has allowed for a more in

depth study, particularly when the samples are information rich. The validity or

meaningfulness of the study is dictated by the richness of the information procured

rather than by the size of the sample (Patton, 2002).

Since a multiple case study was used, the researcher can provide a detailed

account of each. This allows for cross-case comparison, either in a chart summary or

prose (Creswell, 1998). Multiple cases are preferable to single cases, specifically

when cases may not necessarily represent the range of behaviors and/or profiles,

experiences, outcomes, or situations desirable. However, the inclusion of multiple

cases has limited the depth by which each case is analyzed and has also had

connotations for the structure and length of the final report.

52

Through the utilization of purposeful sampling, districts that have an

effective and comprehensive system-wide professional development in DDDM for

school site administrators were selected. Information about districts that fit these

criteria was solicited from county education offices as well as researchers in the field

of education. Districts’ professional development information was also verified via

school district publications. Four school districts were initially considered, and two

districts were chosen as the units of analysis.

Selected districts and their respective school sites were chosen because they

attributed the utilization of DDDM by school site administrators in part to a

successful professional development program. Selection criteria for districts deemed

effective will include proactive rather than reactive measures in their strategic plan

for professional development, which may include a designated individual who plans,

organizes, and conducts professional development for the district’s school site

administrators. Selection criteria for school site administrators within each district

were based on their effectiveness in carrying out the district plan of professional

development in DDDM. Considerations were also made in a district’s willingness to

solicit and network with outside sources in the development of a comprehensive plan

(Patton, 2002).

53

Overview of the Districts

The two school districts, Buck Unified School District and D’Angelo Unified

School District2, were asked to allow me to visit their district offices and school sites

to collect information for this study on the districts’ professional development

program for school site administrators in DDDM.

Buck Unified School District

Buck Unified School District (BUSD) was formed in 1965 when the

Banfield, Universe, Cattle, and Kingston School Districts unified and became known

as Buck Unified School District, and educates approximately 21,000 students. It is

an urban school district that serves the communities of the cities of Amber, Coral,

Hue, as well as portions of Lake, Lapis, and Naples. The District has received

county, state, and national recognition for outstanding programs in counseling,

alternative education, staff development, and labor relations. Staff members have

been selected to participate on state and national educational committees and have

been invited to make presentations at national, state, and local conferences. Students

have been recognized as National Merit scholars, academic decathlon winners, and

participants in the Model United Nations program. Approximately eighty-five

percent of the graduating students go on to higher education.

The Buck Unified School District is governed by a seven member Board of

Education. The district includes nineteen elementary schools, five middle schools,

2 Pseudonyms are used for the purposes of confidentiality.

54

three comprehensive high schools, a college prep 7-12 school, a continuation high

school, infant/children centers, extended-day care, and adult school. The diverse

ethnicity of the community is a reflection of the district’s K-12 population which is

American Indian/Alaskan Native, Asian, African American, Filipino, Hispanic,

Pacific Islander, Portuguese, and White (CDE, 2008).

To meet the diverse needs of its community, they have incorporated

professional development as a key component in phase III of their strategic plan

(2006-2010). Of note is strategic direction 5, professional growth for all employees.

Within that component, specific attention is placed on encouraging a professional

culture of leadership, providing research based professional development,

customized professional development opportunities, promotion of collaborative

learning communities, and recruiting and supporting high quality professionals

(Buck Unified, 2008).

D’Angelo Unified School District

D’Angelo Unified School District (DUSD), formed in 1879, serves a

culturally diverse student population of approximately 30,000 students. The district

serves the entire community of D’Angelo as well as a small portion of La Cross

Flamingo, Lake, and Rose Garden. More than fifty percent of its schools have

received awards of excellence such as National Blue Ribbon and Distinguished

School awards from the U.S. Office of Education and the State Department of

Education, respectively.

55

The district has a Board of Education, a five-member body, which governs 20

elementary schools, four middle schools, three comprehensive high schools, a

magnet high school, one continuation high school, and a developmental center for

multi-handicapped students – a total of 30 schools. DUSD prides itself on an

educational program which is innovative and states that it is part of a community that

is committed toward public schools with more than 72% of voters in recent years

approving a $186 million bond measure to renovate schools.

Furthermore, DUSD has a comprehensive strategic plan which includes

professional development for its employees. Professional development, direction 3

in the strategic plan, notes that it is based on ‘research and proven learning theories.’

It is offered to all employees and it is meant to ‘. . . inspire excellence; promote

skills, knowledge and areas of expertise; and enhance services to students, ultimately

resulting in increasing student learning.’ Professional development at DUSD is said

to be tailored to the specific needs of individuals, allowing time for partnerships,

peer-mentoring, analysis, reflection, and refinement. Such promising practices are

also said to be communicated in groups district-wide. Professional development

targets for individual school sites are based on state and national standards and are

aligned to the school site plan. It is expected that sites produce evidence of meeting

targets via site plans, survey data, and calendar of events. Accompanying those

targets are developed district strategies to support schools in their effort to increase

student achievement. District strategies include: professional development based on

56

site needs, research, and data; resources; pre-service, in-service, and administrative

support; follow-up support for continual planning, reflection, and implementation of

professional development activities.

Data Collection Procedures

According to Merriam (1998), case studies usually rely on the three strategies

of interviewing, observing, and analyzing documents. Usually one or two strategies

are used more than the others. In this study, the primary strategy employed is

interviewing. Observations of professional development and document analysis

were used to support methods for collecting data as well as for the triangulation of

data.

The process of data collection began with individual semi-structured

interview of school district leaders that included: Superintendents, Assistant

Superintendents of Education Services, and Directors. Interviews were conducted

with three principals (elementary, middle, and high school) at each district. A total

of thirteen interviews (one hour minimum) were conducted at both districts. Please

refer to table of all interviews conducted.

57

INTERVIEW PARTICIPANTS DISTRICT POSITION

Buck USD Superintendent Buck USD Assistant Superintendent of Academic

Services Buck USD Director of Child Development &

Special Programs Buck USD Director of Schools Buck USD Supervisor of Curriculum & Professional

Development/Magnet Programs Buck USD Elementary School Principal Buck USD Middle School Principal Buck USD High School Principal D’Angelo USD Superintendent D’Angelo USD Assistant Superintendent of Educational

Services D’Angelo USD Elementary School Principal D’Angelo USD Middle School Principal D’Angelo USD High School Principal Table 1. Interview Participants in Buck and D’Angelo Unified School Districts (2008).

The interviews with district leaders focused on the strategic plan for

professional development vis-à-vis data-driven decision making for school site

administrators, the districts’ method of execution, its monitoring and evaluation

process, and the attendant results. Each interview was a minimum of 60 minutes in

length, tape recorded, and fully transcribed. The protocol for the district leaders’

interviews is included in Appendix A.

The interviews with school site leaders, three from each district, focused on

soliciting their understanding of the district’s strategic plan for professional

development vis-à-vis data-driven decision making for school site administrators, the

districts’ process or method of execution, their evaluation and attendant use of the

58

knowledge and skills gained from the trainings, and the results with respect to the

implementation of DDDM at their school sites. Each principal interview was also

conducted for a minimum of 60 minutes, tape recorded, and fully transcribed. The

protocol for the principal interviews is included in Appendix B.

Data collection included observations of professional development for school

site administrators at the district office. Observations were for the purpose of

determining the types and extent of discussions centered on data use, data-driven

decision making, and knowledge and skills needed in building a culture of DDDM at

the school sites. The extent of the observations was limited to observing and taking

field notes. I was a participant observer; hence, I did not participate in any

discussion. The purpose of the observations and their attendant field notes was to

allow for triangulation with other data (interviews and artifacts).

Lastly, the process of data collection included a third type of data, district and

school site artifacts. Documents collected and analyzed include, but were not limited

to, data binders that included district and school generated reports, state assessment

reports, formal and informal documents from the district or school sites such as

agendas, policies and procedures, and protocols that relate to the districts’

professional development for their school site administrators. When possible, I

photocopied documents for the purpose of cataloging and coding. With all

documents, content analysis was conducted.

59

Data Analysis

Transcribed interviews, observational field notes, and district documents

were coded and analyzed for the purpose of correlating findings with the conceptual

framework of this study, Data-Informed Leadership (Knapp et al., 2006) and its

research questions. More intricate codes were cultivated to ascertain common

themes that span the units of analysis through the use of color coding (Appendix F).

Since I have a variety of data from two units of analysis, utilization of tables and

charts were necessary to organize the process of analysis. Data and document

triangulation was necessary in order to perform a synthesis of lessons learned from

the two sites which will hopefully contribute to the effectiveness in school districts’

professional development in data-driven decision making (Patton, 2002, p. 500).

The process of data coding and analysis was as follows: Prior to reading

interviews, observation notes, and documents, protocols were reviewed, identifying

key phrases that relate directly to the research questions. A list of fourteen codes

was formulated. Interviews were read and margins annotated to coincide with a

number that was given to each code. Interviews were read sequentially from district

to school administrators, one district at a time, comparing and contrasting responses

between district administrators and school site administrators. Codes were

streamlined based on content, as there was overlap in the content that related to each

code. Codes were narrowed down to seven, which later served as the most salient

themes identified in interviews, observation notes, and documents. All documents

60

were read a second time in the same order and this time were color coded based on

the eight codes, which then served as themes. Organization of themes and

integration of Data Informed Leadership framework (Knapp et al., 2006) were as

follows:

• Themes corresponded to each of the four questions

• Themes were correlated with interrelated elements of Data-Informed

Leadership

• Integration of data for each theme, as it relates to the Data-Informed

Leadership framework, is exhibited in the written content of Chapter 4

The below tables diagram the organizational process for coding, analysis, and

organizing data and the integration of Data-Informed Leadership elements with the

themes.

61

Protocol Codes and Assigned Numbers for

Annotating Data

Refined Codes (Color Coded)

Correlation of Theme to Sub-questions

1 Assess Need for Professional Development

Needs Assessment for Professional Development

Methods of Needs Assessment: Sub-question #1

2 Building Capacity

Leadership Capacity Building

Leadership Capacity Building: Sub-question #1

3 Facilitators of DDDM 4 Resources-Training and

Support/Plan

5 Process of DDDM

Support in Process of DDDM/Accessibility to Tools

Process of DDDM: Sub-question #2 Accessibility to Tools: Sub-question #2

6 Process of Data Analysis

7 Technology/Support/Tools

8 Culture of Data Use Culture of Data Use Culture of Data Use: Sub-question #3 9 Types of Data

10 Evaluation Tools Effectiveness of Professional Development

Effectiveness of PD: Sub-question #3 11 Evidence of DDDM

12 District Expectations District Expectations District Expectations: Sub-question #4 13 Gaps

Table 2. Organizational Process for Coding, Themes, and Integration with Sub-questions

62

Table 3. Organizational Chart for Integration of Sub-questions, Themes, and Elements of Data-Informed Leadership by Knapp et al., 2006

Ethical Considerations

I obtained informed consent from all participants prior to conducting any

interviews, observations, and document retrieval to ensure that there was voluntary

participation in the research study. During the entirety of data collection, analysis,

and the reporting process, I complied with the University of Southern California’s

procedures as well as that of the districts and schools being researched. The districts

understood the nature of the study and that, at any time, they could withdraw from it.

Because the study was conducted at the district and school site levels,

anonymity will not be compromised. Data gathered during research, such as taped

63

interviews, interview transcripts, field notes, and documents will be kept confidential

in order to guard the names of all participants from other participants and to honor

anonymity. Information included in my final dissertation will be presented in ways

that mask the individuals’ identities should they choose to remain anonymous after

previewing the entirety of the study. Specifically, I was diligent in adhering to

guidelines for ethical conduct in research. Institutional Review Board (IRB)

approval was obtained through the University of Southern California.

Limitations of the Study

The current study gathered qualitative data through interview protocol,

observational protocol, and document collection in two California school districts.

These districts are identified as having promising practices in professional

development for school site administrators that elicit DDDM at the school site level,

ultimately affecting student achievement. The study of these districts and their

schools were conducted over a short period of time, spanning five months. Factors in

the districts and their respective school sites may affect the pertinence and

transmission of this study to other sites. However, hopefully the themes found in the

study will generalize to other districts since the selection was purposeful and made

with the hope that data gathered would be relevant to all California school districts

based on the requirements of PSAA and NCLB.

64

Summary

This chapter describes the research design and methods of data collection

procedures and analysis that I used in my research. In the forthcoming chapters, I

will present my research findings as well as an accompanying analysis of the same.

65

CHAPTER FOUR

Data Analysis and Data Interpretation

Introduction

In this chapter, I will analyze and interpret data collected from interviews,

observations, and documents with respect to Buck and D’Angelo Unified School

Districts. I viewed the districts’ professional development programs through the

framework of Knapp et al.’s (2006) Data-Informed Leadership, which offers ways of

recognizing and understanding the process or lack thereof by which professional

development is conducted in a particular setting, district and/or school site.

The arrangement of this chapter is organized by the research questions and

their direct connection to the common themes discovered in the analysis of the

research. The following themes were extracted and organized under the

corresponding research questions:

Research Question: How do districts build the capacity of school leaders to engage

in data-driven decision making?

The overarching research question was formulated to guide the study and the

answer to this question will surface within each common theme, but is most

prominent in the themes of Leadership Capacity Building, Support in the Process of

Data-Driven Decision Making, and Culture of Data Use. The sub questions and their

attendant themes are outlined below.

66

Sub Questions:

(a) How is the need for professional development assessed?

Theme #1: Methods of Needs Assessment

Theme #2: Leadership Capacity Building

(b) What types of training and support are provided to school site leaders?

Theme #3: Accessibility to Tools

Theme #4: Process of Data Driven Decision Making

(c) What evidence is there that leaders put new data-driven practices into

place?

Theme #5: Effectiveness of Professional Development

Theme #6: Culture of Data Use

(d) What knowledge gaps still exist?

Theme #7: District Expectations

The answers to the proceeding sub questions via the evidence that was

compiled through their attendant emerging themes were discovered during data

analysis. Note that Data-Informed Leadership will not be prominently outlined

throughout each section but will be weaved in through data interpretation. Therefore,

before delving into the themes, I will discuss the framework.

67

A Framework - Data-Informed Leadership

Knapp et al. (2006) define Data-Informed Leadership as a “more thoughtful

and intentional approach to using data.” They contend that this phrase is a shift or a

redefining of data-driven decision making, wherein data-driven decision making

indicates making decisions based on data. Data-Informed Leadership insinuates that

data affirms that an educational leader brings into the process specific core values,

vision, and a contemplative and deliberate approach into the practice of leadership.

The implication is that data do not necessarily drive decisions, but that the use of

data acknowledges the complexities and ambiguities that play into data use in

educational institutions. This lens will draw attention to why and how professional

development by the district office for school site administrators in data driven

decision making is a more complex endeavor, which entails raising the capacity

within a spectrum of knowledge and skills in order for leaders to be effective in

increasing student achievement.

When applying the lens of Data-Informed Leadership, it is important to

understand that it is utilized in the midst of governmental policy environments (see

p. 46 for a visual of the framework). This policy environment is the catalyst for high

stakes accountability throughout the country and propels districts into refining their

practices to meet these expectations. The ‘anchors’ for data informed leadership are:

leadership focus, core values, theories of action, data literacy, and available data.

The anchors will manifest in the message by the superintendents and their respective

68

district and school site leaders as well as in the district strategic plan. In order for

these anchors to be solid there must be an investment in developing leaders’

expertise and data literacy, both of which were used as screening tools in the

selection criteria for the above-named districts, and they continue to be part of the

learning process of both school districts.

Data-informed leadership is also defined through the culture and cycle of

inquiry loop wherein districts access or search for information, make sense of it, take

action and communicate, learning from action, and then pursue a reframing of a

problem. It is my intention to demonstrate that ‘why’ and ‘how’ of districts

conducting professional development for school site administrators is through

continuous engagement in the culture of inquiry loop. Districts and schools

encourage and motivate the development of cultures of inquiry within its institution

with the intention and actualization of making strides in student achievement.

Furthermore, this culture of inquiry functions symbiotically with the implementation

process and the attendant effect on students, educational professionals, as well as

continuous learning within the institution.

Leaders, specifically district administrators, who by nature are expected to

provide the above-named anchors for effective data use, often times define the focus

(directly or indirectly) for the data that they generate and use with their principals.

Knapp et al. (2006) suggest that the following foci be a priority: (1) Focus attention

and effort on improving student learning, (2) Guide the learning of individual

69

professionals, and (3) Guide what is known as ‘system learning’ – similar to the term

‘organizational learning’ coined by Senge (1999). Since this study focuses on how

districts build capacity of school leaders in data-driven decision making, the above

three priorities were clearly articulated by both units of analysis, and remained at the

forefront when gathering and analyzing data. Data presented will be in support of

the Data-Informed Leadership framework, and it is hoped that specific insights from

the data will expand the framework. I will now turn to the themes that address the

research questions.

Need for Professional Development

The assessment of professional development through the Data-Informed

Leadership framework begins with the mandated requirements of NCLB and PSAA.

For district leaders to begin to evaluate professional development needs, they must

assess not only the external pressure of governmental mandates but also the

knowledge, skills, and intrinsic motivation of the school site leaders to implement

new learning at their respective sites. Other considerations in assessing the need for

professional development is a district’s expectation of school site administrators data

use to improve student achievement and how they will establish the cycle of inquiry

at their sites. The prominent themes that emerged in the data were Methods of Needs

Assessment, how districts went about assessing the need for professional

70

development, and Leadership Capacity Building, how the need is measured relative

to the level of leadership capacity of the school site administrators.

Theme #1: Methods of Needs Assessment

Buck and D’Angelo Unified School Districts see professional development

for school site administrators in data driven decision making necessary and valuable

in meeting the required government policy mandates. This is evidenced by their

inclusion of professional development in the strategic plan, which dictates their

obligation to probe deeply into the needs that exist within professional development

for principals. Both district superintendents have established goals within their

strategic plan in professional development that resolve to forward district efforts to

increase student achievement via a district team effort. Both superintendents have

the understanding that the support of district and school site personnel are necessary,

and that these individuals should hold the belief that rigorous capacity building,

management, and organizational skills and strategies are necessary to increase

student achievement system wide. Moreover, they are both of the belief that

principals share in this understanding by carrying the message back to their school

sites, implementing the plan, making responsible decisions, and most importantly

valuing that learning organizations are about having a steady flow of information

regarding their work, its outcomes, and proactively seeking opportunities to build

their knowledge as well as that of the teachers within their schools. Albeit, the

discovery of the professional development needs are different based on their

71

organizational culture, the leadership focus and strategic plan reflect a concentrated

effort in student achievement.

Buck Unified School District Superintendent’s Message

Buck has a vision that is simple and powerful:

We believe students in Buck should be as well educated as any in the world. We believe all students have the capacity to be high achievers.

We believe people are the cornerstone of our district and students are the reason we are here.

D’Angelo Unified School District

Superintendent's Message

It should be clear that the mission and focus of the D’Angelo Unified School District

is student achievement. Each of us should clearly understand

how we support that process.

Buck Unified School District

Buck Unified School District utilizes a concept formation model when

conducting professional development for their school site administrators, allowing

for innovation in their approach toward responding to government and district

accountability measures. The concept formation model allows creative and critical

thinking, communication, and independent learning. Since the particular concept to

be learned is data-driven decision making, school site administrators would utilize

data, technology, and leadership knowledge and skills. Furthermore, creative and

critical thinking extend from a common set of features within the district and/or

school site, namely best practices. Professional development, however, is not being

72

identified on the basis of a single critical piece in raising student achievement via

data driven decision making or through one specific framework. Buck Unified

School District has a longstanding history of providing school site administrators

with autonomy over their school sites. On the basis that each school site has their

unique community and their set of complexities, school site administrators have

room for innovation. This allows each school site leader to incorporate their beliefs,

values, and experiences within their individual school site environment and within

the parameters of the district strategic plan and the individual single school plan.

The configuration and performance of the individual school also plays a role in

determining professional development training. There is an expectation that school

site leaders, in collaboration with district leaders, will make an assessment of their

site’s professional development needs. The assistant superintendent shares:

We have different levels of ability . . . progress varies in terms of how many schools’ entire staff have been trained and the number of principals trained and their various levels. . . . our roll out last year consisted of a couple of principals who are very tech and data savvy, using data for instructional purposes . . . principals share how to use the data from a principal’s perspective.

Due to Buck’s philosophical spirit of autonomy and innovation coupled with

the district’s desire to implement a more district wide systematic approach to

curriculum and assessment, a decision was made by district administrators to take the

opportunity of the new math adoption training to further streamline their process.

The assistant superintendent continues by reporting:

73

Everybody is attending the new math curriculum training. A good part of that training is assessment and the data that comes with it. So everybody is looking at benchmark assessments, feeding those benchmark assessments into Data Director so they can be used to make judgments about how and what kids are learning. Through document analysis, it was noted that over five hundred teachers are

slotted to attend in-depth training in the newly adopted Houghton Mifflin and

Prentice Hall Math programs from now until mid-March 2009. The training

emphasizes lesson design, pacing and planning, differentiated instruction, and

assessment embedded in curriculum. Of essence are demonstrations in instructional

strategies and student engagement. A key component to this training is

incorporating Data Director3, which has been in use district wide for one year now.

A needs assessment for professional development in DataDirector revealed, in

district documents, the following:

This data and assessment management system allows teachers and administrators to view, disaggregate, and analyze student assessment data. During the first year of use, approximately 920 teachers and administrators accessed DataDirector for information. Buck’s superintendent articulated that despite budget constraints, the need for

professional development is vital, stating, “. . . dollar spent on staff development is

the best dollar spent and so we’ve been strategic.” The specific school site needs

determine the strategic approach for professional development and in such cases

3 DataDirector is an online data management system which serves as a decision-making support and tool.

74

require a plan to be fashioned that would include a deeper analysis of instruction

monitored by benchmark assessments.

Buck USD contracted with educational consultant Dennis Parker to provide

professional development, Effective Schooling Strategies, for their lowest performing

schools in order to increase students' achievement. Parker was contracted via the

High Priority Schools Grant to assist in the implementation of effective schooling

strategies for one middle school and one high school. Also DataWorks, a research-

based educational support provider, is providing support at an elementary school as

part of the high priority program. Through this partnership the teaching staff

receives over forty hours of intensive staff development in the areas of explicit direct

instruction, assessment analysis, and curriculum design.

Professional development is aimed at assisting teachers with tools to ensure

student success in the classroom. Staff will continue to employ successful strategies

including working with consultant Dennis Parker to implement Strategic Schooling

best practices; sharing and discussing test results with students; posting and checking

off the California standards on a weekly basis; using Smart Goals to focus on a

particular standard for four weeks; and sharing data to assist students who need extra

intervention including English language learners.

In order to accommodate for the philosophical belief of autonomy and

innovation within the district and yet meet the professional development needs of the

remaining Buck USD schools, the district has provided other opportunities such as:

75

DuFour’s Professional Learning Communities; Marzano’s training in cultivating and

sustaining effective instructional strategies in the classroom; and, other opportunities

to seek knowledge. It was also noted by district and school leaders that professional

development needs were addressed based on the level of knowledge and skills of the

school site administrators.

Compass program is really for the new administrators . . . the leadership academy meetings are what in most school districts are called the principals meetings . . . Buck is a little different because the leadership development component speaks to all leaders in the district, not just principals . . . a component of the leadership development academy has always been looking at data through a data driven or results oriented organization. The Director of Schools added all principals come in the morning and then

the new principals will stay throughout the afternoon, giving them a full day of

professional development. New principal training includes use of DataDirector.

Buck administrators also noted the need for professional development for school site

administrators to be balanced with structured and unstructured time. The Director of

Schools noted, “The idea is that on the leadership academy day there would be a

structured time for collaboration, and by structured I mean we would provide the

purpose, what we want them to work on collaboratively, talk about what’s working.”

She noted that topics for professional development structured time are gleaned from

discussions that materialize during these meetings. Planning is consistent with what

principals need to know, which is revealed through a cycle of inquiry during the

professional development at the Leadership Academy meetings. The district sees this

76

as being flexible and open to the needs of their school administrators. It was

discovered through the interviews of district and principals that the goal has never

been to have a conventional didactic approach to professional development but rather

to determine the need during capacity building time based on voiced needs. Buck,

however, is gently moving toward being more systematic in the area of benchmark

assessments, the process of which will be discussed in the section for the second sub

question, which delineates the process in supporting data-driven decision making.

The Director of Schools noted, “The planning committee’s goal is really to revisit

topics and issues over and over rather than use the whole shotgun approach of let’s

throw everything out there and see what sticks.” Revisiting issues establishes a cycle

of inquiry that allows for not simply learning, but opportunities to reframe problems,

implement, assess, and allows for communication and collaboration in the process.

For Buck this is a priority when determining the need for professional development.

The Supervisor of Curriculum and Professional Development indicated that

many considerations are made when introducing new concepts, stating:

My experience at the district level is that you must take into account that people change and people move at different rates . . . when you plan for something big, you must plan for people who are ready to go, people who are going to look at it for a while, and people who are really resistors, but even they come along eventually, but you just have to keep that in mind when you do something. In an effort to move school administrators to implement systematic

instruction and assessment practices at their school sites, district administrators along

77

with select school administrators meet to establish future needs for system learning,

the implementation and effects on professional learning, which directly affect student

learning. Via a cycle of inquiry in the district infrastructure, which allows for

ongoing adjustments in the professional development, a beginning of the year

meeting allows for a skeleton outline of professional development for the year. The

team is made up of the Assistant Superintendent of Educational Services, Director of

Schools, Director of Child Development and Special Programs, the Deputy

Superintendent, and select principals throughout the district. The year will begin

with a three to four hour meeting, which sets out to discuss issues as they relate to

the strategic plan and the superintendent’s initiatives. During the course of the year,

this committee meets monthly. The committee conducts ongoing monitoring of the

professional development program with its primary concern this year to meet the

need of establishing a common language across the district. Thus, the committee

will tie professional development into the new math adoption in order to support

schools in the use of data on a more frequent basis. An elementary school principal,

stated:

They’re (district) continually investing in programs and technology to make it easier for us, like Data Director, it’s a very expensive program. It was not an easy choice for our district to spend that money, but the willingness for them to do that so that we and the teachers have that access to data shows their willingness to support us.

78

A middle school reports that the need for professional development is based

on district needs.

. . . 19 new principals . . . the COMPASS program helps through training and support . . . making data driven decisions through teaming . . . an experienced administrator and two new principals. There is coaching from district, and retired administrators. The needs assessment for professional development at Buck, viewed through

the lens of Data Informed Leadership, takes into consideration the complexities in

the knowledge and skill set necessary to be an effective educational leader. Buck

believes that an investment in the data infrastructure (through the purchase of Data

Director) and access (through professional development focusing on instruction and

assessment) are an integral part of the ongoing learning required to improve and

produce results. With this understanding, the superintendent, in an observed

professional development meeting in the beginning of the year, made clear that there

is direct correlation between what is expected and what will be provided in

principal’s professional development based on CPSELs, the California Professional

Standards for Educational Leaders, indicating that expectations and evaluations will

be aligned with standards for administrators just as students and teachers are

evaluated based on standards. Thus, the need for professional development directly

correlates with what school site administrators are expected to know and do.

79

D’Angelo Unified School District

District expectations for learning and continuous improvement are prevalent

at the D’Angelo Unified School District as well. D’Angelo USD utilizes the Focus

on Results framework to guide professional development for district and school site

administrators as well as teachers. It has been consistent in following this

framework for the last four years. Focus on Results is a process within a framework

to actualize improvement in student achievement, which is D’Angelo’s primary goal.

A vital component in the process is the establishment of an instructional team of

teachers at each school site; utilization of student data to form an instructional focus

and to implement research-based instructional practices drawn from each school’s

teaching staff; ongoing student assessment; setting and monitoring of goals; and

intervention for students not reaching goals.

D’Angelo’s superintendent clearly stated that SEAM, a set of questions based

on instructional reflection, should be at the forefront: S: Is Instruction Standards-

Based? E: Are students actively Engaged in the lesson? A: Are students Assessed

regularly to inform and drive instruction? M: Are students reaching Mastery of

standards? These reflective questions allow for an ongoing cycle of inquiry, which

allows for the reframing of problems; thereby, determining the need for professional

development. Rather than the process being school site leader driven, it is teacher-

driven, dictated by teachers’ needs to actualize school improvement. Hence, the

process of professional development for school site administrators includes 90 lead

80

teachers at district-wide meetings, and a total of 291 lead teachers at thirty schools.

These teachers meet monthly for training, engaging in the sharing of best practices,

and evaluating progress along the way. The superintendent’s philosophy is to make

clear to the community the area of challenge, articulating that it is important for “. . .

all D’Angelo children to find success—especially those that have lagged behind in

achievement. Approximately 55% of our Hispanic students, 40% of our Caucasian

students, and 20% of our Asian students are not proficient in Language Arts and

Math . . .”

With Focus on Results selected as a means for improvement, and keeping in

mind the limitation of financial resources, a needs-assessment was made based on

which schools would receive Focus on Results professional development first.

Similar to Buck USD’s need to prioritize, a high school principal noted:

The first schools that went on Focus on Results were going to have to contribute to the cost and we have no Title 1 money and I was not able to contribute to the cost of anything. . . some of those schools had additional pressures of beating state mandated improvement. It was very wise for the district to start with them because they needed all the help and support. The needs assessment for professional development was tiered – while the

district began its implementation of Focus on Results with the first cohort of 13

schools, other school leaders were taking the initiative to engage in the cycle of

improvement through organizational restructuring. In other words system learning

was happening at some schools without the guidance and direction of a formalized

81

professional development. The schools engaged in Focus on Results professional

development and the other schools, not engaged in the process, established very clear

anchors for Data-Informed Leadership. The superintendent established leadership

focus through asking all schools to produce a “Good News Statement” and an

“Urgency Statement.” All schools were expected to work toward improving student

achievement.

On the second year of Focus on Results implementation, the second group

was rolled out, and then the third and last group was rolled out by the third year,

2007-2008. The district is currently in their fourth year of implementation with

cohort one having completed their three years of formal professional development

through Focus on Results. The professional development roll out process for each

group takes a total of three years. The entire process is being led by Focus on

Results consultants who guide the principals at monthly meetings. The end goal is

for existing administrators, and now for administrators in cohort 1, to exercise lateral

capacity building via mentoring, coaching, and training of other administrators in the

knowledge and skills they have gained in their three-year roll out. While engaged in

this process a clear focus based on each school’s urgency statement is formulated.

An assistant superintendent stated,

Each principal establishes their goal. We work with them on the goal, looking at their strengths and weaknesses . . . so district wide if you walk down our hallways, we have a good news statement at all schools and we celebrate what we are doing . . . and then we have an urgency statement.

82

Assessing the need for professional development also happens through the

central office instructional leadership team (CILT). After thinking through their

delivery of support this year, CILT will identify the two highest needs schools, one

elementary and one secondary. The team is made up of the assistant superintendents

and directors. The goal is to target two specific schools identified through data to

assist them in anyway they request, whether it is to get them more data, conduct

more data analysis, or provide them with extra support. The assistant superintendent

also noted that one of the measures used for assessing the need for professional

development is also the schools’ API standings.

We’ve done tentative runs on Data Director, all our high schools grew, and three out of four middle schools . . . two elementary schools that dipped had over 900 API.

He also stated that administrative makeup at school sites is another consideration,

i.e. a school site administrator’s effectiveness and those of their teachers is another

element in assessing professional development needs.

Principals and teachers attend the Focus on Results meetings together, but

district administrators also establish their needs assessment through school site visits

where they engage in discussion and cognitive coaching to see what supports can be

offered and what avenues to pursue. An elementary school principal notes:

Well they (district administrators) were trained sitting right next to us. (They) help us . . . coach . . . come once a month and see our sites . . . they push us and say, tell me how you’re using data, show

83

me what you’ve done this month. Tell me how your teachers are doing . . . I happen to have the Director of HR as my coach.

Building a culture of inquiry requires district administrators to be a part of the

learning process, where learning becomes a system endeavor. Through

conversations, needs outside of Focus on Results may surface, such as was shared by

a middle school principal, “Focus on Results gives you the primer, to decide the

focus, for instance we chose writing. The district will provide a level of support . . .

if the data shows we need more, then we visit the superintendent and he or cabinet

will come up with ways they can support us.”

Support, as a result of the methods for needs assessment for professional

development, is ultimately measured by state expectations. The tables below show

the Academic Performance Index for Buck and D’Angelo as well as their respective

subgroups, and measurable growth from the previous year. Data indicate Buck made

a growth of 11 points, which places it within 5 points of being an 800 district.

D’Angelo USD also shows a growth of 11 points with an overall API of 818. Both

districts have exceeded the State of California’s average API score of 742.

Furthermore, both districts are aware that schools receiving Title 1 funds are subject

to identification as a Program Improvement school based on its inability to make its

AYP in a particular area for two consecutive years. Buck has 13 Title I schools

including 5 in PI status. Four of the five schools met or exceeded their 2008 API

growth scores. Both districts have growth in API for all their subgroups.

84

BUCK USD API

Number of Students

in 2008 Growth API 2008

Growth

2007

Base

2007-08

Growth

15,478 795 784 11

Subgroup API

2008

Growth

2007

Base

2007-08

Growth

African American-not of Hispanic origin 1,471 Yes 733 715 18

American Indian or Alaska Native 33 No

Asian 4,820 Yes 915 906 9

Filipino 1,543 Yes 858 850 8

Hispanic or Latino 6,018 Yes 696 681 15

Pacific Islander 145 Yes 775 760 15

White (not of Hispanic origin) 1,324 Yes 796 786 10

Socioeconomically Disadvantaged 5,962 Yes 697 685 12

English Learners 4,155 Yes 697 683 14

Students with Disabilities 1,457 Yes 564 554 10 Table 4. California Department of Education, Accountability Progress Report, Buck (2008).

Table 5. California Department of Education, Accountability Progress Report, D’Angelo (2008).

D’ ANGELO USD API

Number of Students

in 2008 Growth API 2008

Growth

2007

Base

2007-08

Growth

20,311 818 807 11

Subgroup API

2008

Growth

2007

Base

2007-

08

Growth

African American (not of Hispanic origin) 260 Yes 761 746 15

American Indian or Alaska Native 39 No

Asian 2,713 Yes 919 908 11

Filipino 1,269 Yes 858 845 13

Hispanic or Latino 4,528 Yes 735 725 10

Pacific Islander 24 No

White (not of Hispanic origin) 11,390 Yes 822 811 11

Socioeconomically Disadvantaged 8,630 Yes 755 742 13

English Learners 10,598 Yes 768 749 19

Students with Disabilities 1,747 Yes 581 576 5

85

As evidenced from the above tables, both Buck and D’Angelo made from 5

up to as high as 19 points growth in various subgroup APIs from 2007 to 2008. Both

districts are meeting the performance based accountability challenge set by the state

of California.

Theme #2: Leadership Capacity Building

Knowledge and skills to meet the challenges of performance-based

accountability in public education to increase student achievement is of prime

concern in Buck and D’Angelo USDs. Both districts are acutely aware of the need

and urgency for school site leaders to be proficient in data use and analysis, but more

importantly for teachers so they can gauge academic progress and instructional

practices. The need for professional development is also measured by the school

site administrators’ level of leadership capacity to make the above materialize.

Buck Unified School District

The superintendent of Buck USD approaches capacity building of school site

administrators and teachers as multi-pronged. The district’s approach must be one of

empowerment for school site administrators and teachers.

Just throwing the data out (there) without training principals on how to deliver would be a recipe for disaster. You have to know your audience, you have to walk in their shoes, and you have to do it in the most effective way without shutting them down. An effective way, we discovered, is to not give them the data, but give them the ability to go after the data. They will bring themselves to that gut wrenching moment, how did I do? So we’re not training the principals on the mechanics, but rather how to do it as a presentation to the staff.

86

With delivery being just as important as data use, the superintendent notes,

“The ultimate goal is to awaken the collective intelligence. We don’t expect a

principal to know everything. We expect them to unleash their teachers. The

teachers will automatically go where you want them if it’s presented the right way.”

Professional development at the district level has not always been at its peak;

however, now the format for principals’ meeting is centered-around lateral capacity

building -- colleagues presenting to colleagues. Buck’s superintendent notes, “We

need to be selective about the principals we choose to present and worry about egos

some other time. You know, maybe after I retire, I’ll worry about egos.”

Relationship building, however, is a conduit to increasing the capacity in

principals. Buck’s superintendent notes that frequent school visits is an intrusion,

obligating school sites to treat you like a celebrity. He stated he sees principals all

the time, and they do not mind coming to the district office for meetings, stating “We

are informal.” The level of comfort was noticeable through the first half-day

professional development observation at the district office, where the superintendent

arrived prior to the PD and interacted with the principals. Conversations consisted of

personal and work related topics. This was in line with the evidence gathered from

district administrators that relationships are built due to the fact that they invite

principals to be an integral part of the planning committee. Buck superintendent

stated, “So there is little imposed except that which we impose on ourselves and we

see ourselves as a collective group. Everybody’s idea has merit and we all gravitate

87

to a good idea.” He also noted that high expectations for student achievement pushes

the agenda for leadership capacity building, “My job and the district’s job are to be

the conscience . . . we won’t let you ignore those things even though there is comfort

(in the relationship).” Along with increasing the leadership capacity of principals is

the level of expectation espoused in the superintendent’s leadership focus, “. . . we

do have a bottom line . . . a principal needs a game plan and needs to work with his

or her staff to improve student achievement. We expect that.”

The assistant superintendent indicated that leadership capacity building takes

place at the school through providing additional human resources. Through a grant,

there will be a coordinator at the site level who conducts frequent walk-through and

provides support for staff. This additional monetary resource can put key people in

to facilitate the day to day walk, “because this is really providing accessibility to

teachers in the classroom, allowing for development and refinement in instruction.”

There is also differentiation in capacity building for principals. COMPASS -

Coaching Our Management: Peer Assistant Support System is a program for first

and second year principals and other administrators who may need it. PASS, Peer

Assistant Support System, was developed jointly with the union. A district

administrator stated, “It sounded better than PAR, Peer Assistance Review, and then

we just expanded that type of program to administrators by adding COM (Coaching

Our Management).” Principals receiving COMPASS are assigned to retired

88

principals who are COMPASS coaches. In addition to the monthly meeting, a

couple times a month, retired principals will go to the site for one-on-one coaching.

With close monitoring, guiding, cognitive and reflective coaching from

district administrators, progress is made. He continued by reporting, “We’re

definitely working with our program improvement schools, working with them on

the data, and asking the hard questions.” This year we had two of our PI five schools

meet all their AYP goals,” indicating the level of support is commensurate with the

need. A principal of a school which has gone from PI 1 to PI 2 status asks, “What

are we going to tell the staff, how are we going to convey that message? More

importantly what are we going to do about this trend?” Discussions ensue around

these issues to see where the district can offer support. The district will work to

relieve the principal by freeing up time spent crunching and preparing data.

Capacity building for school administrators consists of district administrators

offering services with the expectation that principals will be proactive and seize the

opportunities to refine their knowledge and skills. For instance, Buck offered

approximately twenty after-school workshops on a variety of topics such as writing a

single school plan, categorical budget, or data analysis. New principals or anyone

interested could come and sit around the table for a one-hour informal presentation.

Other district administrators play a significant role in capacity building. The

Director of Schools has the responsibility of supervising and evaluating the nineteen

elementary school principals. She is also responsible for the COMPASS program.

89

A large part of what she does is design the monthly leadership academy meetings.

She works with others to plan and design the leadership academy which is the

district’s professional development for all of the leaders in the district, not just

principals. Part of the plan for the coming year, for principals, will be devoted to

additional training to help them be more comfortable with data. She gives an

example of a recent problem that required support in the form of capacity building

from district administrators, reporting:

One of our schools dropped dramatically in its API this year and this became a challenge of course for the principal who has a high level of accountability . . . I mean it caused a lot of anxiety. Both I and Director of Special Programs talked with the principal, we first looked at . . . scores individually through Data Director to see if we could see any trends to point out to the principal. The Director of Schools indicated during this portion of the interview that

many different reports were pulled through the district’s expert on Data Director in

preparation to assist this principal in leading her staff to find the answers. The

process they took the principal through was what they expected her to model to her

staff, stating:

What I didn’t want to do is to tell her what to do because her staff needs to see . . . it come from her . . . we spent a couple of hours and I pointed out some of the things that we had seen and with some possible suggestions about what we thought might have happened . . . then we talked about questions she could ask her staff as they are looking at the data (together).

District administrators coach principals in the process of the cycle of inquiry

so they can replicate the process with their staff. Other skills are needed in the

90

beginning of the year such as the Single School Plan, the guiding document of how

schools will determine their work for the year as well as how they will allocate their

resources. The Director of Schools stated that she structures the topics for new

principals through the COMPASS meetings to meet their needs as the year

progresses, stating next week’s COMPASS meeting will be about the Single School

Plan and how to work with school site council, which comes up early in the year and

principals need the knowledge and skills to conduct this successfully.

The COMPASS meetings are also a venue for individual conversations. The

Director of Schools reports this is a powerful way to building relationships,

providing space for reflective coaching, collaboration, as well as counseling, stating,

“. . . there is a lot power in just individual conversation . . . as a way to facilitate

relationships between administrators . . . to discover strengths in order to put the

pieces together, put people together . . . guide in the way you want it to happen.”

Leadership capacity building also takes place at the individual school sites

wherein a culture of trust and respect are built to lead the way to distributive

leadership. One elementary principal indicates “. . .it’s important having the culture

of trust . . . now that’s an ongoing thing, that’s not like something you can say, well

we trust each other now, so we can just move on . . . I’ve learned it has to be rebuilt

almost daily. . . to try and make sure that people trust each other and feel comfortable

with each other.” She indicated that she also has an assistant principal, who is also a

classroom teacher, reporting, “. . . she’s a good sounding board for me.”

91

A middle school principal in Buck USD shared the role a district

administrator plays in role modeling facilitation, indicating that this skill is a

necessary one with principals and teachers. Proper facilitation leads to productive

collaboration, which ultimately raises the level of performance in the day to day

work at the school. She reports:

As administrators we sit in on meetings and coach . . . what I’ve discovered is that we send our teachers to collaborate and we’ve never taught them how to collaborate. At first they were (defensive), but now they understand if you’re doing most of the talking then you’re not facilitating, you’re directing. As soon as we trained people to facilitate correctly, collaboration and trust moved to a different level. Lateral capacity building is the ultimate goal within a district, wherein

colleagues are collaborating and sharing their knowledge and skills with one another.

The principal of a high school describes his journey from working in a large district

and coming to Buck, a much smaller district, stating:

I remember gathering the high school principals and saying this is not the way to do business. Let me show you some of the practices that I have been doing for a while. And we finally got the data in here and that made it very interesting. We created our own benchmarks and we used that data . . . a lot of what is being created now has been emulated from what we’re doing.

A key element to pushing forward the agenda of improvement is for the

district to allow a forum for shared practices among administrators in order to

increase principals’ ability. Capacity building at times works from the school site

level up to the district level, and a superintendent’s support and leadership focus

92

makes all the difference when working through the process of improvement via

leadership capacity building.

D’Angelo Unified School District

D’Angelo’s leadership capacity building is a key component in the Focus on

Results process, wherein the principal is accompanied with the leadership team to the

Focus on Results meetings. The superintendent indicated that the entire instructional

team comes down with the principal on a monthly basis to be coached in the process

of bringing about improvement. The principal and the team carry the message back

to their school. D’Angelo’s superintendent reports:

The vast majority in the district is focused on literacy in some way . . . writing . . . reading comprehension . . . one high school is focused on higher level thinking skills. Everyone has a focus and everyone has a common language. Now as part of the process, each year we train a cadre of teachers and administrators to actually teach the modules as we move through them. The promise was after three years of being in the program, the facilitator, an outside consultant, would back out and we would have built enough capacity into the program. As with Buck, D’Angelo’s superintendent believes that role modeling is vital

to capacity building. This was evident during a professional development

observation, and during stated in an interview:

Unless you as the superintendent are visible to support the process . . . there will be principals who will not take it seriously. I will tell you I am at 98% of those trainings for the instructional leadership team . . . to openly talk about the great work their doing, give a pep talk, encourage them because you have to walk the talk . . . if it’s that important, that the focus is truly student achievement, then why can’t the damn superintendent be down here to be a part of the process?

93

D’Angelo’s superintendent indicates the importance of providing principals

with complete support. He noted that supporting a principal with problematic

situations is providing them with tools to do their job. With the support, however,

is accountability. “. . . on the other hand, if I’m dealing with a principal like on the

graph I showed you (inconsistencies in performance), and I ask him, how are you

going to change this? What’s your plan to address this? After giving all the

training and giving all the support, they can’t articulate a plan to make the changes,

then that’s when they’re done.”

Capacity building develops through vision and leadership focus, but

sometimes it needs to be accompanied from outside sources. D’Angelo’s

superintendent notes that one has to be knowledgeable of the opportunities

available whether it is Focus on Results, Springboard, or any other framework for

improvement based on what one intends to get out of it. He also indicates that the

superintendent must drive the process, standing back and delegating a major

endeavor sends the message to everyone that the effort is of secondary importance,

stating “ . . . if you’re not willing to stand up and spend the time to do it, they won’t

do it. They will shrug it off, you won’t be taken seriously.” This point was

emphasized several times during the interview, demonstrating that building the

culture of inquiry through viable system learning requires anchors of Data-Informed

Leadership – leadership focus, core values, theories of action, data literacy, and

available data.

94

The assistant superintendent emphasized that building capacity is a process

that needs to be given time. He noted that the support system within the district is

both structured and unstructured and formed through triads, wherein three principals

work together as peer coaches, the Focus on Results consultants work with all the

principals, and a district level administrator is assigned to a triad. He stated that it is

his job along with the other assistant superintendent to coordinate Focus on Results

throughout the district. Upon assessing the situation, a suggestion was made to bring

Focus on Results into the principal’s meetings, which would allow for consistency.

He stated, “We actually lessened their time but made it more consistent . . . it

allowed for individual coaching with the principals, the one I evaluate and the one

that I work with on the Focus on Results triad.”

Principals’ leadership skills are necessary to accomplish the expectations set

by the superintendent to ensure effective implementation of Focus on Results. An

elementary school principal notes that to meet the needs of students “. . . the

superintendent is challenging principals to spend 50% of their time in classrooms,

so that we know what’s going on, so that our feedback is much more directed, so

we can have more courageous conversations with teachers about rigor and

connection to the standards.”

The Focus on Results process has facilitated capacity building with respect to

taking a critical look at instruction and its delivery. Some of the conversations that

95

are elicited are what does engagement look like, how does one raise the level of

engagement. She continues by saying,

We had a two day training to be presenters . . . we wrote our own curriculum, what we believed we need to do next to deepen our level of understanding . . . having teachers unpack standards and discuss what it looks like as far as teaching to mastery. What are the lessons? Let’s go through and see what really matches . . . what’s a mismatch . . . not simply going through every activity in the textbook. The Focus on Results consultants train us on how to train our staff how to work.

This elementary school principal corroborated the assistant superintendent’s

statement that capacity building for principals has extended beyond Focus on Results

meetings, but have extended into the principal meetings where best practices are

shared and discussed. The expectation is that principals take the information and go

back and utilize it. This elementary principal also noted that there is an

accountability piece between all the stakeholders, stating:

. . . accountability piece where you have to come back and say what you did with it . . . it’s the same with grade level meetings . . . what do you do that ensures they met during that time unless you are running from classroom to classroom . . . what are they accountable for . . . what are the four questions to be discussed so you can get feedback, and that’s modeled in our principal meetings as well. Leadership capacity building happens through peer-mentoring outside of

district meetings. System learning is evidenced at D’Angelo, similar to what the

high school principal shared at Buck. Learning is intrinsic because school leaders

believe that learning is an ongoing endeavor. A principal shares:

I go to another elementary school every Wednesday to meet one of the principals. . . she really knows what to do with data and how to look at

96

it in a much deeper way . . . we share, what are you going to do when you meet with your teachers? What are you asking them? What is the expectation and how do you follow up? A middle school principal shares similar efforts to build her knowledge and

skills. “The other principal and I have a relationship, a professional relationship,

where we could trust one another to be open . . . vulnerable with one another . . . and

it is the same thing even with my boss the assistant superintendent. . . I trust him but

I could be vulnerable with him and know that it isn’t going to come back to bite me.”

She also emphasizes that capacity building happens through multiple skills:

As you know, an administrator has to have different components in their make up. If you’re just going to use data and using data is your sole criteria for leading your school, you are probably going to end up dying somewhere along the way. You need different skill sets to be able to incorporate data in your leadership skills to lead your staff effectively. If you don’t have people skills, no matter how good you are in data, you’re not going to get anywhere. Data Informed Leadership and having the knowledge to establish the anchors

of leadership focus, core values, theories of action, data literacy, and provide data

coupled with DDDM play a significant role in carrying out the process. This

example is set at the district level where coaching is modeled by district

administrators, where in turn principals model that relationship with staff. A

D’Angelo high school principal shares:

Because of Data Director and Focus on Results, we entered last year and had a team of about four or five, we’ve expanded to eleven. Eleven gives you a much better chance of having those tentacles to draw in more teachers because I firmly believe that the most important conversations do not always happen in the staff

97

development sessions. They happen in the faculty room, at the copy machine, and it’s those exchanges that we now have eleven. This quite honestly was one of my biggest challenges . . . to bring this team to the table in a way that was open-minded. The district recognizes those that are proactive and represent the essence of

system learning. They become role models for other principals. A proactive

endeavor at one high school was distributive leadership through resource specialist

teachers. Distributive leadership through special resource teachers (SRTs) was a

choice, a strategic move, for this principal. She noted that department chairs are

elected but an SRT is not an official position with an official description. She does

what is necessary to move the school and to do what is in the best interest of

students. She conveys this to her staff by stating, “. . . success buys independence.

As long as we are successful, they don’t have to send someone in to tell us how to do

it better, so if we can make progress, our chance of staying independent is greater.”

In summation, the needs assessment for professional development at Buck

and D’Angelo is influenced by the governmental policy mandates. Most

importantly, however, it is based on the desire of district and school site leaders to

live up to the leadership focus that the superintendents have espoused. During the

observations of professional development meetings at both districts, there was a high

level of engagement from the superintendents. Both superintendents remained for

the duration of the professional development, and conveyed through their interviews

98

that through their presence and message it is clear that ‘continual learning’ is vital to

increase student achievement.

Relationship building begins with superintendents, district, and school site

administrators meeting formally and informally during the course of the year.

Formal meetings take place at the district office and at the school sites between

district and school site administrators; however, the cornerstone that exemplifies that

both institutions are learning systems is that school site administrators meet

consistently throughout the year – meetings that are not mandated – meetings that are

born out of the sheer need to engage in the cycle of inquiry. This cycle of inquiry

involves taking action and communicating based on system learning, learning from

the action (through implementation at the school sites), visiting or revising the

problem, accessing or searching various avenues, making sense of the outcomes, and

continuing the cycle.

This cycle of inquiry which allows for ongoing collaboration and refinement

dictates the need for further professional development at both districts with the

understanding and support of all stakeholders. Buck gathers their needs assessment

through inviting school administrators into the process of developing the year’s

agenda for professional development. D’Angelo formulates its professional

development based on a process that plays out through the Focus on Results

framework. A variety of needs are assessed through collaboration at the district

meetings as well as meetings with members of each triad. Both districts have

99

prioritized their professional development by assessing the level of need at individual

school sites. With professional development needs assessed, districts must also have

a well structured plan to train and support their school site leaders.

Training and Support for School Site Leaders

According to Knapp et al. (2006) Data Informed Leadership, training and support

requires an investment in (1) the development of data infrastructures, and (2) the

development of leaders’ data literacy. Investment in the development of data

literacy can be through a relationship with third-party groups, in-house experts, or

certificated programs. Buck and D’Angelo have utilized third party vendors as well

as in-house staff to provide principals with the training and support to meet their

district’s mission and strategic plan goals. Training and support for both districts

was quite prominent in two themes: accessibility to tools and the process of data

driven decision making.

Theme #3: Accessibility to Tools

Buck’s superintendent acknowledges that supporting his principals through

training and support is a direct result of listening to their needs, “We purchased . . .

DataDirector. It is pretty user friendly . . . you can do almost anything with it and we

received extensive training in it. . . the principals came and said, ‘we need this’ . . .

They were eager to have it. After reviewing several programs we decided to go with

DataDirector.” Buck’s superintendent continued,

100

We set up our principals’ meetings as well as a support system beyond the meetings so they are good users of data . . . and eventually we create schools where teachers and the principal are engaged in the process . . . they are a learning community working on it (improvement process) together. The initial process, however, did not begin in this way. The assistant

superintendent recalls her role as a principal seven years ago:

we would take data that was the size of a phone book . . . it was this stack and I would look at it and feel somewhat overwhelmed . . . at this time a couple consultants from [the county office of education] presented at the principal’s meeting and talked about how to use the data. I don’t know if at the time we had a full understanding of how to use it. The assistant superintendent noted that prior to December 2007, Buck

purchased DataDirector and immediately began a series of trainings. Training was

divided into level I, II, and III. Level I was a big overview, level II covered exam

building, and level III was the data analysis piece. The training was conducted by

Buck’s program specialist and technology professional development trainer. With

the intention of providing effective district wide training, a multi-pronged approach

was used by training principals, site level administrators, and school site training for

teachers. The focus shortly thereafter was to move away from district print out of

data binders, which are given to principals. The emphasis was placed on how

principals and teachers can access the data through DataDirector. The assistant

superintendent noted that the focus will now shift to principals bringing their data

with them and talking about how they’re using it. She also shared that accessibility

101

to tools has allowed some principals to be proactive and take data use to higher

levels, reporting:

This is a principal who knows how to access the information and use it in a meaningful way . . . he needs to play a key role in sharing that expertise with other principals . . . an elementary school principal is also very good with technology and data. There are some strong principals who I believe will play a key role as facilitators at our trainings . . . we try to set it up (professional development) where we look at the best practices within our district. Best practices are also learned through Dennis Parker, with an area of emphasis (being) data.

In order to utilize data for improving instruction other tools had to be put in place in

order to establish instructional targets, namely benchmark assessments; however,

these significant tools were not implemented as a mandate. The Supervisor of

Curriculum and Professional Development explains:

We have not had any district level benchmark assessments in any subject. We made it an option and we set it up along with this year’s math pilot . . . We bought them a laptop and a scanner for DataDirector. We are using benchmark assessments provided by the publisher. We’re printing for them the first time around, all of the assessments and all of the answer sheets. We are training a person to use the scanner and paying for a day to scan. Our technology trainer is going to train staff, showing them (what to do) once we get the information into DataDirector, and what they can produce. The assistant superintendent and I are in the process of coming up with a protocol, some guiding questions, for grade level teams to use to evaluate their data once they see it. The receptiveness and level of involvement from school site leaders and their

teachers has been a source of excitement for the district administrative team. The

assistant superintendent noted that as the beginning of the school year approaches

more schools are getting on board, stating “ . . . if all 29 decide to pilot . . . we will

102

find a way to give the support.” She indicated that piloting the math adoption

simultaneously with advancing the process of data driven decision making was a

strategic move, stating “. . . it was more natural to introduce a cause . . . we’ll do a

three day math training for all teachers, and one full day on assessments. During

that time they’ll go deeper into the assessments . . . how to use DataDirector, putting

results into the system, pulling out reports, and looking for trends.”

She stated that every teacher will have access to these assessments and will

be encouraged to try it this year. In 2009-2010 the target will be district benchmark

assessments. A similar approach and training will be used with the language arts

adoption process. The assistant superintendent stated, “We will make the same

statement. We will provide the incentives and we will have a similar if not better

response because they’ll be used to it.” Utilizing the new math adoption coupled

with incentives has allowed Buck to implement DataDirector and launch itself from

a district with no benchmark assessments to a district with benchmarks in one year.

With these new tools in place the district will quickly have to engage in a cycle of

inquiry when trying to support a system of twenty nine schools. The assistant

superintendent stated “. . . we’re also trying to test the system. What does it really

take to do a district benchmark assessment? What type of support do we need to

provide schools, but more importantly is really getting them prepared to look at the

data, to look at their best practices. That’s really the goal -- to make sure curriculum

is covered to a level of mastery and they’re using data to drive instruction.”

103

The superintendent of D’Angelo noted that upon entering the district, it

considered itself to be a data-driven district, but notes, “It really was far from that.”

He stated that it may have been at some schools and in some classrooms, but not

district wide, and it was “. . . virtually non-existent in secondary.” He noted, that to

help facilitate the process, DataDirector was the best data tool given to teachers, and

it was the springboard for establishing an instructional focus at each school.

An interview with the assistant superintendent reveals that the DataDirector

program allows test material to be loaded, information by state, district, and school

site level. It allows teachers, assistant principals, and principals the ability to access

data at their fingertips. He noted that this year was a transition year where they

intend to shrink the data notebook down so “. . . it is not a notebook that just sits on

the desk.”

Both Buck and D’Angelo recognize investment in the data infrastructure and

access for principals and teachers is a necessity. They also understand that this

investment should be followed up with the investment of developing leaders’

expertise and data literacy. Both districts are slowly weaning their school site

administrators from data notebooks, with Buck being on a more accelerated track,

yet offering on-site support if needed. With this in mind, both districts, albeit it in

different ways, are simultaneous providing training and support in accessibility to

tools and in the process of DDDM.

104

Theme #4: Process of Data Driven Decision Making

The process of data driven decision making requires an investment in the

development of data literacy. A crucial component in this investment is to provide

ongoing support to leaders in process of using data. In this particular theme,

answers to the overarching question are addressed. Both districts have implemented

a plan to facilitate this process via third party organizations with Buck using a

variety of resources based on specific school site needs and based on its desire to

provide autonomy to their principals. D’Angelo is utilizing one outside source and

they are currently entering their fourth year in the process. Both districts are

utilizing DataDirector as a conduit to DDDM, with Buck ending its first year and

D’Angelo entering its third year.

Buck Unified School District To highlight Buck’s initiatives and to create awareness within and

surrounding the institution, the superintendent is frequently featured on “The Unite

Show” an appealing and effective communication tool for garnering the support of

district stakeholders. In a recent show, the superintendent spoke of three initiatives,

two of which were: RTI (Response to Intervention) and the state of English

Language Learners (a district that has students who speak 39 different languages).

Both of these initiatives require training and support in the process of DDDM.

As noted earlier, a key training for high priority schools in the district is

professional development offered by Dennis Parker, an outside consultant. Training

105

of Buck principals includes an emphasis on data walls, data charts, and the

importance of how teachers collect, display, and utilize various types of data beyond

CST and CELDT4, i.e. data on student participation, whether students know and

understand the standards being taught. This outside consultant has worked with 4-5

schools throughout Buck on how to look at data at different levels and from different

perspectives. The assistant superintendent is of the belief that principals who

underwent this in depth process with the consultant coupled with DataDirector will

take the lead and bring along other principals in the process. Stating, “Most

principals are functioning at levels that span intermediate to advance.”

Having DataDirector for almost a year, the assistant superintendent states

‘intimate conversations around specific data with site leadership teams’ are taking

place. Last year, at the inception of the DataDirector implementation, conversations

were broader, and the assistant superintendent would meet with the principals before

actually walking their site. The conversation then was about their school plan and

what data would be used to develop school plan goals.

The Director of Child Development and Special Programs confirms by

describing that the enthusiasm for such data tools comes from the fact that there is no

longer a need to sit down with paper and pencil for hours crunching data. Now

4 CST, California Standards Tests, and CELDT, California English Language Development Tests, are two of several standardized tests given to measure progress in student achievement in the State of CA.

106

subgroup data can be accessed through punching a couple of buttons. He indicated

practical application embedded in professional development is the most effective.

Looking at English learners last year, we used DataDirector to print out a report on how many had stalled at the intermediate level on CELDT . . . there’s that glass ceiling and we showed them . . . here’s the table so you can see which kids are moving up and which kids are stalling. Most of the principals ran back to their sites, got on DataDirector, and shared it with their staff. So the key is giving them the tools and letting them take it back and use it.

Data Director, which was purchased last year, enables not just the site

administrators but the teachers to look at their current scores and also at longitudinal

data. The principals have all been trained in how to use DataDirector and how to

work with teachers to support them as they look at their own student data. There is

more support for the program improvement schools to make sure they have their

data, are looking at it, and are being supported in its use. Nonetheless, it is the

district expectation that principals will be proactive in utilizing the tools as a prime

resource for Data Informed Leadership. This was emphasized by several district

administrators, one of whom noted:

We are working with them; we constantly try to show them the benefits of using data. The power of using it to move a teacher who doesn’t agree with you, it’s really hard for them to make excuses when the numbers show what you are saying, the evidence doesn’t lie.

As a result of district professional development training and support for

principals in DDDM, one principal noted that the process of DDDM has been

refined through access to tools such as DataDirector. She talks about measures used

107

at her school site to improve instruction and learning via the training and support

offered by Buck. Teachers look at multiple sources of data, stating that they use the

basic reading inventory (BRI) by Johns, a school wide measure put in place before

they had technological assessment tools from the district. BRI is used school wide,

K-6, to track progress but also to have a way to compare. This allows them during a

grade level meeting or an SST meeting to have a common language. This is done

through a tracking card that follows a student from kindergarten to sixth grade. She

noted that the process of DDDM has evolved through the years and has heightened

with district accountability measures, and some of the means in the past have been

publisher tests or summative tests. She gave an example of DDDM by teachers:

I was with 3rd grade yesterday. They pulled out their chapter test, and were in the process of re-leveling their groups. So every teacher came in with her test, all leveled out by number and type of questions correct . . . deciding which kids were struggling and needed to be bumped down, which kids were showing improvement and could go up a level. They made a shift right there during that 45 minutes . . . mostly formative assessments at this point, thank goodness because it used to be autopsy . . . let’s look back and see what happened.

This approach to DDDM is most effective and is also aligned with the

district’s response to intervention (RTI) initiative. Initiatives are still balanced with

the need to provide principals with a voice in how the improvement process is

carried out at their respective sites. A middle school principal shares the process of

DDDM with respect to the training and support provided by Buck USD.

At the top of every agenda are the school plan goals and then the collaboration team’s learning goal. We do not call anything PLC

108

because people will think it’s a program . . . it’s the way in which you approach things, as a learning community . . . most of our staff has been trained by DuFour on professional learning communities because we had it district wide. My staff received that training and then last year I took a team of 5 to Marzano’s training.

This middle school principal stated that these trainings took her staff deeper

into the process. She stated that upon their return, “They bring back the fire and

passion for these things, then it spreads quickly where as if I’m the only one going

and bringing it back then it’s top down and that doesn’t do us any good . . . because

I can’t teach it all, I need five other people helping me out.” She also conducts

informal assessments of instructional practices, and similar to principals presenting

best practices at principal meetings, she asks teachers to present best practices to

their colleagues at staff meetings.

The sense of urgency, however, is greater at a low performing high school in

Buck USD. The principal of this high school stated that after the purchase and

training on DataDirector, teachers now isolate and segregate data, they know how

many kids they need to move forward, and how many kids are sitting in the bubble.

He stated, “The kids know where they’re going, how many benchmarks they have

passed, and how they’re progressing.”

A possible significant hurdle in moving a district quickly into the realm of

accountability, via DDDM practices, is the relationship that exists between a district

and the teachers’ union. The superintendent and other administrators interviewed

noted that there is indeed a partnership in Buck:

109

There is a partnership where the union really cares about student achievement. It’s not a union that sits there and tells you they only care about teachers and kids are secondary. They actually tell you that they don’t want bad teachers among their staff. I mean they are still here to represent teachers. They will fight you, they’ll come back here to defend the teacher, but their core roots are research. They want you to look at data. They want you to actually do your job as an administrator and they tell you, do your job, if you have a bad teacher, do your job and get rid of them . . . but do it right . . . and they’ll support you.

D’Angelo Unified School District

Support is also a profound element at D’Angelo. Through interviews with the

superintendent, district, and school site administrators it was noted that the process

of DDDM is reinforced through training and support received through the Focus on

Results framework, which is currently in its fourth year of implementation. The

Focus on Results meetings are monthly where the principal and the school’s

leadership team attend and receive in depth training on the use of data. A recent

decision was made to eliminate principal meetings where district simply passed

along information that could be given in a memo. The superintendent noted, “We’re

trying to make them (principal meetings) meaningful training sessions with an

instructional focus.” The Superintendent noted, and this was corroborated by other

interviews with other district administrators and teachers:

The instructional leadership team has become such an important piece of the improvement process, in the best of cases . . . the instructional team drives the process. The principal really almost becomes a facilitator of the process in best practices. So it’s interesting to watch, now that I have principals leaving for various reasons, with a strong instructional focus and a strong team (at a school) a new principal can

110

walk into a situation and immediately hit the bricks running because all that principal needs to do is support that team’s direction. The process of DDDM through Focus on Results has been streamlined.

Principals and their instructional leadership teams, with the assistant superintendent,

described the process as “very structured and organized.” A notebook shared for

document analysis delineated the leadership expectations, process, and tools, funded

through a best practices grant. In addition to the Focus on Results meetings, there

are instructional walk through days, mentioned earlier by principals. The

instructional walk through consists of an assistant superintendent and two other

schools. He reports, “All our schools have defined their best practices . . . and

address one of the weaknesses through data collection . . . the other schools are

versed on what to look for in a pre-walk staff. Then we break up into teams and go

through classrooms and look for evidence of best practices.” He also indicated that

this past summer the district has continued to weave things together in their PGP

(Professional Growth Plans), stating that they’ve integrated Focus on Result into

those plans. The goal is to have plans driven based on data and the results, noting

that “. . . we’re using interim and common formative assessments to evaluate both

pre and post to see how well they’re doing. Now is that implemented district-wide?

No, but we’re in the process . . .”

To provide the training and support to establish uniformity in best practices

across the district, district administrators are coached by Focus on Results

111

consultants. The district has structured staff development so that it will be self

sufficient in about three years wherein the consultants work with a group of writers

in the summer and through on-going training. They write the staff development for

the cohorts, with three cohorts -- they’re all in different places. The trainings will be

written and run strictly by the writing team and then it will be presented by district

administrators. The writing team consists of teachers, principals, and teacher

specialists. And the team has been trained in presentations.

The process of DDDM, however, is tailor-made to each school site. In other

words, it is decided by the instructional leadership team. For example one of the

high schools has writing as their best practice. They’ve done staff development

around writing and so the goal there is to raise the level based on Dr. Douglas

Reeves’ research. School site administrators share how the training and support

they’ve received from the Focus on Results professional development has furthered

the improvement process at their school site.

An elementary school principal notes that Focus on Results has allowed

principals and instructional leadership teams to have common discussions at district

meetings, to work together, and to “Borrow and steal, and ask what are you doing?

How is that working?” She also noted that they plan their own professional

development based on what teachers believe they need for their next steps in order to

help them become more proficient in delivery of instruction.

112

Teachers at this principal’s elementary school get a folder with all their data

at the beginning of the year. They see the standard based report cards which include

all assessment results. They can identify their second language learners, their GATE

students, how students scored on their benchmarks, and how they scored on their

CST’s. She states, “They get the whole picture. We also have an intervention

program, and they’ll figure out which students are going to intervention right up

front.” They meet four times a year to look at grade level data, which are summative

data. Grade levels do a great job planning collaboratively, every grade level, K

through 6th. Similarity in classroom instruction and environment are prevalent. The

school principal notes that “. . . a lot of it comes from sharing best practices.” This

principal attributes success in her school and district wide to commitment, stating: “.

. . our superintendent sits in our meetings with us. Board members come and sit.

There is no question about the level of district office commitment.”

A middle school principal also expresses the impact of the effort D’Angelo

gives in providing training and support in the process of DDDM. She stated that

when she began with the district, 10 years ago, they were already beginning to work

with data and that it has just gotten better and more intense. She noted access to not

only CST data, but district benchmark tests, grade level data, classroom data,

wherein teacher and student growth can be measured, but also growth from school to

school within the district. She noted, “Healthy competition never hurts anyone and

you know very well as a teacher you want to know how well this other school has

113

done? Or how did this other teacher do?” She noted an example of uniformity

throughout the district is in the writing benchmark at every grade level, which is

normed, administered, and graded by each teacher at each grade level. Furthermore,

the district math coordinator does analysis of CST results, benchmark tests by

teacher, school, and district. These reports are provided to the teachers each year.

The lower performing schools have priority to the data and receive their reports first.

She adds that for her school the plan and process for DDDM is school-wide and by

department.

I give them three years worth of data . . . one year is not enough . . . could have been a group of students, two teachers left in the middle of the year… I don’t want to hear those excuses. I want them to look at the pattern. As a department they will come up with a plan and then it is their responsibility. Electives have a responsibility to help Math and English. They will look to see how they can incorporate math or writing into the PE curriculum. Nobody is going to take the praise for succeeding and no one’s going to take the blame. This middle school principal shared that Focus on Results professional

development is rather structured, with planning time allotted in the afternoon. The

team gets together using whatever data they have, using whatever training they have

received, to sit down and plan the next month’s staff development and bank time.

The information learned is taken back and shared with the ILT, Instructional

Leadership Team, which consists of every department chair in addition to anyone

who wants to be part of that meeting. Then the ILT disseminates the information to

everyone else.

114

Another component in the training and support, in the process of DDDM, is

the coach assigned to each triad, which could be an assistant superintendent, a

director, or another principal. The coach would meet with the principal on a monthly

basis to discuss the improvement process at the school as well as talk about the

expectations for next month’s visit. The routine of following up on the process of

improvement is consistent. This is also mirrored in the relationship between the

principals and teachers. For example, a principal shared that she asks departments

to take a look at last year’s data and see if they still glean the same problems in the

same sub strands. Trend data reinforces the point that the challenge is not the

students, but rather the instruction that is happening in the classroom.

On the north side of town, near the mountains, is one of D’Angelo’s high

schools, which was confident in its high performance. The high school principal

shares, “In 1999 when we got our first STAR test results . . . a strong highly qualified

school was 800. There was an assumption that we would come in at 800 or maybe

810 or 820, who knew, but there was this strong sense that we were the kind of

school that an 800 would indicate.” She noted that the first results came in at 759,

and the next year they went down one point. The superintendent at the time called

the administrators together and asked, “What do we need to do to get these high

schools moving?” This high school principal along with her co-principal put

forward the idea of a standard resource teacher (SRT) who for one period a day

would focus on academics, looking at the blue prints, and making sure that the

115

curriculum matched, that the instruction given to the students matched the state blue

print. They began the process first with English and Math, now they have this in all

four subject areas: English, Math, Social Science, and Science. Data analysis at this

time was conducted via data notebooks provided by the director of testing. The

principal would take a team of 6 teachers, the department chairs, down to the district

office to look at the data, analyze them, and plot a course. In going through this

exercise, they learned from the data that their students were not doing well on say for

instance the genetics section of the life science exam. They may discover that

genetics was worth a phenomenal amount on the test but was only covered briefly in

the former part of the school year or perhaps the textbook didn’t cover specific

material that was tested. Armed with specific knowledge and skills, they were

equipped to gauge and carry out instruction effectively.

As the district adopted Focus on Results framework and DataDirector for

their data management system, the principals noted that they were able to delve

deeper into what was happening, particularly with the subgroups. The high school

principal reports, “We started this improvement process and last year our API was

861 and this year we’re predicting 870 or higher, so we have made substantial

progress through a very deep look at data.” She noted that the process of

improvement involves pulling together teams to discuss data that concern them and

district administrators allotting time for school teams to look at data. She states the

116

Focus on Results process not only allows them to look at the data but to put together

a course of action on how to use the data.

The process of DDDM is embedded in practical application but in research as

well. Focus on Results training for principals includes professional reading and

discussion, such as the Marshall Memo. A principal notes, “. . . for the leadership

team, I think it’s essential. I always try to find articles that are focused on where we

are and what we are trying to do and that reinforces (the professional development at

the school site). People who are studying this have determined that this is going to

make a difference, we need to keep going down this road . . . it’s a reinforcer.”

There was consistency in the interview, observation, and document data

analysis from Buck and D’Angelo regarding the evolutionary process that occurs in

the adoption of assessment tools as well as processes for DDDM. Both districts

made an investment in the data infrastructure as well as in developing leaders’

expertise and data literacy. Through these investments, both districts were able to

develop cultures of inquiry in their respective institutions. These investments

consisted of both districts purchasing DataDirector and contracting outside vendors

and using in-house experts to drive the process of DDDM. Both districts have

principals who were proactive and advocate, pushing the process along. Some

principals at each of the districts realized that distributive leadership was necessary

to effect real change in building a culture of data use and analysis, with a process of

ongoing inquiry. With the districts making a concerted effort to provide professional

117

development to increase student achievement, how do they gauge the level of

implementation and its effectiveness?

Evidence that Leaders have put New Practices into Place

In an effort to build data expertise by principals, Buck and D’Angelo

demonstrate it is not enough to simply invest in data infrastructure and in training

and support. Both districts realize that the crucial question is how can the

effectiveness of such investments be measured? A foundation, for gathering

evidence that tools are effective and that processes are being learned, is to create an

internal accountability system within the institution. Interviews with district

administrators shed light on how the effectiveness of professional development for

school site administrators in DDDM is measured through creating a culture of

inquiry and an ongoing cycle of inquiry. Answers to the overarching question are

presented in themes 5 and 6.

Theme #5: Effectiveness of Professional Development

Effectiveness can be measured on a variety of levels, such as on a rubric of 1

to 4 with set criteria, according to the superintendent of Buck USD. He gives an

example, “Principals are trained on how to present data to teachers. A principal goes

back to their school site and makes the presentation. Is that effective staff

development? Certainly on one level it is because it’s being utilized. Utilization can

be checked off. Then if one were to ask if the utilization is effective, rating effective

118

utilization would be to say it is making a difference in student achievement.” He

noted that looking at student achievement is a bit removed because now you go from

the district to the school to the teacher and then to the students. He also noted that

“long gone are the days of the Madeline Hunter model, where evaluation is based on

teacher behaviors. There needs to be a relentless focus on student behaviors, asking

the question: What are the students doing?” Furthermore, “Staff development that

never reaches the students is like not having done any staff development at all.”

Thus, a plan with action steps from a district level must be formulated to

determine the effectiveness of the district’s professional development. Buck relies

on achievement data to dictate discussions that probe or elicit action from principals.

For instance, the superintendent notes attention will be paid to a particular grade

level at a specific school which has excelled in a specific subject. The principal of

the school will be asked about the instructional strategies utilized. On the other

hand, there could be a school with a grade level that’s floundering, and then the

questions would be: What do you think happened? What kind of staff development

did you do? Was there a disruption? Once those questions are asked, district

administration will pull back. If a principal conducted staff development, then the

question would be: Why wasn’t it effective? He noted that it is not about laying

blame because “we all have that gut ache” but there is a need to ask the staff

development question, “What got in the way of student achievement?” As the

interview proceeded, Buck’s superintendent stated that the ideal professional

119

development should be a trainer of trainers’ model, which he believes most only give

lip service to, noting: “We never really do a staff development that says when you

train people, you bring the box in, you set the projector up, you access adult learning

file. Simply do the thing we are trying to get them to do.” He noted that this model

of professional development should be at the principal and teachers level. In other

words, how does one work a room of 30 teachers or 30 students? This idea revisits

the notion that data alone does not ‘drive’ action rather that Data-Informed

Leadership fostered from the experiences and wisdom of leaders is more likely to be

effective in addressing the challenges of ineffective professional development or

ineffective principals in carrying out the knowledge and skills learned. However,

Buck’s superintendent also notes that there are other elements which serve as a

hindrance.

I think we need a longer school day and a longer school year. I think it’s ridiculous what we have now. Theoretically, the way we test is, I’ll say it, stupid. At best we have 180 days and we test 90 percent of that year, so what 10 percent do we get to take out of the standards. The answer is nothing. The state should say that we are going to test at the end of the year, a minimum of 4 days at the end of the school year, after everything is over. How about 100 percent of the year to teach 100 percent of the curriculum? That’s real simple . . . if the state has to pay for 4 more days of school well then so be it, but test after the instruction is completed.

External elements play a factor; however, the assistant superintendent is

convinced that the professional development she personally received, during her

principalship, through Buck’s leadership academy was effective. She passed this

120

knowledge onto her assistant principal, who is now the principal of a Title 1 high

achieving elementary school for the third year in a row. “We’ve evolved as a district

over time and I would say that my capacity as a site administrator really came from

the training that was provided to me during our principal leadership academy.”

Building profound relationships within the district through a step within the cycle of

inquiry, specifically ‘taking action and communicating’ goes a long way in district

administrators accessing the information they need to measure the effects of

professional development. The assistant superintendent notes that the principal of

this high performing Title 1 elementary school conducts data chats with each student,

calling in every student, one by one, and talking to them about the evidence of their

performance level. She emphasizes to the students that the only score they need to

beat is their own. Along with the data chats are the milestones where success is also

celebrated. The assistant superintendent, however, noted that the cycle of inquiry is

not simply about random inquiry that is elicited from a district administrator or

shared by a school site administrator. She links the focus of her inquiry directly to

initiatives and that invariably becomes a measure of professional development

effectiveness. She gives an example,

Last year we had two initiatives, English Learners and Response to Intervention. In terms of measuring the effectiveness of professional development, I went to the sites and asked them to share the percent of students who moved up a level? What does that (ELD program) look like? What do you need in terms of materials, professional development? So from collecting the data, it became apparent that we needed to do some site workshops. We take the professional

121

development on the road, to that site, and target what that site needs because each school in district has a different need and a different structure. We try to customize our professional development.

The assistant superintendent noted that sometimes the district leadership team

will walk through a school site. As they walk through the classrooms they are

looking for best practices that are making a difference in the classrooms. Targets

are set for the next visit, where the principal has a conversation with the teachers to

get them on board to meet those targets. School site observations accompanied with

data are used to look at best practices and to shine the spotlight on areas of need.

More importantly there is an internal accountability between the stakeholders

within Buck, which includes the district, school sites, and the union. This year

they’ve introduced a new evaluation system for principals based on the CPSELS, but

even with the old evaluation system the caliber of the principal’s efforts is

demonstrated through their own efforts. Principals were expected to set up goals.

These goals are connected to the Single School Plan (SSP). The SSP is married to

the strategic plan so that district and schools are synchronized. Principals write goals

not only for academic achievement but for other areas such as staff development,

fiscal resources, and community support. This allows them to reflect on their

achievements throughout the year prior to meeting with district administrators. The

assistant superintendent notes, “You know who the top principals are . . . you know

because they are taking schools higher and higher every year.” These principals set

up goals and produce the evidence that they’re achieving those goals; however, “. . .

122

if you are a principal who’s not getting it done, the teachers are going to complain to

the union. . . we have a good partnership with them, they’ll let the superintendent or

deputy superintendent know, saying you know what, we have a problem here.”

Principals who are effective and utilize best practices are proactive and furthermore

take the step of sharing their knowledge with their colleagues.

The principal of a Title 1 school (with an API over 800) was asked to present

to her colleagues. This principal, according to the Director of Child Development

and Special Projects, has “. . . taken data, created that sense of urgency, and then

used it to show progress . . . she gets everybody behind her.” This type of leadership

is extraordinary but requires a skill set that could be evaluated through the new

evaluation system, connected to CPSELS, in that documentation and a portfolio will

demonstrate evidence of each element.

According to the Director of Schools, however, the district cannot forget

about the leadership development aspect of professional development, those

elements that are beyond CPSELs. She stresses the importance of having the

knowledge and skills needed to manage change, innovation, dealing with issues of

ethical leadership, and issues of communication. She notes that these are the ‘big

umbrella’ issues that are not directly addressed in CPSELs. She notes, “I think often

times educators forget that there are things to be learned about leadership or

organizational theory from the business side.” She references Pullman and Bowman

and Deal, noting “These are people who really speak to the power of leadership and

123

leadership with a big L.” She notes the fact that these can be taught or at least

brought to ones level of consciousness so they are aware of the effect. So they can

be strategic about things they do that are specific to their role as a leader, not

necessarily an educational leader but as a leader. She states that this could be a huge

pitfall and if not addressed, principals will end up being simply managers of schools,

“That’s not going to help us to deal with all of the challenges that education is facing

right now.”

Evaluation of an abstract concept, such as leadership skills, is gleaned by

Buck district administrators from collaborative interactions that take place at

leadership academy meetings or observations of professional development held at

the individual school sites. The Supervisor of Curriculum and Professional

Development notes that access to the principals is always a challenge, noting they

are not always the best at reading and digesting all the e-mail. To have them as a

captive audience at leadership meetings allows for observation. It also is an

opportunity to listen to their comments and ideas. She notes it is not that big of a

group, there are 29 principals. This allows a more intimate setting to engage

principals, but the most valuable observations, are those at the school sites.

Through a culture of inquiry by district administrators they garner how

principals take action and make sense of the improvement process. A principal

describes how she carves out time for collaboration and intervention:

124

Some of the actions we’ve taken are providing library and P.E. back to back, two subjects that can be monitored by a credentialed teacher with the kids going 45 minutes to the library and 45 minutes to P.E. The whole grade level goes at one time so the teachers are all released during that hour and 45 minutes for grade level collaboration. Intervention is also done between the bells because we noticed for years students in need of intervention don’t come after school. I really have a between the bells philosophy because I feel that’s really the only thing I have control of. Her philosophy, which is observed by district administrators, is based not

only on her philosophy as an educator but through her organizational leadership

skills. The practice is effective because it has shown results in student performance.

I think one of the reasons we are successful is we do look at each subgroup. We went over 800 two years ago, then to 801, then 815, this year 843, but the most exciting thing is our subgroups which are recognized by the state, which is our English Language Learner population and our low socioeconomic students, and our Asian students. All those groups are over 800 and that just happened this year. The low socioeconomic and the English Language Learners had 48 to 49 point gains this year and went over 800 so we are thrilled. I was more excited about having those subgroups go over 800 than I was about the overall API score.

This principal goes on to explain the effects of a variety of professional development

offered by the district have made a difference at her school site. She noted that a

school must embrace the concept of professional learning communities. Several

staff members at her school site spent several days in summer institute training,

allowing for time to make plans on how this would look at their school by looking at

pieces such as grade level planning and the grouping of students to help them at each

level. At the same time as the PLC training, her staff was introduced to RTI, which

125

was fully embraced. It was a combination of these professional development

opportunities that made things work at their school. She stated that training through

AB430 made her view things in a different perspective: “I came into teaching at a

time when there was really no curriculum. I walked into my classroom the first day.

. . I was hired during the 20 to 1 year and I was given a white board, teacher’s desk,

and a chair and I really didn’t have much more than that.” She believes that the shift

to standards is impressive, but moreover the concept of fidelity is a stretch for a

district which prides itself on innovation. She states, “This year we’re not going to

deviate. We are not going to do lessons out of order . . . I said to them (staff), I can’t

even believe I’m saying this to you right now, but because this is our learning year,

let’s just do it. Let’s just follow it.”

A middle school principal notes the district gauges professional development

effectiveness by referencing the school plan, the district plan, and where data were

used, stating, “. . . our district, to its credit, is very trustworthy and administrators

will visit your school, and will bring up (issues). Let’s talk about this, but nothing is

ever tangibly handed in.” She notes “. . . it’s easy for them to see if you are or are

not implementing a plan” stating that all district administrators are highly engaged in

the leadership academy professional development sessions. The district utilizes an

informal measurement based on leadership academy discussions, guiding the

discussion by addressing data use and then using the administrative discussions as a

sounding board for what each site is doing. The data from principals mirrors the

126

informal data collection methods espoused through the interviews by district

administrators. This middle school principal admitted that not handing in evidence

and engaging in a professional dialogue allows for “a culture of trust.”

As stated earlier by the superintendent, the effectiveness of professional

development is measured by the effect on students. This middle school principal

indicated “Because we’ve made data non-threatening, the kids are starting to log

things and talk about their own scores.” She describes the culture of trust as

‘spreading’ with everybody logging everything and students charting their own

progress and teachers are charting student progress and talking about it freely. “I am

finding that the students aren’t as afraid to talk about their own data. Well how did

you do on that test? Why did you answer that question that way? So I think the

culture of trust in using data is a big thing because data are not about us, it’s about

those kids in the classroom and if we train them to use it, then we know they have

ownership.” She continues, “I know that the big push is to have teachers read data.

Data is of no relevance until students can understand how it impacts them too and so

I think we forget that in the equation and until we put it back (in their hands), data is

not going to be as relevant for schools.”

Like Buck, D’Angelo’s superintendent states that there are multiple measures

that demonstrate whether a school site administrator is meeting the expectations of

the professional development training. “There are multiple indicators of whether the

principal is truly doing a good job. We have a unique evaluation process of the site

127

principals in that they are evaluated in October each year, rather than the end of the

year and the state test results are part of the evaluation process. Everyone seems fine

with it . . . But I can tell you it’s a norm now. I don’t feel like the principals are

overly anxious. They’re going to have to do a study to ensure that what they’re

doing is effective. And we actually have a model that they’ll use and we’ll take them

through it.”

An elementary school principal reports that one of the expectations from the

district is to be in the classrooms 50% of the time. “I do. I cover their classroom, so

they are in another classroom. I want them to be able to see best practice. I am not

always in their room watching. Sometimes, I take their classroom while they go

teach a lesson in another.” She continues by indicating that another expectation from

the district is that each site determines its best practice. They provide their own

professional development, which needs to be research-based. They may need help

from the district for references or people within the district that can provide the

professional development, but there is no one overseeing professional development

like when there was a professional development center. Everyone is doing whatever

they want. They all do something different based on their instructional focus.

Furthermore, Focus on Results includes regular walk-through with a large

group of people from different schools. Four times a year they have a walk through,

a team composed of other schools to observe, focusing on specific questions. The

128

first year began with evidence of school focus and the following year evidence of

best practices. Last year the walk through was evidence of SEAM.

To role model district best practice, this elementary school principal conducts

internal walk through at her school site, where they (teachers) all go together.

“We’ve been doing internal walk through after school during a bank day, where they

can go see one another’s classroom. Every meeting I have is in a different

classroom.” During a walk through the principal will use evidence to ask hard

questions. Part of the plan for this year is with teachers who are struggling with

instruction. They will be required on a monthly basis to visit other classrooms in

order to refine their instructional practice, look at pacing, and at the level of rigor.

The middle school principal shares that the effectiveness of professional

development manifests and is gauged by the level of ongoing dialogue between the

principal and the assigned district administrator and/or the Focus on Results

consultant. District administrators look for evidence during the walk through, which

elicits more inquiry. The improvement process in D’Angelo mirrors the cycle of

inquiry as demonstrated by Data-Informed Leadership. The process is supported and

then witnessed by the district. One middle school principal shares:

Every Wednesday, for 4 to 5 years, we (principal and teachers) met, talked, strategized. . . now district has helped and picked up the cost of two people . . . they are not pulled out, they are not at the district, they are at the school. We have been working with Marzano’s most effective instructional strategies. . . how is this different from what you learned last year, how is World War II different from World War I . . . you are building a wealth of knowledge.

129

Another mechanism put in place at this Title 1 middle school is a program for

students with three or more F’s, wherein by the end of the year the total number of

F’s for the entire group of students has diminished drastically. It is said to be “a

highly successful program involving student tutors.” Student tutors are placed in

twenty to one algebra classes and twenty to one intro classes. The school is unable

to hire adult tutors so students tutor, who have a gap in their class schedule or study

hall. The current staff consists of teachers who were once student tutors. The school

offers a community service medallion to anyone who does 100 hours of service.

Last year’s graduating class gave 55,000 hours of service to the community and a lot

of it was service to fellow students, a tremendous help to low achieving students.

Evidence of effective professional development is manifest in the form of

student success. One high school principal states there are significant reviews in

ninth grade. There is a district writing benchmark which is studied carefully and

then in first semester of the tenth grade, all the tenth grade teachers give a former

CAHSEE5 essay. Subs are brought in for the day and all English teachers score

those essays. This high principal states, “When English teachers get together and

score, it is amazing staff development.” After this, they do significant reviews in

every tenth grade English class. An administrator or an SRT, special resource

teacher, visits every tenth grade English class to explain the CAHSEE and what role

5 CAHSEE, California High School Exit Examination, a test which all public school students are required to pass to earn a high school diploma.

130

it plays in their educational process. For students who haven’t passed, they offer an

intensive program in the summer. The test results and graduation rates attest to the

effectiveness of this school’s improvement process.

Furthermore, research based assessment strategies are conducted and deemed

effective. The common formative assessments are teacher created at the school,

given to the students, and returned quickly, then re-taught based on the results of the

assessments. The principal attests that this is when “you get the gain.” She notes

that staff development days this year will be focused on common formative

assessments and the goal is to get four in place this year for each subject area. The

principal praises her staff, “I think one of the keys to this school’s success has been

teacher involvement which is one of the big pieces on Focus on Results. We sit and

look at what’s strong and what needs to be fixed.” This high school principal utilizes

the power of distributive leadership to assure effectiveness in the strategies and to

assist in guiding the improvement process.

The genuine efforts and resolve of district and school site administrative staff

at Buck and D’Angelo are coupled with the recognition that research based

professional development is the conduit to school improvement efforts. Professional

development effectiveness is measured via formal and informal assessment

(evaluation and discussion/observation), by formative and summative assessment

(walk-through evidence and CST data). The measurement of effectiveness of

principals mirrors the same accountability piece that is in place for teachers via the

131

professional teaching standards and students via the California State Standards. It

must be noted that the effectiveness of the professional development manifests

through the cycle of inquiry -- accessing (knowledge) or searching, sense-making,

taking action and communicating, learning from action, problem (re) framing. This

cycle is possible through districts expending great effort in building cultures of

inquiry, and specifically in this era of accountability -- a culture of data use.

\Theme #6: Culture of Data Use

There is a sense of urgency to accelerate the competency of educators in the

use of data, based on NCLB expectations for 2013-2014. Accessibility of data tools

and its attendant knowledge and skills for administrators and teachers is important

and necessary to begin the process of ultimately developing a sustained culture of

data use. The culture of data must also be supported by system-wide learning based

on results from the implementation and effects on student, professional, and system

learning (an element of Data-Informed Leadership).

Buck Unified School District

At Buck system learning is not an option and is embraced by most. System

learning, however, does not mean loss of autonomy. Buck’s superintendent highly

regards the anchors of Data-Informed Leadership, core values and theory of action,

when considering system learning for those within the district. Values by which he

makes decisions have implications based on a theory of action that he shares with

132

key players in the organization. With the understanding of how Buck operates and a

method for how it can intercede to improve, he shares:

For example the reading program . . . we actually have three reading programs at the elementary schools. Now that can be a problem when you are trying to do overall staff development; however, the plus is that teachers and principal at that site, the educators, the colleagues at that site, feel empowered to make the best decision for their community. Schools are made up of two kinds of communities. The community they serve and the community they are themselves and if you get a program that the teachers work between the bells on . . . you will get compliance. So teachers and the principal that own the program are going to devote more to it and take it beyond compliance, we hope, through to greatness so that’s why we encourage, not allow, we encourage that kind of school by school decision-making.

The core value of autonomy in the use of curricular research-based materials

is an example of one of the anchors of Data-Informed Leadership, an extension of

core values is a theory of action. For Buck’s superintendent, the empowerment of

principals and their staff goes a long way in opening the door for establishing a

culture of data use. At Buck this has been one way of building trust and respect

between the district and teachers. Buck’s superintendent proudly describes how

partnerships facilitate ease of use in building a data use culture:

We have a partnership with our teachers union that is a decade old. We don’t fight the same way other people do. We don’t pay lawyers for grievances and neither do they. They devote tens of thousands of dollars -- I’m talking hard cash dollars, to providing staff development to the teachers in our district. It’s rather amazing. I meet weekly with the union president.

He reminisces about an occasion when one of the union leaders came in from the

national organization, looked at the teachers and said ‘I don’t know who’s been

133

talking to you but you’re not working hard enough,’ and this was a union leader

talking to teachers, so that partnership provides incredible human resources but also

absolute material resources. He went on to share that his predecessor, who came

from outside of California, told him that “there are no prima donnas at Buck, just

hard working people.” Their experiences dictated that most districts had islands,

people who horded good news and that isn’t the culture at this district. “People want

to share something that’s working. They’re freely and eagerly sharing.” He

continues:

Everyone knows we need to use data. We’re a district that wants to be honest and take a look at ourselves. But my role is to keep that in front of us and require us to take an honest look, focus on the gap with Hispanic and African American students. The gap is there, I’m going to show it (at the Principal’s Leadership Academy), and I am going to ask, ’Now what do we do? After adopting RTI, we analyzed the data after students had failed . . . even though we didn’t want to do that, and we have (received) wonderful responses to that. We achieved a level of improvement. What we neglected is the data that says we should be impacting the original classroom instruction. The superintendent is of the firm belief that data should drive classroom

instruction. “One must ask, ‘How can African American kids be successful in this

class? How can Hispanic kids be successful?’ Not after school, not on weekends,

but in this class that you are teaching.” Data are a significant part of the decision

making process. Data use has been an evolutionary process, where in the beginning

it was prepared by the district and given to administrators. Now DataDirector allows

data to be at administrators and teachers’ fingertips. This entire shift happened in a

134

year time period. Discussion about trends and looking for patterns within current

subgroups of students are topics of conversation. If there are patterns within content

areas, identification of those trends in the data are made.

With this in mind, they began to look at data trends by students and tried to

put a face to a name, to really personalize the data. The other thing they did was to

pull out blueprints and to look at the percentages for the blueprints. Assessments

focus on how did students perform in bands and then correlate that information and

look for discrepancies. After that they look at best practices in specific areas, and

how to duplicate a practice beyond one classroom. Such conversations became the

culture of the school and the district. The assistant superintendent recalls the time

she was a principal in the district, stating:

Analyzing subgroup data with staff and noting the gap with the African American subgroup, and because I’m African American . . . they saw it, and a teacher was whispering. I told the staff, ‘let’s say it, because if we are not comfortable saying it, then we’ll pretend it’s not there.’ I told lots of personal stories. I really wanted teachers to see these students as if they were their own and I believe this is what began to change the culture. We began the language of adopting kids, having data chats with them, pulling kids one by one to our desk . . . making data number one, having them set goals. This was a result of district training . . . training with Dennis Parker.

Culture of data use began prior to the installation of DataDirector. Teachers

have received training such as Success for All6, DIBELS7, and other reading

6 SFA, Success For All, is a reading program which can be paid for by Title I and State Compensatory Education funding.

135

assessment tools as a source of data to measure student progress and make

determinations about strengths and challenges, and methods of instruction and

intervention. The Supervisor of Curriculum and Professional Development noted

that sites have used local measures for a number of years. She anticipates that it will

take time to get everyone to a level of proficiency where they are able to enter and

access data through DataDirector. It is anticipated with time, training, and support

this will become a norm district-wide.

An elementary school principal notes that she only meets with teachers

during grade level collaboration once a month. She states that it’s difficult for a

principal to get to all those meetings, but also that she doesn’t want teachers to feel

that she’s “looking over their shoulder.” She wants them to feel empowered not as

if she’s a tyrant looking down on them. Furthermore, she is aware that many

surrounding districts operate differently than Buck. She notes that this year is a pilot

year for benchmarking in the district, and attributes the late timeline of implementing

benchmark exams to the culture of innovation in the district. She notes that

principals are really given a lot of freedom to look at their school site and say, ‘this is

my unique population and what can I do to help them specifically, instead of a

blanket approach. She notes that this has its good and bad, “. . . because you seem

7 The Dynamic Indicators of Basic Early Literacy Skills (DIBELS) are a set of procedures and measures for assessing the acquisition of early literacy skills from kindergarten through sixth grade..

136

like you’re out there on your own quite a lot. However, if you do come up with a

great idea, you can get some support to back you up.” This year Buck starts math

benchmark assessments, and this elementary school principal’s team has requested to

be part of the benchmarking process. This step into to trimester benchmark exams

begins the cross over from a culture of innovation to establishing a deeper culture of

DDDM. Trust is a factor in the process.

Through interviews and observations, the importance of trust and respect are

evident. District administrators as well as school principals value a climate of trust

and respect through relationship building. With one principal noting that building a

climate of trusting relationships is important “. . . when I do propose something, or if

someone comes with an idea, we can all say, hmm, you know, should we try . . . if

she says it’s going to work, its going to, you know, we should try it.” She indicates

that this is a big consideration that is often times overlooked, going on to say “The

groundwork has to be laid in order to make big changes.”

A culture of trust and respect opened the door for one principal to introduce

the concept of fidelity in the Houghton Mifflin language arts curriculum, appealing

to the teaching staff to participate in action research to see if indeed this will make a

difference in student achievement. This suggestion was born out of AB430 in-

service that the principal attended, which noted that a disservice was being meted

upon some students by not allowing them to benefit from explicit and direct

instruction. At the end of the day, after teaching the series with fidelity, the school

137

had a 31 point gain. This principal noted, “So I’m a believer because what it really

does is give every kid a level playing field.”

The proficiency level of students can be determined by how they learn, and

may be similar to that of teachers. Each grade level chooses assessment tools and

not all will use the same thing. For instance when teachers are clustered producing

and analyzing data some teachers learn through a more interactive environment.

Other teachers are more proficient and already have their data and have produced

nice graphs through DataDirector, a tool which they were not privy to a year ago.

The evidence of a culture of data use is prevalent when teachers are utilizing

common assessments, and one sees students charting their own progress.

A middle school principal reports, “If it’s just the teacher charting the

progress, the student doesn’t know what it is and it doesn’t have any relevance.” The

culture of data use is strengthened by the benefits by which it provides teachers to

gauge instruction as well as by the way it allows students to regulate their learning.

Another principal is of the belief that the culture of data use begins with an

introduction from the district level, but ultimately there is one purpose and one

purpose alone, student achievement. If utilized incorrectly it can be detrimental to

building that culture. If its purpose is to improve instructional practices, then it will

help in moving students to the next level. A principal reports that the district began

using data based on surveys given within the community and at the schools. The

superintendent presented the data based on these surveys, sharing community

138

responses, and administrators would learn to look at this type of data. Then, district

data would be presented, noting that it did not score as high as expected, and the

superintendent reporting that this is an area that needed work. He would ask the

question, “How are we going to fix this so that the next year when we take this

survey, we are going to do better?” This type of conversation brought data to the

forefront and was the beginning of building a culture of data use. Data were then

presented based on the strategic plan goals, such as the data on ELL students. He

would ask questions such as, “How many students were being re-designated?” In

turn principals brought that data back to their schools. Each principal would present

the data that were pertinent to their school and staff.

A school site administrator views the process of building a culture of data use

as informal; although, it was introduced at the beginning of this year through

CPSELs, which will be the basis of principal evaluation. Because of the ease of use

of Data Director, it is the consensus of those administrators interviewed that

everyone has bought into it in a short period of time. This air of acceptance was also

noted through professional development observations conducted during this research.

A high school principal described the culture of data use as synonymous with a

culture of an internal accountability within each teacher . . . stating that this has

always been the goal. He noted that data has no blame, but that everyone is held

accountable for their data, which is a cultural change in his school, stating “Now, the

culture of the place is that everybody looks at everybody’s data. This principal notes

139

that the ultimate message is that the principal is not going to come get us, but rather

we’re not going to let each other down.”

This high school principal describes how the culture of data use is evident on

the walls of the classrooms. It can be seen with teachers, how many standards can be

covered in a day, how many times a teacher is seen sitting at the desk when you’re

walking through. Data are used to track the number of classrooms administrators

visit each day. Data can also reveal student effort, such as how many times

homework was not turned in, a test score, and what it takes to be proficient.

Building a culture of data use requires use across the board in everything a school or

district does, and in every conversation. “This year it finally feels as if it’s working

and moving.” The first year the start was slow, showing data, not making much

noise, emulating the district’s approach to introducing data use. Then the next

meeting they showed a little more. Teachers became comfortable, and without

telling them, they began sharing it with each other.

D’Angelo Unified School District

D’Angelo’s superintendent shares how a deeper culture in data use at his

district was established. He indicates that a framework made all the difference in

establishing a deeper culture allowing for consistency district wide. After searching

around and being aware of the Focus on Results process, through various

connections and success stories at another district, where he was close friends with

the Superintendent. He began the process of trying to get the principals to buy into

140

the program. Also a funding source was available to make this process materialize to

encourage and support entry into the program. The process then began with thirteen

schools. The superintendent was told by an assistant superintendent that trying to

develop a district wide program was going to be a waste of time and would never

happen because the district had provided so much autonomy to school sites. With a

complete focus on student achievement, the second year nine more schools were

brought in, and then last year the remainder of the schools. The Focus on Results

framework, which is research based, allows for differentiation from school to school.

It helps each school identify the instructional focus by using its own existing data.

The superintendent notes that there were challenges along the way, with the

union attempting to sabotage the implementation process; however, because it is so

teacher driven and teachers who love what they’re doing, loved being a part of the

process. The process allows teachers the opportunity to provide leadership. Some

schools have three to four teachers on their leadership team, with others having as

many as twelve. The superintendent has given principals the room and space to do

that. He noted, “They change behaviors as they trust you and believe what you are

doing is good for kids and making their lives better. So, you’ve got to show them

this truly is the case; otherwise, they’ll close the door.”

The doors at school sites are open three to four times a year for walk through,

which creates dialogue, builds climate of trust and respect, and creates peer pressure.

A D’Angelo elementary school principal notes that culture of data use is observable

141

through the data posted in classrooms, with every single teacher having their graphs

posted on how students are doing. Students will verbalize their goals, based on a pie

graph that may be printed on a monthly basis, noting “We’re going for the green

because green is proficient. Blue is advanced proficient.” Furthermore, every single

student is on the office wall, which demonstrates level of proficiency in their school

wide goal of writing. Culture of data use means providing time for teachers to meet

and collaborate on data. Also time is allotted for celebration when students advance

a level. Release time is provided for teachers to meet once a month to score writing

and to have that collaborative time. After scoring is done, post-it notes get moved on

the office data wall.

Building a culture of data also means communicating with parents and

getting them on board as well. “We talked to parents about it. Everybody has a

script on back to school night. The message is exactly the same. The teachers have

a common language as well.” The principal shared a story, “I had one parent that

questioned the office data chart (indicating his opposition to the information being

public). The conversation shifted later to, I’d rather my son was proficient.”

Building a culture of data use is a result of deeper and wider involvement. “I

have a Focus on Results Team that we call FORT. There is representation from

every grade level. I have 17, from a staff of 29, 17 of them are on this committee.”

This committee meets twice a month with the principal. The principal notes that the

most powerful piece is for teachers to see a best practice that’s implemented by a

142

teacher who has out-scored everybody else, where teachers are able talk and share

those practices.

Like Buck, data use and goal setting must filter down to the classroom, to the

student level, and should include parent involvement. Letters are sent home to

parents, CST information is shared between teacher and students, and a plan of

action is outlined. Each teacher does it differently, but ultimately each student has a

goal. Building a culture of data use requires a school site administrator to know and

understand the level of use in data with each teacher at each grade level.

A middle school principal shares her perspective on the establishment and

building of a culture of data use. She states that it is founded on the frequency of

data analysis and planning. She notes that data are reviewed at the beginning of the

year. Time is blocked every Friday with two of the four being used for a department

meeting and the other a staff meeting. At this time common formative assessments

are analyzed and discussed, which includes coming up with a plan. The other two

Fridays they work independently or as teams on creating new lessons or strategies to

help students. Department percentage goals are established as well as individual

teacher goals. Common and expected practices to build a culture of data use include

commendations of teacher work to build respect and trust, and also to confirm

successes of research based practices.

A culture of data use is made stronger by bringing more voices to the table.

One D’Angelo high school principal notes, “ . . . this new model . . . involving more

143

people . . . gives us more voice and more people to think at the table and it makes the

table a richer place.” She also notes that the culture and depth of data analysis can

be bolstered by joining an initiative with evidence. For instance, if the emphasis is

Algebra, to provide teams with discussion data on Algebra 1, so that a group of

principals can all sit and discuss Algebra 1, or if the conversation is closing the

achievement gap with Latino students to provide that data for discussion.

Evidence of DDDM practices surface in the above themes of effective

professional development and culture of data use. Evidence was noted from the data

collected in the interviews and observations, and through triangulation of district,

school, and observational data. Under the umbrella of core values and theories of

action, effective professional development and a culture of data use are implemented.

Ultimately the execution of the same provides focus and rationale for participating in

a cycle of inquiry. The challenge is more than a technical one, limited to the use of

DataDirector; it broadens to acquisition of knowledge and skills to assist students in

academic achievement. Furthermore, and most importantly it facilitates capacity

building amongst educators to share their effective strategies with one another. The

end goal for both districts is to establish a district-wide culture wherein the capacity

of school site leaders is such that each school within the district utilizes data

conscientiously in daily practice.

144

Existing Knowledge Gaps

Knowledge gaps, however, for principals exist and can result from a number

of reasons. The following has been discovered through research: a fear regarding

consequences of systematic data analysis; other beliefs, aside from instructional

practices, that get in the way of student achievement; refusal to expose a lack of

knowledge about analyzing and interpreting data; deficiency in investment in a data

management system or professional development; and, a disinterest in making a

change in practices that are necessary to develop a culture of DDDM.

At Buck and D’Angelo, the district offices have utilized Data-Informed

Leadership by investing in DataDirector, a data management system. They have also

invested in human resources to facilitate use of the data system at the school sites as

well as to offer training and support in implementation on an ongoing basis. Both

districts have been forthright about the sense of urgency to increase student

achievement, despite or perhaps in light of functioning in an accountability system

that could lead to punitive action. Both districts are cognizant that a lack of capacity

to engage in DDDM through a cycle of inquiry can hamper Data-Informed

Leadership by district and school site administrators. Both districts have a goal or an

expectation that to increase student achievement, a district must increase the capacity

of knowledge and skills of principals to trigger a district-wide effect. To pursue this

goal effectively, a district has to set clear expectations but also be able to recognize

where the challenges lie. The below table demonstrates that challenge areas lie in

145

approximately 30-40% of students, in Buck and D’Angelo, who have yet to score

proficient in language arts and mathematics.

District

Enroll-

ment

Students Performance

%

English

Learners

% Free or

Reduced

Meals

%

Minorit

y

Largest

Ethnic

Group

% Schools

Making API

Targets

Made

AYP

%

Proficient

Lang Arts

%

Proficien

t Math

BUCK

USD 20,860 20.40% 39.50% 88.10% Hispanic 68% No 57.90% 61.50%

D’Ange

lo USD 27,035 24.60% 41.30% 42.20% White 72% No 63.80% 68.50%

Table 6. California Department of Education, Accountability Progress Reporting (2008)

The absolute reality of the percentage of students, not scoring proficient,

lends itself to the realization that professional development to positively affect

instruction in the classroom is a necessity. The approach, in addressing and

presenting this need, will determine the ease of acceptance.

Theme #7: District Expectations

The superintendent of Buck notes that a contractual feature that has existed

for quite a number of years is that individual schools can make time allotments for

professional development. Furthermore, it takes 75% of the staff vote and the

principal to make it happen. The principal has veto power, noting that even if 100%

of the staff votes and the principal opposes, the principal has the power. Of course,

he states that this would not be a good practice. Generally speaking, the principal

will introduce a concept to the leadership team with the evidence that supports it, and

then follows this with a proposal. He notes that invariably if done right, there is no

controversy. “If you get people voting no and you want to do it anyway . . .

146

Alexander Poe said, ‘A man convinced against his will is of the same opinion still.’

You can’t get people in a headlock and make them like what you’re asking them to

do. The model we like to practice, and it’s contractual, is the leadership team gets

together, suggesting to bank some time for collaboration with a purpose of student

achievement. He also notes that he doesn’t know of any school within the district

that has banked time and does anything else. They could be working on a particular

focus for their school site, not necessarily the same focus, but almost all schools

bank time. He did note that this is the case in elementary schools; however, at the

middle and high schools it is a challenge and there is no set model for secondary. He

also stated that he does not recall any negative votes on banking minutes at any

school.

Even with a leadership team consensus to bank time for collaboration, the

superintendent stated that the expectations are clear, a school site administrator must

have the following: a game plan for the year based on data analysis, leadership

knowledge and skills on how to impact staff, know how to ask questions based on

data, and be able to guide staff to delve into the answers for the question, ‘What do

we do now?’ If a principal is unsuccessful, and the school is not prospering, he

would make changes at the school. This change would not necessarily be stated as

the principal’s fault, but from a perspective that both you and I have been

unsuccessful in moving the school. If it appears that the principal is not going to

leave voluntarily, then the superintendent would not hesitate in moving

147

administrative and/or teaching staff. Reconstitution has been implemented,

particularly when a school is stuck at a low level. Buck’s superintendent does admit

that test scores do matter, noting that beyond the politics, the California standards are

good standards, and that the tests do reflect the standards.

The assistant superintendent at Buck is also of the philosophy that buy in

from teachers is an important factor. The minimum expectation is a review of CST

data in order to make instructional refinement, but also a look at data for the current

school roster in order to establish target goals for those students. She notes that each

school is at a different level with data use with some utilizing it in professional

development, grade level meetings, broadcast programs, and some go as far as to

personalize it to help students gain awareness of their own scores.

Expectations are also articulated during professional development in the

leadership academy wherein an initiative is addressed, i.e. English Language

Learners, subgroup achievement gap, etc. and principals are taken through the

inquiry cycle of what will they do to work with their staff on this challenge.

Sometimes you will find principals saying that they have three or four teachers who

don’t even know what a computer is, and then others who have teachers that are

savvy with using data.

Nonetheless, district administrators, in charge of supervising and evaluating

principals, have been given the charge of increasing accountability. The

expectations were outlined in a professional development observation that was made

148

early in the year, wherein CPSELs, the California Professional Standards for

Educational Leaders, was introduced as an evaluation instrument. One of the

elements of CPSELS is using data to make decisions. Along those lines, the district

expects that they would all have a level of facility and ease of use with DataDirector,

to guide their teachers. This, of course, is with the expectation that school site

administrators be proactive particularly since the district has invested in a data

management system and provided training and support at the district and site level.

During interviews with principals regarding their understanding of district

expectations, they innumerate the following: One middle school principal states,

“They have never said you have to do this. What they do is provide you the

opportunities and you as a principal better take the leadership role. So they trust that

we believe in the process and that we’re going to follow through with our staff with

the training and support that they’ve offered in terms of DataDirector training and

professional development. There are people at our disposal that can help guide us . .

. but as far as giving directives, that’s just not the way we function.”

District expectations at D’Angelo are also geared toward instructional focus

and API growth. The principals are expected to attend district meetings twice a

month, once with their instructional leadership team and one meeting with just

principals. D’Angelo superintendent notes that the meeting with just principals is to

“get them going down the right direction or path.” Thereafter, the district

instructional team works with the principals. The superintendent notes, “I’m starting

149

to see and believe that even though they have an instructional focus, they all now

know how to use the data to begin to change classroom teaching.” He goes on to

explain that he provides them with “a great deal of latitude” with respect to their best

practice, indicating various strategies or skills that teachers use to refine instruction.

He points out that schools that have engaged in Focus on Results for three years will

need to revisit and compare their best practices and investigate whether indeed the

practice is truly best practice. During our interview, he displayed longitudinal

graphs for each school. He pointed out the API growth for the elementary schools,

noting “You can see consistent patterns . . . see how steep some of the patterns are . .

. on this one there is no consistency. So . . . the pressure is on district wide to show a

consistent pattern of growth.”

The superintendent notes that principals are expected to utilize the resources

that have been provided. He indicated that providing a data binder is ‘passé’, and

that principals are expected to go online and procure data and be ready to discuss

them based on the topic of the day. He stated that the district focus is to truly make

principals instructional leaders. He stated that they are expected to spend at least one

day a week in the classrooms with a goal of 50% of the week, stating “If you don’t

want to do that, don’t come work here. That’s the way it is. I don’t want to hear

excuses about parents. I don’t want to hear excuses about other issues. If you’re not

going to spend the one day a week in the classroom, this is absolutely critical, you

won’t make improvement.”

150

The superintendent notes that he role models involvement at principal

meetings. He expects principals to do the same at their sites, “If you’re doing your

job, then your team will do theirs.” He asks principals, ‘Do you want to trust the

coaching process to someone else, or do you want to do it?’ Because if they’re (your

designee) not doing it, you are going to be the one to suffer.” Furthermore, the open

dialogue that ensues at the leadership meetings is expected to filter down to the sites

to bring the schools along in the process.

While information is being conveyed to the sites, there is strong union

presence that also needs to be considered. In D’Angelo, there are two schools in

particular that are very union oriented – one with no push back and the other with

considerable push back. It was noted that leadership skills are an extremely

important factor in bringing the school along in the process, and the majority of

principals have been able to do this successfully.

An assistant superintendent notes that district expectations are framed

publicly with the good news statement being that the district is achieving at an 800

level, district wide, as well as the highlights of distinguished and blue ribbon

schools. The urgency statement is that the district still has 60 percent of Latino

population not achieving at grade level as well 40% of our Caucasians and 20% of

the Asian population. He states, “We recognize this, we talk about it at board

meetings, it’s out in the open, as you walk through the hallways, you will see it all.”

151

He notes that the superintendent has made sure that the message is prominent and

that everything in the district centered on student achievement.

The expectations of revolving all work around student achievement manifests

in the ability for administrators to have difficult conversations when necessary. The

assistant superintendent describes that the district uses improvement plans better than

most districts. He notes that there are teachers on improvement plans with an

expectation that they improve their teaching. At the same level, if an administrator is

not doing what they need to do, there are some hard conversations that district

administrators pursue. The superintendent through his leadership focus expects the

instructional focus to be implemented without deviation. He attends to the

principals, visits sites, and has conversations with them as well. The conversations

sometimes revolve around keeping their eye on the ball, and not allowing anything to

pull them away from the instructional focus, which is the priority.

Principals understand this very clearly and keep an eye on the instructional

focus. This translates into making sure time is being used wisely but also that time

for instruction and student practice is not being wasted. One elementary school

principal notes that she has dissuaded her teaching staff from giving book reports

because of the time it takes to grade them. “If a book report is done, one must ask

how much of it is instructional based, what is being graded, and what’s the

purpose?” She constantly asks teachers to identify the purpose, expecting that when

she walks into the classroom, a lesson or assignment is based on an instructional

152

purpose or a connection to something valid. The principal expects teachers are using

results from DataDirector to guide the purpose.

Although there is no particular mandate of how or when teachers use

DataDirector, open dialogue and conversation raises the level of expectation. A

principal states, “Principals that use it (DataDirector) more frequently, have the

discussion with their colleagues, they are making much more progress . . . and those

practices are shared with the superintendent. Just like at the principals’ meetings . . .

we hammer . . . we model . . . how is goal setting helping in your school, look this

one bumped up 54 points . . . tell us what you are doing. There is not a single person

there who doesn’t want to do what that person is doing.” Thus role modeling at

district professional development is emulated at the school site level to raise the level

of expectation by highlighting successes.

Even though it is a district expectation to utilize data, data use is not tied to

teacher evaluation. One middle school principal describes a conversation she has

with the teaching staff. “I think the district’s expectations for data use are simple,

that we use it to help guide our instruction. To help students be successful. It’s a

tool. It could be used wisely depending on the teacher or the principal. You’re

always very sensitive to the fact that we cannot hold the teachers accountable by

using the data in their evaluations, but that doesn’t mean we can’t have conversations

with that in mind. “You know what -- we need to step it up.” She notes that the

message is students need to be proficient and that the excuses of the past cannot

153

continue. She encourages data use to help find solutions in that the data are what

they’re being held accountable for.

The district expectations are intertwined with a principal’s ability to lead their

school site. A principal confirms information from an earlier interview with a

district administrator regarding union involvement and push back. She notes:

They want people (principals) who are able to use data to lead, but they are also looking for someone who has empathy, compassion, and the professionalism to be able to get a staff to follow them into the trenches, because some of our schools are in the trenches. Right now we are having some union issues . . . staff and administration is unaligned . . . How are we going to cross the line and work together as a team? Data is there to be used productively and wisely, and the district expects you to have the skills to be able to use it wisely. With district expectations for principals to have the know-how to build a

team, a middle school principal has a candid conversation with her staff, explaining

that as the year progress growth should be evident. Using the data to show

percentages, she states:

From your 150 students, I’m expecting that maybe you have 10% in the far below basic, you’ve moved 30% into the below basic, and moved another 40% here into the basic, and then you moved the other 30% into proficient. If you can show me the growth, I am going to be happy with you, and you’re going to be happy with yourself.

A high school principal sums it up by stating that data utilization is important

to making progress and guiding the instructional focus to meet target goals. Bottom

line, however, “You have to hire great teachers. The one thing that I always do, I

always look at their college transcripts and if I’m hiring them to teach English or

154

history, I want to see A’s and B’s and if I am not seeing them I do not hire them.”

The knowledge and capability of the teaching staff is commensurate with the high

expectations that the district sets for principals to increase student achievement at

their site. This high school principal notes that teachers provide instruction to a

spectrum of students and within a range of classes. “You (as a teacher) not only

teach honors 10, you teach literacy for success. So we don’t have the situation in

which all of our best teachers are teaching our high level kids and they are not

working with our struggling students, so for the most part that’s across the board.”

Levels of Data-Informed Leadership are apparent at the district and school

levels in Buck and D’Angelo. There is strong evidence from interviews and

professional development observations that principals are provided with tools and

support to meet the expectation that data use, analysis, and interpretation are a non-

negotiable to promote data driven decision making activities to improve instruction

at their respective schools.

Conclusion

This chapter represents the discovery, analysis, and interpretation of a host of

data on district provided professional development for school site administrators in

data driven decision making in the Buck and D’Angelo Unified School Districts.

This method was useful in studying educational innovations within the district and a

valuable approach in the quest to answer the “how” and “why” questions. The

155

research questions focused on the identification of need for professional

development, the choice and implementation of professional development, and its

effectiveness and attendant challenges. After collecting and analyzing the three

types of data (interview, observations, and documents) essential themes surfaced and

these themes were subsequently associated with the sub questions formulated for this

study. The sub questions answered how the districts assessed the need for

professional development, the types of training and support provided school site

leaders, evidence that leaders put DDDM practices into place, and what knowledge

gaps exist to date. The common themes within each sub question represented the

most dominant and significant data which connected to the research questions.

The professional development offered to school site administrators at Buck

and D’Angelo Unified School Districts was formulated through a thoughtful process,

which has had impact on the culture of the organizations, permeating school sites,

albeit at different levels, with regard to data driven decision making. Evidence was

discernible in the models of ‘best practice’ at school sites within each district. Of

note were well-established routines of Data-Informed Leadership. School leaders

and their staff modeled elements of Data-Informed Leadership garnered gains in

student achievement through DDDM by the superintendents establishing and making

clear the anchors for Data-Informed Leadership.

Anchors include: a well outlined and pronounced strategic plan and

initiatives that are communicated to all stakeholders; core values are reinforced by

156

the superintendents at professional development meetings through their words and

their actions, specifically their presence and engagement at all professional

development meetings; and, theories of action are pronounced in their choice of

professional development based on their respective organizational cultures. Lastly,

available data are procured and integrated through DataDirector, a data management

system, and through human resource support in the district office. Data literacy is

integrated into the professional development component at the district and school site

levels. Aside from the anchors for Data-Informed Leadership, are the various

elements that establish an ongoing process in creating a learning organization.

Buck and D’Angelo have invested in data infrastructure and access as well as

in professional development to develop the expertise and data literacy of leaders.

Through this process, they continue to build cultures of inquiry, and lasting culture

of inquiry allows for school sites to implement a plan of action and observe the

effects on student, teacher, and organizational learning. There is awareness, by

district and school site administrators, wherein the goal is to engage in a culture of

inquiry, where the district continually role models and expects principals to emulate

the cycle of inquiry at their schools.

It is clear from the research at Buck and D’Angelo that district leaders are

stridently aware of the benefits of data, and they have been constant and consistent in

the last 5-7 years in escalating the process. There is increasing sustainability to

efforts at both districts, albeit through different theories of action. Buck USD

157

maintains an approach of autonomy and innovation at the school sites with regard to

research based instructional materials and professional development. D’Angelo

USD’s theory of action also maintains autonomy; however, based on the size of the

district and the desire to streamline the message, the entire improvement process is

funneled through a district-wide framework, Focus on Results.

The mainstay of accountability expectations by federal and state policy

environments has not only changed the culture within the districts but that of

education in general. District leaders are increasingly demanding a scientific

approach of inquiry by their school site leaders, and expect that this approach is

implemented at their schools. Both Buck and D’Angelo are determined to sustain

system learning through a culture of inquiry district-wide at an optimal level. In the

context of educational organizations being systems of learning, it is evident that both

Buck and D’Angelo Unified School Districts utilize Data-Informed Leadership as an

influential conduit between existing performance and desired outcomes.

The practices at Buck and D’Angelo merge with Data-Informed Leadership,

the framework utilized in this study. The concept of Data-Informed Leadership

includes a wide array of issues and questions requiring reflection by leaders when

crafting their approach to advancing practices and dealing with challenges in data

use to increase student achievement. In essence, data driven decision making does

not exist in a vacuum, and indeed Data-Informed Leadership is required to glean the

knowledge and skills, which at times seem like nuances, in educational leadership.

158

Data-Informed Leadership broadens the scope of expertise, beyond DDDM, required

for advancing student performance.

159

CHAPTER FIVE

Summary and Implications of Findings

Introduction

Data driven decision making (DDDM) has become part and parcel of an era

of K-12 educational accountability based on mandates by the federal and state

governments. Invariably, this has resulted in a heightened level of accountability at

the school district level. There is an expectation for district leaders to have the

knowledge and skills to recognize and implement all the modalities necessary to

meet the pressures of performance-based accountability to increase the quality of

public education for all students. A viable strategy to developing and sustaining

performance-based accountability district-wide is to increase the capacity of school

site leaders through systematic and comprehensive professional development.

The purpose of this study was two-fold -- to add to the literature and to

provide a roadmap for adopting practices which lead to a comprehensive and

systematic professional development for school site administrators in data-driven

decision making. With external and internal accountability measures, districts are

compelled to invest in professional development for school site administrators in

data driven decision making; thus, empowering each school leader with the

knowledge and skills to engage in a process of improvement to raise student

160

achievement. This begs the question, “How do districts build the capacity of school

leaders to engage in data-driven decision making?”

This qualitative study was conducted in Buck and D’Angelo Unified School

Districts, both in the state of California, in order to answer the above question as well

as related sub-questions: How is the need for professional development assessed?

What types of training and support are provided to school site leaders? What

evidence is there that leaders put new data-driven practices into place? What

knowledge gaps still exist? The method of data collection to answer these questions

involved interviews of district and school site administrators, observations of

principal professional development, and document analysis.

Through the above methods it was discovered that both districts since the

inception of accountability mandates have understood the importance of utilizing

data to engage in the improvement process. Of note, however, it was not until these

districts procured the necessary assessment tools to produce reports that they were

able to delve deeply into retrieving and analyzing data and developing focused action

plans. Furthermore, both districts have extremely high expectations of their school

site administrators’ use of data to drive decisions about instruction; however, both

districts are aware that the level of knowledge and skills vary from one school site

administrator to another. Answers to the research questions were achieved through

protocols which allowed professional development processes to surface. Evidence

revealed these districts to engage in: established methods to assess professional

161

development need, training and support of their principals, developed methods to

measure implementation of new learning in data driven decision making, and

recognized methods to evaluate existing knowledge gaps.

The findings in this study have strong and relevant connections to research

delineated in the literature review of chapter two. It is also important to note that the

entirety of the data gathered and analyzed was examined within the framework of

Data-Informed Leadership (Knapp et al., 2006). The intent of viewing this research

in the context of this particular framework was to allow for a wider scope in

understanding the complexity of educational leadership. Hence, it is believed that

effective leaders not only gather and analyze data, but draw from their experiences

and from accumulated knowledge of their organizational culture. In addition, these

leaders also exhibit good judgment and wisdom prior to the development and

implementation of a plan. Ultimately, the data revealed that mastery of data-

informed leadership skills, by the superintendents, was the channel by which district

and school site administrators were empowered to gain and implement new

knowledge and then to extend it by effectively empowering others through

distributive leadership. With that in mind, below are the salient points in this

research study and how they connect to prior research.

162

Connections to Prior Research

The results of this study engage existing literature on professional

development for school site administrators in data driven decision making,

leadership and learning organizations, and leadership development models. Chapter

two examined research on professional development for school site administrators in

relation to three crucial areas: Accountability in K-12 Education, The Role of Data-

Driven Decision Making in K-12 Educational Reform, and Professional

Development to Build Capacity of School Site Administrators in DDDM. In the

following sections, I will delineate the connection between these areas of research

and the findings from this study.

Accountability in K-12 Education

The sub-section for Accountability in K-12 Education, found in the literature

review, described consistencies between NCLB and California’s PSAA, specifically:

student performance is based on standardized test results, a focus is placed on

performance of numerically significant subgroups, and interventions are mandated to

help those schools that are low-performing or program improvement (PI) schools. A

study by AIR indicated AYP linked to NCLB was more challenging since it requires

schools to meet proficiency targets versus California’s API target which is focused

on growth. It went on to purport that a hindrance to school improvement was the

professional development component: “. . . the district role (can hamper) school

improvement, the influence of instructional coherence, and internal school capacity”

163

(AIR, p. I-5). Another substantial finding was the difference that emerged between

schools that utilized an extensive amount of data and schools that did not. Growth

schools were described as using data frequently and extensively to guide instruction,

while low growth schools used it to identify students not performing well (p. IV-13).

Findings from Buck and D’Angelo USD noted that overall there was

consistent and progressive growth in schools within their districts, attributing growth

at some of the schools to the implementation of effective DDDM practices. Through

interviews, district administrators noted that initiatives were focused on targeting

areas of challenge, with a keen focus on subgroups and content areas, coupled with a

convergence and sharing of best practices. School site administrators utilized staff

development meetings and grade level meetings to not only identify students not

performing, but more importantly focused on methodologies for intervention, i.e.

carving out time for intervention during the course of the day, sharing and agreeing

to utilize best practices in instruction, identifying particular standards not mastered

and consistently analyzing data to strategically regroup students based on need.

Other research in the literature review, under Accountability in K-12

Education, noted the importance of adherence to federal and state expectations, but

at the core of advanced practices in a school system is the interdependence between

district and school sites. Knapp et al. (2006) demonstrate this relationship in their

Data-Informed Leadership diagram, which establishes that it is incumbent upon

school districts to create a comprehensive school system by building the capacity of

164

those concerned with DDDM through a culture of inquiry. In this study, it was

noted that Buck and D’Angelo superintendents and district administrators have

eliminated extraneous agenda items at principal staff development meetings and

have targeted content to student achievement via school improvement. Principals

were clear about the tapered message of student achievement and their responsibility

to disseminate that message to the teachers as well as implement best practices of

DDDM at their school sites. Furthermore, accountability is fostered at district

professional development through an exchange of learning which extends vertically

and laterally. This exchange has translated to accountability that extends to the

student level. Both districts have couched the proficiency level of students in the

idea that success occurs when students are keenly aware of their performance and

progress. Thus data are broadcast through public announcements and the presence

of school-wide and classroom data walls, extending the line of accountability to

include district, principals, teachers, students, parents, and community.

To further corroborate research in accountability, RAND researchers propose

that the majority of educators believe data are useful for informing improvements in

teaching and learning. However, it is noted that DDDM does not lead to effective

decision making, but school improvement happens with the following actions:

promotion of various data, allocation of time, training on data analysis and

formulating action plans, identifying and implementing solutions, partnering with

institutions that support data use, assigning people to filter data, and user-friendly

165

technology and data system. All of these actions exist within the elements that effect

and contribute to Data-Informed Leadership, and were found in the research

conducted in Buck and D’Angelo. Further to this, the acknowledgement and

implementation of the above is spearheaded through the mastery of leadership

strategies used by the superintendent and cabinet.

The pace and tone for sustainability are set by district leadership.

Sustainability is a result of institutionalization, involving a layered process

embedded in structural routines and rituals within an organization (Datnow, 2005).

District staff, at Buck and D’Angelo, is readily available to offer support and training

at the district office and on-site. Furthermore, allocation of time for practices that

move student achievement forward exist in both districts and are: principal meetings

geared toward school improvement through the use of data analysis and the

formulation of action plans; partnerships formed at both districts with outside

consultants have provided professional development or a framework for school

improvement through data use; and, implementation of a user-friendly data system,

DataDirector. The superintendents’ messages reflect a priority of student

achievement and that the job of the people within the district is to make this happen.

The implication of practices, based on accountability, indicate that data driven

decision making plays a tremendous role in the process of improving performance.

166

The Role of Data Driven Decision Making in K-12 Educational Reform

The sub-section for The Role of DDDM in K-12 Educational Reform in the

literature review indicates that DDDM is defined as: “. . . teachers, principals, and

administrators systematically collecting and analyzing various types of data . . . to

guide a range of decisions to help improve the success of students and schools”

(Marsh et al., 2006, p.1). Further to this, Fullan (2002) stated, “Only principals who

are equipped to handle a complex, rapidly changing environment can implement the

reforms that lead to sustained improvement in student achievement (p.1).” The

implication is that a process must be put into place for DDDM to play a prominent

role in educational reform, but with a caveat – that principals have the leadership

‘know how’ to put this process in place and to make it sustainable.

Buck USD has integrated the use of data with the consideration of other

important knowledge and skills through the new principal’s evaluation system – a

system based on CPSELs. Furthermore, professional development offered by the

district for its principals is based on student achievement which directly correlates

with data use or the use of assessment tools. One means of sustainability will

manifest through the integration of district wide math benchmark assessments and

then will follow with language arts benchmark assessments the following year

(corresponding with the adoption cycle). D’Angelo USD has integrated

sustainability in DDDM by undergoing a three year roll out process for each school

site utilizing the Focus on Results framework. D’Angelo considers this a

167

comprehensive and systematic district wide professional development for the

purpose of achieving equilibrium in the knowledge and skills of principals across the

board. Professional development for principals and their leadership team occurs on a

monthly basis, and this year a second principals’ meeting, eliminating ‘nuts and

bolts’ information, will incorporate the Focus on Results framework for continuity

sake. There is awareness at both districts that the role of data driven decision

making is integral to the work of the district and its school sites.

In a study about leaders in the schools and system, a case is made that

DDDM should be the norm “. . . stating explicitly that data use is non-negotiable”

(Datnow et al., 2007, p. 71). At Buck and D’Angelo, professional development, site

visits, coaching and mentoring sessions (by district administrators and peers) are

based on the analysis of data to guide school improvement. Action plans, evaluation,

and monitoring of the improvement process at each school are based on the story

told by the data. Schmoker (2006) notes, that from a system perspective, the

superintendent can promote a culture of data use through the implementation of

professional learning communities. Moreover, data discussions at professional

development and with district administrators build capacity for school site

administrators to continue the process at their respective school sites.

Buck USD’s superintendent and cabinet have placed a deliberate focus on the

use of data in their message, conversations, and method by which they enquire about

the state of a school. Based on Buck’s culture of innovation, the superintendent has

168

very clear and high expectations for principals but does not dictate the professional

development they seek for their site. Based on this expectation, all school site

administrators interviewed reported receiving in depth professional development for

their teachers, i.e. Effective Schooling Strategies by Dennis Parker, Professional

Learning Communities by DuFour, and training by Marzano in cultivating and

sustaining effective instructional strategies in the classroom. These professional

development opportunities for those particular school sites have garnered results in

student success as measured by a gain in API score and progress in closing the

achievement gap for specific subgroups. Moreover, principals with best practices

that have acquired success at their school site provide staff development at the

principal meetings; thus, building a culture which values learning system-wide.

Schmoker (2006) notes that districts who have more knowledge and skills

capacity in DDDM have deepened and widened the scope of those learning through

distributive leadership. From the interviews and observations at Buck and D’Angelo

USD, it is clear that distributive leadership is being implemented and it has provided

a foundation for collaborative data discussions school wide and at grade levels,

wherein various data are discussed, analyzed, and action plans formulated. Best

practices in distributive leadership were prominent at D’Angelo USD through Focus

on Results, a district wide professional development, wherein the principal along

with the entire leadership team attend monthly Focus on Results professional

development. The process in and outside of these meetings, similar to Knapp et al.

169

(2006) cycle of inquiry, creates discussions which allow for adjustments in

instruction and sharing of best practices. Most schools in D’Angelo USD have

demonstrated consistent and steady gains in API and some schools have shown

progress in closing the achievement gap for specific subgroups. Administrators at

both districts, however, admit that there is still much work to do and learning gaps

still exist amongst students, teachers, and administrators.

Mac Iver and Farley (2003) note that among some of the best practices to

build capacity within a school district includes implementation and training in data

tools, effective DDDM in curriculum and instructional practice, and supporting

schools in analyzing data and developing action plans. Both Buck and D’Angelo

USDs are aware that the use of DDDM is a tiered process that exists at the district,

school, and classroom levels. Superintendents at both districts are consistently

engaged with district and school site administrators and sustain an unwavering

message of student achievement based on data. Discussions between district

administrators and principals during regular school site visits as well as observations

(walk-through) center around: best practices in data use, best practices in

instruction, and progress in the implementation of action plans. Both districts have

staff which provides district and on-site training in data tools.

Literature on the role of DDDM in K-12 educational reform by Coburn and

Talbert (2006) describes the barriers in establishing a comprehensive system of

DDDM: beliefs related to evidence/research and previous reform policies, varied

170

responses to accountability policy at the various levels of a school system. The

issues are in the coexistence of conflicting views at the various levels of the system.

These differences could hamper the use of data to make decisions that will ultimately

improve the quality of education for students. Both districts indicated that there

were barriers that hampered the process of DDDM. Through interviews with school

site administrators, a Buck principal noted that camaraderie is difficult in a district

that values innovation, noting that it sometimes brings feelings of isolation and an

inability to share best practices across the district based on similar efforts. On the

other hand, there are opportunities within the district for principals to present their

findings of best practices to their colleagues at principal meetings. Furthermore,

camaraderie has been formed through collaboration during principal meetings and

amongst some of the school site administrators who meet on their own time to share

leadership practices and strategies for implementing DDDM effectively at their

school sites. D’Angelo, however, has worked the sharing of practices through its

Focus on Results framework, wherein triads are formed amongst the schools.

Schools formally visit one another, conduct walk-through to find evidence of best

practices as it relates to a school’s instructional focus. Like Buck, some of

D’Angelo’s school site administrators meet with their colleagues on their own time

to discuss leadership practices to advance their teachers in refining their instructional

practices for the purpose of school improvement. At times a barrier at D’Angelo

USD is the teacher’s union, which has pushed back on components of the DDDM

171

process, the implementation of DataDirector is an example. Whereas the

relationship between the teacher’s union and Buck USD is a partnership, a

significant aberration from the typical district-union relationship, wherein the union

supports the district with raising the expectations for teachers and offers heavy

financial support for teachers’ professional development.

Supovitz and Klein (2003) concluded that the American system of education

does not allow for the structures and avenues to engage in the in-depth inquiry

necessary to improve student learning outcomes. On the contrary, there were

examples at both Buck and D’Angelo that indicate structures can and have been put

in place to allow for in-depth inquiry. A principal in Buck USD notes that she has a

‘between the bells’ approach. Teachers are given time to do in-depth inquiry,

looking at data and restructuring small group instruction in a grade level meeting,

while students engage in a block of time spent in computer, PE, and music. Another

principal in D’Angelo USD has taken an unconventional route to provide time for a

teacher within a content area to serve as a resource specialist teacher, where one

period a day would focus on academic content, making sure instruction, blue prints,

and state test match. This was done first in English and Math, and then in Social

Studies and Science. Principals who have mastered the leadership strategies

necessary to ‘move’ a school to higher levels are evident at both Buck and

D’Angelo. This includes the school site administrator’s ability to create a school

172

culture and increase the capacity of teachers to use data in making decisions about

learning and instruction.

Professional Development to Build Capacity of School Site Leaders in DDDM

The subsection of the literature review, entitled Professional Development to

Build Capacity of School Site Leaders in DDDM, indicates that in a number of

studies administrators have been noted to make most decisions without the use of

data (Coburn, Honig, and Stein, in press). In another study done by Corcoran,

Fuhrman, and Belcher (2001) it states none of the districts that were evaluated

utilized data to make decisions about professional development. Datnow et al (2007)

adds routine of data use is not simply to analyze, interpret, and utilize data to

formulate an action plan for the SSP, but to create a culture of data use at the school

site.

There was evidence in interviews, observations, and document analysis of

Buck and D’Angelo USD that both districts understand that the most crucial

investment is in the development of human capital. Both districts provide principals

with professional development that focuses on the development of data literacy and

analysis skills, which is also a significant element in Data Informed Leadership.

Also significant is the element of building cultures of inquiry, wherein both districts

have widened the scope of professional development to extend outside of the district

office’s professional development center.

173

Superintendents and other district administrators consider cognitive coaching

and mentoring of their principals a significant element in professional development –

a necessary component to the implementation and sustainability of DDDM at the

school sites. Peer-mentoring is also encouraged and occurs within both districts. As

Darling-Hammond & Orphanos (2006) indicate, it is necessary for school site

administrators not to simply be managers but organizational leaders with highly

developed skills to understand and transform schools in organizational change.

However, according to Schmoker (2006), school site administrators’ professional

development has for the most part taken the form of transmitting procedural

knowledge which often times does not pertain to practical application. This type of

professional development leaves recipients in a vulnerable position, one that

insinuates dependence on external sources rather than empowerment and ownership.

Buck and D’Angelo utilize external sources; however, Buck couples external

sources with a strong relationship between the district and the schools to establish

internal accountability. Also, principals who have acquired the knowledge, skills,

and mastery of leadership strategies are sharing their best practices through

professional development provided to their colleagues. D’Angelo USD’s external

source is professional development that introduces a framework rather than

procedural knowledge, allowing for a weaning of the process to leaders in the

organization who are being trained to become expert in the framework. An emphasis

174

to integrate professional learning with practical application is evident in both

districts.

Researchers, as indicated in the literature review, describe professional

development to have relevance when it is intertwined with activities that reflect

instructional improvement and are in harmony with the mission and vision of the

organizational system. Learning should take place in a venue that creates individual

and organizational habits and structures, encouraging learning on a continuum

(Fullan, 1990). This requires an intense relationship between the district office and

its schools, which was evident at both Buck and D’Angelo. Professional

development was noted as one of the most important roles of the district. This is

prominent in the strategic plans of both districts. In addition, both superintendents

value and adhere to existing research in professional development by establishing the

following: interpretation of data is a shared process to allow for sound instructional

decisions, instructional strategies are highlighted through discussion and observation,

team leadership is engaged through distributive leadership at school sites, decision

making is based on evidence, and alignment of curriculum is addressed at the district

and site level. All of the aforementioned processes can be implemented through

structures; however, structures and processes alone must fall under the umbrella of

building organizational culture simultaneously with professional development to

assure harmony amongst all stakeholders. Of significant note is both superintendents

175

have presence, high visibility, and involvement in most professional development

meetings.

With high involvement and focus on professional development that

concentrates on DDDM, Darling-Hammond (2002) asks, Are school site

administrators actions based on intrinsic reasons or because they must conform to

external pressure? She states that pressure must be accompanied by external

mandates that incite a mind-shift that DDDM is part of systemic or organizational

improvement that empowers educational leaders to have a collection of tools to plan

and make decisions. In correlation with this research, it was substantiated through

interviews and observations that pressure by the superintendents are applied through

high expectations, performance results, and candid conversations rather than through

castigatory messages. As a matter of fact both superintendents and their cabinet

circulated throughout the entirety of the professional development sessions, engaging

in the process, as well as connecting with principals through light-hearted

conversations and/or ‘shop talk.’ Assistant superintendents in educational services

and their directors continue this thread of expectation through timely and frequent

site visits throughout the year to monitor and evaluate process as well as to offer

support, coaching, and mentoring. The implication is that a principal will rise to the

occasion through insights uncovered by new knowledge gained, experiences,

collegial sharing, and proactive measures to construct meaning, as well as

relationships that foster interdependence between district and school sites. District

176

superintendents and their district administrators set the pace and cycle for data

literacy, inquiry, and action. Creating this type of understanding between district and

site leaders requires trust and respect through relationship building and the mastery

of strategies in Data-Informed leadership. Nonetheless, to this thesis is an antithesis.

Based on a number of researchers from the literature review in chapter two,

district leaders do more than make decisions based on data. Data, therefore, serve as

reinforcement for improvement and is one of many skills that district leaders must

have to move their organization into one that is a ‘learning’ organization. This idea

is aligned to the framework of Data Informed Leadership, wherein a culture of

inquiry manifests from the implementation of system learning (Knapp et al., 2006).

While moving a district culture to one of inquiry (via relationship building), district

leaders present evidence and utilize strategies to set or establish the purpose for

change. In Buck and D’Angelo the presentation of evidence and the utilization of

strategies are demonstrated through the conversations, planning, and decisions that

occur through the district leadership teams, a close sphere of individuals that the

superintendent works with on an intimate basis. Once a concise plan is formulated

this is conveyed to all stakeholders via the strategic plan. The mode of

communication with the public is much different than that with the leaders who will

assist in making sure this plan manifests throughout the district; however, certain

messages are consistent internally and externally. Buck USD’s superintendent

espouses the district’s sense of urgency through specific initiatives that it

177

communicates through the ‘Unite Show’ and other media. D’Angelo USD’s

superintendent communicates a ‘Good News’ and an ‘Urgency’ statement that is

visible throughout the district and through various media. The mode and methods

utilized by the superintendents and their sphere of individuals is a calculated one,

characterized by intentional transparency and founded on evidence (data).

Superintendents at both districts along with their cabinet relay the same message, a

message spearheaded by the superintendent. This also serves as a role model for

district administrators, school site administrators, and fosters external and internal

accountability for the work to be done. It should also be noted the message given by

the superintendent is expected to resonate from each district administrator and school

site administrator in order for improvement to be system-wide. This was evidenced

in the interviews and observations at both districts in the espoused message at

principal meetings as well as at the school sites.

Internal accountability exists through “learning at work — learning in context

occurs, for example, when principals and their leadership teams are members of a

district's inter-visitation study team for which they examine real problems — and the

solutions they have devised — in their own systems” (Fullan, 2002, p. 19). This type

of learning, evidenced at D’Angelo USD, fosters shared knowledge and commitment

to improve the culture of inquiry within the organization. It also allows for a social

setting that is conducive for performance monitoring by district administrators who

facilitate in a triad, reinforcing DuFour and Burnette’s (2002) approach of “What

178

gets monitored, gets done.” It also serves as a support system within the organization

for principals, allowing for problem-solving and strategizing. This process

strengthens the interdependency between district leaders and school site leaders

(Fink & Resnick, 2001).

Professional development for school site administrators, based on the above

research, is fragmented by the combination of state and district efforts, fragmented in

that the state withholds funding to offer its own professional development, namely

AB 430. District efforts, however, lend to a professional development model which

allows for breadth and depth as a result of sequence and continuity. Based on this

research, efforts in professional development for school site administrators should be

localized to the district level wherein learning is customized directly affecting

classroom instruction and improvement in student outcome.

According to Elmore and Burney (2000) a recommendation for effective

professional development for school site administrators is based on: monthly day

long meetings; consistent principal site visits by district personnel to monitor

progress and provide guidance; principal study groups; new principal support

groups; and, strong organizational leadership through a comprehensive professional

development plan. Based on the data collected, Buck and D’Angelo have

implemented a rendering of the above. Below is a chart comparing findings across

both the Buck and D’Angelo Unified School Districts as it pertains to effective

practices in DDDM professional development for school administrators.

179

EFFECTIVE PROFESSIONAL DEVELOPMENT PRACTICES IN DDDM FOR SCHOOL ADMINISTRATORS

Buck Unified School District D’Angelo Unified School District Superintendent presents evidence and uses DIL strategies to establish purpose for change.

Superintendent presents evidence and uses DIL strategies to establish purpose for change.

Need for PD assessed through gathering and analyzing data.

Need for PD assessed through gathering and analyzing data.

PD plan based on needs assessment integrated into Data Informed Leadership (drawn from experiences and knowledge of organizational culture).

PD plan based on needs assessment integrated into Data Informed Leadership (drawn from experiences and knowledge of organizational culture).

Based on knowledge of district’s innovative organizational culture, a strategic approach was used. A variety of research based PD practices in DDDM exist and one element of district-wide practice implemented with a plan to add a second element on the following year.

Based on knowledge of an effective research-based framework for DDDM, Focus on Results, was implemented district-wide.

Superintendent and close sphere of district administrators and principals work intimately to formulate a concise PD plan to be executed.

Superintendent with close sphere of district administrators and outside consultants work intimately to formulate a concise PD plan to be executed.

Mode and method of communicating PD plan by Superintendent and close sphere of district administrators is consistent, based on evidence, and intentional transparency.

Mode and method of communicating PD plan by Superintendent and close sphere of district administrators is consistent, based on evidence, and intentional transparency.

Organizational learning and culture of inquiry exists through internal accountability and fostered through a commitment to improve practices.

Organizational learning and culture of inquiry exists through internal accountability and fostered through shared common knowledge and commitment to improve practices.

Distributive leadership is modeled by Superintendent and District Administrators.

Distributive leadership is modeled by Superintendent and District Administrators.

Process of training, support, monitoring, and evaluation is consistent and conducive to strengthening inter- dependency between district and schools.

Process of training, support, monitoring, and evaluation is consistent and conducive to strengthening inter-dependency between district and schools.

Table 7. Effective Professional Development Practices in DDDM for School Administrators

180

Evidenced, however, was a commonality between the superintendents at both

districts, which made the difference in the effectiveness of preparing school site

administrators in data driven decision making practices. It was the mastery of

leadership strategies to move their respective organizations into a cycle of inquiry

based on data-driven decision making.

Implications for Future Research

The effectiveness to conduct professional development that raises the

capacity of school site administrators to lead via data driven decision making has

been highlighted in the research conducted at Buck and D’Angelo Unified School

Districts as well as the research noted in the literature review. Whereas this study

offered valuable insight in the framework, processes, and strategies to implement

systematic and comprehensive professional development in DDDM for principals,

future research is needed. Research is needed to intensify the understanding of how

superintendents and their district administrators coach school leaders in the mastery

of leadership strategies. Below are suggested areas for future qualitative research

studies on professional development for school site administrators in Data Informed

Leadership, a professional development that integrates DDDM with a mastery of

leadership strategies.

181

• Analysis in the consistency of district administrators to coach and assist

principals in deepening the process of data-informed leadership

• Analysis of principals within a district to establish a culture of inquiry

through distributive leadership

• Evaluation of a district’s process to design and monitor mastery of leadership

strategies used by principals to increase teachers’ use of best practices in

instruction

• Evaluate school models, operating within the school day, to allow for

professional development that includes: team meetings focused on

instructional improvement through a cycle of inquiry (accessing and

analyzing data, formulating and discussing actions, and opportunities to

re-frame)

The above areas to be researched would be best conducted through intensity

sampling choosing one unit of analysis (a district) which deems to have a

comprehensive and systematic approach to professional development. The study

would be across the board, with a greater number, if not all, district and school site

administrators studied within the district, for an extended length of time through

interviews, observations, and document analysis. This would allow for a more in

depth study (Patton, 2002).

182

Implications for Policy and Practice

Based upon the findings of this study in a merger with prior research, a

district aiming to establish a comprehensive and systematic professional

development program to support site administrators in effective use of data should

consider the following:

Mastery of Leadership Strategies in Data-Informed Leadership

DDDM as a routine within a district does not exist in a vacuum, and certainly

it is the ability to utilize data to inform leadership at the district, school site, and class

room levels, which makes all the difference. The superintendent who sets the vision,

tone, and dictates the pace of organizational learning must possess, master, and

exercise the appropriate leadership strategies to launch the organization into a cycle

of inquiry and sustain momentum, so that learning is continuous. This Mastery of

Leadership Strategies was evidenced and role modeled by Buck and D’Angelo

Unified School District superintendents and their respective cabinet members

through the context of each district’s established organizational culture.

Professional development to build the capacity of school site leaders in DDDM

should include the Mastery of Leadership Strategies for Data Informed Leadership,

which are:

• Utilize evidence-based practices to establish the need for professional

development

183

• Implement an effective agenda for organizational change through distributive

leadership, which is first modeled by the superintendent and district

administrators

• Identify crucial actions for principals to be successful leaders, role model

these actions through district professional development, and then monitor

implementation during site visits

• Utilize effective and consistent communication through cognitive coaching,

mentoring, and problem solving to build collaboration, respect, trust, and to

manage conflicts

• Evaluate, diagnose, and engage in cycle of inquiry to create high

performance school-wide through a pre-determined construct

• Develop enduring communication (with all stakeholders) to attain a firm

grasp of the school culture to effectively bring organizational vision to

reality, again role modeled at the district level

• Synthesize all elements of a school’s performance, not just “bottom line”

data to impact the culture and improvement process for student

achievement

• Allow time to identify and integrate new learning to build capacity school-

wide

184

Table 8. Mastery of Leadership Strategies in Data Informed

`MA

STE

RY

OF

LEA

DE

RS

HIP

STR

ATE

GIE

S IN

DA

TA IN

FOR

ME

D LE

AD

ER

SH

IP

Utilize evidence-

based practices to establish the need for professional developm

ent (PD

) in data-driven decision m

aking (DD

DM

).

Execute

organizational change through m

odeling distributive leadership, w

hich functions w

ithin the confines of a fram

ework for

professional collaboration.

Identify crucial actions for principals to be successful leaders, and then offer training, support, m

onitoring, and evaluation of im

plementation.

Restructure

comm

unication through cognitive coaching, m

entoring, and problem

solving to build collaboration, respect, trust, and to m

anage conflicts.

Create a culture

of inquiry by m

odeling the cycle of m

aking sense of learning, taking action, learning from

action, and problem

re(fram

ing).

Calculate

comm

unication to im

part a stream

lined m

essage of student achievem

ent and to garner w

ide support to bring about organizational vision.

Synthesize all

elements of a

district’s perform

ance, not just “bottom

line” data to im

pact the culture and im

provement

process for student achievem

ent.

District

administrators base

PD

for principals in D

DD

M on student

achievement data

and research-based best practices. A

cohesive and receptive group of principals, district-w

ide, responsive to the im

plementation

of a DD

DM

fram

ework at their

school sites.

Superintendent and

district adm

inistrators are present and engaged in all principal P

D

meetings and school

site visits. Principals,

district-wide, receive

and implem

ent D

DD

M fram

ework

via distributive leadership and professional collaboration.

Superintendent and

district adm

inistrators build capacity of principals by serving as facilitators, cognitive coaches, m

entors, and consultants in P

D

and monthly school

site visits. Investm

ent is made

in data infrastructure, access, and literacy.

Interdependency betw

een district and school site leaders is prom

inent. C

omm

unication strategies are evident betw

een principals and teachers. S

ymbiotic

relationship in system

learning betw

een district and schools are m

arked.

Cycle of inquiry

is part of the professional learning fram

ework,

which takes

place at district and school site P

D. S

chools instructional leadership team

use cycle of inquiry in grade level m

eetings.

District and schools

have a pronounced m

essage reflecting achievem

ents and urgent needs for im

provement.

Schools and

classrooms use

various means of

comm

unic-ation, i.e. data w

alls, charts, chats, new

sletters, and school events.

District celebrates

wide range of

endeavors. District

effects and makes

considerations and adjustm

ents based on state and local policy environm

ent. Leadership focus, core values, theories of action, data availability and literacy are prom

inent.

Table 8. M

astery of Le

adership S

trategies in Data Inform

ed Leadership

185

General principles can be extracted from this study, as its intent was to offer

rituals and routines in effective professional development and to provide districts

with a road map to support their school site leaders to confidently acquire the

knowledge and skills to implement an evidence-based culture at their schools.

Anchors for data-informed leadership (Knapp et al., 2006), with a resounding

consistency in the leadership focus by superintendent, district administrators, and

principals is imperative. Each tier, district, school, and classrooms, should

understand the theory of action by which system learning occurs, whether it is

through a predetermined school-wide professional development or through a district

wide framework. Data literacy should be a priority through the investment in a data

infrastructure, such as DataDirector, for use by all administrators and teachers.

Furthermore, support and tools should be provided for effective implementation.

Mastery of leadership strategies for Data Informed Leadership is vital to the

effectiveness of the professional development programs. The conduits are: the

ability to communicate and build trust and respect to bring together the support and

genius of all stakeholders; the investment in data tools and support; and, the

investment in third party sources to develop expertise in establishing an

infrastructure which engenders system learning for district administrators, principals,

teachers, and students, which includes, but is not limited to distributive leadership.

186

Conclusion

The implementation of a systematic and comprehensive professional

development to increase the capacity of school site administrators in DDDM

indicates the level of performance based accountability expected through state and

federal mandates. While this study provides well-timed and relevant information

into the process by which districts provides school leaders with professional

development in DDDM, a greater extent of research is necessary to ensure that

mastery of leadership strategies to ensure proficient utility of DDDM is prevalent

district wide. Visibly, the benefits of professional development are far reaching and

necessary to develop the expertise of school site leaders’ ability to establish a culture

of data literacy and data use to increase student achievement. At the end of the day,

the level of expertise amongst school leaders will be a reflection of district

leaderships’ ability to conduct effective professional development, and this will be

measured by the degree in which a school effectively increases students’

performance levels.

187

REFERENCES

American Institutes for Research (AIR). Evaluation Study of the Immediate Intervention/Underperforming Schools Program of the Public Schools Accountability Act of 1999 (2005). Retrieved February 26, 2008 from http://www.air.org/publications/pubs_ehd_school_reform.aspx

Blasé, J. & Blasé, J. (2004). Handbook of Instructional Leadership: How Successful

Principals Promote Teaching and Learning. University of Georgia, Athens, GA: Corwin Press.

Bloom, G., Castagna, C. & Warren, B. (2003). More than mentors: Principal

coaching. Leadership, May/June 2003. Sacramento, CA: Association of California School Administrators.

Buck Unified School District (2008). Strategic Plan (2006-2010). Retrieved on

May 25, 2008 from http://myBUCKusd.org/pdf/strategicplan.pdf Burke, J. C. (2004). Achieving accountability in higher education: Balancing public,

academic, and market demands. In J. C. Burke (Ed.). The Many Faces of Accountability. (pp. 1-24). San Francisco, CA: Jossey-Bass.

Butler, J.A. (1992). Staff Development. School Improvement Research Series (SIRS). Retrieved December 6, 2007 from http://www.nwrel.org/scpd/sirs/6/cul2.html

California Department of Education (2007). 2007 Adequately Yearly Progress

Report Information Guide. Retrieved March 2, 2008, from http://www.cde.ca.gov/ta/ac/ay/documents/infoguide07.pdf

California Department of Education, Accountability Progress Reporting (2008),

retrieved from: http://www.cde.ca.gov/ta/ac/ar/index.asp Coburn, C.E., Honig, M.I., & Stein, M.K. (in press). What is evidence on districts’

use of evidence? In J. Bransford, L., Gomez, D., Lam, N., Vye (Eds.) Research and practice: Towards a reconciliation. Cambridge: Harvard Educational Press.

Coburn, C.E. & Talbert, J.E. (2006). Conceptions of evidence use in school districts:

Mapping the terrain. American Journal of Education 112(4), 469-495.

188

Corcoran, T., Fuhrman, S. & Belcher, C. (2001). The district role in instructional improvement. Phi Delta Kappan, 83(1), 78-84.

Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage. Darling-Hammond , L. (2002). The right to learn (pp. 148-176). San Francisco:

Jossey-Bass. Darling-Hammond, L. & Orphanos, S. (2006). Leadership Development in

California. Retrieved on December 20, 2007, from: http://irepp.stanford.edu/projects/cafinance.htm

Datnow, A. (2005). The Sustainability of Comprehensive School Reform Models in

Changing District and State Contexts. Educational Administration Quarterly, Vol. 41, No. 1, 121-153.

Datnow, A., Park, V. & Wohlstetter, P. (2007). Achieving with Data. Los Angeles,

CA: Center of Educational Governance, University of Southern California.

Dembosky, J., Pane, J., Barney, H. & Christina, R. (2005). Data Driven Decision-making in Southwestern Pennsylvania School Districts. Retrieved on December 20, 2007, from: https://www.rand.org/pubs/working_papers/2006/RAND_WR326.pdf

DuFour, R., DuFour, R., Eaker, R., Many, T. (2006 ). Learning by Doing – A Handbook for Professional Learning Communities at Work. Indiana: Solution Tree Earl, L. & Katz, S. (2002). Leading schools in a data rich world. In K. Leithwood,

Pl. Hallinger, G. Furman, P. Gronn, J. MacBeath, B. Mulforld & K. Riley (Eds.) The second international handbook of educational leadership and administration. Dordrecht, Netherlands: Kluwer.

EdSource, California School Finance, Accountability Overview. Retrieved on

March 27, 2008 from http://www.edsource.org/edu_acc.cfm Elmore, R. F. (2002). Bridging the gap between standards and achievement,

Washington, DC, Albert Shanker Institute. Retrieved July 12, 2003, from http://www.nsdc.org/library/results/res11-02elmore.html

189

Elmore, R.F. & Burney, D. (2000). Leadership and learning: Principal Recruitment, induction and instructional leadership in Community School

District #2, New York City. Pittsburgh, PA: University of Pittsburgh, HPLC Project, Learning Research and Development Center.

Elmore, R. & Rothman, R. (1999). Testing, teaching, and learning: A guide for states and school districts. Washington, DC: National Academy Press.

Fink, E., & Resnick, L. (2001, April). Developing principals as instructional leaders. Phi Delta Kappan, 82, 598-606. Fullan, M. (1990). Staff Development, Innovation, and Institutional Development. Changing School Culture Through Staff Development, edited by Bruce

Joyce. Alexandria, VA: Association for Supervision and Curriculum Development (p. 3-25).

Fullan, M. (2000). Change Forces: The Sequel. Philadelphia, PA: Falmer. Fullan, M. (2002). The Change Leader. Educational Leadership. Retrieved March

28, 2008, from http://www.sacle.edu.au/files/ChangeLeader.pdf Hale, E.L. & Moorman, H.N. (2003). Preparing School Principals: A National

Perspective on Policy and Program Innovations, Institute for Educational Leadership, Washington, DC and Illinois Education Research Council, Edwardsville, IL.

Heritage, M. & Yeagley, R. (2005). Data use and school improvement: Challenges

and prospects. In Herman, J. L. & Haertel, E. H. (Eds.), Uses and misuses of data for educational accountability and improvement (pp320-339). Malden, MA: Blackwell Publishing.

Knapp, M., Swinnerton, J., Copland, M. & Monpas-Huber, J. (2006). Data-

Informed Leadership in Education. Seattle, WA: Center for the Study of Teaching and Policy, University of Washington. 56 pages, from http://depts.washington.edu/ctpmail/PDFs/DataInformed-Nov1.pdf

Leithwood, K., Louis, K.S., Anderson, S. & Wahlstrom, K. (2004). How leadership

influences student learning. Retrieved on January 16,2008, from http://www.wallacefoundation.org/NR/rdonlyres/52BC34B4-2CC3-43D0-9541-9EA37F6D2086/0/HowLeadershipInfluences.pdf

190

Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education. RAND Corporation. Retrieved December 14, 2006 from

http://www.rand.org/pubs/occasional_papers/2006/RAND_OP170.pdf Merriam, S. B. (1998). Qualitative research and case study applications in

education. San Francisco, CA: Jossey-Bass Publishers. National Staff Development Council (n.d.) Learning to Lead, Leading to Learn:

Improving school quality through principal professional development. Retrieved, April 30, 2008, from http://www.nsdc.org/library/leaders/leader_report.cfm

Oberman, I., Hollis, J. & Dailey, D. (2007). Bringing the State and Locals Together:

Developing Effective Data Systems in California School Districts, SpringBoard Schools.

O'Day, J., Bitter, C., Kirst, M., Carnoy, M., Woody, E., & Buttles, M., et al. (2004).

Assessing California's accountability system: Successes, challenges, and opportunities for improvement (Policy Brief 04-2). Berkeley, CA: Policy Analysis for California Education. Retrieved March 26, 2008, from http://pace.berkeley.edu/policy_brief.04-2.pdf

Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc.

Peterson, Kent (2002). The Professional Development of Principals:

Innovations and Opportunities. Educational Administration Quarterly, Vol. 38, No. 2 (April 2002) 213-232.

Schmoker, M. (2006). Results Now. Alexandria, VA: Association for Supervision and Curriculum Development.

Senge, P.M. (1999). The Leaders New Work, Building Learning Organizations. In

Hickman, G.L. (Ed.) Leading Organizations: Perspective for a New Era (pp. 439-449). Thousand Oaks, CA: Sage Publications, Inc.

Supovitz, J. A. & Klein, V. (2003). Mapping a course for improved learning: How

innovative schools systematically use student performance data to guide instruction. Philadelphia, PA: Consortium for Policy Research in Education. Retrieved February 5, 2005, from http://www.cpre.org/Publications/AC-08.pdf

191

Togneri, W. & Anderson, S. E. (2003). Beyond islands of excellence: What districts can do to improve instruction and achievement in all schools. Washington, DC: Learning First Alliance.

U.S. Department of Education. (n.d.). Executive summary of the No Child Left

Behind Act of 2001. Washington, D. C.: U.S. Department of Education. Retrieved February 24, 2008, from http://www.ed.gov/rschstat/eval/disadv/tassie1/index.html

U.S. Department of Education (1998). San Francisco Unified School District, San

Francisco, California: Profile of Award-Winning Professional Development. Retrieved on May 15, 2008, from

http://www.ed.gov/inits/teachers/96-97/sanfran.html

Woody, E. L., Buttles, M., Kafka, J., Park, S., & Russell, J. (n.d.). Voices from the field: Educators respond to accountability. Berkeley, CA: Policy Analysis for California Education (PACE). Retrieved March 3, 2005, from http://pace.berkeley.edu

192

APPENDIX A

District Administrator Interview Protocol8

Note: I will begin the interview by informing the interviewee about myself and my study. An explanation will be provided regarding the tape recording of the interview and that their responses will be strictly confidential. They will also be informed that if there is something they would like to say off tape, I will oblige by stopping the tape midstream for their commentary.

1. Please describe your role in the district (background). 2. Please describe the support the district provides to school site administrators in

terms of data use and/or data driven decision making (research question 1, 2, and 3). Probes: professional development, specialized data, reports, etc. How often? What are school sites expected to do with these data?

3. Please describe the district’s professional development plan to assist school site administrators in the collection and analysis of data? Technology, data the district collects, whether from schools directly or from other sources, such as state assessment data (background and research question 1, 2 and 4, indirectly, and 3 directly). Probe: How often and what types of professional developments are offered in data collection and analysis? Please describe the types of professional development offered to school site administrators in developing a culture of data use at their respective schools sites? How often do these types of trainings occur? Does the professional development plan include coaching or mentoring?

4. How does the district measure the success or effectiveness of its professional development plan for school site administrators? (research questions 2, 4 and 5) 5. Are the selected school site leaders to be studied typical of other school site leaders in the district in terms of their establishment of school culture in data use? (research questions 2 and 4, and question 3 indirectly)

Probe: How? If not, what are the challenges?

8 District Administrator Protocol Interview Questions were adapted from Dr. Amanda Datnow’s Protocol

193

6. What is your role, as a district administrator, in building the capacity of school site administrators in the use of data and in establishing a culture of data use at their school sites? (research questions 1, 2, 3 directly, 4 and 5 indirectly)

Probe: • Is professional development comprehensive and sequential? • Is professional development tailor made based on the needs of

individual school site administrators • Is there a cycle of inquiry in the district’s infrastructure to allow for

ongoing adjustments in professional development

7. At a minimum, what do you expect school site administrators to use from the professional development offered as a means of student outcome improvement? (research question 2 and 4)

Probe: • Are these expectations system-wide? • How do you expect school site administrators to execute the plan of

improvement at their school site? • How will you know that the expectations have been met?

8. What problems have you seen schools run into in trying to use data for decision making? (research question 5) Probe:

• How do you measure the need for additional professional development (research question 4)?

• How are the problems addressed?

9. What problems have you seen school site administrators run into in trying to establish a culture of data use at their school site? (research questions 2 and 5)

10. Is there anything else you think might assist in raising test scores in the districts’ schools? (research questions 2 and 5)

11. What do you see as the next steps for the district’s professional development program in supporting schools’ use of data? (research questions 5)

12. What advice would you have for other district administrators about how they might provide a comprehensive professional development program to support site administrators in effective use of data? (research questions 1-4) Note: I will conclude the interview by thanking the participant for his/her time and

valuable input.

194

Research Questions

Cross-references of interview questions with the five research questions: 1. How do districts build the capacity of school leaders to engage in data-driven

decision making?

2. How is the need for professional development assessed?

3. What types of training and support is provided school site leaders?

4. What evidence is there that leaders put new data-driven practices into place?

5. What knowledge gaps still exist?

195

APPENDIX B

Administrator Interview Protocol9

Note: I will begin the interview by informing the interviewee about myself and my study. An explanation will be provided regarding the tape recording of the interview and that their responses will be strictly confidential. They will also be informed that if there is something they would like to say off tape, I will oblige by stopping the tape midstream for their commentary.

1. Please describe briefly the history of the school (background). 2. Please describe the district and school’s journey into data-driven decision

making (all research questions, indirectly). Probes: history of data use, process of becoming more data-driven

3. Please tell me about your school’s performance in terms of accountability

measures. (research question 3 and 4, indirectly) Probes:

• How has the school’s API progressed and have targets been met? • How has the school’s AYP progressed and have targets been met? • Are there other district measures used to assess schools? � If so, what are they and how has the school done?

4. If targets have been met/progress has been made: To what do you attribute

your school’s growth? (research question 3 and 4, indirectly) 5. If data is mentioned: How are data used at your school site? (research

question 3, directly) If data is not mentioned: From reading the data about your school on the California Department of Education website, data seems to play a part in the school’s improvement in accountability measures. How are data used at your school site? (research question 3, directly, and 4 indirectly)

6. What types of data are analyzed at this school? (research questions 1, 2, 3, and

4 indirectly) Probes: state assessments, district assessments; school, classroom,

standardized, teacher-created

9 Administrative Protocol Interview Questions were adapted from Dr. Amanda Datnow’s Protocol

196

7. How are data analyzed at this school site? (research question 3 and indirectly 2 and 4) Probes: staff development, grade level meetings, administrative efforts, teacher efforts, professional learning communities

8. What are the district’s expectations of data use? (research questions 1 and 2, indirectly) Probe: Why do you believe that school administrators use data? (rewards/sanctions)

9. How does your district support the use of data by administrators? (research

question 1 and 2, directly) Probes: Does the district provide you with professional development? If so,

does the district expect administrators to analyze data individually? Does the district provide administrators time to analyze data together?

If yes to either question: • Does the district provide options about use of this time or is an agenda or

some other guidance/structure provided? • Do administrators provide the district with minutes or other evidence of

their data use implementation, analyses, and discussions at school sites? • How does the district determine what type of professional development in

data-driven decision making is needed by school administrators? How is the training and support provided (i.e., staff meeting time, site visit, mentoring, coaching, etc.)

10. Does the district provide a comprehensive professional development plan to

assist school site administrators in data use, data driven decision making, and building a culture of data use at the school sites? (research question 2)

If yes: Please explain the types of training provided. 11. Are there individuals at the district who provide support to you in the use of

data and/or building a culture of data use at your site? (research question 2) If yes: Please describe what that person does. 12. What problems have you run into in trying to use data for decision making?

(research questions 2, 3 and 4) Probe: What types of training and supports does the district provide?

13. Is there anything else you think might be making a difference/helping to raise

test scores at this school? (research question 2 and 1, indirectly)

14. What do you see as the next steps in your school’s use of data? (research questions 2 and 3, indirectly)

197

15. What advice would you give to me, as a school site administrator, about how I

might begin to use data or building a culture of data use for decision making to increase student achievement at my school site? (research questions 1 and 3, and 4 indirectly)

Note: I will conclude the interview by thanking the participant for his/her time and

valuable input.

Research Questions

Cross-references of interview questions with the four research questions: 1. How do districts build the capacity of school leaders to engage in data-driven

decision making?

2. How is the need for professional development assessed?

3. What types of training and support is provided school site leaders?

4. What evidence is there that leaders put new data-driven practices into place?

5. What knowledge gaps still exist?

198

APPENDIX C

Observation Protocol – District Professional Development Data Discussion

Date: District: Time: Administrators present: DDDM Professional Development

Discussion Observation List Notes/Comments

During the discussion, I will tape record and take running notes of the meeting. I will formulate specific records of whether the following items are discussed and/or used:

• Informal school/student assessment data

• Formal school/student assessment data

State assessment

District-created assessment

School-created assessment

Publisher-created assessment

• School Site Professional Development

• Professional Learning Communities

• Grade Level Meetings

• Coaching/Mentoring

Other questions or items I will look for: Was there a formal agenda? If so, who generated it?

Did administrators bring actual records and/or assessments to the discussion? Did administrators present and/or collaborate based on actual school site data?

199

APPENDIX D

Preliminary Codes

1. Assess Need for Professional Development

2. Building Capacity

3. Facilitators of Data Driven Decision Making (DDDM)

4. Resources – Training and Support/Plan

5. Process of DDDM

6. Process of Data Analysis

7. Technology/Support/Tools

8. Culture of Data Use

9. Types of Data

10. Evaluation Tools

11. Evidence of DDDM

12. District Expectations

13. Challenges

14. Gaps

200

APPENDIX E

Salient Codes

1. Needs Assessment for Professional Development

2. Leadership Capacity Building

3. Support in Process of Data Driven Decision Making

4. Accessibility to Tools

5. Culture of Data Use

6. Effectiveness of Professional Development

7. District Expectations

8. Challenges