Deliberate Practice Revealed - paec.org · Deliberate Practice Revealed ... principals and make certain that they are receiving “just in ... about Deliberate Practice to readers

1

Deliberate Practice Revealed

Recently, we have been providing school districts across the state of Florida technical assistance support as they endeavor to implement the Florida Department of Education’s model principal evaluation framework. Toward that end, we have presented to school and district leaders throughout the state of Florida about the elements of the Florida School Leaders Assessment (FSLA) framework (e.g., the research base upon which the model is built, the leadership domains of practice and related areas of proficiencies and indicators that describe highly effective and effective school leader practices, the steps to scoring school leaders performance, and the related school leader growth plan or “Deliberate Practice” model that requires school leaders to focus on “thin slices” of leadership practice to improve student results). During these presentations, participants routinely asked such questions as:

How does the concept of Deliberate Practice connect to the FSLA framework in general and specifically to the FSLA Indicators?

How do we measure the impact of leadership practices?

How do we use the “cause” data (e.g., measures of leadership practice) to make just-in-time adjustments to leadership practices?

How do we monitor the implementation of the Deliberate Practice efforts of principals and make certain that they are receiving “just in time, just for them, just for where they are in their learning process, and just what they need to move forward” (Hattie, 2012, p. 20) feedback?

Questions such as these caused us to think more deeply about the concept of Deliberate Practice. As a result, we spent many hours reviewing professional growth plans (noting the absence of clearly articulated, measurable leadership strategies or practices, despite the claims of school leaders that unambiguous leadership practices have been identified), worked with school leaders to help them understand how they might measure their leadership impact (with little success), and have tried different methods of showcasing how the results of Deliberate Practice might be depicted and used to inform leadership practice using a variety of bar charts or graphs.

Our challenge was to help school leaders understand that the exercise was more than an exercise in compliance by simply filling in the blanks of the Deliberate Practice Template. Rather, we discovered new ways to help school leaders understand the relationship of the items within the Deliberate Practice Template (see Table 1) to one another and the potential impact on their practice. Overcoming this challenge helped us help them realize that successfully crafting a Deliberate Practice Growth Plan must first be preceded by deliberate thought as to the reasoning behind the various sections of this planning document. When school leaders engage in thoughtful self-assessment of the FSLA Indicators from which they identify prioritized indicators of practice, connect the knowledge and skills within these indicators to targeted needs within their school improvement plan, consider what professional development support they require specific to the knowledge and skills reflected in the prioritized indicators, and then develop one or two measurable leadership strategies that cause the school leader to apply the required knowledge and skills within these FSLA “Indicators” to solve complex

2

problems central to the core business of teaching and learning (e.g., highlighted within the school’s improvement plan), then school leaders are thinking deeply and deliberately about Deliberate Practice.

Consequently, the purpose of this “thought paper” is to expose our thinking and learning about Deliberate Practice to readers in an effort to help them think about how they might want to approach the task of making their leadership visible by becoming students of their own effects and creating Deliberate Practice efforts that are representative of quality action research initiatives and successfully complete the Deliberate Practice Growth Target Template reflected in Table 1.

School Leader’s Name and Position:_____________________________________________________________________________________

Evaluators Name and Position: _________________________________________________________________________________________

Target for school year: 2012-13 Date Growth Targets Approved: ___________________________________________________________

School Leader’s Signature: _______________________________________

Evaluator’s Signature___________________________________

Deliberate Practice Growth Target #: ___ (Insert target identification number here, the check one category below)

( ) District Growth Target ( ) School Growth Target ( ) Leader’s Growth target

Focus issue(s): Why is the target worth pursuing? (SMART Goal from SIP)

Growth Target: Describe what you expect to know or be able to do as a result of this professional learning effort.

Anticipated Gain(s): What do you hope to learn?

Plan of Action: A general description of how you will go about accomplishing the target.

Progress Points: List progress points or steps toward fulfilling your goal that enable you to monitor your progress. If you goal

1.

2.

3

Notes:

Table 1: Deliberate Practice Growth Target

To begin with, the state has described deliberate practice as a process in which the leader and the evaluator identify one to four specific and measurable priority-learning goals related to teaching, learning, or school leadership practices that impact student learning growth. This description closely parallels Geoff Colvin’s (2008) description of the same topic in his book entitled, Talent Is Overrated: What Really Separates World-

3

Class Performers from Everybody Else. Colvin suggests that deliberate practice requires individuals to identify certain “sharply defined elements of performance that need to be improved, and then work intently on them” (p. 68). Colvin goes on to say that a hallmark of great performers (i.e., the great soprano Joan Sutherland or Tiger Woods) is that they “isolate remarkably specific aspects of what they do and focus on just those things until they are improved; then it’s on to the next aspect” (p. 68).

To practice deliberately, you must focus on areas that need to be improved. It takes courage and honest introspection to identify areas of your performance that need work. In our FSLA technical support presentations to school leaders we have recommended a prioritization process called “funneling” to help them isolate specific aspects of their performance on which to focus their improvement efforts. The funneling process consists of five steps and is depicted in Figure 1.

In order to make certain that school leaders understand the practice of “narrowing” the data and focusing on specific aspects of leadership practice, it may be helpful to think in terms of a flow through filter funnel. In the area of science, a filter funnel is a laboratory funnel that scientists use to separate solids from liquids via the laboratory process of filtering. In order to achieve this, a disk shaped piece of filter paper is usually folded into a cone and placed within the funnel. A mixture of solid and liquid is then poured through the funnel. The solid particles being too large to pass through the filter paper are left on the paper, while the much smaller liquid molecules pass through the paper to a vessel positioned below the funnel. Thus producing a filtrate.

In everyday terms, we witness this “filtration” process every time we make coffee for example where hot water is poured over ground up coffee beans. The hot water coffee-filtrate is then deposited in the coffee pot leaving behind the less important used coffee grounds trapped in the filter. Like scientists in the laboratory, educational practitioners would be wise to apply an educational “filter funnel” to their deliberate practice process. That is, school leaders could use an educational “filter funnel” through which they pour a variety of causal data (e.g., leadership practices reflected within The FSLA Indicators), which then passes through multiple filters that invokes a simplifying or narrowing heuristic to the leader’s self-assessment data by separating less important data particulates from the more important data filtrate. This data “filtering” process allows

Figure 1: Narrowing the Leader’s Focus

Narrowing the Leader’s Focus

Lowest scored indicators

ALL Domains

Lowest scored “weighted”

Indicators Domains 1 & 2

Lowest scored

Indicators

Proficiency

Area 4

Self-assessment of all 45 Indicators

3-5 High-leverage Leadership Indicators

4

school leaders to focus their improvement efforts on those few FSLA Indicators (leadership knowledge and skills) that are most directly aligned to increases in student achievement.

The first step of this “filtering” or narrowing process directs the school leader to conduct a self-assessment of all indicators for which they are being held accountable. The leader’s self-assessment process is recorded on the FSLA Short Form (a listing of all four domains, 10 proficiency areas, and 45 indicators) subsequent to a review of the narrative descriptions for each indicator on the corresponding FSLA Medium and/or Long Form along with the “word picture” descriptions of Effective practice within the Long Form Rubric. That is, for each indicator, the leader compares their existing practice in this indicator area to the practice specified within the FSLA narrative and rubric descriptions and then records their level of performance (i.e., Highly Effective, Effective, Needs Improvement, or Unsatisfactory) on the FSLA Short Form (see Table 2, column 1 for example).

In the second step, the leader identifies the lowest scored indicators in ALL four FSLA Domains by drawing a circle around those low performance indicators on the FSLA Short Form. It is important to note that the lowest scored indicator in Domains and Proficiency Areas are not necessarily just those scored at the “Needs Improvement” and/or the “Unsatisfactory” levels, as “Effective” performance in some of those areas may well be the lowest performance score (see Table 2, column 2 for example).

Third, considering all of the “circled” indicators, the leader highlights the lowest scored “weighted” indicators (i.e., those 25 indicators that are identified in the state’s FSLA Phase In options) in Domains 1 and 2. We are recommending that the leader focus on these two domains of practice given the fact that the knowledge and skills reflected in these indicators of practice are closely aligned to the research base as potentially having the greatest impact on increases in student achievement (see Table 2, column 3 for example).

Next, with the lowest scored “weighted” indicators in Domains 1 and 2 in mind, the leader should identify the lowest scored indicators in Proficiency Area 4. Proficiency Area 4 deals with Faculty Development and focuses school leaders on such practices as staff recruitment, retention, and on developing an effective and diverse faculty and staff; on evidence, research, and classroom realities faced by teachers; and on linking professional practice with student achievement to demonstrate the cause and effect relationship; facilitate effective professional development; monitor implementation of critical initiatives; and secure and provide timely feedback to teachers so that feedback can be used to increase teacher professional practice (see Table 2, column 4 for example).

The suggestion to focus on Proficiency Area 4 above all the rest (the narrow part of the funnel) is made for two reasons. First, with the successful passage of Senate Bill 736, the Florida State Legislature designated “Instructional Leadership” as being a required competency of all Florida School Leaders and therefore a requirement within the evaluation process. Senate Bill 736 described instructional leadership as including but not limited to “performance measures related to the effectiveness of the instructional

5

personnel in the school…recruitment and retention of effective and highly effective classroom teachers…improvement in the percentage of classroom teachers evaluated at the effective and highly effective level.” Second, research (Robinson, 2011) supports the fact that school leaders’ regular and expert application of “instructional leadership” (p.10) practices has a rather large effect size (0.84) and provides some empirical support for calls to school leaders to be actively involved with their teachers as the “leading learners” of their school. With student background factors controlled, the more that teachers report their school leaders (usually the principal) to be active participants in teacher learning and development, the higher the student outcomes.

The last step of this FSLA Indicator “narrowing” process requires the school leader to thoughtfully reflect on the remaining indicators and identify a few “sharply defined elements of their performance” (Colvin, 2010, p. 68), indicators, that they believe offer the greatest opportunity to leverage their knowledge, and skills (e.g., those reflected within these specific indicators) in multiple leadership performance areas (other FSLA Indicators) to improve student achievement. We suggest school leaders focus on a limited number (e.g., three to five) of indicators because we have found in our work with educators across the country that overload (e.g., simultaneously focusing on many things) and fragmentation (e.g., the loss of cohesion and unity) are as Michael Fullan (2000, p. 6) discovered, "The main enemies of large-scale reform” (see Table 2, column 5 for example) and blunt leaders’ efforts to make swift, dramatic improvements in schools.

#1: Self-Assessment of All

25 Indicators

#2: Lowest Scored Indicators in All

Domains

#3: Lowest Scored “Weighted”

Indicators Domains 1 & 2

#4: Lowest Scored Indicators

Proficiency Area 4

#5: 3-5 Prioritized Indicators

1.1-E 1.2-NI 1.3-E 2.1-NI 2.3-E 3.1-E 3.2-E 3.3-NI 3.6-NI 4.2-NI 4.3-NI 4.4-E 4.5-E

4.6-E 5.3-E 5.4-NI 6.1-E 6.4-HE 7.1-E 7.2-HE 8.3-E 10.1-HE 10.2-E 10.3-E 10.4-HE

Domain 1 1.2 2.1 Domain 2 3.3 3.6 Domain 1 4.2 4.3 1.2 2.1 5.4 Domain 2 Proficiency Area 4 Domain 3 3.3 3.6 4.2 4.3 4.2 4.3 6.1 7.1 4.2 4.3 4.4 4.5 4.5 3.6 8.3 5.4 4.6 1.2 Domain 4

10.2 10.3

Table 2: Results of A Fictitious “Funneling” Process

As a result of the “funneling” process described within the past several paragraphs, the school leader has now established her or his Deliberate Practice “Leader’s Growth

6

Target;” Those three to five FSLA Indicators and the knowledge and skills, which are richly described in the corresponding FSLA Long Form Rubrics. During our technical support trainings with districts we have routinely used a fictitious example (see Table 2 below) of a “funneling” process for a new school leader by the name of Ray whose self-assessment was based on the state’s Phase In Option #1 of 25 Indicators. Florida School Districts that decided not to “Phase In” the implementation of the state’s model could perform the same “funneling” process depicted in Table 2 with all 45 Indicators.

As depicted in Table 2, Ray’s self-assessment of the 25 indicators on which his 2012-13 evaluation will be based can be seen in the far left column, which is followed by the lowest scored indicators in all Domains as reflected in the second column. Next, Ray highlighted his lowest scored “weighted” indicators in Domains 1 and 2 in the third column and the subsequent reduction of indicators as a focus within Proficiency Area 4 in the fourth column. Inasmuch as the data in the first four columns of this table are fairly straightforward and require no further explanation, this is not the case with the data in the prioritized indicators in the last column. Clearly the prioritized indicators of 4.2, 4.3, and 4.5 make sense as they are situated squarely in Proficiency Area 4 as the “funneling” process suggests. But why would Ray and his supervisor not simply include the other two FSLA Indicators in Proficiency Area 4 (4.5 and 4.6)? Furthermore, what might have been the thinking behind including Indicators 3.6 and 1.2, which are situated in other proficiency areas? Here is one possible explanation behind the reasoning.

Ray along with his supervisor determined that by focusing on Indicator 4.2, Feedback Practices, 4.3, High Effect Size Strategies, and 4.5, Facilitating and Leading Professional Development related to writing (e.g., a School Growth Target as will be discussed shortly) that he would inadvertently be addressing FSLA Indicators 4.4, Instructional Initiatives and 4.6, Faculty Development Alignments. The addition of FSLA Indicators 1.2, Performance Data and Indicator 3.6, Faculty Effectiveness were logical inclusions as Indicator 1.2, Performance Data, requires Ray to demonstrate knowledge and skills in the use of not only student but adult performance data and Indicator 3.6, Faculty Effectiveness, will cause Ray to focus on the effectiveness of his instructional faculty as they move to use the research on effective writing strategies. That is, these two indicators compliment the previous three indicators that are part of Proficiency Area Four (4).

While determining the Leader’s Growth Target is essential, it is insufficient without also considering the other essential component of any improvement effort, the identification of a SMART goal that the school leader intends to accomplish by applying the knowledge and skills reflected within the prioritized three to five FSLA Indicators. Together, the two factors (School Improvement Plan SMART Goal and the Leader’s Growth Target) equal the leader’s Deliberate Practice Growth Plan (see Figure 2).

7

Figure 2: Deliberate Practice Formula

The leader’s SMART Goal should be drawn directly from their School’s School Improvement Plan. For example, assume that Ray’s school had identified the following Objective as part of the school’s Writing Goal, “By June 2013, 70% of eighth grade students will meet the DOE writing criteria in grades 6th – 8th will attain a Level 3.5 or above.” In addition, the staff at Ray’s school has also identified the following strategies in order to accomplish the writing goal and objective:

Implement “Writing Workshops” where students attend weekly to work on deficient skills.

Provide differentiated instruction within the classroom setting utilizing net TREKKER (i.e., an Internet search engine especially designed for K-12 schools. It allows for targeted searches of online resources that have been evaluated for educationally relevant content, aligned to state standards, reviewed by teachers for quality, with options to search by reading level, for related lesson plans and support materials, and much more), B.E.E.P. (i.e., Broward Enterprise Education Portal, an electronic resource center provided by Broward County School District for students), and the Writing Workshop.

Now that Ray has “narrowed” his “growth target” to three to five FSLA Indicators and has identified a SMART Goal from his school improvement plan he can complete the first portion of the Deliberate Practice Growth Target Template as depicted in Table 3. Note, for illustration purposes, Table 3 depicts just the first several lines of the entire Template (see Table 1). Notice how Ray was able to determine () that his Deliberate Practice was both a School Growth Target (i.e., working on a SMART Goal from his school improvement plan) and a Leader’s Growth Target (i.e., reflects 3-5 priority indicators on which Ray will be focusing—some of which will require new learning). That is, school leaders don’t necessarily need to create multiple Deliberate Practice Growth Target Templates. In fact, intentionally keeping the personal improvement plan tightly focused in scope constitutes a research based best practice.

School Improvement Plan SMART

Goal

Leader's Growth

Target (3-5 FSLA

Indicators)

Deliberate Practice

Growth Plan

8

School Leader’s Name and Position: Ray Smith, Principal, Marsh Middle School

Evaluators Name and Position: Dr. Julie Hellman, Director of School Improvement and Accountability

Target for school year: 2012-13 Date Growth Targets Approved:




( ) District Growth Target () School Growth Target () Leader’s Growth target


By June 2013, 70% of eighth grade students will meet the DOE writing criteria in grades 6th – 8th will attain a Level 3.5 or above

Growth Target: Describe what you expect to know or be able to do as a result of this professional learning effort.

Indicator 1.2 – Performance Data: The leader demonstrates the use of student and adult performance data to make instructional leadership decisions. Indicator 3.6 - Faculty Effectiveness: The leader monitors the effectiveness of classroom teachers and uses contemporary research and the district’s instructional evaluation system criteria and procedures to improve student achievement and faculty proficiency on the FEAPs. Indicator 4.2 - Feedback Practices: The leader monitors, evaluates proficiency, and secures and provides timely and actionable feedback to faculty on the effectiveness of instruction on priority instructional goals, and the cause and effect relationships between professional practice and student achievement on those goals. Indicator 4.3 - High Effect Size Strategies: Instructional personnel receive recurring feedback on their proficiency on high effect size instructional strategies. Indicator 4.5 - Facilitating and Leading Professional Learning: The leader manages the organization, operations, and facilities to provide the faculty with quality resources and time for professional learning and promotes, participates in, and engages faculty in effective individual and collaborative learning on priority professional goals throughout the school year.

Table 3: Sample Focus Issue and Growth Target Statement

Next, Ray must address at least three challenges. First, he must be clear as to what exactly he is expected to know and be able to do to perform at the “Effective” or higher performance level(s) for each of the prioritized indicators. Second, Ray must determine with which of this knowledge and these skills does he feel competent and what for him constitutes new learning and therefore will require professional development support? Last, he must figure out how he might apply the knowledge and skills reflected in the five prioritized FSLA Indicators (e.g., specific leadership strategies) to help his staff implement the “writing strategies” (i.e., implement Writing Workshops and differentiated writing instruction) and accomplish the stated writing objective within his school improvement plan?

During our technical support workshops we have used a process called “unpacking” to help school leaders be clear about what they must be able to demonstrate to be considered “Effective” or higher in any particular FSLA Indicator of practice. To “unpack” any FSLA Indicator, school leaders would begin by identifying the Proficiency Area in which the Indicator is located (see Table 4 for details) along with the brief description of the Proficiency Area (which they can simply cut and paste from their FSLA Evaluation document). The Indicator that is being “unpacked” follows the brief description of the Proficiency Area. Following the identified Indicator is a brief description of that Indicator, which, as with the Proficiency Area description, can be cut and pasted from the FSLA document.

9

Essentially, the “unpacking” process requires leaders to underline the nouns (the knowledge), circle the verbs (the skills) in the description of an Indicator, and then determine the level of cognitive knowledge and skills within the indicator by “Blooming” them (e.g., use Bloom’s Taxonomy to determine the cognitive level). Table 4 depicts an “unpacking” process for FSLA Indicator 4.2 that school leaders might want to consider so that they are clearly aware of the embedded knowledge and skills in the indicators they have prioritized for their Deliberate Practice work (Note, rather than circling the verbs we have gray shaded the verbs in this illustration in Table 4, Indicator 4.2 Feedback Practices). While Table 4 illustrates the “unpacking” process for FSLA Indicator 4.2, it is important to note that the same process would need to be replicated for each of the prioritized indicators. In other words, using Ray’s example, he would need to replicate the “unpacking” process for the remaining four indicators (i.e., Indicator 1.2, 3.6, 4.3, and 4.5) in order to see a complete picture of the expected knowledge and skills reflected in these five indicators.

10

Knowledge What school leaders

need to know about nouns

Skills What school leaders need to be able to do

verbs

Level of Cognitive

Knowledge and Skills

Illustrative Example of Impact What the school leader might produce to demonstrate impact

Proficiency Monitor (proficiency) 1 A schedule that results in frequent walkthroughs and observation of teaching and learning

Evaluate (proficiency) 5 Rubrics that distinguish among proficiency levels on evaluation indicators are used by the leader to focus feedback on needed improvements in instructional practice.

Timely and actionable feedback

Secures (timely and actionable feedback) 1 Evidence the leader has a system for securing feedback from teachers specific to prioritized instructional practices.

Provides (timely and actionable feedback) 5 Samples of written feedback provided teachers regarding prioritized instructional practices.

Effectiveness of instruction Secures (timely and actionable feedback on the effectiveness of instruction)

1 Evidence the leader has a system for securing feedback from teachers specific to their knowledge and skills of effective instruction

Provides (timely and actionable feedback on the effectiveness of instruction)

5 The leader provides feedback that describes ways to enhance performance and reach the next level of proficiency.

Priority instructional goals Secures (timely and actionable feedback on priority instructional goals)

1 Evidence the leader has a system for securing feedback from teachers specific to priority instructional goals

Provides (timely and actionable feedback on priority instructional goals)

5 Samples of written feedback provided teachers regarding priority instructional goals

Cause and effect relationships between professional practice and student achievement…

Secures (timely and actionable feedback on cause and effect relationships between professional practice and student achievement)

1 Teacher produced graphs or charts that depict the degree to which key instructional practices are implemented and the impact or not on student achievement

Provides (timely and actionable feedback on cause and effect relationships between professional practice and student achievement)

5 School leader produced graphs or charts that depict the degree to which key instructional practices are implemented by teachers and the impact or not on student achievement

Table 4: Sample Unpacked FSLA Indicator 4.2—Feedback Practices

Proficiency Area: 4—Faculty Development Effective school leaders recruit, retain, and develop an effective and diverse faculty and staff; focus on evidence, research, and classroom realities faced by teachers; link professional practice with student achievement to demonstrate the cause and effect relationship; facilitate effective professional development; monitor implementation of critical initiatives; and secure and provide timely feedback to teachers so that feedback can be used to increase teacher professional practice. Indicator: 4.2—Feedback Practices The leader monitors, evaluates proficiency, and secures and provides timely and actionable feedback to faculty on the effectiveness of instruction on priority instructional goals, and the cause and effect relationships between professional practice and student achievement on those goals.

11

Once Ray has revealed the knowledge and skills he is required to demonstrate to be “Effective,” as stated earlier he must determine what of this knowledge and skills he knows and can do and what will require additional training in the form of professional development. In Ray’s case he might determine that while he has a superficial understanding of feedback, based on the rubric descriptions of effective feedback and “cause” and “effect” data that he would need some additional support in these areas. For example, he might want to participate in one of the district’s professional development offerings that address the issues of feedback and cause and effect data. Additionally, he might secure a copy of John Hattie’s book, Visible Learning for Teachers to read about these concepts thereby building his capacity in these growth areas.

The next step in the “unpacking” process requires the school leader to translate the knowledge and skills reflected in the three to five FSLA Indicators into one or two specific and measurable leadership strategies. These one to two leadership strategies become the “Action Plan” the leader intends to implement, monitor, and measure against improvements in student achievement (e.g., writing). Prior to offering examples of strategies it is important to define what we mean by the phrase—leadership strategy. A leadership strategy is: Observable, subject to frequent public testing, and measurable (e.g., quantifiable; able to be gauged); An obvious translation (i.e., paraphrase) of the three to five prioritized “unpacked” FSLA Indicators; Constructed with formative language (e.g., increase the percent of, reduce the percentage of, increase the amount or number of, etc.), as they are intended to be “dipstick” measures over time, rather than goal statements; and Time-bound: they should indicate how often the public testing will occur.

To return to the fictitious example of Ray, an illustration of a leadership strategy that Ray would develop might look something like this and inserted in Table 5,

Increase the percent of faculty implementing differentiated writing instruction at the “Proficient” or higher level based on both direct observation and teacher self-assessment monthly using a locally developed rubric.



Table 5: Sample Plan of Action Statement

This leadership strategy statement paired with the school improvement plan goal of improving student achievement in writing establishes a hypothesis that begs to be tested. For instance, if Ray and his teachers demonstrate monthly increases in faculty who proficiently (or higher) implement differentiated writing instruction (using a locally developed rubric), as described in this strategy statement, then they would expect to see similar increases in the percent of students scoring at the “Proficient” or higher levels of a locally developed writing assessment using a site-developed rubric. Consequently, the two essential “Progress Points” (see Table 6) that Ray and his staff

12

will want to monitor is:

The percent of staff implementing at the “Proficient” or higher level differentiated writing instruction monthly

The percent of students scoring at the “Proficient” or higher level on a writing assessment monthly


1. The percent of staff implementing at the “Proficient” or higher level differentiated writing instruction monthly 2. The percent of students scoring at the “Proficient” or higher level on a writing assessment monthly

Table 6: Sample Progress Points Statements

Moreover, this illustrative example of a strategy meets the definition of a leadership strategy in all aspects as well as incorporating the knowledge and skills of all five FSLA Indicators. That is, the strategy is observable as it is subject to frequent (monthly) public testing (securing from teachers the degree to which they are implementing differentiated writing instruction), and measurable (if there are 30 faculty members on Ray’s staff it’s the percent of Ray’s faculty who rate themselves at the “Proficient” or higher level of the rubric any given month). Next, the strategy requires Ray to monitor faculty proficiency or effectiveness (implementation of differentiated instruction) and evaluate, which staff are implementing the instruction at the proficient or higher level of a rubric (Indicator 1.2). Because the leadership strategy indicates that monthly increases in faculty proficiency should occur, Ray must not only give timely and actionable feedback (Indicators 3.6, 4.2 & 4.3) to teachers about the differentiated writing instruction he is observing them use, he also must secure from his teachers their self-assessment of the degree to which they are implementing the same writing practices (Indicators 3.6, 4.2 & 4.3) so that he can use this data to support the improvement of his teacher’s writing instruction (Indicators 1.2 & 4.5).

As a result of implementing this leadership strategy, two key learning intentions or “Anticipated Gains” would evidence themselves (see Table 5). First, Ray along with his faculty would become keenly aware of the barriers to proficient student writing. Second, they would develop an awareness of the instructional practices that appear to most help them overcome those barriers as well as those don’t have a positive relationship to improvements in student writing. Third, Ray would learn whether or not his selected leadership strategy was having the desired impact or not. These three points can be summarized into the category “Anticipated Gains” in Table 7.


Become keenly aware of the barriers to proficient student writing

Develop an awareness of the instructional practices that appear to most help them overcome those barriers as well as those don’t have a positive relationship to improvements in student writing

Determine whether or the selected leadership strategy was having the desired impact or not

Table 7: Sample Anticipated Gain(s) Statements

A fully completed Deliberate Practice Growth Template, which combines Tables 3, 5, 6, and 7, is depicted in Table 8.

13

School Leader’s Name and Position: Ray Smith, Principal Swam Middle School

Evaluators Name and Position: Dr. Julie Hellman, Director of School Improvement and Accountability

Target for school year: 2012-13 Date Growth Targets Approved: September 28, 2012




( ) District Growth Target () School Growth Target () Leader’s Growth target


By June 2013, 70% of eighth grade students will meet the DOE writing criteria in grades 6th – 8th will attain a Level 3.5 or above Growth Target: Describe what you expect to know or be able to do as a result of this professional learning effort.

Indicator 1.2 – Performance Data: The leader demonstrates the use of student and adult performance data to make instructional leadership decisions. Indicator 3.6 - Faculty Effectiveness: The leader monitors the effectiveness of classroom teachers and uses contemporary research and the district’s instructional evaluation system criteria and procedures to improve student achievement and faculty proficiency on the FEAPs. Indicator 4.2 - Feedback Practices: The leader monitors, evaluates proficiency, and secures and provides timely and actionable feedback to faculty on the effectiveness of instruction on priority instructional goals, and the cause and effect relationships between professional practice and student achievement on those goals. Indicator 4.3 - High Effect Size Strategies: Instructional personnel receive recurring feedback on their proficiency on high effect size instructional strategies. Indicator 4.5 - Facilitating and Leading Professional Learning: The leader manages the organization, operations, and facilities to provide the faculty with quality resources and time for professional learning and promotes, participates in, and engages faculty in effective individual and collaborative learning on priority professional goals throughout the school year.


Become keenly aware of the barriers to proficient student writing

Develop an awareness of the instructional practices that appear to most help them overcome those barriers as well as those don’t have a positive relationship to improvements in student writing

Whether or not the selected leadership strategy was having the desired impact or not




1. The percent of staff implementing at the “Proficient” or higher level differentiated writing instruction monthly 2. The percent of students scoring at the “Proficient” or higher level on a writing assessment monthly

Table 8: Completed Deliberate Practice Growth Target Template

A sample of how a school leader might depict their efforts to improve student achievement by leveraging a handful of “sharply defined elements” of leadership performance is illustrated in Table 9. Notice that this fictional illustration, both the percent of teachers implementing differentiated writing instruction (i.e., the leadership strategy) monthly (teacher self-assessment data only) is compared to student performance on a monthly writing assessment and scored using a common writing rubric. Graphing adult cause data (what the adults, teachers and leaders, in the school do that strongly relates to improved student effects) and its impact on student effect data (results of student performance in writing) is useful in several ways. First, the graph serves as a constant reminder to school leader of the degree to which their most important leadership practices (those identified within the “funneling” process) are

14

having an impact on their core business—teaching and learning. Second, publicly reviewing this data with faculty, perhaps during a monthly faculty meeting, is an excellent way to celebrate “short-term wins” and discuss unplanned losses (when and if the identified strategy may not be working as it was intended). Third, by graphing the relationship of adult actions to student achievement, school leaders can promote the collective efficacy of their staff. And, a graphic display of this nature can be used with supervisors during formative evaluation settings as they spawn numerous constructive conversations about teaching, learning, and leadership. For instance, graphs of this nature might cause either the school leader or her supervisor to ask such questions as:

What might be some of the “key learnings” about instructional practices, which seem to have the greatest impact on student ________ (insert the content area)? Which do not seem to have a positive impact?

What are some of the barriers to improved student achievement in ________ (insert the content area)?

What are some of the adjustments in leadership practice you have to make as a result of supporting teacher growth in ___________ (insert the content area)?

What additional support might the supervisor provide the school leader based on the leader’s Deliberate Practice results?

What percentage of your staff is “Proficiently” implementing the identified high-effect size instructional practices? What support strategies are you providing staff who are not yet implementing essential instructional practices proficiently?

15

Table 9: Sample Monitoring Graph Depicting Monthly Leadership Strategy and Student Assessment Results

September October November January February March April May

Leadership Strategy#1 5% 20% 40% 45% 52% 70% 73% 85%

Student Assessment #1 30% 40% 48% 54% 65% 69% 72% 82%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Perc

en

t P

rofi

cie

nt

Frequency of Monitoring Progress Points

Leadership Strategy vs Student Achievment

16

Our research suggests that the degree of implementation of initiatives, including the Deliberate Practice Growth Target as an initiative, by teachers and school leaders is a

powerful variable that is often independent of the brand name of the program. Great programs poorly implemented have little opportunity for impact. Great programs layered on top of one another with insufficient time and attention also have little opportunity for

impact. Consequently, frequent monitoring and measuring by leaders as to the degree to which initiatives are implemented constitutes an essential leadership practice. Toward that end, organizations must create an implementation rubric that reflects the

degree to which there is evidence that the Deliberate Practice is taking place at the school level.

An implementation rubric is a means of describing what Deliberate Practice “looks like” (both at the developmental and implementation levels) when fully implemented (Hall & Hord, 2006). The following Deliberate Practice Development and Implementation Rubric (see Figure 3) paints a series of “word pictures” of the school leader behaviors and practices and also describes the behaviors and practices as school leaders move from the “Unsatisfactory” variation toward the “Highly Effective” variation.

What Is the Purpose of An Implementation Rubric?

An implementation rubric helps everyone (e.g., school leaders, central office staff, etc.) involved in implementing the initiative develop a common understanding of where they are headed or having the end in mind. Having the “end in mind” helps educators focus their efforts and avoid the problem of everyone doing things differently under the name of the initiative. Additionally, an implementation rubric helps individuals and organizations figure out where they are and what they need to do to move toward full implementation. For example, individual school leaders as well as central office leadership can use the rubric to determine progress in developing and implementing Deliberate Practice and decide where to target specific training that will be most effective in moving school leaders toward the desired state.

How Will We Use the Implementation Rubric?

School leaders should use these two documents to determine what variation (e.g., Highly Effective, Effective, etc.) within either the development or the implementation of Deliberate Practice predominates throughout the district. School leaders’ reflections should be as candid as possible when determining their level of performance; this is not a rating tool, but a tool of self-reflection that will help school leaders determine where they are in the development and implementation process.

As a measurement tool, central office leaders may elect to have school leaders self-assess in the Implementation Rubric monthly. By summarizing the data they collect (e.g., the specific variations, which predominate in their school leaders) central office leaders can model for their staff how to use data to inform behaviors and practice.

17

Deliberate Practice (DP) DEVELOPMENT Rubric Deliberate Practice Priorities: The leader and the evaluator identify 2 to 4 specific and measurable priority learning goals related to teaching, learning, or school leadership practices that impact student learning growth. One or two targets are recommended.

Highly Effective Effective Needs Improvement Unsatisfactory

All criteria for the proficient category have been successfully met. In addition:

The leader…

Assists other leaders in helping them develop a proficient DP

Uses the concepts contained within the DP template to influence other school improvement planning documents

Frequently shares what they are learning (both successes and failures) about the effective development of DP plans with other schools, departments, or districts to maximize the impact of the leader’s personal learning experience

The leader…

Establishes a Focus Issue for their personal growth that aligns with a SMART (e.g., specific, measurable, ambitious, relevant, and time bound) Goal within their School Improvement Plan

Identifies a few (2-5) sharply defined elements of their performance, FSLA indicators, that offer the greatest opportunity to leverage their knowledge based on research, and skills (based on research) in multiple leadership performance areas that are designed to improve student achievement as Growth Targets

Captures multiple (2-5) key learning intentions or Anticipated Gains both student achievement and adult practices that would evidence themselves as a result of implementing their Plan of Action

The leader…

Establishes a Focus Issue that is either unrelated to one of the school’s SMART Goals or attempts to focus on more than one SMART Goal within the school improvement plan

Identifies a few (2-5) FSLA indicators as Growth Targets however, the indicators selected may not be in areas the research supports as high-effect size leadership practices

Labels a few key learning intentions or Anticipated Gains but they are limited to student achievement gains only

Constructs a Plan of Action statement that is not observable and/or measurable. Additionally, the statements more resemble a list of activities to be done than strategies to be measured

The leader…

Is unable to produce a completed draft of their DP or several elements of the DP template are left blank and/or

Selects a Focus Issue that is not connected to critical targets identified within the school improvement plan

Identifies more than five FSLA indicators as Growth Targets and the indicators selected are not in areas the research supports as high-effect size leadership practices

Labels a number of learning intentions or Anticipated Gains and they are all limited to student achievement gains only

Develops a Plan of Action however the statements are simply a list of tasks to be done and while they are observable, they defy measurement in any meaningful way

Figure 3: Deliberate Practice Development and Implementation Rubric

18

Deliberate Practice (DP) Development (continued)

Practice DEVELOPMENT Rubric

Highly Effective Effective Needs Improvement Unsatisfactory The leader… (continued)

Constructs one to two observable, subject to frequent public testing, and measurable (e.g., quantifiable; able to be gauged) leadership strategies as their Plan of Action. These strategies are an obvious translation (i.e., paraphrase) of the three to five prioritized “unpacked” FSLA Indicators. These strategies are constructed with formative language (e.g., increase the percent of, reduce the percentage of, increase the amount or number of, etc.) and are Time-bound (e.g., they indicate how often the public testing will occur).

Pinpoints both a leadership practice and a student achievement performance area that they intend to monitor, measure, and use to determine next steps (e.g., instructional and leadership) monthly as Progress Points

The leader…

Names Progress Points that tend to be tied to large scale student assessments (e.g., FCAT) rather than locally developed assessments that can be measured, monitored and used by the staff frequently to make adjustments to their practices (instructional and/or leadership)

The leader…

Names Progress Points that are tied to large scale student assessments (e.g., FCAT) rather than locally developed assessments that can be measured and monitored by the staff frequently

19

Deliberate Practice (DP) IMPLEMENTATION Rubric

Highly Effective Effective Needs Improvement Unsatisfactory All criteria for the proficient category have been successfully met. In addition:

The leader…

Shares the results of their action research with faculty, what they are learning, and how that learning will influence leadership practices in the future

Publicly reports, including plans and oral presentations, a frank acknowledgement of prior personal and organizational failures, and clear suggestions for system-wide learning resulting from those lessons

This leader regularly shares the results of their action research along with some of the things they are learning about leadership practices and the connection to student achievement with other schools, departments, or districts to maximize the impact of the leader’s personal learning experience

The leader…

Demonstrates a positive relationship between the adult and the student Progress Points

Produces clear and consistent evidence that they are monitoring and measuring both the leadership strategy or strategies as well as the impact on student achievement monthly

Documents the changes in leadership practice that is occurring monthly as a result of the monitoring

Publicly displays the graphic depiction of the degree to which the achieved leadership strategies-in-action compare to the impact on student achievement

The leader…

Provides evidence some of the Progress Points (student data) were monitored but not adult (cause) data

Produces evidence that they are monitoring and measuring student effect data, but are inconsistent in monitoring and measuring leadership data. Consequently it is difficult to determine the degree to which the specified leadership practices are impacting student achievement

Participates in the action research process, and limited evidence of changes based on data

Has not yet created a graphic display of their action research

The leader…

Demonstrates no significant effort to work on the targets.

Demonstrates an indifference to data, no changes in leadership practice compared to the previous year. The data screams “Change!” and the leader’s actions say, “Everything is fine.”