8/11/2019 NACSA Authorizer Evaluation Report
1/38
National Association of Charter School Authorizers
June 25, 2013
NACSA Authorizer EvaluationReport
Authorizer
District of Columbia Public Charter SchoolBoard
Board Chair
John H.Skip McKoy
Executive Director
Scott D. Pearson
8/11/2019 NACSA Authorizer Evaluation Report
2/38NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Contents Rating Categories 3
Rating System 3
About the Authorizer 4
Executive Recommendations 5
1. Application Decision-Making 7
2. Performance Management Systems 17
3. Performance-Based Accountability 26
4. Autonomy 31
Sources 37
Biographies 38
8/11/2019 NACSA Authorizer Evaluation Report
3/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Rating CategoriesAuthorization quality is rated in two categories:
Established
Refers to the authorizers practices as set out
on paper whether by policy, protocol, or other
means. It also addresses the way that the
authorizer communicates information about its
practices to relevant stakeholders within the
authorizing agency and to schools. This category
rates the authorizer based on what it plans to
do.
Applied
Refers to the authorizers practices as applied.
This category rates the authorizer based on what
it actually does, in practice.
Within each part of the evaluation, the rating
categories are defined more specifically withrespect to the authorizers responsibilities in that
area.
Rating SystemFor each category (established or applied), the
authorizer receives a rating as follows:
Well-Developed
Commendable in that it meets or exceeds
NACSAs Principles & Standards.
Approaching Well-Developed
Fundamentally sound in that it contains most
aspects of a well-developed practice but requires
one or more material modifications to meet
NACSAs Principles & Standards.
Partially Developed
Incomplete in that it contains some aspects of a
well-developed practice but is missing key
components, is limited in its execution, or
otherwise falls short of satisfying NACSAsPrinciples & Standards.
Minimally Developed
Inadequate in that the authorizer has minimally
undertaken the practice or is carrying it out in a
way that falls far short of satisfying NACSAs
Principles & Standards.
Undeveloped
Wholly inadequate in that the authorizer has not
undertaken the practice at all or is carrying it out
in a way that is not recognizably connected toNACSAs Principles & Standards.
8/11/2019 NACSA Authorizer Evaluation Report
4/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
About the Authorizer
The District of Columbia Public Charter School Board (PCSB) was established in 1996 and
approved its first charter school to open in 1997. The DC Council passed mayoral control
legislation in 2007 that dissolved the DC Board of Education (DCBOE), making PCSB the
sole authorizer in DC. The legislation transferred 18 charter schools then supervised by the
DCBOE to PCSB. Currently PCSB authorizes 57 charter schools with an additional seven
schools in pre-operational status.
PCSBs mission is to provide quality public school options for D.C. students, families, and
communities through: a comprehensive application review process; effective oversight;
meaningful support; and active engagement of its stakeholders.
PCSB is overseen by a seven-member board. Board membership has been relatively
consistent over recent years. All of the current members have served on the board for the
last three years. One position has been held vacant for two years.
The staff currently numbers 31 and is organized by areas of authorizing expertise within the
following departments: executive leadership, school performance, human capital and
strategic initiatives, communications, and finance and operations. Many of the staff at PCSB
are new, as the organization saw significant staff turnover in 2011 and 2012 after long-time
leader, Josephine Baker, stepped down as executive director. The new executive director,
Scott Pearson, shifted the structure of the organization from a school-based model to its
current specialized model.
The 57 operating charter schools under the authority of PCSB are a mix of single-site and
multi-site schools. In recent years PCSB has seen an increase in the number of national
charter management organizations (CMOs) showing interest in opening schools in the
Washington, D.C. area. Almost 20 percent of the operating charter schools have an
alternative focus or serve special student populations (e.g. early childhood education or
adult education).
PCSB regularly evaluated the schools in its portfolio through the Performance Management
Framework (PMF) and categorizes them as Tier 1, Tier 2, or Tier 3. Tier 3 schools may be
eligible for a charter revocation hearing. Since the inception of the PMF, a number of Tier 3
charter schools have voluntarily closed or have not been renewed and the overall
achievement of students in schools authorized by PCSB has increased.
8/11/2019 NACSA Authorizer Evaluation Report
5/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Executive Recommendations
Maintain Practice
The evaluation has identified the following policies and/or practices that the authorizer
should maintain:
Continue utilizing the robust set of best-in-class performance management systems thatestablish high expectations for academic, financial, and organizational performance.
These systems- the Performance Management Framework (PMF) and CHARM scores- are
thoughtful, thorough, and place a premium on school autonomy.
Continue the organizational structure of teams with specialized knowledge of different
aspects of authorizing.
Maintain a strong premium on charter school autonomy, as evidenced in policies and
which was affirmed by school operators.
Continue to operate, especially at the Board of Directors level, free from conflicts of
interest and outside interference.
Prioritize Practice
The evaluation has identified the following changes that should be priorities for addressing
weaknesses or gaps in authorizing practices:
Revise evaluation rubric to have clearly defined standards for approval. Conduct
evaluator training to ensure that those standards are consistently referenced in
evaluation and are the foundation for comments and recommendations for approval.
Clearly document its intervention policies including, but not limited to, defining the
types and triggers of evaluation and consequences for failure to improve.
Clarify how the PMF factors into intervention, revocation, and renewal decisions.
Consider developing policies that would allow it to grant further autonomy to high
performing schools.
Ratings SummaryEstablished Applied
Application Decision-Making Approaching Well-Developed Approaching Well-Developed
Performance Management Systems Well-Developed Approaching Well-Developed
Performance-Based Accountability Well-Developed Approaching Well-Developed
Autonomy Well-Developed Well-Developed
8/11/2019 NACSA Authorizer Evaluation Report
6/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Develop Practice
The evaluation has identified the following ways in which the authorizer should further
develop promising practices:
Compile its various performance measurement tools- PMF, CHARM score, and Equity
Reports- into one document to create a robust picture of school performance from
various angles.
Continue developing alternative PMFs for nonstandard schools (Early ChildhoodEducation, Adult Education, Alternative Schools).
Continue to proactively meet with schools that will be up for renewal within the next
couple of years to clarify accountability goals and get ahead of the process.
8/11/2019 NACSA Authorizer Evaluation Report
7/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Application Decision-Making
Does the authorizer approve applications based on applicants
demonstrated preparation and capacity to open and operate a qualitycharter school?
Established:Approaching Well-Developed
Applied:Approaching Well-Developed
Summary Assessment
Overall, the District of Columbia Public Charter School Board (PCSB) demonstrates sound
authorizing practices for application decision-making. PCSB through its application materials
request information from applicants that is aligned with NACSAsPrinciples and Standards
for Quality Charter School Authorizing. Specifically, applicants must provide information on
the educational program, budget and finances, board development and composition, and
operational capacity. All applications receive a thorough review by internal staff who
demonstrated significant knowledge and capacity to conduct thorough and rigorous
evaluations of the applications. Due to a recent strategic reorganization of the office, PCSB
no longer uses external experts for the application evaluation. While the current staff is
large and knowledgeable, the process does not have a consistent independent, externalperspective on the evaluation. PCSB does, however, consult with experts, including
members of their board, to review applications with a specific focus.
PCSB recently introduced a separate process for experienced operators. This application
process is reserved for school operators who currently run successful schools or networks.
It is part of a strategic shift to encourage the participation of national Charter Management
Organizations (CMOs). The experienced operator process focuses on the track record of
success of the schools or CMOs as well as plans for the proposed charter school including
information on the proposed educational program and development of an independent
board. The evaluation of these applications includes the growth plans of the CMO which
allows PCSB to evaluate the fiscal health and capacity of the organization, overall.
Evaluators have sufficient time to review the application and materials and schools have
ample time between authorization and opening day to plan. It was noted both during
observations and review of materials that the evaluations are not sufficiently rooted in key
criteria or standards. Although the comments on the applications were substantive, they did
not specifically reference how the application did or did not meet the standards for approval.
8/11/2019 NACSA Authorizer Evaluation Report
8/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Recommended Actions
Revise evaluation rubric to have clearly defined standards for approval. Conduct
evaluator training to ensure that those standards are consistently referenced in
evaluations and are the foundation for comments and recommendations for approval.
Consider use of external consultants to provide independent, external analysis of
proposals. An investment has clearly been made in developing PCSB internal staff and
they are capable of conducting most application reviews; however, some reviews may
require content-specific knowledge.
8/11/2019 NACSA Authorizer Evaluation Report
9/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Detailed Analysis
1.1
Application Materialsand Process
The authorizer provides clear
guidance and requirements
regarding application materials
and submission requirements and
runs a clear and well-structured
application process with realistic
timelines.
Established:
Well-Developed
Applied:
Approaching Well-Developed
Analysis
PCSB is very thorough in both setting standards and expectations for
application review as well as communicating them to applicants. The
timeline, which is published in the charter application, sets forward a
detailed calendar for charter applicants that affords sufficient time for
the completion of all required tasks. The process commences in
September with the release of the application guidelines and
concludes with final recommendations approximately eight months
later in May. PCSB makes applicants aware of key dates (e.g.
information sessions and interviews) at the beginning of the process
and adheres to those dates.
To assist applicants (and reviewers) in ensuring that all application
materials have been submitted, PCSB has created a Charter
Application Filing Form. The form provides a checklist for all required
sections and attachments and allows applicants to submit missing
documents in a short but reasonable timeframe.
After reviewing samples of completed evaluations and observing
Defense Day it is evident that the evaluation standards and rubric
need further development. Charter application evaluations are
conducted almost exclusively by PCSB staff. There is an all hands on
deck approach to reviewing the applications that ensures that staff
are given sufficient time and resources to conduct a thorough
evaluation. The authorizer has a rubric that uses a multi-tiered system
to evaluate the written application. Although reviewers offered
substantive comments, they were not clearly aligned to the rubric or
evaluation criteria.
Better defining the standards for approval, and incorporating those
into the norming session, will produce evaluations and
recommendations that are more consistent across reviewers (in terms
of standards and format) and more focused. All evaluators participate
in an evaluation criteria norming session where the standards for
evaluation are shared along with some other basic information (ex.
different types of assessments). The evaluation criteria are also
shared with applicants and feedback is given to applicants after the
review has been completed.
The evaluation process consists of two in-person interviews for all
applicants: one conducted by staff and the second conducted by the
Board in a public hearing. PCSB also conducts a site visit for
experienced operators. Each of these processes could be further
developed. The documents we reviewed for the interview prep listed a
long set of questions but it was unclear what the purpose was
(clarification, assessing capacity to implement). Similarly, it was not
clear what rubrics or standards are used to assess the site visits to
schools run by experienced applicants.
8/11/2019 NACSA Authorizer Evaluation Report
10/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
1.2Educational Program
The authorizer has thorough
requirements and rigorous
evaluation criteria for the
proposed educational program
including the vision and mission
statements, educational
philosophy, curriculum and
instruction, teaching skills and
experience, calendar and daily
schedule, target population,
enrollment, and plans for
educating students with special
needs.
Established:
Well-Developed
Applied:
Approaching Well-Developed
Analysis
The level of quality is consistently high throughout PCSB charter
application, including the educational program. The section related to
the educational program is robust and requests information from
applicants regarding the proposed schools mission and vision,
educational philosophy, educational needs of target student
populations, academic and operational goals, curriculum, methods of
instruction, standards for achievement and promotion, and how they
will meet the needs of all students.
PCSB, in its review, places significant emphasis on goals and academic
targets. The D.C. charter school law places much weight on a schools
success or failure in meeting the goals and targets outlined in its
charter. In fact, this appears to be the primary criteria for making
high-stakes decisions around renewal and revocation. As such, the
application focuses on the goals a great deal and considerable
guidance is given to applicants about the types of goals they should
be setting.
In practice, tracking and monitoring charter school goals over the
term of the charter, which in D.C. is 15 years, has proved challenging.
Interviews with staff revealed that some goals became irrelevant andwere no longer tracked by the school or the authorizer. Recognizing
this challenge, PCSB is encouraging applicants to use their
Performance Management Framework (PMF) for its academic goal
setting. Though the authorizer cannot mandate this, they have done
significant engagement with the current charter community and have
emphasized it in the application to encourage participation. Adopting
the PMF for academic goals is a good step toward ensuring
consistency and rigor in the schoolsgoals as well as simplifying the
monitoring process.
A review of sample completed evaluation rubrics demonstrated that
reviewers sought a high level of explanation of the connection
between the educational program and goals of the school and its
stated mission. This was evidenced both by specific questionsregarding the use of multiple methodologies for instruction and
curricular materials as well as in the budget section questioning if
the budget would support the trainers required by one of the
programs. PCSBs evaluation of applicants target student population
was also very rigorous, at times identifying demographic assumptions
of the proposed target population that were incorrect and potential
impact that it might have on the proposed program. Overall, reviewer
comments were substantive and demonstrated thorough content
knowledge. They did not, however, reference the stated standards for
evaluation.
8/11/2019 NACSA Authorizer Evaluation Report
11/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
1.3Organizational Plan
The authorizer has thorough
requirements and rigorous
evaluation criteria for the
proposed organizational plan
including the effective governance
and management structures and
systems (including staffing);
founding team members
demonstrating diverse and
necessary capabilities; and
understanding of legal
requirements related to opening
and operating a charter school.
Established:
Well-Developed
Applied:
Approaching Well-Developed
Analysis
The level of quality is consistently high throughout PCSB charter
application, including the organizational plan. The application requires
a very thorough organizational plan that is singularly directed toward
guaranteeing successful opening and ongoing operation of the school.
PCSB requests a sufficient level of detail to ensure that information is
provided for all levels of the organization from the board, to school
leadership, to students.
PCSB requires extensive information related to the governance and
ongoing management of the school. The D.C. charter school law
requires an independent board of trustees to govern each charter
school. Furthermore, each board must be comprised of at least a
majority of D.C. residents and at least two parents of children
attending the school. The application asks for a board development
plan or specific board members if already identified. Questions are
designed to assess whether the board will be sufficiently diverse and
capable to effectively govern the school. The application also requires
submission of board policies and by-laws and PCSB will not grant final
charter authorization until they have been submitted and accepted.
The management or leadership structure of the school is also givensignificant weight. Applicants are asked to identify the key leadership
roles for the school. If candidates have been identified already,
information is requested on the background and experience of the
proposed leader. If key staff have not been identified, PCSB requires a
detailed plan for the recruitment, selection, and hiring of the
candidates. PCSB requests additional information related to operations
including a plan for the recruitment and admission of students to
ensure that the school will be fully enrolled on day one. Applicants
must also identify a facility or propose a plan for the identification of a
facility. A final charter will not be granted until a facility has been
identified and approved.
Similar to previous sections, reviewers provided substantive
comments (specifically about the lottery process including dates,sibling preference, etc.) but the comments were not rooted in the
evaluation criteria.
8/11/2019 NACSA Authorizer Evaluation Report
12/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
1.4Business/Financial Plan
The authorizer has thorough
requirements and rigorous
evaluation criteria for the
proposed business plan including
financial viability of the plan
demonstrated through budget
projections that are aligned with
the proposed educational
program.
Established:
Well-Developed
Applied:
Partially Developed
Analysis
PCSBs business plan requirement is consistent with the overall high
level of quality throughout the charter school application. For all
schools, PCSB requests specific information on anticipated funding
levels and sources (both government and private), anticipated costs,
and contingencies. The application requests information that is
focused on the successful opening of a school (ex. contingencies if
funds are delayed or lower than expected) as well as continued
success in operation (five year budget including goals and objectives).
Additionally, all schools are required to adopt Generally Accepted
Accounting Principles for the maintenance of their financial records.
The process is appropriately differentiated for experienced operators.
The application requests three years of audited financial statements
and most recent internal financial statements for the organization as a
whole. The authorizer should consider adding the requirement for a
forward looking business plan for all CMOs to ensure the
organizations fiscal health will support its projected growth.
The application evaluations that were reviewed did not demonstrate
the same level of scrutiny as many of the other sections. For instance,
in one rubric, the finance section was entirely blank. Although therewere passing references to whether the budget would support the
proposed educational plan of the school, that analysis was not evident
in the review of the finance sections.
8/11/2019 NACSA Authorizer Evaluation Report
13/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
1.5Capacity
The authorizer has thorough
requirements and rigorous criteria
for evaluating the applicants
capacity to implement the school
plan effectively, including but not
limited to a substantive in-person
capacity interview with the
applicant group.
Established:
Partially Developed
Applied:
Approaching Well-Developed
Analysis
The authorizer has a rigorous application process overall, however,
some elements of assessing a founding teams capacity are under-
developed. The application is strong in requesting sufficient
information regarding the founding teams. They are required to
provide biographies or resumes as well as state the teams purpose
for wanting to start a school in the community. Additionally, all
applicants participate in an in-person interview as part of the
application process.
Although all applicants receive an in-person interview, there are no
stated guidelines or criteria for the interviews or expectations around
participation from the founding team members. The rubric offers
reviewers a section to note potential questions for the interview but
the evaluation team did not observe any guidance around the content
or purpose of these questions. The interview preparation document
included a lengthy list of questions but it was not evident how the
questions were organized for the actual interview.
PCSB does conduct some due diligence of applicants, though it
appeared to be ad hoc. During observations ofDefense Dayit was
evident that reviewers had researched some founding teams beyondthe applications they received, but it was not clear that there were
guidelines for doing so. Similarly, site visits are conducted when
possible at campuses of experienced operators, though no rubric was
observed for those visits.
8/11/2019 NACSA Authorizer Evaluation Report
14/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
1.6Priorities andApplication Adaptations
The authorizer adapts the basic
application as necessary based on
identified needs including
specialized applicant types that
are commonly received and/or
desired program types.
Established:
Approaching Well-Developed
Applied:
Approaching Well-Developed
Analysis
The application process does not yet fully reflect the diversity of the
school types offered in D.C. While the staff and evaluators appear to
have the content knowledge and expertise to assess a variety of
proposed school types, the application itself is not modified to address
them. Similarly, the same rubric and evaluation criteria are applied to
all schools, regardless of type. This could pose challenges for the
authorizer in appropriately determining the quality of the proposed
program. D.C.s charter community is incredibly diverse and serves
students of almost all ages from pre-K through adult education. It also
has alternative schools that defy easy classification and serve a wide
range of students from special education to drop-outs. PCSB has also
received applications for blended and virtual models and the
authorizer is actively seeking to add a number of national CMOs to its
mix of home-grown schools.
Recently, PCSB has attempted to modify its application to address the
different needs of certain school types by creating an experienced
operator application. Though some revisions to this application have
been noted elsewhere in this report, the move signifies PCSBs
recognition of the differences in school and operator types.
Staff and stakeholders indicated, in interviews with the evaluation
team, a desire for a process to reflect the needs of the city and
various communities in the application. Currently, the application is
singularly geared toward assessing the objective quality of the
proposed program. Although applicants must demonstrate need for
the school, the evaluation does not thoroughly considerdemand for
the proposed school in one area versus another. PCSB is in the early
stages of working to identify and incorporate these priorities in their
application and evaluation process.
8/11/2019 NACSA Authorizer Evaluation Report
15/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
1.7Decision Alignment
The authorizer makes application
decisions that are informed by
and align with documented
evidence and analysis of the
extent to which the plan satisfies
approval criteria and the extent to
which applicants demonstrate
strong preparation and capacity to
establish and operate a quality
charter school.
Established:
Well-Developed
Applied:
Well-Developed
Analysis
PCSB has a demonstrated track record of basing approval and denial
decisions of new applicants on the quality of the proposal. Through
interviews at all levels (staff, leadership, and the Board) it was
consistently reported that decisions were not influenced by outside
factors and were based solidly on the overall quality of the proposed
school and the recommendations of staff. With the new organizational
structures, staff prepares recommendations with supporting
information and data to the Board for their consideration. Interviews
with staff and board members revealed a high level engagement with
the Board in reviewing the recommendations and supporting
documents as well as an appropriate amount of deference to staff
recommendations.
Over the three year history of application decisions that was provided
and reviewed, the overwhelming majority of Board votes followed staff
recommendations. There were, however, some instances when the
Board deviated from staff recommendations. Although these instances
were rare, staff were probed during interviews and provided
satisfactory explanations.
8/11/2019 NACSA Authorizer Evaluation Report
16/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
1.8Transparency
The authorizer has transparent
processes for both application
evaluation and application
decision-making.
Established:
Well-Developed
Applied:
Well-Developed
Analysis
Overall, PCSB is very thorough in communicating both its process for
new school applications as well as the expectations for approval.
Supporting materials that ensure transparency in the application
process include a timeline, guidance for how to apply, checklists for
applicants, standards for evaluation, its Performance Management
Framework, and training given to evaluators. Much of this information
is included in the application itself as well as on the website.
Additionally, PCSB provides much of this information at the beginning
of the process allowing applicants a complete view of the process and
sufficient time to plan and prepare all portions of the application.
The timeline specifies a date for a public information session at the
start of the process for prospective applicants. Additionally, staff
reported that they are available to assist applicants in the tactical
elements of submitting a charter school application.
Given the Boards solid track record of following staff
recommendations and the clear communication of criteria, applicants
have a satisfactory sense of what it takes to gain approval. This
sentiment was echoed by many in the charter leader round-table that
was held. In the event that an applicant is denied, PCSB staff hostfeed-back sessions with the applicant to detail the areas where the
application failed to meet the standards. Several applicants have
reapplied after an initial denial and received approval suggesting
that the feedback offered is substantive and relevant.
8/11/2019 NACSA Authorizer Evaluation Report
17/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Performance Management Systems
Does the authorizer have effective systems for establishing and monitoringschool performance expectations and for holding schools accountable asnecessary to protect student and public interests?
Established:Well-Developed
Applied:Approaching Well-Developed
Summary Assessment
PCSB has created a robust set of best-in-class performance management systems that
establish high expectations for academic and operational performance. These systems- the
Performance Management Framework (PMF), Equity Reports, and CHARM scores- are
thoughtful, thorough, and place a premium on school autonomy. The authorizer has
demonstrated an unwavering focus on quality and protecting the interest of students and
families. The PMF has given PCSB an effective tool for communicating school performance
against a set of common measures. The PMF has begun to reshape the charter landscape in
Washington, D.C. Since its inception the number of high performing schools has increased
and parents are overwhelmingly choosing to enroll in higher performing schools. This shift
to include a focus on standardized accountability measures has caused tension with some
charter operators and advocates who believe charters should only be held accountable for
mission-specific goals. PCSB has deftly walked the line between increasing accountability
and overstepping its statutory authority and should continue to work with school operators
and advocates to address their concerns.
PCSBs financial monitoring tools, particularly the CHARM score, are truly exceptional. These
tools strive to create a robust and comprehensive picture of each schools financial health
and communicate findings in a clear and straightforward manner. The process of developing
the CHARM score metrics has helped schools better understand expectations for financial
management and has resulted in stronger financial performance in the portfolio. PCSB is
also piloting Equity Reports this year, which report a number of metrics such as attendance,
attrition, and academic results by subgroup to help illuminate how well schools are serving
different groups of students. Information from these reports will inform decisions about the
level of monitoring each school receives. This is another example of PCSB using the power
of setting high standards and shining a light on results to influence school actions to serve
the interests of students.
8/11/2019 NACSA Authorizer Evaluation Report
18/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
The current leadership of PCSB has strengthened the authorizers approach to performance-
based accountability. For example, in recent years PCSB has scaled back and simplified its
intervention policy, as part of a new approach that is both effective and sensitive to school
autonomy. While the authorizer has established certain triggers that automatically move
schools to a revocation hearing, it has taken a more flexible approach to working with
schools that are out of compliance or in need of improvement. While the authorizers current
approach to school intervention is reasonable, it is not clearly enough defined in policy.
PCSB has only recently begun conducting a charter renewal process, beginning in 2011. Thelength of the charter terms and the evolution of school and authorizer practices have caused
some difficulty in the renewal process. Many schools modified or abandoned goals over time
and the authorizer did not always clarify or consistently track performance against charter
agreement goals. The current administration of PCSB has made a painstaking effort to
clearly identify the appropriate accountability goals to include in the renewal process but
some stakeholders report that they are still unclear on what goals they will be evaluated
and the criteria for renewal.
Recommended Actions
Continue to prioritize the development and piloting of PMFs for nonstandard schools
(early childhood education, adult education, alternative schools).
PCSB should clearly document its intervention policies including, but not limited to,defining the types and triggers of evaluation and consequences for failure to
improve.
The authorizer should continue to proactively meet with schools that will be up for
renewal within the next couple of years to clarify accountability goals and get ahead
of the process.
8/11/2019 NACSA Authorizer Evaluation Report
19/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Detailed Analysis
2.1 Contracting
The authorizer has a timely and
efficient process for developing
and executing the charter
agreement.
Established:
Well-Developed
Applied:
Well-Developed
Analysis
PCSB has well-developed systems and processes for establishing
performance contracts with the schools it authorizes. The authorizer
has developed a comprehensive standard charter agreement that is
customized with school-specific information and performance goals.
While the School Reform Act does not require the authorizer toestablish a charter agreement within a set timeframe, PCSB has not
had any documented issues with executing charter agreements in a
timely manner. Relevant pieces of the charter petition are
incorporated into the charter agreement and the authorizer has a clear
policy and process for schools to request charter amendments for
material changes to the charter. The charter amendment policy clearly
defines what constitutes a material change, including mission or
educational philosophy, articles of incorporation or by-laws, grade
levels served, and governance structure. School operators reported
overall satisfaction with the charter amendment policy and how it has
been implemented.
PCSB has established a robust oversight system that allows it to
monitor compliance with the terms of the contract. For example, theauthorizer utilizes industry specific software to track compliance with
regulatory reporting requirements. This has allowed the authorizer to
efficiently and effectively manage a heavy compliance load and work
with schools to resolve compliance issues when they arise. The
authorizer has also reorganized its staff structure to better track
charter compliance by creating the Equity and Fidelity Team that is
specifically tasked to monitor each schools compliance with the terms
of the charter.
8/11/2019 NACSA Authorizer Evaluation Report
20/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
2.2 School Opening
The authorizer ensures that
approved schools are prepared
adequately for opening.
Established:
Well-Developed
Applied:
Well-Developed
Analysis
PCSB has a clear and comprehensive checklist for monitoring new
schools readiness to openand their capacity to serve students with
disabilities. The checklist includes expectations for school enrollment,
staffing, health and safety, facilities, as well as instructional and
special education programs. Authorizer staff meet with new schools at
their site to complete the checklist and document specific items that
need to be in place to successfully open. If a school is unable tosatisfy the requirements they are given additional time and authorizer
staff will visit repeatedly until the checklist is complete.
In practice, many schools have struggled to receive their Certificate Of
Occupancy in a timely manner. Authorizer staff report that this is an
unpredictable process and that PCSB has helped schools obtain their
COOs on several occasions. If a school is unable to obtain their COO,
however, PCSB will not permit them to open.
8/11/2019 NACSA Authorizer Evaluation Report
21/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
2.3 Ongoing Monitoring
The authorizer has an effective
process for monitoring education,
financial, and organizational
performance of the schools it
authorizes.
Established:
Well-Developed
Applied:
Approaching Well-Developed
Analysis
The authorizer has developed a system for ongoing monitoring of
school performance that is effective, efficient, and respectful of
autonomy. PCSB divides monitoring processes into three areas-
academic performance, financial performance, and compliance
monitoring. Together, these monitoring practices allow the authorizer
to stay reasonably informed of each schoolsperformance and to
proactively address areas of concern as they arise.
The Performance Management Framework (PMF) was created in 2009
to facilitate the evaluation of charter school performance against a
common set of academic performance metrics. The authorizer
involved stakeholders in the development of the PMF and even
scrapped an earlier version of the framework due to feedback from
school operators and charter advocates. The PMF establishes three
tiers of academic performance and includes measures on
achievement, growth, and key gateway (i.e. graduation rates) and
leading (attendance, retention) indicators. The authorizer uses data
from the PMF to create an annual School Performance Reports
document that is distributed widely in both print and electronic
formats. The School Performance Reportsare well designed and
communicate a complex set of information in a clear and
straightforward manner. The authorizer has several task forces in
place to develop appropriate measures for nonstandard schools (early
childhood education, adult education, and alternative schools), which
comprise 40 percent of the portfolio. Completion of these metrics
should remain a top priority.
The PMF has had a significant and immediate impact on the quality of
the D.C. charter sector. Over the last two years the number of Tier 1
schools has increased while the number of Tier 3 schools has declined
(several Tier 3 schools have even voluntarily closed). The authorizer
has also made information from the PMF widely available to parents,
who seem to be using it to inform their choices. Demand for Tier 1
schools, as evidenced by student enrollment and wait lists, is
extremely strong while demand for Tier 3 schools is weak.
PCSB has created an exemplary system to monitor charter financial
performance. The centerpiece of PCSBs financial monitoring is the
innovative CHARM score. These scores are based on a sophisticated
set of financial indicators pulled from audited financials and
benchmarked against other PCSB charter schools. CHARM scores that
are outside the norm trigger site visits and additional monitoring by
PCSB. The vast majority of schools with low CHARM scores in the first
year made significant improvements and were moved off the
concerned list by the second year of implementation. The authorizer
has not yet made CHARM scores public but is considering doing so this
year.
The new site visit protocol focuses exclusively on instructional quality
in schools, using a rubric that was designed in consultation with school
operators, moving away from a more wide-ranging and compliance-
based approach in previous iterations. These new rubrics provide
valuable information about both the quality of instruction at the school
and the alignment of the instructional program with the vision of the
school, which is factored into intervention and renewal decisions.
8/11/2019 NACSA Authorizer Evaluation Report
22/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
2.4 SchoolIntervention/Revocation
The authorizer has effective
policies and practices for school
intervention and revocation and
conducts merit-based
interventions, including revocation
where appropriate, in response to
clearly identified deficiencies in
the schools record of educational,
organizational and/or financial
performance.
Established:
Approaching Well-Developed
Applied:
Approaching Well-Developed
Analysis
PCSB has integrated the PMF into its monitoring strategy, using
information from School Performance Reportsto determine
interventions, and schools with three consecutive years in Tier 3
status are automatically brought before the Board for a charter
revocation hearing.
Over the last several years PCSBsapproach to school intervention hasshifted from a hands-on, tiered intervention system to a less
centralized process that emphasizes school autonomy and public
accountability. While the authorizer has established certain triggers
that automatically move schools to a revocation hearing, it has taken
a more flexible approach to working with schools that are out of
compliance or in need of improvement. For example, schools that are
out of regulatory compliance will receive notices about the problem
from the authorizer and will be brought before the Board if the
situation is not remedied in a timely manner. Schools with low
academic performance can be placed on a probationary status
requiring additional monitoring and program improvement planning.
PCSB previously had one comprehensive intervention policy whereas
now interventions are a part of several different policies (the data
submission policy, the special education audit trigger policy, etc.).
While the authorizers current approach to school intervention is
reasonable, it should consider documenting it in a single
comprehensive policy to give schools clarity about what would lead to
an intervention.
PCSB has the authority to approve which auditors work with charter
schools and has used the CHARM process to proactively educate them
to achieve a more uniform quality and standard of audit reports. After
the first year of working with auditors the number of reportable
findings in audits fell by one third, suggesting that PCSBs clear
expectations of financial management are having a positive impact on
charter schools. This is an excellent example of how charter
authorizers can use high expectations to impact the quality of the
charter sector.
8/11/2019 NACSA Authorizer Evaluation Report
23/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
2.5 Renewal
The authorizer runs a well-
structured renewal process
including clear requirements; a
meaningful opportunity for the
school to present information and
respond to the authorizers
findings; clear communication;and prompt notification of
decisions.
Established:
Approaching Well-Developed
Applied:
Approaching Well-Developed
Analysis
PCSB has only recently begun conducting a charter renewal process,
beginning in 2011, because of its 15-year term charter agreements.
The authorizer has created a comprehensive document that details the
renewal process, including specific timelines and criteria in the 2012-
2013 Charter Renewal Guidelines document. This document provides
step-by-step instructions for preparation of the charter renewal
application and encourages schools to adopt the PMF as their goals fortheir next contract. Some school leaders report a reticence to adopt
the PMF as their accountability goals in the contract as it is a living
document and subject to change beyond the schools control. PCSBs
renewal process gives schools the opportunity to request a public
hearing to respond to the cumulative record of performance and make
the case for their renewal.
In practice, the length of the charter terms and the evolution of school
and authorizer practices have caused some difficulty in the renewal
process. Many schools modified or abandoned goals over time and the
authorizer did not always clarify or consistently track performance
against charter agreement goals. The current administration of PCSB
has made a painstaking effort to clearly identify the appropriate
accountability goals to include in the renewal process but some
stakeholders report that they are still unclear on what goals they will
be evaluated and the criteria for renewal. The authorizer should
continue to proactively meet with schools that will be up for renewal
within the next couple of years to clarify accountability goals and get
ahead of the process.
8/11/2019 NACSA Authorizer Evaluation Report
24/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
2.6 Closure
Following non-renewal, revocation
or voluntary return of the charter,
the authorizer has an effective
plan for and ensures orderly
closure of schools.
Established:
Well-Developed
Applied:
Well-Developed
Analysis
The authorizer has a comprehensive school closure policy that clearly
identifies roles and responsibilities for all parties in the orderly closure
of a school. The plan provides for securing of records, securing of
public property, orderly wind up of the schools finances, and
communicating with affected families.
The authorizer works proactively to coordinate closure efforts withboth D.C. Public Schools and the Office of the State Superintendent
and meets on-site with the affected school early in the process to
ensure strong financial controls and accountability. The authorizer has
dedicated significant resources to working with families affected by
school closures to ensure every student is placed in a new school. To
this end, the authorizer hires part-time specialized staff to handle
closures, both at the administrative level and to work directly with
families through the closure process.
8/11/2019 NACSA Authorizer Evaluation Report
25/38
8/11/2019 NACSA Authorizer Evaluation Report
26/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Performance-Based AccountabilityDoes the authorizer have rigorous, appropriate standards by which it holdsschools accountable for results? Are decisions made with the intent tomaintain high standards and protect the students and the publicsinterests?
Established:Well-Developed
Applied:Approaching Well-Developed
Summary Assessment
PCSB establishes very high standards of performance for its schools and uses these
standards to hold schools accountable for results. PCSB has a track record for enforcing its
standards and making difficult decisions regarding charter status and has revoked charters
for academic, financial, and organizational reasons. The authorizer holds schools
accountable for mission-specific goals established in the contract and regulations defined in
state and federal law. PCSB has also developed the PMF to evaluate schools on a set of
common academic and financial performance indicators. While this information is used toguide monitoring and accountability decisions, it is not part of the criteria for charter
renewal or revocation. Many school operators reported confusion over the role of the PMF in
the renewal process. PCSBs renewal decisions also indicate a commitment to upholding
high standards, with all schools recommended for renewal ranking in the top two tiers of the
PMF (with the exception of one alternative school that will be eligible for alternative
measures that are now being piloted).
As an organization that makes high-stakes, data-based decisions, PCSB must be able to rely
on the quality of its data. In order to fulfill its monitoring and oversight requirements PCSB
requires schools to submit large amounts of data on a regular basis. Many school operators
report that the current system is time consuming and often results in data errors. PCSB has
made an effort to address these problems but school operators indicated that they continue
to persist.
Recommended Actions
Clarify how the PMF factors into intervention, revocation, and renewal decisions.
Work with school operators to develop more effective data collection processes.
8/11/2019 NACSA Authorizer Evaluation Report
27/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Detailed Analysis
3.1 EducationalPerformance
The authorizer holds schools
accountable for academic
performance using objective and
verifiable measures that address,
at a minimum, student
achievement, student growth, and
post-secondary success as the
primary measures of school
quality.
Established:
Well-Developed
Applied:
Approaching Well-Developed
Analysis
PCSB establishes high standards of educational performance for all of
the schools it authorizes. These expectations are clearly defined for
each school in the charter agreement and in the PMF. As established
in law, the goals defined in the charter agreement guide renewal andrevocations decisions made by PCSB. The individualized goals of the
charter agreements, however, limit the authorizers ability to assess
the overall and relative level of performance of schools in its portfolio.
The PMF was created to overcome this challenge and articulates a high
bar for performance. This information is used to inform the public
about the performance levels at each school and help guide PCSB
decisions on school monitoring and is not used as the basis for
renewal or revocation decisions, which are driven solely by each
schools ability to prove that it has met the goalsdefined in the
charter agreement.
In practice, the PMF has caused some confusion among charter
operators about accountability because PCSB has integrated
information from the PMF into its monitoring process. PCSB hasinstituted a policy that any school remaining on Tier 3 for three
consecutive years will automatically face a charter revocation hearing.
Though the actual revocation is based upon the schools ability to
prove that it has met the goals established in the charter agreement,
this is not clear to the majority of the stakeholders interviewed for the
evaluation. To alleviate this confusion the authorizer should clarify
exactly how the PMF is used in accountability decisions.
8/11/2019 NACSA Authorizer Evaluation Report
28/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
3.2 Financial Performance
The authorizer holds schools
accountable for financial
performance using appropriate
near term and sustainability
measures as the primary
indicators of a schools financial
viability.
Established:
Well-Developed
Applied:
Well-Developed
Analysis
PCSB uses the CHARM score to effectively monitor the financial
performance of each school. The CHARM score includes a number of
appropriate measures of financial performance, using data from
audited financial statements. These measures include key ratios such
as current assets position, capitalization ratio, and months of cash on
hand. Like the PMF, the CHARM score is not used as renewal or
revocation criteria, rather it guides staff decisions on what schools
need additional monitoring and support. Schools also regularly submit
financial statements as part of their board packets, with schools in
good standing required to submit on a less frequent basis. Charter
schools are required to submit four years of audited financial
statements as part of the renewal application.
All of the information on financial performance factors into staff
recommendations for renewals or revocations. Staff and board
decisions are guided by the fiscal management criteria listed in the
charter agreement as reasons for charter revocation or nonrenewal; a
pattern of non-adherence to generally accepted accounting principles
and no longer being economically viable. PCSB has closed a number of
charter schools for failing to meet these terms of the charter
agreement.
8/11/2019 NACSA Authorizer Evaluation Report
29/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
3.3 OrganizationalPerformance
The authorizer holds schools
accountable for compliance with
organizational performance
requirements established in law or
the charter contract, including
educational programrequirements, governance and
reporting, financial management
and oversight, and operational
requirements related to students,
employees, and the school
environment.
Established:
Partially Developed
Applied:
Partially Developed
Analysis
PCSB has comprehensive systems for monitoring organizational
performance and compliance but the effectiveness of one of these
systems is questionable. Schools are required to upload a variety of
data as part of the organizational performance monitoring process.
While the EpiCenter system for uploading board documents and other
forms received positive reviews from the stakeholders we interviewed,
a number of school operators reported significant problems uploading
their enrollment, attendance, and discipline data using the ProActive
software, in some cases causing erroneous information to be reported,
and requiring time-consuming problem solving and resubmissions.
Authorizer staff acknowledged that there were problems with the
ProActive system in the past, but believed the problems have largely
been resolved. This is incongruent with reports from school operators
that they are still struggling with the system. Given that this data is
used in accountability decisions and is incorporated into the PMF it is
imperative that data integrity be strong beyond reproach.
8/11/2019 NACSA Authorizer Evaluation Report
30/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
3.4 Decision Alignment
The authorizer makes
accountability decisions that are
informed by and align with
documented evidence and
analysis of the extent to which the
school satisfies performance
expectations. The analysis
presented to decision-makers is of
high quality and the merits of the
decisions themselves show
decision-making is based on
thoughtful analysis ensuring that
only the charter schools that meet
or exceed expectations are in
operation. (Note: this section
focuses on decisions by the
authorizer other than the
application, which is addressed in
1.7.)
Established:Well-Developed
Applied:
Approaching Well-Developed
Analysis
The authorizer has a demonstrated track record of making evidence
based decisions regarding charter status. All charters receive
automatic reviews after 5 and 10 years of operation and the
authorizer uses data from the PMF and compliance monitoring to
determine what schools receive additional site visits. PCSB has
created a comprehensive guide for charter renewal that walks school
operators through the process step by step. The guide clearly states
that renewal decisions are based on criteria defined in the charter law-
adherence to the terms charter contract, including evidence that the
school has met the academic performance goals included therein.
PCSB staff review the renewal application and make recommendations
to the Board that include well-reasoned and data rich arguments for or
against renewal. PCSB has considered 11 charter renewal applications
and approved 10. The school that was not renewed was rated a Tier 3
school and had clearly not met the academic outcomes defined in the
charter. Of the 10 schools that have been awarded renewal, nine of
them are rated in Tier 1 or Tier 2- indicating a strong alignment
between renewal decisions and quality indicators. The one Tier 3
school that was renewed is an alternative school that will be included
under an alternative set of PMF measures in the near future. The factthat PCSBs portfolio includes a number of nonstandard schools
highlights that the adoption of alternative PMF measures must remain
a top priority of the Board.
8/11/2019 NACSA Authorizer Evaluation Report
31/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Autonomy
Do schools have the autonomy to which they are entitled?
Established:Well-Developed
Applied:Well-Developed
Summary Assessment
PCSB fundamentally understands its role as authorizer is tasked with affording its schools
the autonomy to which they are entitled while holding them accountable based on the law
and the terms of their contract. In interviews with the executive director and several
members of the Board, we heard that preserving charter school autonomy was among their
chief tasks. Overall, this balance is maintained across PCSBs authorizing practices. The D.C.
charter school law is clear in the autonomies afforded charter schools and the way in which
they may be held accountable. PCSB has tried to ensure that they are able to evolve in the
way that they are holding schools accountable while still acting in accordance with the law.
The chief test to this has been the introduction of the Performance Management Framework.
Though the authorizer went through several iterations and sought charter input and
feedback, they still face resistance to it and charges that it infringes on the schools
autonomy and right to be held accountable to the terms of its contract. PCSB is addressing
this in a way that appears fair and in accordance with the law by using the PMF as a guidewhile ensuring that decisions on current schools are based on the terms of their contract.
Looking forward, they are working to encourage new schools to adopt the PMF as their
academic achievement goals in their charter application and contract.
PCSB employs EpiCenter to manage compliance requirements for the schools. The system
allows them to collect required compliance items from schools in a way that is streamlined
and efficient. Some elements of their ongoing academic monitoring are very specific and can
tend toward a focus on inputs, including a Danielson-based evaluation of teacher practice,
however, we did not hear push back on this from the schools or other stakeholders. In fact,
many of the schools and stakeholder groups reported that PCSB largely respects the
autonomy of schools and that most appreciate some of the more involved feedback fromschool visits and reviews.
Recommended Actions
Continue to move toward output based accountability standards and decisions.
Continue engagement with stakeholders on PMF and ensure that its application is in
compliance with law and respectful of school autonomy.
8/11/2019 NACSA Authorizer Evaluation Report
32/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Detailed Analysis
4.1 Autonomy
The authorizer defines and
respects the autonomies to which
the schools are entitled based on
statute, waiver, or authorizerpolicy. The authorizer does not
reduce school autonomy unless
there is a compelling reason to do
so.
Established:
Well-Developed
Applied:
Well-Developed
Analysis
The Charter Agreement signed by PCSB-authorized schools explicitly
states the autonomies afforded it by the D.C. charter school law.
Specifically, the agreement identifies the laws with which it must
comply as well as stating that all public charter schools are exemptfrom the regulations governing District of Columbia Public Schools.
The monitoring requirements set forth by PCSB are in line with the
autonomies afforded charter schools in D.C.
In addition to the autonomies outlined in the law and charter
agreement PCSB proactively seeks to minimize the compliance burden
on schools. Given the unique structure and status of the District of
Columbia there are numerous entities with jurisdiction over a
relatively small number of schools. PCSB has collaborated chiefly with
the Office of the State Superintendent of Education (OSSE) to improve
the efficiency of the monitoring and data collection. The two agencies
have worked to identify redundant or unnecessary compliance items
and to streamline the processes where possible. For instance,
previously, newly approved charter schools had to submit a lengthyapplication to OSSE for start-up funding. The two agencies have now
merged their processes, making it more efficient for applicants as well
as the agencies.
8/11/2019 NACSA Authorizer Evaluation Report
33/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
4.2 Educational Program
The authorizer defines and
respects school autonomy over
the educational program.
Established:
Well-Developed
Applied:
Well-Developed
Analysis
PCSB appropriately respects a schools ultimate control over its
educational programs. The authorizer evaluates the programs largely
based on outputs and whether it is achieving the goals set forth in its
charter agreement. At this time, PCSB does not identify any priorities
for educational programs (ex. blended learning) at the time of
approval. The ongoing monitoring and evaluation of the schools also
respects the autonomy the schools have to execute their educational
program. In interviews with the executive director he identified some
instances in the past when Program Development Reviews (PDRs)
resulted in findings that were highly subjective and reflected the
biases and opinions of the observer, rather than an objective
evaluation of the quality of the program. He has sought to remedy this
by redesigning the evaluation process and reorganizing the staff.
Some schools, as well as the charter association, asserted that the
introduction of the PMF was encroaching on the autonomy over their
educational programs because it failed to account for mission-specific
goals and uniformly applied the same evaluations to all schools
regardless of the unique features of a particular educational program.
Although the PMF does not specifically include mission-specific
measures, schools are still evaluated by meeting the goals in theircontracts, which include mission-specific goals.
It appears that schools are allowed to make minor changes to their
programs without approval from PCSB; however, more substantial
changes require a review from the authorizer. In those instances
where the change warrants review, it is unclear what the standard for
approval is.
8/11/2019 NACSA Authorizer Evaluation Report
34/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
4.3 Financial Management
The authorizer defines and
respects school autonomy over
financial operations.
Established:
Well-Developed
Applied:
Well-Developed
Analysis
PCSBs financial oversight is informed largely through the collection of
monthly or quarterly financial reports, audited financial statements,
annual budgets, and IRS Form 990s. Additionally, financial data is
evaluated using the authorizers CHARM framework which is geared
toward assessing the financial health of the organization. All of this
information is communicated thoroughly to the charter schools. The
processes used by PCSB for financial oversight are in line with their
authority and preserve the schools autonomy to make budget
decisions that they believe best support the mission and program of
the school. The financial policies also ensure that PCSB can
appropriately hold schools accountable.
Recently PCSB moved away from its GPA system for financial
oversight and introduced a new model called CHARM. PCSB is taking
steps to use the results of CHARM along with the PMF in their
comprehensive evaluations of schools. Finally, interviews with charter
school leaders yielded unanimously positive statements in support of
the financial oversight offered by the CFO and staff.
8/11/2019 NACSA Authorizer Evaluation Report
35/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
4.4 Conflicts of Interest
The authorizer operates free from
conflicts of interest.
Established:
Well-Developed
Applied:Well-Developed
Analysis
PCSB functions as an independent body at every level. Board
members are free of conflicts that could compromise their ability to
act independently on charter decisions. Although members are
appointed by the mayor and (newly) approved by Council, they
demonstrated a clear independence from any influence, perceived or
actual, by these groups.
There was no evidence that there were any structures in place within
the authorizer that created a financial incentive for keeping any
particular school open. There is an authorizer fee as prescribed by
law, though it is minimal.
8/11/2019 NACSA Authorizer Evaluation Report
36/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
4.5 Earned Autonomy
The authorizer periodically
reviews compliance requirements
and evaluates the potential to
increase school autonomy based
on flexibility in the law,
demonstrated school
performance, and other
considerations.
Established:
Partially Developed
Applied:
Approaching Well-Developed
Analysis
Charter schools in D.C. have a high baseline of autonomy granted to
them in their charter agreements. This autonomy is understood and
respected by the authorizer, as evidenced through interviews with
staff, board members, and other stakeholder in addition to evidence in
their authorizing practice. We did not find, however, that there is a
clear or well-defined path to earned or increased autonomy. The law
does not appear to limit the differentiation of autonomy, though it
does not expressly permit it either.
There has been some movement toward earned autonomy with the
advent of the PMF. PCSB has started using a schools Tier Rating on
the PMF to determine the frequency with which it is evaluated, with
higher performing schools receiving less frequent evaluations.
Specifically, there is differentiation in monitoring for schools based on
school performance. For instance, high performing schools receive
QSRs at greater intervals, are afforded preference in enrollment
increase requests, and are required to submit quarterly vs. monthly
financials. Likewise, lower performing schools or schools with flags
identified through the data submission and special education policies
will trigger increased monitoring and site visits. All schools go through
a common process when requesting expansion or renewal.
The authorizer has only recently launched its differentiated monitoring
process, so it was not possible to fully evaluate the details of the plan,
what the standards are, and how evenly they are applied. PCSB
recognizes charter school autonomy has a key element of its work and
seems poised to move further in that direction, where appropriate.
PCSB also has strong will from schools and other stakeholders
encouraging them to move further in the direction of increased
autonomy.
8/11/2019 NACSA Authorizer Evaluation Report
37/38
NACSA Authorizer Evaluation Report: District of Columbia Public Charter School Board
Sources
Application Decision-Making
Charter School Application-2013
Sample Charter School Application
Interview Materials & Schedule
Application Evaluation Process-2012
Review & Evaluation TemplatesThree Year Record of Application Decisions
Monitoring Operations
Charter Agreement-Draft
Charter School Annual Reports (2011-2012)
Charter Amendment Handbook & Policy
Fiscal Policy Handbook
Renewal Guidelines Overview (2012-2013)
Authorizer Monitoring Progress Reports
Compliance Materials
Pre-Opening Site Visit ChecklistQualitative Site Review Protocol-2013
Template of SPED Desk Audit
PCS Audits (2010-2011)
Performance-Based Accountability
Charter School Performance Reports-2012
Performance Management Framework-2011
Case Histories
Charter Applications
Charter AgreementsCharter Amendment Requests (2004; 2006;
2012)
Charter Renewal Applications
Program Development Review Reports
(2001-2011)
Qualitative Site Report-2013
Audits & Annual Reports (2010-2012)
Performance Analysis (2008-2009)
5th Year Charter Reviews
Study Review Report
Background
PCSB School Leader Survey Results
PCSB Self-Evaluation Survey Results
Authorizer Data Summary Template
D.C. Charter School Law
Organizational Chart-2012
Organizational Change Narrative
8/11/2019 NACSA Authorizer Evaluation Report
38/38
Biographies
Justin Testerman leads the implementation efforts of the Tennessee Charter School
Incubator as the chief operating officer, driving strategy and communicating with staff,
fellows, and partners to create an environment conducive to charter school growth and
success in Tennessee. Prior to coming to Nashville, Testerman served as the director of
education programs for Volunteers of America in Minnesota. He has long been involved on
the national charter school scene as a board member of the National Association of Charter
School Authorizers and as a charter school specialist in the Minnesota Department of
Education. Testerman began his career in education as a middle school teacher in Newark,
N.J. through Teach For America.
Carly Bolger is a consultant focused on supporting districts, state departments of education,
and other agencies to design and implement high-quality, strategic authorizing practices.
She has seven years of authorizing experience at both the district and state level. Carly was
part of the School District of Philadelphia's efforts to launch its Renaissance Schools
initiative, a process that selected high-performing charter operators to manage the district's
lowest performing schools. She has also headed the authorizing office for the state of New
Jersey and the Office of New Schools at Chicago Public Schools. Carly received a B.A. in
economics from Miami University and a Master in Government Administration from the
University of Pennsylvania.