Data Collection

WORKSHOP # 2

Vision to Know and Do :Helping Educators Use Data Effectively

ASBO International 2005 Annual Meeting

FRIDAY, OCTOBER 21, 2005

Boston, MA

WORKSHOP AGENDA

Learning about CoSN’s Data-driven Decision Making Initiative: Vision to Know and Do

Small Group Exercise

Reporting Out and Reaching Consensus

VISION TO KNOW AND DOwww.3d2know.org

Launched in 2003 by CoSN to help educators use data effectively. This initiative is : • a highly-respected source of up-to-date,

unbiased information • an easy to use mechanism for educating

school leaders• a nationally-recognized framework for sharing

knowledgeSupported by Founding Partners ETS, IBM and SAS with additional support from Co-nect, Dell, Pearson Education, Plato Learning; PowerSchool , SchoolNet, and Texas Instruments. Scholastic Administr@tor is the Media Partner.

DEFINITION

Data-Driven Decision Making

A process of making choices based on appropriate analysis of relevant information

DATA DRIVEN DECISION MAKING PROCESS

Determine your desired outcome Define your business processes Identify data available

VISION TO KNOW AND DO

Released Vision to Know and Do: The Power of Data as a Tool in Educational Decision Making and From Vision to Action: How School Districts Use Data to Improve Performance ,in depth examinations of the issue

Created a rich website, www.3d2know.org Developed a self assessment tool to estimate a district’s

readiness to use data-driven decision models Convened a Congressional Seminars in Washington, DC Issued quarterly newsletter, Vision to Know and Do

newsletter

CURRENT CONTEXT

Data collection, analysis and reporting are critical components of No Child Left Behind (NCLB).

School districts must collect more data, in more detail and disaggregate them.

State-level systems and support are being developed for collecting and integrating student assessment data with demographic information.

NATIONAL EDUCATION TECHNOLOGY PLAN

Toward A New Golden Age in American Education calls upon states, districts and schools to

establish a plan to integrate data systems; use data from both administrative and

instructional systems to understand relationships;

ensure interoperability;  and use assessment results to inform instruction.

NCLB

"Information is the key to holding schools accountable for improved performance every year among every student group…Data is our best management tool.  I often say that what gets measured, gets done.  If we know the contours of the problem, and who is affected, we can put forward a solution.  Teachers can adjust lesson plans.  Administrators can evaluate curricula.  Data can inform decision-making.  Thanks to No Child Left Behind, we're no longer flying blind." 

Secretary of Education Margaret Spellings

MOVING BEYOND THE MANDATE

Current environment is an opportunity to:

use data to transform teaching, learning and administration.

inform decisions about everything from class schedules to textbook reading levels to professional development budgets.

provide a rationale for decisions that parents, teachers, taxpayers, and students can understand.

TEN REASONS TO BRING DATA INTO DECISIONS

1. Assess the current and future needs of students2. Decide what to change3. Determine if goals are being met4. Engage in continuous school improvement5. Identify root causes of problems6. Align instruction to standards.7. Provide personalized instruction.8. Track professional development9. Meet accountability provisions of NCLB10. Keep constituents informed about progress.

DATA AND HURRICANE KATRINA

The Mississippi Student Information System—which stores the past four years of student records—is serving as a backup for districts that have lost crucial equipment and software, and the information they contain.

Mississippi districts receiving evacuated students from other districts in the state are tapping the state’s database for course records, grades, Carnegie units, and special classifications, thus smoothing those students’ transitions into new schools.

Perceived Benefits of Technology (Digital Leadership Divide,CoSN/Grunwald Associates Survey)

71%

71%

70%

68%

67%

61%

60%

52%

51%

41%

74%Helps provide timely data for decisions

Helps school support staff do their jobs moreefficiently

Increases administrator productivity

Helps educators communicate with parents &community

Motivates students

Gives students important life skills

Increases teacher productivity

Helps to address disabled needs (assistivetechnologies)

Helps educators individualize instruction

Promotes / provides academic equity among students

Increases student achievement as measured by tests

DATA-DRIVEN DECISION MAKINGDistricts That Employ A Data-driven Decision Making Process

(Digital Leadership Divide: CoSN/Grunwald Survey)

Large

Medium

Small

Total 78%

89%

76%

62%

17%

9%

18%

27%

6%

6%

11%

3%

0% 25% 50% 75% 100%

Yes

No

Don't know/ unsure

CHANGING THE MINDSET

Why does education data make educators so uncomfortable? To only slightly exaggerate, for the past 150 years data was something a third party required you to gather about yourself so they could embarrass you with it three months later. Or so teachers and administrators believed. Data was something higher-ups would use to catch you doing something wrong.

No longer. As Montgomery County MD Superintendent Jerry Weast elegantly says, in his district, he uses data to catch you doing something right. That’s the good news. The better news is that the idea is catching on.

Vision to Know and DoThe Power of Data as a Tool inEducational Decision Making

Highlights school districts using data analysis systems to improve student outcomes

Identifies factors for successfully integrating data into decision making processes

Calls for more emphasis on data-driven decision making as a way to prepare students with 21st century educational skills

Data Rich Districts

Baldrige winners are examples of school districts moving a step beyond of NCLB requirements by integrating data reporting into a culture of continuous improvement.

Data Rich District:Rural Response to Local Expectations

Name: Chugach School District

Location: Prince William Sound, Alaska

Enrollment: 214 students

Chugach Improvement Process

Create a snapshot of the current status Hold town meetings to shape a shared vision Implement balanced instructional model Write standards in a continuum from pre-

kindergarten through 16 Determine assessments aligned to standards Change reporting process for children, parents,

teachers Phase in with continuous improvement

Name: Community Consolidated School District 15

Location: Palatine, Illinois, northwest of Chicago

Enrollment: 13,000 students

Data Rich District:Integrated Planning

CCSD15 Improvement Process

Set measurable goals and targets Collect data using electronic methods Deliver information to decision makers

(classroom, building, district) Identify clearly levels of performance and

opportunities for improvement

Name: Pearl River School District

Location: Rockland County, New York

Enrollment: 2,467 students

Data Rich District:Students Choose Public Schools

Define district goals, objectives and projects

Collect data using format and informal check points

Check stakeholder satisfaction Analyze promptly and share results Compile analyses annually Evaluate performance and achievement

Pearl River Improvement Process

Data Rich Districts:Lessons Learned

It takes time. It has to start at the top. Progress has to be measurable Business models are starting points. Community outreach is essential. Data –driven decision making can be a

powerful tool in changing student outcomes and promoting continuous improvement.

NEW!!

From Vision to Action: How School Districts Use Data to Improve Performance

for school district leaders and K-12 educators seeking ways to implement a data-driven decision making process

DATA RICH DISTRICTS

From Vision to Action draws from interviews with more than 30 experts around the country

Detailed profiles of Lemon Grove School District (CA), Fulton County Schools (GA) and Cleveland Municipal School District (OH).

Profiles and examples in From Vision to Action provide others with examples and techniques.

DISTRICTS INTERVIEWED

DistrictSchool

s Grades Enrollment

Pearl River School District, NY 5 K-12 2,591

**Lemon Grove School District, CA 8 K-8 *4,588

Palo Alto Unified School District, CA 17 K-12 *10,341

Consolidated Community School District 15, IL 20 K-8 *13,057

Beaufort County School District, SC 26 K-12 18,500

Grossmont Union High School District, CA 18 9-12 *24,447

Plano Independent School District, TX 65 K-12 52,063

**Cleveland Municipal School District, OH 125 K-12 *69,000

**Fulton County Schools, GA 88 K-12 75,188

Montgomery County Public Schools, MD 192 K-12 140,492

Clark County School District, NV 301 K-12 280,600*Indicates 2003-2004 enrollment.

FULTON COUNTY, GA

Getting started: District wide strategic planning process in place

Implementation: Cross functional teams help school site with data use and analysis

Results: All elementary schools met AYP targets in 2003-4

LEMON GROVE

Getting started: ubiquitous access to technology and focus on literacy

Implementation: easy to use reports of multiple measures

Results: 3 out of 4 Title I schools declared high achieving

CLEVELAND MUNICIPAL SCHOOL DISTRICT

Getting started: data warehouse developed with site specialist access

Implementation: Data teams discuss and analyze results to apply interventions

Results: Reduced unexcused absences in one school from 9% to 2%

KEY THEMES

Implementing a successful school district data-driven decision making process requires a collaborative team approach.

The process is continual and cyclical moving from the collection of data , to reporting and analysis and finally to using data for targeted interventions.

Technology tools can be effectively utilized in the process.

TEAM APPROACH TO SUCCESS

design and implementation involves the IT department, curriculum and instruction, assessment ,evaluation, and professional development with oversight by the superintendent.

integration into classroom practice requires the buy-in of teachers, principals and site-based support staff

Data Information=Technology Tools

TECHNOLOGY MATTERS

TECHNOLOGY MATTERS

Technology is constantly changing Information is the result of passing

Data through Technology Tools Technology, by itself, has no value Results are primary goal

ReportsData Warehouse

Reporting and Analysis ServicesTurning data into useful information

TrainingLearning how to use data to make informed decisions.

State and Federal ReportingMeeting reporting compliance

DisseminationSharing data with the community (ie: report cards)

School Interoperability

Framework & IMS

Components of a Data Based Decision Making System

SIS

Finance

Assessment

Instruction

Applications

Personalized Instruction

Source: US Department of Education, 2003.

DATA DRIVEN DECISION MAKING PROCESS

A process of three functional areas: collection, integration and dissemination of

data; analysis and reporting of data; and process and procedures for acting on the

data.

DATA COLLECTION AND INTEGRATION CHECKLIST

Is a data warehouse in place? Are the technical and human support tools in place

to move data from warehouse to schoolhouse? Are systems developed to integrate data into

instruction? Are teachers using assessments to measure

progress? Are curriculum and assessments aligned to

standards?

DATA ANALYSIS CHECKLIST

Do teachers have access to data in an easy-to-use format soon after assessment?

Does the district support the process with analytical tools and trained staff to give decision makers confidence in the data and tools?

Are teachers trained to use item analysis to understand student outcomes and instructional effectiveness?

USING DATA CHECKLIST

Does the district provide the tools and training to interpret and query data?

Have data teams developed a process for identifying, recommending and implementing intervention based on data?

Do district and school-site change agents support teachers and their use of targeted interventions?

PROGRESS IN DATA DRIVEN DECISION MAKING

Majority of districts are making progress in collection of data and most are working towards analysis and reporting.

Challenging task remains providing teachers with proven strategies for targeted interventions in the classroom.

CHALLENGES REMAIN

Lack of training 50% Interoperability 42% Lack of understanding about what to do

with the data 39% Absence of clear priorities 36% Failure to collect data uniformly 35%

FAILURE TO COLLECT DATA UNIFORMLY

Capture data to meet requirements7% no action, 28% early, 20% proficient

Extract data for analysis 12% no action, 35% early, 15% proficient

Process for intervention strategies12% no action, 35% early, 13% proficient

Not just a tech issue: alignment and leadership are essential

*CoSN Self Assessment Tool.

Grunwald Associates Page 0

Barriers to More Effective Data-Driven Decision MakingLack of training and interoperability are the main barriers to more effective data-driven decisions. A lack of understanding in what to do with the data is a barrier that’s particular to large and medium districts.

Barriers to Data-Driven Decision MakingTo what degree does your district consider the following to be barriers to

a more effective data-driven decision making process? (N=353)(Percent Responding 8, 9, or 10 on 10-point scale) Large (A)

School District Size

Significantly higher

Medium (B) Small (C)

50%

42%

39%

36%

35%

31%

24%

24%

22%

Lack of training

Interoperability - systems that are unable toshare/exchange data

Lack of understanding in what to do with thedata

Absence of clear priorities on what datashould be collected

Failure to collect data in a uniform manner

Outdated technology / legacy systems

Low quality data - inaccurate or incomplete

Timing of data collection

User interface is too complicated tounderstand reports

47%

44%

41%

36%

38%

34%

24%

23%

19%

55%

42%

44%

37%

35%

30%

25%

25%

24%

42%

36%

17%

28%

28%

22%

20%

23%

21%

C

C

LESSONS LEARNED

Select your technology team based on expertise and ability to perform

Technologies employed must be based on the business goals and not the other way around

Let the technology team do their job with frequent check points to the business model

ONLINE ASSESSMENT TOOL

Take a simple 10-question assessment is available at 3d2know.cosn.org/assessment/survey.cfm

Receive an immediate response Learn where you are in the process of

becoming a data rich district

NEW: FAQ’S

Visit www.3d2know.org View the full list of FAQ’s Coming Soon: Moderated discussion

around these questions

FOR ADDITIONAL INFORMATION www.3d2know.org

What's New Publications Best Practices Other Resources Subscribe to the

Newsletter

GROUP EXERCISE # 1

Select a problem that you are currently facing in your school or school district

Indicate the types of data you will need to make a decision

Identify how you will proceed Come up with an implementation plan,

including barriers encountered

GROUP EXERCISE #2

Report out your findings Share your experiences Capture the findings ---

We want to present these to a larger audience and post on www.3d2know.org

ABOUT COSN

CoSN Mission To promote the use of information technologies and the Internet to ensure technology has a positive effect on learning by focusing on leadership development, coalition building, advocacy and emerging technologies.

Audience Key technology leaders at the school district, state and national level

CoSN GOALS

Leadership Development: supporting school leaders to ensure technology has a positive effect on learning

Advocacy: creating an effective voice on education technology issues

Coalition Building: building partnerships and collaborative efforts around the use of technology in schools

Emerging Technology: exploring meaningful uses for new technologies in education

COSN LEADERSHIP INITIATIVE

New!! Accessible Technologies for All Students Projecthttp://www.accessibletech4all.org/

increasing achievement and success for all students through the unlimited and effective use of accessible technologies

Taking Total Cost of Ownership (TCO) to the Classroom http://classroomtco.cosn.org/Helping School Leaders Budget More Accurately for Education Technology

Safeguarding the Wired Schoolhousehttp://safewiredschools.cosn.org/Resources to Help School Administrators Understand Internet Safety Strategies

3D: Vision to Know & Dohttp://3d2know.cosn.org/Enabling Educators to Think Strategically about the Use of Data Driven Decision-Making

Cyber Security for the Digital Districthttp://securedistrict.cosn.org/Ensuring Security of School Networks

Irene K. [email protected] President202-861-2676 x112

1710 Rhode Island Ave., NW #900Washington, DC 20036www.cosn.org