Students Track Their Own Learning with SAS® Data …support.sas.com/publishing/authors/extras/kosturko_data_notebook.pdf · Students Track Their Own Learning with SAS® Data ...

Students Track Their Own Learning with SAS® Data Notebook

Lucy Kosturko SAS Institute, Inc.

[email protected]

Jennifer Sabourin SAS Institute, Inc.

[email protected]

Scott McQuiggan SAS Institute, Inc.

[email protected]

Keywords: Self-regulated learning, Mobile learning, Digital notebooks, Metacognition,

Motivation, Learning strategies, Communication

Abstract

The adoption of the Common Core State Standards for many schools implies greater academic rigor on

behalf of the students and an expectation of modernized, thought-provoking instructional methods

from teachers. Ultimately, students are expected to not only master academic content, but also

demonstrate proficiency for skills associated with academic independence and self-regulated learning.

Extensive research has shown that these behaviors are associated with greater academic motivation and

success (Zimmerman, 1990). One successful method for developing self-regulatory behaviors is

encouraging students to set explicit, measureable goals, use quantitative and qualitative means to track

progress, and utilize data-supported evidence to reflect on academic achievements and refine previous

goals (Zimmerman, 2008). Therefore, the purpose of this research is to investigate the utility of a Digital

Data Notebook—an aggregated digital space for setting, tracking, and reflecting on academic goals—for

improving student’s self-regulatory skills. Similar non-digital tools have been successfully implemented

in the classroom (Bryne & Schaefer, 2006) and research-based evidence supports the use of resources

targeting specific, individual facets of self-regulated learning (Zimmerman, 2008). However, to date little

empirical work has been devoted to analyzing the efficacy of digital tools for the regulation of learning

goals. Consequently, this work will explore the use of a Digital Data Notebook for fostering student self-

regulatory behaviors through direct modelling and capitalizing on the affordances of technology for data

tracking and visualization.

Theoretical Framework

The ultimate mission of the Common Core State Standards Initiative is to ensure all students leave high

school with the necessary skillset to thrive in both career and college settings (Common Core State

Standards, 2010). Therefore, the Standards challenge students to not only build and master academic

content but also become independent, high-level thinkers. Students should leave high school equipped

with the cognitive and metacognitive abilities required of the lifelong learner. In short, today’s students

should embody the characteristics of a self-regulated learner.

Self-regulated learning is a term used to describe the behaviors of students who actively control their

own learning by “[sustaining] cognitions, behaviors, and affects that are systematically directed toward

the attainment of goals” (Schunk, 2008, p. 245; Schunk & Zimmerman, 2003; Winne & Hadwin, 1998;

Pintrich, 2000). Self-regulated learners are not only equipped with a sufficient set of learning strategies,

but also have the motivational control to put forth the necessary effort to engage in these cognitive

processes (Pintrich 2000). Therefore, self-regulation is often represented by three components:

metacognitive awareness, strategy use, and motivation (Zimmerman, 2000). While students often show

great variance in their self-regulatory behaviors (Ellis & Zimmerman, 2001), proficiency of such skills has

shown to be influenced and improved through direct instruction and modelling (Zimmerman, 2002).

Furthermore, there is evidence that students who are better able to regulate their learning in an

intentional and reflective way often demonstrate greater academic motivation and achievement

(Zimmerman, 1990).

Although encouraging self-regulation is critical for the Common Core era student, the new curriculum is

intentionally ambiguous in techniques for increasing these behaviors. According to official

documentation, “The Standards do not mandate…the full range of metacognitive strategies that

students may need to monitor and direct their thinking and learning. Teachers are thus free to provide

students with whatever tools and knowledge their professional judgment and experience identify as

most helpful for meeting the goals set out in the Standards” (Common Core State Standards, 2010).

With respect to developing self-regulatory skills, research-based best-practices suggest teachers should

turn to contextualized methods that require students to explicitly engage in behaviors such as setting

specific, measureable goals, diligently monitoring progress toward such goals, engaging in strategies

that encourage metacognitive awareness, and reflecting on previous actions (Zimmerman, 1990; Schunk

& Swartz, 1993; Schunk & Zimmerman, 2003; Schunk, 1990).

One method for modelling and developing students’ self-regulatory behaviors proposed by the Baldrige

Education Criteria for Performance Excellence is through the use of a student data tracking system or

data notebook (Byrne & Schaefer, 2006; Baldrige, 2013). Data notebooks provide a single aggregated

space where students can set their own measureable goals, evaluate and reflect on their progress, and

refine their goals accordingly. Empirical studies have shown significant differences in academic

motivation between students who set and track their own goals and students who are assigned goals

(Zimmerman, 2008). Furthermore, as reported by Byrne and Schaefer (2006) several qualitative reports

such as “it is very empowering for students, giving them ownership of their education” support the use

of data notebooks in the classroom.

Beyond self-regulatory behaviors, the Common Core State Standards emphasize the use of data and

evidence as a necessary component for effective communication at all grade levels. Specifically, anchor

standard CCSS.ELA-Literacy.CCRA.SL.4 and anchor standard CCSS.ELA-Literacy.CCRA.Sl.5 respectively

state that students should “present information, findings, and supporting evidence such that listeners

can follow the line of reasoning and the organization, development, and style are appropriate to task,

purpose, and audience” and “make strategic use of digital media and visual displays of data to express

information and enhance understanding of presentations” (Common Core State Standards, 2010). With

respect to reflecting upon and detailing one’s academic progress, explicit goal setting and monitoring

provides an excellent platform for utilizing data and evidence for communication.

Therefore, the present research sought to provide preliminary findings for the following research

questions: 1) How do teachers perceive Digital Data Notebooks affect students’ self-regulatory behaviors

and academic achievement?, and 2) How are students using Digital Data Notebooks as a data point

when communicating educational progress?

Research Method

Participants

The SAS Data Notebook app was developed in partnership with teachers from a large socioeconomically

diverse elementary school in the Midwest (644 students, 61% free/reduced-price lunch); therefore,

participants for this study were recruited from the partner elementary school, as well as a blanket call

for participation extended to other teachers using SAS Data Notebook in their classroom via social

media outlets.

A total of eight participants were included in the study ranging in teaching experience, educational

technology experience, and level of comfort with technology in the classroom (see Figure 1).

Participants also represent a range of grade levels from kindergarten to 8th grade. All participants had

experience using a non-digital form for data notebooks in their classroom prior to implementing SAS

Data Notebook.

Figure 1. Participant teaching and technology experience and level of comfort with using technology in the

classroom.

Materials

Teacher Demographic Inventory is a 7-item survey to gather information about participants’ teaching

experience, technology use, and familiarity with traditional data notebooks (Appendix A).

SAS Data Notebook (Figure 2) is a free tool that brings data notebooks from the traditional 3-ring binder

to the iPad. Data Notebook provides tools for students to set personal goals, monitor their own learning,

reflect on previous work, and communicate their progress to teachers and parents using real data.

Within the notebook, students can use the built-in templates to create mission statements, set goals,

generate checklists, reflect using plus/deltas, create and practice spelling lists, and plot histograms. Data

Notebook also includes a blank page and scratch paper template that enables students to load pictures,

drawings, and more into their notebook. Students can also add sections for additional organization.

Thus, students’ goals, daily progress, and achievements are aggregated in one place for easy access

allowing for reflection and communication with parents and teachers.

0

1

2

3

4

5

P1 (10+) P2 (6-10) P3 (10+) P4 (10+) P5 (6-10) P6 (10+) P7 (6-10) P8 (3-5)

Rat

ing

Participant (Teaching experience in years)

Technology Experience Technology Comfort

Figure 2. SAS Data Notebook.

Teacher Reflection Survey is a 15-item, researcher-constructed metric designed to collect use,

satisfaction, efficacy, communication and observational data regarding the use of SAS Data Notebooks

over the course of the 2013-2014 school year. Questions were answered using a 5-point Likert-scale,

and open-ended reflections in order to extend and understand the effect of using the Data Notebook

app had on students’ self-regulatory behaviors (Appendix A).

Procedure

Over the course of the 2013-2014 school year, teachers and students utilized the SAS Data Notebook

tool as part of their regular coursework. Depending on how the teacher decided to implement Data

Notebook, students updated their notebooks by collecting and archiving data, revising goals, and

reflecting on performance on a daily or weekly basis. Notebooks were also reportedly used during

parent-teacher conferences as a data point for student reviews. Although all teachers at the target

school were required by administration to implement Data Notebook in their classroom, only those who

choose to participate in the research study were asked to complete the Teacher Demographic Inventory

and Teacher Reflection Survey. Participants were asked to complete the surveys in the spring of 2014.

Results

Using a mixed-methods exploratory design, quantitative ratings of overall satisfaction, student self-

regulatory behaviors, and communication with parents were triangulated with qualitative reports to

further validate the data.

How do teachers perceive Digital Data Notebooks affect students’ self-regulatory behaviors and

academic achievement?

To understand teacher’s perceptions of self-regulatory behaviors, participants were asked to specifically

reflect on the components of self-regulated learning: metacognition, strategy use, and motivation

(Zimmerman, 2000). As reflected in Figure 3, on average, participants felt SAS Data Notebook did have a

positive effect on their students’ metacognition (M = 3, SD = 1.18), strategy use (M = 3.63, SD = 1.44),

and motivation (M = 3.63, SD = 1.40). In line with these findings, participants commented that their

students “are eager to enter their data and see their scores on the graphs” and “like to track their data

and see progress.” With regard to metacognition, one teacher reported that having “successful data to

support it” was useful when “students are trying to meet their weekly and daily goals.” Finally, teachers

felt that using Data Notebooks was beneficial to overall academic performance (M = 3.75, SD = 1.38).

Figure 3. Participants’ perceptions of SAS Data Notebook’s effect on students’ self-regulatory behaviors and

academic performance.

How are students using Digital Data Notebooks as a data point when communicating educational

progress?

Similar results were found for using SAS Data Notebook as a means to communicate educational

progress (Figure 4). On average, participants felt the app was beneficial for aiding their communication

with parents and students (M = 3.38, SD = 1.68). One noted that “Keeping track of how each child does

is helpful when I meet with each student. During our conference time, we use their data from SAS to

discuss weekly and daily goals.” Another stated, “During conferences, I shared the students’ SAS work

with their parents. Also, I use their data to share with parents during the school year.” Furthermore,

participants felt the app helped students communicate with their parents about their academic

performance (M = 3.89, SD = 1.33). One teacher pointed to the portability of the iPad as particularly

useful for communication stating that “It's nice to have the data for each student right there on their

iPad. It can also be shared with parents daily when the child takes his/her iPad home each night.”

0

1

2

3

4

5

P1 P2 P3 P4 P5 P6 P7 P8

Rat

ing

Participant

Motivation Strategy Use Metacognition Academic Performance

Figure 4. Participants’ perceptions of SAS Data Notebook’s effect on teacher-student/parent

communication and student-parent communication.

Implications

The educational rigor mandated by the Common Core State Standards challenges students to go beyond

what has been traditionally required by state curricula. Furthermore, as students approach college and

career readiness, they are expected to demonstrate academic independence through both cognitive and

metacognitive actions and embody the characteristics of a self-regulated learner. Such qualities (e.g.,

motivational control, metacognitive awareness, and effective learning strategy implementation) are

beneficial for thriving in the Common Core era.

While self-regulatory behaviors are believed to be acquired over time through guidance and practice,

research suggests students, even those of the same age, demonstrate a wide range of proficiency (Ellis

& Zimmerman, 2003). Such findings motivate the need for more explicit, contextualized self-regulated

learning instruction. Flexible tools, such as SAS Data Notebook, provide a great resources for

encouraging students to take control of their own learning across the content areas as reflecting in the

findings from this study. Overall, integrating the SAS Data Notebook app was perceived to have a

positive effect on students’ self-regulatory behaviors and overall academic achievement. Furthermore,

students’ Data Notebooks were posited as a useful tool for communicating educational progress. In fact,

only one of the participants indicated she does not plan to integrate SAS Data Notebook in her

classroom instruction next year.

However, it should be noted, implementing programs like SAS Data Notebook must be done with care.

Students’ self-regulatory skills are developed overtime and proficiency often requires direct instruction

and modelling (Zimmerman, 2002). Therefore, instruction around the use of these tools and the

motivation for setting goals, monitoring progress, and reflecting on results is critical. In fact, two of the

eight participants in this study felt SAS Data Notebook did not improve their students’ self-regulatory

skills; however, these teachers pointed to conceptual misunderstanding as a potential problem. One

noted that she, “will spend more time in August emphasizing how to stay organized and updated. This

0

1

2

3

4

5

P1 P2 P3 P4 P5 P6 P7 P8

Rat

ing

Participant

Teacher-Student/Parent Student-Parent

will alleviate issues that I have encountered this year.” The other said he will, “have more upfront

training for students to be more independent with their Data Notebook.”

Finally, the results should be generalized with care as there are several, notable limitations. First, the

sample used for this investigation was very small and does not necessarily represent the larger

population of classrooms utilizing the SAS Data Notebook app. Furthermore, the degree to which Data

Notebook was integrated over the course of the 2013-2014 school year was not controlled nor was the

manner in which students used the app; therefore, direct comparisons between the participants should

be made with caution. Moreover, it is important to consider students’ self-regulatory skills prior to using

the Data Notebook app in order to better understand the direct effect of using the tool throughout the

school year. Lastly, the measures used for this study were researcher-constructed and self-report in

nature leaving room for measurement error. More rigorous investigations with more vigilant

experimental control, sufficient power for more sophisticated statistical analyses, and validated

measures for each of the study’s variables are necessary to ensure the validity of these preliminary

findings. Additionally, as self-regulated learning behaviors develop overtime, it would be interesting to

understand the differences in effectiveness of using Data Notebook between grade levels. Nonetheless,

design considerations for SAS Data Notebook are grounded in research-proven principles and the

findings from this exploratory study show promise for supporting the development of students’ self-

regulatory skills.

References

Byrne, S. & Schaefer, C. (2006). The Baldrige Program: Self-Assessment for Continuous Improvement.

Principal, 24-27.

Ellis, D. & Zimmerman, B. J. (2001). In H. J. Hartman (Ed.), Enhancing self-monitoring during self-

regulated learning of speech (pp. 205-228). Dordrecht: Kluwer.

National Institute of Standards and Technology (2013).

Baldrige Criteria for Performance Excellence 2013-2014. Washington, DC. National Governors

Association Center for Best Practices & Council of Chief State School Officers. (2010).

Common Core State Standards for English language arts and literacy in history/social studies, science,

and technical subjects. Washington, DC: Authors.

Pintrich, P. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. Pintrich, &

M. Zeidner (Eds.), Handbook of self-regulation. San Diego, CA: Academic Press.

Schunk, D. H. (2008). Attributions as motivators of self-regulated learning. In D. H. Schunk & B. J.

Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 267-

286). New York: Routledge.

Schunk, D. H. & Schwartz, C. W. (1993). Goals and progress feedback: effects on self-efficacy and writing

achievement. Contemporary Educational Psychology, 18(3), 337-354.

Schunk, D. H. & Zimmerman, B. J. (2003). In W. M. Reynolds & G. E. Miller (Eds.) Self-regulation and

learning (Vol. 7, pp. 59-78). New York: Wiley & Sons.

Winne, P. H. & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A.

C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 279-306). Hillsdale, NJ:

Erlbaum.

Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: an overview. Educational

Psychologist, 25, 3-17.

Zimmerman, B. J. (2002). Becoming a self-regulated learner: an overview. Theory into Practice, 4(2), 64-

70.

Zimmerman, B. J. (2008). Goal setting: A key proactive source of academic self-regulation. In D. H.

Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and

applications (pp. 267-286). New York: Routledge.

Appendix A

Teacher Demographic Inventory

1. What grade(s) do you teach? (Pick all that apply)

Kindergarten

1st grade

2nd grade

3rd grade

4th grade

5th grade

6th grade

7th grade

8th grade

Specialist

Administrator

Tech Coordinator

2. How long have you been teaching?

1-2 years

3-5 years

6-10 years

10+ years

3. Rate your level of experience with using technology in the classroom.

1 - No Experience

2 - Novice

3 - Moderate Experience

4 - A lot of Experience

5 - Expert

4. Rate your comfort level with using technology in the classroom.

Very Uncomfortable

Uncomfortable

Uncomfortable at times

Comfortable

Very Comfortable

5. What technology(ies) do you use in the your classroom? (Pick all that apply)

iPad/iPod/iOS devices

SmartBoard

Andriod mobile devices

Chromebooks

Desktop computers

Laptop Computers

No technology/I am not a classroom teacher

6. On average, I use SAS Data Notebook in my classroom:

Daily

2-3 times per week

Once a week

Never

7. Before using SAS Data Notebook, I used: (Pick all that apply) (Required.)

3-ring binders

other

I did not use data notebooks in my classroom

Teacher Reflection Survey

1. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' motivation.

Disagree

Slightly disagree

Neither agree nor disagree

Slightly agree

Agree

2. Explain.

3. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' study habits/strategies.

Disagree

Slightly disagree

Neither agree not disagree

Slightly agree

Agree

4. Explain.

5. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' metacognition.

Disagree

Slightly disagree

neither agree nor disagree

Slightly agree

Agree

6. Explain.

7. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my students' academic performance.

Disagree

Slightly disagree

Neither agree nor disagree

Slightly agree

Agree

8. Explain.

9. To what extent to you agree with the following statement? I believe SAS Data Notebook improves my communication with parents about students' academic performance.

Disagree

Slightly disagree

Neither agree not disagree

Slightly agree

Agree

10. Explain.

11. To what extent to you agree with the following statement? I believe SAS Data Notebook improves students' communication with their parents about their academic performance

Disagree

Slightly disagree

Neither agree nor disagree

Slightly agree

Agree

12. Explain.

13. I plan to integrate SAS Data Notebook into my classroom again next school year.

No

Yes

Undecided

14. If yes, what will you change for next year? What will you keep the same?

15. Do you have any best practices, lessons learned, or advice for integrating SAS Data Notebook into the classroom?

Submit Your Answ ers