Top Banner
CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech
30

CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Dec 29, 2015

Download

Documents

Muriel Hines
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

CS6604 Spring 2012Notes on

Algorithm Visualization

Clifford A. ShafferDepartment of Computer Science

Virginia Tech

Page 2: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

“State of the Field”

• Hundreds of visualizations are freely available on the Internet

• Studies on the effectiveness of AVs– Many studies show no significant difference– But AVs have been shown to help in some

implementations– One conclusion is that creating/using effective AVs

is possible but not easy

• Many faculty wish to use Avs – but there is not as much use as this would indicate.

Page 3: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

What AVs are Available?

• A collection of links available at http://algoviz.org• Links to over 500 visualizations• Nearly all AVs now written in Java

– Applets vs. applications• Stand-alone vs. collections

Page 4: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Who Makes Them?

• Single authors, one-off implementations (1-5)– 30%

• Small shops, sustained over a few years– Typically a faculty member and a few students– 5-10 visualizations– 10%

• Larger teams, longer term investment– Team built, maybe funded– 25%

• Major Projects– integrated package or shared look-and-feel– 35%

Page 5: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Is There Adequate Coverage?• No

– Sorting, search trees, and linear structures overwhelmingly dominate

– Coverage for more advanced topics is spotty

134

60

4638

29

14 13 12

4

0

20

40

60

80

100

120

140

Sort A

lgor

ithm

s

Searc

h Stru

ctur

es

Linea

r Stru

ctur

es

Graph

Algo

rithm

s

Other

Alg

orith

ms

Other

Dat

a Stru

cture

s

Compr

essio

n Alg

s

Searc

h Algo

rithm

s

Mem

ory M

anag

emen

t

Page 6: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

What Is Their Quality?

• A majority have no pedagogical value– These give the user no understanding of how the data

structure or algorithm works– Will be of little use in the classroom

• We would recommend less than one quarter of what we have seen for any purpose

• Even the better visualizations usually have serious deficiencies– Animation only: Users are passive observers– Tree structure visualizations tend to show what

happens, but not how– Limited interactivity

Page 7: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Is the Field Improving?

• Pros:– A growing body of literature on best practices

to create effective AVs– Community starting to organize (AlgoViz)

• Cons:– Recent projects are no more in tune with

coverage gaps than old projects– No apparent movement in creating

repositories

Page 8: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Is the Field Active?• Appears to be a reduction in “one-off”

development. (Drop in student projects)– Fewer CS students– Less interest in Java– But these trends might reverse

• But steady activity in the larger groups.

Page 9: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

AVs: The Problem• AVs have high faculty and student

favorability ratings

• But most faculty don’t use them much in courses

Page 10: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Informal Survey Results• Warning: Self-selected responders

• Are AVs useful?– Strongly Agree: 12– Agree: 17– Neutral: 1

• A (bare) majority indicated that they used some sort of visualization with class

Page 11: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Survey: Impediments to Use• Lack of knowledge/time to find good

AVs: 13

Page 12: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Survey: Impediments to Use• Lack of knowledge/time to find good

AVs: 13

• Time to make good AVs: 2

• Difficulty integrating in class: 9

• Lack of time within class constraints: 2

• Uncertainty about quality outcomes: 1

• Content not relevant to my classes: 1

Page 13: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Overcoming Impediments

• Reassurance about what AVs are good

• Ideas on how to use AVs

• Reassurance about how a given AV can be used successfully in class

• Ability to connect to developers

Page 14: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

AVs: The Solution is Community• http://algoviz.org/

– Build a community of users/developers– Better disseminate best practices

information

• Project Support– NSF CCLI grant– NSF NSDL grant– Connections to NSDL/Ensemble project

Page 15: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

AlgoViz.org• A collection of links to over 500 AVs

• Annotated bibliography of over 500 research papers

• Forums, field reports

• OpenAlgoViz

Page 16: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Are AVs of Pedagogical Value?

• Instructors generally think so

• Students usually say they “like” them

16

Page 17: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Metastudy: 2002

• Reviewed 24 prior studies on pedagogical effectiveness related to AVs– Generally of an individual system or AV

• Results of 24 studies:– 11 found significant (positive) results– 10 did not find a significant result– 2 entangled prediction with visualization– 1 study found a negative result!

17

Page 18: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Epistemic Fidelity Model

• There is an “objective truth”

• Experts carry a model of this truth in their heads

• For data structures, graphics are especially helpful in representing this model

• Therefore AVs should be especially helpful in transferring this model to students.

18

Page 19: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Cognitive Constructivism

• Individuals construct their own knowledge from subjective experiences

• When they become engaged in learning, they actively construct new understandings from new experiences

• Therefore, passively watching AVs won’t have much effect– Students must become actively engaged– The technology should be a tool for knowledge

construction.19

Page 20: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Classification

• The studies represented a wide range of activities and methods

• Looking deeper, reclassify the independent variables:– Epistemic Fidelity: 10– Cognitive Constructivism: 14– (others too few to measure)

• CC has the highest percentage of positive studies

20

Page 21: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Results

• CC: 71% statistically signficant

• EF: 30% statistically significant

21

Page 22: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

CC Activities

• Construct own input sets

• Make predictions about future states

• Program the algorithm

• Answer questions about the algorithm

• Construct own visualization

22

Page 23: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Level of Effort

• Compared whether the two treatments required similar “cognitive effort” vs. different levels of effort– Equivalent effort: 33% significant– Not equivalent: 71% significant

• Construct algorithm/visualization takes time• Note that just taking time need not correlate to

learning

23

Page 24: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Procedural vs. Conceptual Knowledge

• Procedural only: 67% [10/15]

• Procedural and Conceptual: 67% [2/3]

• Conceptual only: 38% [3/8]

24

Page 25: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Study Measures

• Post-test only: 54%

• Pre- to Post-test difference: 78%– But most of these studies came from one

source

25

Page 26: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Study Conclusions

• How students use AV is more important than what they see

• Pre-test/post-test experiments on procedural knowledge show most improvement

• Technology is effective when it is used for active engagment

26

Page 27: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Bloom’s Taxonomy

• Knowledge (facts)

• Comprehension (of the facts)

• Application (mechanically use the facts)

• Analysis (interpreting the facts)

• Synthesis (using facts at higher level)

• Evaluation (ability to make judgments)

27

Page 28: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Engagement Taxonomy

• Naps Working Group 2002– No viewing– Viewing– Responding– Changing– Constructing– Presenting

• Relates to Bloom’s Taxonomy28

Page 29: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

Extended Engagement Taxonomy

• Myller, et al.– No viewing* (textbook)– Viewing* (video)– Controlled Viewing (slideshow)– Entering Input (Define the input to execute)– Responding* (answer questions)– Changing* (direct manipulation)– Modifying (Modify existing AV)– Constructing* (create the AV)– Presenting* (Teach the material)– Reviewing (Give a review of AV)

29

Page 30: CS6604 Spring 2012 Notes on Algorithm Visualization Clifford A. Shaffer Department of Computer Science Virginia Tech.

2009 EvaluationUrquiza-Fuentes/Velazquez-Iturbide

• Analyzed 33 successful evaluations• Evaluation:

– Usability (half of evaluations – often shallow)

– Learning outcomes (other half)

• Many studies compared Viewing, Changing, or Constructing vs. Non-Viewing

• A few compared Changing or Constructing vs. Viewing

• Learning improvements in 75% of studies30