Algorithm Visualization Algorithm Visualization Reflections and Future Directions Reflections and Future Directions John Stasko John Stasko Information Interfaces Research Group Information Interfaces Research Group School of Interactive Computing & GVU Center School of Interactive Computing & GVU Center Georgia Tech Georgia Tech [email protected][email protected]http:// http:// www.cc.gatech.edu/~stasko www.cc.gatech.edu/~stasko
52
Embed
Algorithm Visualization · •Two main learning approaches: –Conceptualize algorithm at a high level (English discourse). Students referred to this as being able to describe it
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Algorithm VisualizationAlgorithm VisualizationReflections and Future DirectionsReflections and Future Directions
John StaskoJohn Stasko
Information Interfaces Research GroupInformation Interfaces Research Group
School of Interactive Computing & GVU CenterSchool of Interactive Computing & GVU Center
–– AnimAnim vs. no vs. no animanim, followed by test, followed by test
•• Results: Just having animation doesn’t make Results: Just having animation doesn’t make learning happenlearning happen
•• Reflection: Difficult for student to leverage Reflection: Difficult for student to leverage animation of complex algorithm when they animation of complex algorithm when they don’t understand algorithm and visual mapping don’t understand algorithm and visual mapping yetyet
•• Reflection: The better students benefit more…Reflection: The better students benefit more…
•• Results: Benefits to animation, though Results: Benefits to animation, though
unevenuneven
•• Reflection: Interaction is the keyReflection: Interaction is the key
–– Students who enter their own data sets into Students who enter their own data sets into
the algorithm benefited from animationthe algorithm benefited from animation
Lawrence, Badre, & StaskoVL ‘94
ICER – Sept. ‘07 21
3. Binomial Heap & DFS3. Binomial Heap & DFS
•• Classical experimental design with Classical experimental design with
prediction addedprediction added
–– Is animation like prediction?Is animation like prediction?
–– Does it help the student to anticipate what Does it help the student to anticipate what
will happen next and learn from that?will happen next and learn from that?
•• Results: Mixed findings, some supportResults: Mixed findings, some support
•• Reflection: We need a new methodologyReflection: We need a new methodology
Byrne, Catrambone & StaskoComputers & Ed ‘99
ICER – Sept. ‘07 22
4. Binomial Heaps4. Binomial Heaps
•• Change to HW scenarioChange to HW scenario
–– Provide student with learning objectives, materials, Provide student with learning objectives, materials, and questions up frontand questions up front
–– Give unlimited work timeGive unlimited work time
•• Results: Significant benefit to animationResults: Significant benefit to animation
–– Students worked longerStudents worked longer
•• Reflection: Animations appear to help Reflection: Animations appear to help motivationmotivation
–– Make a complex algorithm less intimidatingMake a complex algorithm less intimidating
–– Students with animation were “happier”Students with animation were “happier”
Kehoe, Stasko & TaylorIJHCS ‘00
ICER – Sept. ‘07 23
5. Samba5. Samba
•• Students built their own animations Students built their own animations
during algorithms classduring algorithms class
–– Performance on final exam on questions Performance on final exam on questions
about those algorithms was excellentabout those algorithms was excellent
–– Anecdotal evidence that this helped Anecdotal evidence that this helped
understandingunderstanding
–– Students spent a lot of time tweaking Students spent a lot of time tweaking
graphicsgraphics
StaskoSIGCSE ‘97
ICER – Sept. ‘07 24
Unreported WorkUnreported Work
•• InIn--class assessment of class assessment of quicksortquicksort
•• Worked with focus groups of studentsWorked with focus groups of students
•• Developed task analysis methodology for Developed task analysis methodology for
ensuring that animations were illustrating ensuring that animations were illustrating
the “right” thingsthe “right” things
ICER – Sept. ‘07 25
Little Victories…Little Victories…From *******@felix.cc.gatech.edu Wed Feb 25 19:59:22 1998Date: Wed, 25 Feb 1998 19:52:17 -0500To: [email protected]: Susan ****** <*******@cc.gatech.edu>Subject: cs3158
I know you always wanted feedback on using algorith m animations...Well, I have another instance where I think it help ed me...
I was taking a sample Computer Science GRE,a question was asked that had five sorting algorith ms as thepossible answers.
I was pondering the question and thinking about the algorithms- some that we've been exposed to for years are easy to remember,almost inherent. But then I came to "shellsort," a nd I stopped fora second, going "aw geez... that was a complicated one..." butthen suddenly I could see that picture of the anima tion in my headand could remember it as clear as a bell.
Thought you'd enjoy it....Susan ******
ICER – Sept. ‘07 26
Let’s Step Back a MomentLet’s Step Back a Moment
•• Some important questions:Some important questions:
–– What does it mean to understand an What does it mean to understand an
algorithm?algorithm?
–– What makes a good algorithm animation?What makes a good algorithm animation?
–– How should algorithm animations be used?How should algorithm animations be used?
–– ……
ICER – Sept. ‘07 27
UnderstandingUnderstanding
•• What does it mean for a student to What does it mean for a student to
understand an algorithm?understand an algorithm?
•• What are we evaluating in these studies?What are we evaluating in these studies?
Ponder
ICER – Sept. ‘07 28
Understanding an Algorithm isUnderstanding an Algorithm is
•• Knowing which tasks, or types of tasks, for which the algorithm Knowing which tasks, or types of tasks, for which the algorithm is suited or appropriateis suited or appropriate
•• Knowing how to “run” the algorithm, knowing its procedures, Knowing how to “run” the algorithm, knowing its procedures, operations, steps and chronologyoperations, steps and chronology
•• Knowing the algorithm’s purpose and what it can accomplishKnowing the algorithm’s purpose and what it can accomplish
•• Analyzing the algorithm’s complexity and efficiencyAnalyzing the algorithm’s complexity and efficiency
•• Having the ability to code or program the algorithmHaving the ability to code or program the algorithm
•• Being able to articulate about the algorithm in one’s own highBeing able to articulate about the algorithm in one’s own high--level (natural language) wordslevel (natural language) words
•• Having the ability to transfer knowledge about it to another Having the ability to transfer knowledge about it to another algorithm or domainalgorithm or domain
•• Knowing where the algorithm fits into the bigger Knowing where the algorithm fits into the bigger taxonomy/methodology of algorithms in generaltaxonomy/methodology of algorithms in general
ICER – Sept. ‘07 29
Typical Student ApproachTypical Student Approach
•• Focus group findingsFocus group findings
•• Two main learning approaches:Two main learning approaches:
–– Conceptualize algorithm at a high level Conceptualize algorithm at a high level (English discourse). Students referred to (English discourse). Students referred to this as being able to describe it in their own this as being able to describe it in their own terms terms (top down)(top down)
–– Start with pseudo code and get good grasp Start with pseudo code and get good grasp of how it works mechanically, including of how it works mechanically, including tracing on data sets tracing on data sets (bottom up)(bottom up)
Newstetter
ICER – Sept. ‘07 30
Learning OrderLearning Order
•• A. Knowing which tasks, or types of tasks, for A. Knowing which tasks, or types of tasks, for which the algorithm is suited or appropriatewhich the algorithm is suited or appropriate
•• B. Knowing how to “run” the algorithm, B. Knowing how to “run” the algorithm, knowing its procedures, operations, steps and knowing its procedures, operations, steps and chronologychronology
•• C. Knowing the algorithm’s purpose and what C. Knowing the algorithm’s purpose and what it can accomplishit can accomplish
•• D. Analyzing the algorithm’s complexity and D. Analyzing the algorithm’s complexity and efficiencyefficiency
•• E. Having the ability to code or program the E. Having the ability to code or program the algorithmalgorithm
•• F. Being able to articulate about the algorithm F. Being able to articulate about the algorithm in one’s own highin one’s own high--level (natural language) level (natural language) wordswords
•• G. Having the ability to transfer knowledge G. Having the ability to transfer knowledge about it to another algorithm or domainabout it to another algorithm or domain
•• H. Knowing where the algorithm fits into the H. Knowing where the algorithm fits into the bigger taxonomy/methodology of algorithms in bigger taxonomy/methodology of algorithms in generalgeneral
•• Perhaps better: For which will algorithm Perhaps better: For which will algorithm
animation have greater learning animation have greater learning
potential?potential?
ICER – Sept. ‘07 32
Assess AA UtilityAssess AA Utility
•• A. Knowing which tasks, or types of tasks, for which the A. Knowing which tasks, or types of tasks, for which the algorithm is suited or appropriatealgorithm is suited or appropriate
•• B. Knowing how to “run” the algorithm, knowing its procedures, B. Knowing how to “run” the algorithm, knowing its procedures, operations, steps and chronologyoperations, steps and chronology
•• C. Knowing the algorithm’s purpose and what it can accomplishC. Knowing the algorithm’s purpose and what it can accomplish
•• D. Analyzing the algorithm’s complexity and efficiencyD. Analyzing the algorithm’s complexity and efficiency
•• E. Having the ability to code or program the algorithmE. Having the ability to code or program the algorithm
•• F. Being able to articulate about the algorithm in one’s own higF. Being able to articulate about the algorithm in one’s own highh--level (natural language) wordslevel (natural language) words
•• G. Having the ability to transfer knowledge about it to another G. Having the ability to transfer knowledge about it to another algorithm or domainalgorithm or domain
•• H. Knowing where the algorithm fits into the bigger H. Knowing where the algorithm fits into the bigger taxonomy/methodology of algorithms in generaltaxonomy/methodology of algorithms in general
Ponder
ICER – Sept. ‘07 33
What Makes a Good Algorithm Animation?What Makes a Good Algorithm Animation?
•• What are the elements of an effective What are the elements of an effective
visualization/animation?visualization/animation?
•• How does one design a visualization that How does one design a visualization that
students can learn from?students can learn from?
Ponder
ICER – Sept. ‘07 34
Good AA DesignGood AA Design
•• Very little work done on this topicVery little work done on this topic
•• Design has largely been a very individual, Design has largely been a very individual,
understanding of misunderstandings, understanding of misunderstandings,
story telling, artistic sense, …story telling, artistic sense, …
ICER – Sept. ‘07 35
Multiple ViewsMultiple Views
•• Can be quite valuable but also Can be quite valuable but also
problematicproblematic
–– We can only look one place at a time (people We can only look one place at a time (people
may not notice the text at the bottom of one may not notice the text at the bottom of one
important view)important view)
ICER – Sept. ‘07 36
HeapsortHeapsort Case StudyCase Study
•• Taught psychology faculty colleague the Taught psychology faculty colleague the
heapsortheapsort algorithm using an existing algorithm using an existing
animationanimation
•• Q & A session uncovered many problems Q & A session uncovered many problems
with the animationwith the animation
–– 20+ changes made20+ changes made
ICER – Sept. ‘07 37
HeapsortHeapsort ChangesChanges
•• Show numbers appear in array view Show numbers appear in array view first (reading them in). Then place first (reading them in). Then place them in the tree view and (somehow) them in the tree view and (somehow) highlight the correspondences. highlight the correspondences.
•• Put better text region somewhere for Put better text region somewhere for telling viewer what is happening. telling viewer what is happening.
•• Use circle with small lines out sides Use circle with small lines out sides (indicating (indicating subtreesubtree) to tell where a heap ) to tell where a heap is being checked.is being checked.
•• Change color of node "dropping down" Change color of node "dropping down" in in heapifyheapify operation. Rather than operation. Rather than putting the "H" there.putting the "H" there.
•• Put some text like "Comparing key Put some text like "Comparing key values" and "Swapping with biggest“ in values" and "Swapping with biggest“ in tree view when that occurs. Similarly, tree view when that occurs. Similarly, put some text in array view to draw put some text in array view to draw viewers attention to swaps there.viewers attention to swaps there.
•• Put up text to tell when valid heap has Put up text to tell when valid heap has been built. been built.
•• Put up text to indicate when root and Put up text to indicate when root and
last leaf are being swapped. last leaf are being swapped.
•• Change color of node when it has been Change color of node when it has been
moved out of tree and is in place (do in moved out of tree and is in place (do in
both views). both views).
•• Take circle away at this later stage of Take circle away at this later stage of
algorithm. algorithm.
•• Change color of node moving down in Change color of node moving down in
heapifyheapify..
•• Pause longer after Pause longer after heapifyheapify finishes. finishes.
•• Put text on both sides of line in array Put text on both sides of line in array
view, indicating regions that are sorted view, indicating regions that are sorted
and unsorted respectively. and unsorted respectively.
•• Take out that "Elevate #" text at top.Take out that "Elevate #" text at top.
•• Move line over in array view to very left Move line over in array view to very left
end.end.
Video Demo
ICER – Sept. ‘07 38
The Learner is Your UserThe Learner is Your User
•• Understanding an algorithm well Understanding an algorithm well
handicaps you in designing an animation handicaps you in designing an animation
of itof it
•• We should learn from HCIWe should learn from HCI
–– Have learners as part of your design teamHave learners as part of your design team
–– Test it with other learnersTest it with other learners
ICER – Sept. ‘07 39
Algorithm Animation’s PlaceAlgorithm Animation’s Place
•• How should we use algorithm animations How should we use algorithm animations
for learner benefit?for learner benefit?
–– What’s the scenario of use?What’s the scenario of use?
–– When?When?
–– Where?Where?
–– How? How?
Ponder
ICER – Sept. ‘07 40
Potential Use ScenariosPotential Use Scenarios
•• In classIn class
–– When and how?When and how?
•• LabLab
•• OfflineOffline
ICER – Sept. ‘07 41
Typical ClassTypical Class
•• A discussion of the motivation for an algorithm and how it mightA discussion of the motivation for an algorithm and how it mightbe used in the real world. Frequently, analogy is used to give be used in the real world. Frequently, analogy is used to give the students a feeling for the practical value of the algorithmthe students a feeling for the practical value of the algorithm
•• A discussion of the high level methodology and general A discussion of the high level methodology and general operations of an algorithm. Here, a very informal description ioperations of an algorithm. Here, a very informal description is s provided, attempting to give the students a "feel" for the provided, attempting to give the students a "feel" for the algorithmalgorithm
•• A discussion of the detailed methodology and operations of an A discussion of the detailed methodology and operations of an algorithm, with a focus on tracing the algorithm in pseudoalgorithm, with a focus on tracing the algorithm in pseudo--code, code, understanding its control flow and memory usageunderstanding its control flow and memory usage
•• A discussion of the algorithm's computational complexity, that iA discussion of the algorithm's computational complexity, that is, s, an analysis of its space and time requirementsan analysis of its space and time requirements
•• A discussion revolving around the presentation of an algorithm A discussion revolving around the presentation of an algorithm animation illustrating the algorithm's methods and operationsanimation illustrating the algorithm's methods and operations
ICER – Sept. ‘07 42
RepresentationRepresentation
•• Note that each lecture activity involves a Note that each lecture activity involves a
different kind of representationdifferent kind of representation
•• Each uses a set of artifacts and Each uses a set of artifacts and
discussion terms from a different discussion terms from a different
domain/languagedomain/language
ICER – Sept. ‘07 43
ComponentsComponents
•• Motivation/ExampleMotivation/Example
•• HighHigh--level level
methodologymethodology
•• Detailed Detailed
methodologymethodology
•• Complexity analysisComplexity analysis
•• Algorithm animationAlgorithm animation
•• RealReal--world things, world things,
artifacts, problemsartifacts, problems
•• English discourseEnglish discourse
•• Pseudo code or Pseudo code or
programming lang.programming lang.
•• Math notationMath notation
•• Graphics and Graphics and
visualizationvisualization
ICER – Sept. ‘07 44
ReflectionsReflections
•• Running the sessions amazingly instructive to Running the sessions amazingly instructive to
meme
–– Often saw just how an animation wasn’t helping a Often saw just how an animation wasn’t helping a
studentstudent
•• Two papers reflect on the evaluation findings of Two papers reflect on the evaluation findings of
many researchersmany researchers
–– “Meta“Meta--EvalEval of AV,” of AV,” HundhausenHundhausen, Douglas, & , Douglas, & StaskoStasko, ,
JVLCJVLC , June ’02, June ’02
–– ““AlgoAlgo VisVis,” ,” StaskoStasko & & HundhausenHundhausen, appears in , appears in CS CS Education Research, Education Research, ’04’04
ICER – Sept. ‘07 45
ResultsResults
•• Findings definitely mixed, but some studies do Findings definitely mixed, but some studies do
show benefitsshow benefits
–– ------> Potential for pedagogical aid is there, but just > Potential for pedagogical aid is there, but just
can’t throw animation at algorithm and expect it to can’t throw animation at algorithm and expect it to
helphelp
–– Interaction & engagement is keyInteraction & engagement is key
•• Blindly watching algorithm animation not really helpfulBlindly watching algorithm animation not really helpful
•• Student must interact with animation and be engagedStudent must interact with animation and be engaged
–– Animations help motivation, can make algorithm less Animations help motivation, can make algorithm less
intimidatingintimidating
ICER – Sept. ‘07 46
Future WorkFuture Work
•• What’s still missing in algorithm What’s still missing in algorithm
visualization research?visualization research?
–– Further evaluation (somewhat)Further evaluation (somewhat)
–– Simpler animation construction (somewhat) Simpler animation construction (somewhat)