DEPTH OF KNOWLEDGE (DOK) What are students being asked to do? How complex is the student thinking?
Jan 23, 2016
DEPTH OF KNOWLEDGE (DOK)What are students being asked to do?
How complex is the student thinking?
Depth of Knowledge (DOK) is
• A scale of cognitive demand (thinking) to align standards with assessments
• Descriptive of the task students are being asked to do
• Based on the research of Norman Webb, University of Wisconsin Center for Education Research and the National Institute for Science Education
Not the same as difficulty
To align instructional activities with the cognitive rigor needed to master the standards
Is used by Smarter Balanced to indicate the complexity of the assessment task/item
Applicable to all subject areas and at all grade levels
Varying Level of Cognitive DemandELA
Retell or summarize Little Red Riding Hood in your own words.
What is your opinion about the intelligence of the wolf (in Little Red Riding Hood)? Justify using details/evidence from the story.
Varying Level of Cognitive Demand
Mathematics
Is 4/5 closer to 1 than 5/4?
4/5 is closer to 1 than 5/4. Show why this is true on a number line.
How are the new assessments changing?
MATHEMATICS ELA / LITERACY
DOK 3 DOK 4 DOK 3 DOK 4
Current Assessments <2% 0% 20% 2%
New SBAC Assessments 49% 21% 43% 25%
Yuan & Le (2012); Herman & Linn (2013), from Linda Darling-Hammond Assembly Testimony, 3.6.13
Smarter Balanced Depth of Knowledge Goals for Items
Sample Item Specification
Sample Item Specification
Write a definition of Cognitive Rigor on your DOK note-taking guide.
Read “The Ant and the Grasshopper”– develop one basic question and one “rigorous” question.
Share your questions with a partner
Rating Depth of Knowledge (DOK)
In your classroom:• What part of your classroom instruction is spent in each
DOK Level?
• Which DOK levels are assessments focused on?
• What lessons / activities engage students in the different DOK Levels?
• What DOK level would you assign your “Ant and The Grasshopper” questions?
Webb’s 4 Levels of Cognitive Complexity
• DOK 1: Recall & Reproduction• DOK 2: Basic Application of Skills/Concepts• DOK 3: Strategic Thinking and Reasoning• DOK 4: Extended Thinking
LEVEL 1: RECALL
• Requires recall of information, such as a fact, definition, term, or simple procedure and/or performing simple procedures.
• Involves only basic initial comprehension, not an analysis or interpretation.
• Students are to work with specific facts or definitions. The activity requires a shallow understanding of the topic or text.
LEVEL 2: Skills and Concepts •Includes the engagement of some mental processing beyond recalling or reproducing a response
•Items require students to make some decisions as to how to approach the question or problem
• Actions imply more than one mental or cognitive process/step
Recall vs. Skills & Concepts
ELA
Describe the physical features of a place.
Identify and summarize the major events, problem, solution, and conflicts in a text.
Recall vs. Skills & Concepts
MATH
How many minutes did it take Selena to do 3 problems?
Selena continues to work at the same rate. How many problems will she do in 40 minutes?
Selena’s
Recall vs. Skills & ConceptsScience
Identify parts of the cell
Distinguish between a plant cell and an animal cell
LEVEL 3: Strategic Thinking
•Requires deep understanding exhibited through planning, using evidence, and more demanding cognitive reasoning
• The cognitive demands are complex and abstract
• An assessment item that has more than one possible answer and requires students to justify the response would most likely be a Level 3
3. Strategic Thinking
ELA
Analyze or evaluate the effectiveness of literary elements (plot, setting, conflict, point-of view …).
3. Strategic ThinkingMathematics
Solve a multi-step problem, and provide support with a mathematical explanation that justifies the answer.
Beto and Jose were playing a game. At the end of each game, the loser gave the winner a penny. After a while, Jose won 3 games and Beto had 3 more pennies than he did when he began. How many games did they play.
Extended Thinking: Level 4
•Requires high cognitive demand and is very complex
•Students are expected to make connections, relate ideas within the content or among content areas, and select or devise one approach among many alternatives on how the situation can be solved
•Due to the complexity of cognitive demand, DOK 4 often requires an extended period of time
Extended ThinkingELA/Social Science
Write an analysis of an issue. Present multiple points of view and argue for a position in a way that could stimulate the read to consider new perspectives.
Given a problem situation, research and present multiple solutions. Be sure to provide alternative responses and reasons for adopting each.
Extended Thinking
ELA/Science
Construct an explanation based on evidence for how the availability of natural resources, occurrence of natural hazards, and changes in climate have influenced human activity. (NGSS HS-ESS3-1.)
Extended ThinkingMath
This full item is in your handouts
Task Thinking DOK Level
Collecting data samples over several months Recall 1
Organizing data in a chart Skills / Concept 2Using charts and graphs to make predictions and justify conclusions with evidence
Strategic Thinking 3
Developing a generalized model from data, graphs and analysis and applying it to a new situation or to solve a new problem
Extended thinking 4
DOK an Introduction
• Take a few minutes to read through the descriptions or bullets for the levels of DOK 1-4
How does your original definition of Cognitive Rigor Change?
Rate the Examples Level of DOK
• Compare and contrast desert and tropical climates.
• Gather, analyze, organize and interpret data from multiple sources and draft a reasoned response.
• Use a dictionary to define the meaning of words
• Explain the cause-effect of historical events.
Try it!
Rate the Examples Level of DOK
• Identify the elements in a specific compound
• Solve multi-step problems and provide support with mathematical explanation that justifies the answer.
• Determine the area of a triangle given a drawing or labels.
• Evaluate the effectiveness of different exercises and diets on the health of a population.
Try it!
Rate the Level of DOK
• Classify plane and 3-D figures.
• Specify a problem, identify solution paths, solve the problem and report the results.
• What fruit did the witch give to Snow White?
• What is your opinion about the intelligence of the Wolf in Little Red Ridding Hood? Justify your answer using evidence from the story.
• Analyze and explain the multiple perspectives or issues within or across time periods, events, or cultures.
Try it!
Sort the problems by DOK Levels
• Choose a set of cards (ELA or Math)
• With your partner, sort the 8 cards based on the DOK level
• Compare your results with your table partners
Look at the basic question and rigorous one that you developed on for “The Ant and The Grasshopper”
Can you create questions for all four DOK levels?
Do it….
The depth of knowledge level is NOT determined by
the verb, but by the context in which the verb is used
and the complexity of thinking required.
Depth of Knowledge
Same Verb – 3 Different Levels of DOKDOK 1- Describe three characteristics of
metamorphic rocks. (Requires simple recall)
DOK 2- Describe the difference between metamorphic and igneous rocks. (Requires cognitive processing to determine the differences in the two rock types)
DOK 3- Describe a model that you might use to represent the relationships that exist within the rock cycle. (Requires deep understanding of rock cycle and a determination of how best to represent it)
Caution 1
One Statement Across DOK LevelsRecall &
Reproduction(DOK 1)
Skills & Concepts/Basic
Reasoning (DOK 2)
Strategic Thinking/ Complex Reasoning
(DOK 3)
Extended Thinking/ Reasoning
(DOK 4)
All four use the verb identify.
Students will identify information in a passage that is supported by fact.
Students will identify the appropriateness of an argument using supporting evidence.
Students will identify interrelationships (themes, ideas, concepts) developed in more than one literary work.
Students will identify essential information needed to accomplish a task.
One Statement Across DOK LevelsRecall &
Reproduction(DOK 1)
Skills & Concepts/Basic
Reasoning (DOK 2)
Strategic Thinking/ Complex Reasoning
(DOK 3)
Extended Thinking/ Reasoning
(DOK 4)
Students will extend patterns, find the missing term(s) in a pattern or describe rules for patterns (numbers, pictures, tables, words from real-world and mathematical problems (Grade 5 – DOK 3).
Find the next three terms in the pattern and determine the rule for the following pattern of numbers: 1, 4, 8, 11, 15, 18, 22, 25, 29, …
Find the next three terms in the pattern, determine the rule for finding the next number in the pattern and make or find a model for the pattern: 1, 1, 2, 3, 5, 8, 13, 21, 34, …
Draw the next figure in the following pattern:Find the next three
terms in the following pattern: 2/7, 4/7, 6/7, 8/7, ….
DOK is about intended outcomenot about Difficulty
DOK is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product.
• Adding is a mental process• Knowing the rule for adding is the intended outcome
that influences the DOK• Once someone learns the “rule” of how to add, 4 +
4 is DOK 1 and is also easy• Adding 4,678,895 + 9,578,885 is still a DOK 1 but
may be more “difficult”
Points to Consider•DOK 1 + DOK 1 + DOK 1 = 1
• Depths of knowledge classification is based on the task, not the student
• DOK is different from task/item difficulty
•DOK is not a grade level indicator
• DOK ratings aid in alignment of standards and assessment, and therefore instruction
Remember DOK is...
… descriptive… about how deeply a student has to
know the content in order to respond… NOT the same as difficulty… NOT the same as Bloom’s Taxonomy… NOT the same as Costa’s Levels
Review Video
Compare & Contrast CST and SBAC
• Choose 4 CST questions and assign a DOK level. Cite your evidence.
• Choose 4 SBAC questions and assign a DOK level. Cite your evidence.
What is the most significant difference between the CST and the SBAC Assessment?
Discuss the implications for instruction.
How Many of Each Type of Problem?
Colorado Department of Education
Creating DOK Questions and Activities
at all levels
DOK 1, 2, 3, 4Level 1—Identify the tree
Level 2—Explain the functionof the leaves
Level 3—Explain how a drought might affect the growthof the tree
Level 4—Design an investigation of seedling growth to determine the best fertilizer for this type of tree.
Oak Tree
DOK 1, 2, 3 and 4
Level 1—Identify this utensil
Level 2—
Level 3—
Level 4—
I pledge allegiance to the flagof the United States of America,and to the republic for which it stands one nation under God, indivisible, with liberty and justice for all.
Level 1—
Level 2—
Level 3—
Level 4—
DOK in the Classroom
Applying DOK to Instruction• The intended student learning outcome determines the
DOK level
• Assessments, oral questions and classroom activities can all be assigned DOK levels
• Instruction and classroom assessments must reflect the DOK level of the objective or intended learning outcome
As you plan instructional activities, ask:
• What is its purpose?
• What is the implied / intended rigor?
• When in the (lesson / unit) could this be used?
• Which standard(s) does it correlate with?
• Will student responses tell a teacher what to do next in instruction?
From Karin Hess
DOK:• Questions at lower levels are usually more appropriate
for:• Evaluating students’ preparation and comprehension• Diagnosing students’ strengths and weaknesses• Reviewing and/or summarizing
• Usually questions at upper DOK levels are appropriate for:• Encouraging students to think deeply and critically• Problem solving• Encouraging Discussions• Stimulating students to seek information on their own
From Karin Hess
DOK is for all students
• Often struggling students are denied access to higher level questions because they still have difficulty with skills
• However, higher level thinking questions are sometimes easier for struggling students to answer because open-ended questions have more entry points and require more “think time” by the entire class
From Karin Hess
You can ask higher DOK questions• Require students to manipulate prior information
• Why do you suppose…?• What can you conclude from the evidence?
• Ask students to state an idea or definition in their own words
• Ask questions that require a solution to a problem
• Involve students in observing and describing an event or object• What do you notice? Tell me about this. What do you see?
• Ask students to Compare or contrast
From Karen Hess
Reflect on your learning…
• Revisit your definition of rigor – has it changed or been refined? How?
• What is one way you might apply these ideas to your work?
• How might you shift your classroom instructional and assessment practices?
• What existing materials could you examine for a range of cognitive rigor?
From Karin Hess
Questions and Thoughts
What questions and thoughts do you still have regarding DOK?