Page 1
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 3
ISSN: 1758-2199
Two taxonomies are better than one: towards a method of analysing a variety of domains and types of thinking in an assessment
Jackie Greatorex, Research Division, Cambridge Assessment, UK
Jo Ireland, Research Division, Cambridge Assessment, UK
Victoria Coleman, Research Division, Cambridge Assessment, UK
Corresponding author: Jackie Greatorex
Email: [email protected]
Abstract
It has been established practice to analyse the domains and types of thinking involved
in undertaking educational activities using a taxonomy. Many taxonomies focus mainly
on cognition and are therefore unsuitable for analysing non-cognitive domains, these
are often assessed by means other than traditional written assessments. The aims of
this research were to find a taxonomy suitable for analysing a variety of domains and
types of thinking which might in turn be suitable for use across a variety of assessment
tasks and to ascertain whether assessment experts perceived it to be useable. To find
a suitable taxonomy, several were evaluated against pre-defined suitability criteria.
While no individual taxonomy met the criteria, the taxonomies of Marzano and Kendall
(2008) and Hutchins et al. (2013) were merged to form a combined taxonomy which
was used for analysing domains and types of thinking. Six assessment experts
analysed selected assessments including a written examination, a presentation and a
reflection using these taxonomies to inform the research. Findings included the
taxonomic analysis facilitated comparisons between different assessments, all of the
assessments tested a high level of thinking but the interpersonal domain was only
evident in the presentation and reflection assessments. The experts generally found
the combined taxonomy accessible and suitable for analysing the domains and types
of thinking and it could be used successfully to research the alignment between
assessment(s) and/or classroom activities.
Keywords
taxonomy, demand, domain, assessments
Page 2
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 4
ISSN: 1758-2199
Introduction
Since the seminal work of Bloom, Engelhart, Furst, Hill and Krathwohl (1956), it has
been established practice to use taxonomies to analyse the domains and types of
thinking employed by students undertaking academic activities. Taxonomy refers to a
system for classifying domains and types of thinking. Domains are an area of
knowledge or activity;
• The cognitive domain focuses on thinking, including thinking about thinking
(also referred to as metacognition).
• The affective (intrapersonal) domain relates to interests, attitudes, emotions
and values.
• The psychomotor domain centres on physical movements.
• The interpersonal domain concentrates on relationships between people.
Types of thinking are different cognitive activities such as recalling or analysing while
demand generally refers to the level of challenge involved in undertaking a task. There
are nuanced variations of the definition of demand for different taxonomies. In some
taxonomies one type of thinking is considered to be more cognitively demanding than
another. For example, analysing concepts is more demanding than recalling concepts
used to study the topic (Bloom et al., 1956).
There are many reasons to analyse the domains and types of thinking involved in
curriculum and assessment. For example designing assessment strategies by starting
tests or topics with less demanding tasks (such as simple description tasks), before
increasing the demands in the tasks as students become more experienced (such as
evaluative examination questions) and checking that the assessment assesses the
domains and types of thinking required in the curriculum (Ebadi and Shahbazian,
2015; Luebke and Lorié, 2013; Webb, Herman and Webb, 2007). Such practices are
used at all levels of education (primary to post-compulsory) and the work generally
involves experts using a taxonomy to judge which domains and types of thinking
students use when they respond to a task.
Awarding organisations in England and the national regulator of qualifications (Ofqual)
often analyse the cognitive demand of syllabuses such as examination questions and
Page 3
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 5
ISSN: 1758-2199
mark schemes (Pollitt, Ahmed and Crisp, 2007; Shaw and Crisp, 2012). Such analyses
are used to compare the cognitive demands of questions. For example:
• With mark schemes (Shaw and Crisp, 2012).
• From one year with questions from another (Crisp and Novaković, 2009).
• From one assessment provider with questions from another (QCA, 2008).
Such analyses are used in validation work to consider:
• The alignment between question papers and mark schemes (Shaw and Crisp,
2012).
• Whether a variety of demands were placed on the students (Greatorex, Shaw,
Hodson and Ireland, 2013).
Studies tend to focus on the cognitive domain and omit other domains, but the
affective, interpersonal and psychomotor domains are also important as they are all
used in education (Bichi, Hafiz and Abdullahi, 2017; Hiong and Osman, 2013) and/or
employment (Bandaranaike and Willison, 2015; Nair, 2012). For example, the affective
domain is used in architecture curricula (Savic and Kashef, 2013), the cognitive
domain is tested in accounting examinations and used by clinicians (Davidson, 1996;
Laxmisan et al., 2007), the interpersonal domain is employed in many army roles
(Carpenter, Wisecarver, Deagle-III and Mendini, 2005, Hutchins et al., 2013) while the
legal profession use the psychomotor domain in court (Cradduck and Thomas, 2016).
All domains were seen as necessary and included in this research.
The research was done by a team of three researchers working for a national
assessment board and the aims of the project were therefore to:
• find a taxonomy for analysing a variety of domains and types of thinking.
• ascertain whether the taxonomy is suitable for use with a variety of
assessment tasks (not just cognitively orientated written tasks).
• ascertain whether experts perceived the taxonomy to be useable.
Applying Taxonomies
In previous taxonomy research, the participants (those applying the taxonomy) were
often experts with substantial experience in assessing the target field (Chan, Tsui,
Page 4
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 6
ISSN: 1758-2199
Chan and Hong, 2002; Näsström and Henriksson, 2008; Webb et al., 2007). However
many studies gave sparse, if any, description of the experts’ experience in the target
field (for example, Chan et al., 2002; Ebadi and Shahbazian, 2015; Edwards, 2010;
FitzPatrick and Schulz, 2015; Kendall et al., 2008; Palmer and Devitt, 2007). In these
studies, two or three experts tended to use a single taxonomy to categorise items such
as assessment tasks, learning outcomes etc. More unusually, where a larger number
of experts took part (20 was the highest number found in the review of relevant
literature) their experience was given in some detail (Herman, Webb and Zuniga,
2007).
Participant Perceptions
Depending on the situation participants using taxonomy in research may be students,
teachers, researchers or assessors. Whilst there are many taxonomies and a wealth
of taxonomy research, there are only a few studies which investigate participant
perspectives. For example, Dubas and Toledo (2016) researched student perceptions
of the learning objectives that were produced a taxonomy of educational objectives
(Marzano and Kendall, 2007). Nakyam, Kwangsawad and Sriampai (2013) drafted a
curriculum and associated assessments using Marzano and Kendall’s taxonomy and
once the curriculum was implemented the students were surveyed and their views on
the curriculum were found to be positive. Crowe et al. (2008) used Bloom’s taxonomy
in several ways including developing assessments, and tasking students with writing
and classifying test items using the taxonomy. They also gathered student perceptions
on how Bloom’s taxonomy was applied in particular courses and whether it facilitated
learning. However, studies investigating how easy a taxonomy is to use, and whether
it is applicable to the area being studied are rare, yet it is important to know whether a
taxonomy is difficult to apply or appropriate for a particular subject before it is widely
used as this facilitated good educational practice.
Arising from the above literature, the study was designed in three consecutive parts:
Part 1: Selecting a taxonomy for analysing a variety of domains and types of
thinking.
Part 2: Ascertaining whether the selected taxonomy is suitable for use with a
variety of assessment tasks, not just written examinations.
Page 5
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 7
ISSN: 1758-2199
Part 3: Ascertaining the usability of the selected taxonomy from the
participants’ perspective.
The studies were conducted sequentially. A taxonomy was selected in part 1 of the
study, used in part 2 of the study, and finally evaluated by participants in part 3 of the
study. Findings from the earlier studies informed the later studies.
Study 1: Selecting a taxonomy
Aim
The aim of this part of the study was to select a taxonomy to analyse a variety of
domains and types of thinking.
Procedure
The first step was to devise criteria against which we would evaluate taxonomies and
select the most suitable one. We decided our taxonomy should be:
• authoritative and based on current established theory and/or have an
empirical basis to ensure it was robust.
• broad enough to incorporate the cognitive, psychomotor, affective and
interpersonal domains for analysing a variety of domains and non-cognitive
tasks.
• hierarchical or cumulative, with higher levels or categories to distinguish the
more demanding tasks and different types of thinking.
• have been used in assessment in an academic, vocational or applied setting.
(This criterion would ensure that the taxonomy is suitable to use in a variety of
assessment tasks, as per the study aims).
• be applicable to the studied assignments assessed objectives.
Using a taxonomy with no conceptual linkage to the assessment objectives is bad
practice (Johnson and Mehta, 2011) so one researcher searched for and read
literature about taxonomies noting whether the taxonomy met our criteria.
Creating a Combined Taxonomy
It is beyond the scope of this paper to describe all the taxonomies we studied in detail,
but it is important to summarise the selected taxonomies used in Studies 2 and 3. After
Page 6
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 8
ISSN: 1758-2199
investigating a range of taxonomies (Anderson et al., 2001; Atkinson, 2013; Biggs and
Collis, 1982; Carpenter and Wisecarver, 2004; Harrow, 1972; Hauenstein, 1998;
Hutchins et al., 2013; Klein, DeRouin and Salas, 2006; Marzano and Kendall, 2007;
Marzano and Kendall, 2008), it was evident that none met all of the selection criteria
we had set - see above and Table 3 for the criteria. The taxonomy from Marzano and
Kendall (2007, 2008) came nearest and while it omitted the interpersonal domain it
was felt this would prove the most useful. This is summarised in Table 1.
Level Process Description
6 Self-System Thinking
Identifies learning motivation
Identifies emotional responses
Identifies improving ability or understanding
Assess own learning
5 Metacognition Gauge own level of accuracy
Determine own understanding
Monitor own progress toward a goal
Outline learning goals and plan to achieve them
4 Knowledge Utilization
Investigate outcomes
Experiment to find different outcomes
Test theories
Solve Problems and make decisions
3 Analysis
Inform of consequences
Make generalisations
Spot errors
Categorise
Note similarities and differences
2 Comprehension
Draw up information
Design information outputs
Combine information and summarise
Structure information
1 Retrieval Displaying more complex processes
Simple procedural actions
Recollection of simple information
Producing information
Recognising information appropriateness
Identifying information and assessing for accuracy
Table 1: Summary of Marzano and Kendal’s New Taxonomy (2008)
Therefore we chose Marzano and Kendall’s (2007, 2008) taxonomy for use in
combination with Hutchins et al. (2013), taxonomy to provide a comprehensive overall
Page 7
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 9
ISSN: 1758-2199
taxonomy which allowed us to include an up-to-date articulation of the interpersonal
domain.
Marzano and Kendall’s (2007, 2008) taxonomy has two dimensions: (1) knowledge
domains; the things we know or remember, and (2) mental processing how we
understand the things we know or remember. Within the knowledge dimension there
are three domains: information (factual knowledge); mental procedures (how to
mentally carry out a task); and psychomotor procedures (how to physically carry out a
task). While there is no hierarchical relationship amongst these domains, each domain
comprises of further sub-categories, which are hierarchically organised.
Within the mental processing dimension there are three thinking systems delineated
into six levels which mentally process types (domains) of knowledge (Marzano and
Kendall, 2007, 2008). The highest or most complex level is the self-system (Level 6),
which refers to an assembly of interrelated beliefs, attitudes and emotions that control
attention and motivation. These determine the candidate’s level of engagement with a
new task. When a candidate embarks on a new task, the metacognitive and cognitive
systems are activated. The metacognitive system (Level 5) exercises executive
control as it monitors, evaluates and regulates cognition or thinking. Metacognition
involves stipulating goals and planning how to achieve the goals.
Category Abilities needed to….
Interpersonal
communication skills
Express and integrate information in social interaction. This
encompasses listening, speaking, writing, sending/receiving non-verbal
signals in an empathic, attentive, responsive and confident way.
Relationship building skills Develop and keep relationships, to support others, and build strong
beneficial unions as well as manage and resolve conflicts.
Peer leadership skills Coach, advice, motivate and empower group members. Happily interact
with a team, gain trust, dynamically join in problem solving.
Social/behavioural agility
skills
Monitor and understand our own and other’s behaviours and alter self-
presentation to influence the interaction.
Table 2: Summary of Marzano and Kendal’s New Taxonomy (2008)
The remaining four levels process the knowledge required to do the task and form the
third Thinking system: cognitive (Marzano and Kendall, 2007, 2008), knowledge
utilisation (Level 4) entails using knowledge in a particular situation, analysis (Level 3)
Page 8
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 10
ISSN: 1758-2199
involves reasoning which creates new thought and information, comprehension (Level
2) refers to the interpretation of knowledge into a mode appropriate for storing in long
term memory, retrieval (Level 1) is remembering information and drawing it into
consciousness ready for use. All six levels are subdivided into operations, which are
cumulatively organised.
The cognitive levels in Marzano and Kendall’s taxonomy (2007, 2008) are delineated
in terms of the cognitive control of the thought processes needed to complete tasks.
The demand of the tasks is not related to the cognitive demand intrinsic to the task,
but to how intentional the students must be in their thinking of an answer to tasks. In
this context, the term intentional refers to deliberate and conscious thinking rather
than, for example, practised recall which has become automatic
Hutchins et al. (2013) created a taxonomy from literature about training and skills,
including negotiation and conflict resolution. We judged this to be a taxonomy focusing
on the interpersonal domain. The main skill groupings in the taxonomy are:
relationship building skills; peer leadership skills; interpersonal communication skills;
and social/behavioural agility skills. Each main grouping comprises additional
subcategories of skills. There is no hierarchy within the taxonomy (see Table 1).
However, we found no interpersonal taxonomies which were hierarchical.
Page 9
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 11
ISSN: 1758-2199
Suitability Criteria The taxonomy must
An
ders
on e
t al. (
20
01)
Atk
inso
n (
20
13)
Big
gs a
nd C
olli
s (
19
82)
Carp
ente
r an
d W
isecarv
er
(2004)
Harr
ow
(19
72)
Haue
nste
in (
19
98)
Hutc
hin
s e
t a
l. (
2013)
Kle
in e
t a
l. (
2006)
Marz
ano a
nd
Kend
all
(20
07,
2008)
Be established ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Be based on theory ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Include the (meta)cognitive domain
✓ ✓ ✓ ✓ ✓ ✓
Include the psychomotor domain
✓ ✓ ✓ ✓ ✓
Include the affective domain
✓ ✓ ✓
Include the interpersonal domain
✓ ✓ ✓
Be hierarchical or cumulative
✓ ✓ ✓ ✓ ✓ ✓
Be used in assessment of academic achievement
✓ ✓ ✓ ✓
Evidence of the taxonomy being used in assessment of academic achievement
FitzPatrick
and
Schulz
(2015)
Rembach
and Dison
(2016)
Yang
(2013)
Dubas and
Toledo (2016);
Faragher and
Huijser (2014)
Be used in assessment of vocational or applied knowledge and skills
✓ ✓ ✓ ✓
Evidence of the taxonomy being used in assessment of vocational or applied knowledge and skills
Mayer
(2002)
İlgüy, İlgüy,
Fişekçioğland
Oktay (2014)
Chan,
Chan,
Tsui
and
Kwok
(2014)
Pate and Miller
(2011)
Haolader, Ali
and Foysol
(2015)
Edler, Eberman,
Kahanov, Roma
and Mata
(2015)
Table 3: Judging taxonomies against suitability criteria
Note, ✓ denotes that the taxonomy meets the criteria, and a blank cell denotes that it does not meet the criteria.
Page 10
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 12
ISSN: 1758-2199
Turning to the issue of how we combined the taxonomies, cognition
(interpreting/understanding communication signals) and intrapersonal ability
(managing emotions, beliefs) are key mediating variables for the deployment of
interpersonal skills (Klein et al., 2006). As mentioned in the introduction, it is important
to examine domains beyond just the cognitive domain, as these are also used in both
education and employment. Therefore, we theorised that the levels of mental
processing (Marzano and Kendall, 2007; 2008) also apply to the interpersonal domain
(Hutchins et al., 2013). Greatorex and Suto (2016) made the same theorisation. The
taxonomies focus on each individual candidate and can apply to a variety of academic
and vocational subjects (Hutchins et al., 2013; Marzano and Kendall, 2007, 2008).
Within our combined taxonomy there are three dimensions of demand: (1) the number
of domains; (2) the knowledge category; and (3) the level of mental processing. The
demand of a task can be increased by including more domains, using a higher
knowledge category, or using a higher level of mental processing. Similarly, the
demand can be decreased by reducing the number of domains tested, requiring a
lower knowledge category or a lower level of mental processing.
In conclusion, the selected taxonomy was a combination of Marzano and Kendall
(2007, 2008) and Hutchins et al. (2013). We theorise that the levels of mental
processing described by Marzano and Kendall (2007, 2008) also apply to the
interpersonal domain from Hutchins et al. (2013). This is shown in Table 4: Overview
of the combined taxonomy (Hutchins et al., 2013; Marzano and Kendall, 2007, 2008).
Thinking system/level of
mental processing
Domain
Information Mental
procedures
Psychomotor
procedures Interpersonal
Self system
Meta-cognitive system
Knowledge utilisation1
Analysis1
Comprehension1
Retrieval1
Table 4: Overview of the combined taxonomy (Hutchins et al., 2013; Marzano and Kendall, 2007; Marzano and
Kendall, 2008) Note: Together these levels of mental processing form the Cognitive system.
Page 11
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 13
ISSN: 1758-2199
Study 2: Using the selected taxonomy
Aim
The aim of this stage was to ascertain whether the new combined taxonomy was
suitable for use with a variety of assessment tasks and if the data generated from
applying the taxonomy could be used to compare the:
• Domains, categories and operations elicited by each task.
• Demands of various assessments.
Method
Six experts in summative assessment of the target field were recruited as participants
in the study. Here, expert refers to individuals with substantial experience of
summative assessment, particularly in marking and determining grade boundaries.
Each expert categorised the questions from the assessments under the same
conditions using the combined taxonomy from Study 1. The discipline/level was
purposefully omitted to protect the anonymity of participants. This is due to the small
number of experts within this specific area of international education.
The following summative assessments from a secondary level international academic
qualification were used:
• An assignment: for which students study a topic then write a report based on
their studies.
• A reflection: (reflective article) about their team’s solutions to a problem
which the team researched.
• A presentation: about their own research and proposed solutions to the
aforementioned problem.
• A written exam: comprising questions which required short and long
responses.
These summative assessments were used as they varied in style and were not all
timed written tests.
Page 12
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 14
ISSN: 1758-2199
Five Example Exam Questions
Five example questions were devised by the researchers using guidance in Marzano
and Kendall (2007, 2008). The example questions were designed specifically for the
research and the orientation exercise and are shown in Appendix 1.
Procedure
Preliminary work
Johnson and Mehta (2011) explain that applying a taxonomy relies on the participants
being able to relate their subject knowledge to the taxonomy. They therefore advise
that, before applying a taxonomy, researchers map the dimensions in the taxonomy
to the subject. Therefore, the assessment objectives from the above-mentioned
summative assessments were mapped to the selected taxonomies. There were two
steps to this procedure:
a) A researcher systematically went through the verb in each assessment
objective and noted the level of mental processing it corresponded to in
Marzano and Kendall (2007, 2008), by matching it to the example verbs, or
where they were synonyms of the verbs. Marzano and Kendall (2007, 2008)
give examples of verbs from educational objectives connected with each level
of mental processing. For instance, the verb ‘state’ is frequently used in
learning objectives for retrieval (Level 1).
b) A researcher noted the object of each assessment objective and judged which
domain(s) were targeted by the assessment. Marzano and Kendall (2007,
2008) argue that the object of educational objectives signpost the domain(s)
of knowledge on which an assessment concentrates.
The results of the mapping are given in Table 4, which shows the assessment
objectives mapped to the combined taxonomy showing the combined taxonomy was
suitable for analysing assessment tasks.
Orientating experts to the taxonomy
One researcher briefed the experts on the combined taxonomy to engender a shared
understanding of the taxonomy categories. They then applied the combined taxonomy
categories to the five example questions with facilitation by the researcher. Here the
Page 13
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 15
ISSN: 1758-2199
aim was not for experts to agree on which questions fitted into a taxonomy category
but for orientation.
Applying the taxonomy
The experts were told they needed the assessment tasks (question papers or
equivalent), the written document summarising the combined taxonomies and a
document for recording responses. It was also explained that the exercise was about
the types of knowledge and thinking elicited by the assessment tasks. The researchers
asked the experts to work individually and to re-familiarise themselves with the
summary of the combined taxonomies. The researchers advised that some
assessment tasks may use one domain and others may use more. They were then
asked to do the following for each assessment task in turn starting with the first
question of the written examination:
• Read the assessment task.
• Decide which domain(s) the assessment task was likely to elicit.
• For each domain, decide which thinking operation(s) the assessment task
was likely to elicit.
• Record the decisions.
In summary, the experts individually categorised each assessment task using the
combined taxonomy.
Analysis
To categorise assessments with operations and domains we calculated the number of
experts allocating an assessment task (examination question) to an operation and
domain category. Several studies use any agreements between two experts to
indicate where an item is classified within the taxonomy. Examples include Edwards
(2010); Kendall et al. (2008); Näsström and Henriksson (2008). Other studies, with
between six and twenty coders, take between 56% to 67% of coders assigning an item
to a particular taxonomy category, to indicate where an item is classified within the
taxonomy (Herman et al., 2007; Webb et al., 2007). In this study when four of the six
experts said that a question elicited a particular domain or operation the question was
then added to the respective domain or operation.
Page 14
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 16
ISSN: 1758-2199
This allowed an assessment of whether experts thought that assessment tasks elicited
a variety of domains, categories, levels of processing and operations. The orientation
task was designed to develop a shared understanding of the taxonomy but not
necessarily a shared interpretation of the assessment tasks.
Results and Discussion
The assessments were compared on the basis of the domains selected by the experts
and the results are shown in Table 5. Both the written examination (questions One to
Four) and the assignment evoked two domains: information and mental procedures.
The presentation and reflection were the most demanding as they educed all
domains. In this situation, one assessment being more demanding than another is
satisfactory as there is no requirement for the assessments to be of equal demand.
Domain Category
Qu
esti
on
1
Qu
esti
on
2
Qu
esti
on
3
Qu
esti
on
4
Assig
nm
en
t
Pre
sen
tati
on
Refl
ecti
on
Information
Principles ✓ ✓
Generalizations ✓
Time sequences
Facts ✓ ✓ ✓ ✓
Vocabulary terms ✓ ✓
Mental
procedures
Macro-procedure ✓ ✓ ✓ ✓ ✓
Tactics
Algorithm
Single rule
Psychomotor
procedures
Complex combination rules ✓
Foundational procedures
Simple combination rules
Interpersonal
Communication skills ✓ ✓
Peer leadership skills
Relationship building skills ✓
Social agility skills ✓
Table 5: Questions that elicited each category
Page 15
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 17
ISSN: 1758-2199
The assessments were then compared in terms of the categories elicited within each
domain. The assignment and presentation prompted a higher information category
than the written examination with the assignment and presentation equally
demanding. Each assessment stimulated the highest category of mental procedures.
The presentation achieved the highest psychomotor procedures category making it
the most demanding assessment in the psychomotor procedures domain. The
combination of assessment tasks drew from a range of domains and categories. The
set of questions elicited each domain and nine of the 16 categories, including both
high and low categories (see Table 5). This suggests a good assessment regime as
the questions required a breadth of knowledge and there were less demanding tasks
where lower ability candidates were more likely to achieve, and more demanding tasks
to stretch higher ability candidates.
There was a relationship between question type and the domain(s) assessed, as might
be expected. Three of the questions (1, 2 and 4) from the written examination each
elicited one domain, either the information or mental procedures domain. The other
written assessment tasks the assignment and the reflection each stimulated
candidates to use two domains. The presentation prompted examinees to use all four
domains. It seems that a greater variety of domains were elicited by questions and
assessments requiring longer constructed responses using a medium other than a
written examination. There was a relationship between the type of question
(openness of the constructed response) and whether higher or lower categories were
elicited. Questions 1 and 2 (those requiring the shortest responses) elicited lower
categories within the information domain. Questions 3 and 4, the assignment,
presentation, and reflection each prompted candidates to use the highest category
in a domain. Generally, open constructed response questions educe the higher
categories within the domain.
Page 16
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 18
ISSN: 1758-2199
Table 6: Questions which elicited each operation
The assessments were compared in terms of the operations elicited by the
assessments. The written examination questions evoked lower operations than the
assignment (see Table 6). The presentation and reflection educed higher
operations than the assignment. Therefore, the presentation and reflection were of
Level Operation
Qu
esti
on
1
Qu
esti
on
2
Qu
esti
on
3
Qu
esti
on
4
Assig
nm
en
t
Pre
sen
tati
on
Refl
ecti
on
Self system
Examining overall motivation ✓
Examining emotional response ✓ ✓
Examining efficacy ✓
Examining importance ✓ ✓ ✓
Meta-cognitive
system
Monitoring accuracy ✓ ✓ ✓
Monitoring clarity ✓ ✓
Process monitoring ✓ ✓ ✓
Specifying goals ✓ ✓
Knowledge
utilisation
Investigating ✓ ✓ ✓
Experimenting
Problem solving ✓
Decision making ✓ ✓ ✓ ✓
Analysis
Specifying ✓ ✓
Generalising ✓ ✓ ✓ ✓
Analysing errors ✓ ✓ ✓
Classifying ✓ ✓ ✓
Matching ✓ ✓ ✓
Comprehension Integrating ✓
Symbolizing
Retrieval
Recognition ✓ ✓
Recall ✓ ✓
Execute ✓ ✓ ✓ ✓
Page 17
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 19
ISSN: 1758-2199
the greatest demand. In this situation, one assessment being more demanding than
another is suitable as there was no requirement for the assessments to be of equal
demand. It is striking that the longer constructed response questions, particularly
those drawing on group work, elicited higher operations (see Table 6). In other words,
the longer constructed questions and the questions drawing on group work placed
higher demands on candidates.
In summary, the data generated by the taxonomic analysis allowed us to compare the
domains, categories and operations elicited by each task, and compare the demand
of various assessments. Therefore, the combined taxonomy was suitable for use with
a variety of assessment tasks.
Study 3: Experts evaluate the utility of the taxonomy
Aim
To ascertain whether experts perceived the taxonomy to be useable.
Method
The same experts from Stage 2 subsequently completed an online questionnaire
including three closed response questions about the accessibility, appropriateness
and ease of use of the taxonomies. There was also an open-ended question that
asked for a list of knowledge and skills not covered by the Thinking systems.
To analyse the closed response questions the frequency of responses were tabulated
for positive or negative responses. For the open questions, the responses were
summarised or quoted, as agreed with the experts and themes drawn out.
Results
The accessibility, appropriateness and ease of use of the taxonomies were evaluated
using three closed response questions. All or most of the participants found each
aspect of the taxonomies accessible (see summary in Appendix 2). All the participants
felt that most aspects of the taxonomies were appropriate for the target examination
questions (see summary in Appendix 2). The only aspect which they felt was not
appropriate was the psychomotor procedures domain. Generally the participants
found it neither easy nor difficult to analyse the domains and thinking systems involved
in the tasks (see summary in Appendix 2). Overall, the taxonomy was found by the
Page 18
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 20
ISSN: 1758-2199
experts to be useable in our study and therefore may be considered for use with similar
assessments.
Experts were asked to list the knowledge and skills from the assessment which were
not covered by the domains or thinking systems. Responses indicated that three areas
of knowledge/skills were omitted. Firstly “Identifying terms/phrases,” (Expert 2) from
questions 1 to 4 was not evident but it could be argued it is covered by recognition.
Secondly “Much of the information domain appears to be focused upon utilising
knowledge and not gleaning knowledge,” (Expert 4). This comment is accurate, but
‘gleaning knowledge’ is arguably captured in the operation experimenting and
investigating in the cognitive system. Thirdly, the field assessed by the examination
questions “assesses ability to utilise a system above that of self,” one might term this
‘society system.’ This supports the notion that knowledge and skills from the
qualification were largely incorporated in the domains or thinking systems.
Overall discussion and conclusions
The research had two key purposes. These were to ascertain:
• Whether the taxonomy combining Hutchins et al. (2013) and Marzano and
Kendall (2007, 2008) could be used to analyse domains, types of thinking and
demand.
• Whether assessment experts perceived it to be useable.
The main findings were that the combined taxonomy:
• Could be used to analyse domains, types of thinking and demands.
• The experts’ found the combined taxonomy accessible and appropriate for a
variety of assessments and subjects.
There are several limitations to the study. Firstly, the research focused on only one
subject, although the experts felt that the taxonomy could be used in other subjects.
Secondly, the participants may not be a representative sample of the population of
assessment experts in a variety of subjects, and therefore it is unclear how far their
participant perceptions can be generalised. Conversely, there is no clear reason why
they would be significantly atypical of assessment experts.
Page 19
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 21
ISSN: 1758-2199
Regardless of these limitations, our findings prove useful. The combined taxonomy
has some advantages over CRAS (Complexity, Resources, Abstractness, Strategy),
which is the established and widely used tool for analysing demands in school
examinations. CRAS focuses on cognitive demands and, consequently, has limited
usefulness with examinations which claim to prepare students for employment, an
industrial sector, or a particular occupation. This is because many careers draw upon
non-cognitive as well as cognitive domains (Bandaranaike and Willison, 2015;
Carpenter. et al., 2005; Cradduck and Thomas, 2016; Davidson, 1996; Hutchins et al.,
2013; Laxmisan et al., 2007; Nair, 2012; Savic and Kashef, 2013). In contrast, the
domains in the combined taxonomy broadly align with those in the literature. Also
CRAS may not align with the non-cognitive aspects of assessments, which are key
features of applied and vocational qualifications. The combined taxonomy may be
more relevant to such qualifications.
The domains, types of thinking and demand invoked by a variety of item types, such
as short/long constructed response, and modes of assessment, such as written
examination and presentations, were explored and compared using the procedures
and the combined taxonomy. In the future this technique could be applied to
comparability studies, which include a variety of question types. Such comparability
studies may compare examination questions from different years or assessment
providers. The combined taxonomy, together with the research procedure, effectively
identified if a variety of demands from a range of domains were placed on students as
well as highlighting whether there were questions of relatively low demand on which
less able candidates might achieve. It also identified relatively high demand questions
which will challenge able candidates. Thus, this approach could be applied to
validation studies, particularly when the assessment includes non-cognitive domains.
The research was unusual in gathering participants’ views on whether the combined
taxonomy facilitated the analysis of domains, types of thinking and demand.
Participants understanding of the concepts drew from their expertise and the
orientation task. Such work feeds into key assessment work on validity and
comparability, as already mentioned. For the combined taxonomy to aid analysing
domains, types of thinking and demand it must be readily applicable and appropriate
for the subject. The assessment experts’ views were largely positive on all of these
Page 20
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 22
ISSN: 1758-2199
issues. This suggests that the combined taxonomy is a useful tool for analysing
domains, types of thinking and demands.
We followed established procedures for experts applying taxonomies. Examples
include experts in the examination questions participating in the research and
familiarising participants with the taxonomy used. The present research allowed
experts to record as many categories and operations as they wished for each item, as
the item might elicit more than one domain and could be answered at various levels
of thinking. For instance, it is likely that doing a presentation uses interpersonal skills
and information. This approach aligns with prior research (Johnson and Fuller, 2006;
Näsström and Henriksson, 2008). In other examples, experts were asked for one
taxonomy category per item (Ebadi and Shahbazian, 2015). Arguably, future research
could ask for participants to identify each domain category and the highest operation
elicited in association with that domain category. It could then be assumed that the
question might also elicit all lower categories and operations.
Greatorex and Suto (2016) theorised that the thinking systems and operations
(Marzano and Kendall, 2006, 2007, 2008) may be used to process interpersonal
knowledge. The present research provides evidence to support that theory. Finally,
the findings are important for qualification development. Taxonomies are often used
as guidance for writing educational objectives, items and mark schemes as well as
aligning these during the development process (Marzano and Kendall, 2007, 2008).
As the participants felt the combined taxonomy may be appropriate for many
qualifications and subjects, it follows that it may be useful in guiding syllabus and
item/mark scheme writing in many areas. In this regard our findings accord with those
of Greatorex and Suto (2016).
In conclusion, we found that the taxonomies of Hutchins et al. (2013) and Marzano
and Kendall (2007, 2008) can be combined. Moreover, the combined taxonomy was
successfully used to compare the domains, categories and operations elicited by a
variety of tasks and assessments, as well as compare the demand of such tasks and
assessments. Furthermore, this taxonomy is accessible and appropriate for a variety
of assessments and subjects.
Page 21
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 23
ISSN: 1758-2199
References
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E.,
Pintrich, P. R., Raths, J. and Wittrock, M. C. (2001) A taxonomy for learning, teaching
and assessing: a revision of Bloom’s taxonomy of educational objectives. New York:
Longman.
Atkinson, S. P. (2013) Taxonomy circles: Visualizing the possibilities of intended
learning outcomes. Learning and Teaching Working Paper. Available at:
https://spatkinson.files.wordpress.com/2015/01/taxonomy-circles-atkinson-aug13.pdf
Bandaranaike, S. and Willison, J. (2015) Building capacity for work-readiness:
Bridging the cognitive and affective domains. Asia-Pacific Journal of Cooperative
Education, Special Issue, Vol. 16(3), pp. 223-233.
Bichi, A. A., Hafiz, H. and Abdullahi, S. (2017) Evaluating secondary school students’
science achievement: Implication for curriculum implementation. International Journal
for Social Studies, Vol. 3(1), pp.113-121.
Biggs, J. B. and Collis, K. F. (1982) Evaluating the quality of learning – the SOLO
taxonomy. 6th ed. New York: Academic Press.
Bloom, B. S., Engelhart, M. D., Furst, E. D., Hill, W. H. and Krathwohl, D. R. (1956)
Taxonomy of educational objectives: the classification of educational goals. Handbook
1, Cognitive domain. London: Longmans.
Carpenter, T. D. and Wisecarver, M. M. (2004) Identifying and validating a model of
interpersonal performance dimensions. Alexandria: Army Research Institute for the
Behavioral and Social Sciences.
Carpenter, T.D., Wisecarver, M. M., Deagle III, E. A. and Mendini, K. G. (2005)
Special Forces interpersonal performance assessment system. FEDERAL
MANAGEMENT PARTNERS INC ALEXANDRIA VA.
Chan, C. CH., Chan, K., Tsui, M.-S. and Kwok, A. (2014) Towards a competence-
based evaluation framework: The personal growth of volunteers in a child injury
prevention program Journal of Social Work, Vol.14(5). doi:
10.1177/1468017313479080.
Page 22
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 24
ISSN: 1758-2199
Chan, C. C, Tsui, M. S., Chan, M. Y. C. and Hong, J. H. (2002) Applying the
Structure of the Observed Learning Outcomes (SOLO) taxonomy on student’s
learning outcomes: an empirical study. Assessment & Evaluation in Higher
Education, Vol. 27(6), pp. 511- 529. doi: 10.1080/0260293022000020282.
Cradduck, L. Thomas, M. (2016) From the waist up: developing psychomotor skills
for the court room. International Journal of the Legal Profession, pp. 1-21. doi:
10.1080/09695958.2016.1226844.
Crisp, V. and Novaković, N. (2009) Is this year's exam as demanding as last year's?
Using a pilot method to evaluate the consistency of examination demands over time.
Evaluation and Research in Education, Vol.22(1), pp. 3-15. doi:
10.1080/09500790902855776.
Crowe, A., Dirks, C. and Wenderoth, M. P. (2008) Biology in Bloom: Implementing
Bloom’s taxonomy to enhance student learning in biology. CBE—Life Sciences
Education, Vol. 7, pp. 368-381. doi: 10.1187/cbe.08-05-0024.
Davidson, R. (1996) Cognitive complexity and performance in professional accounting
examinations. Accounting Education, Vol.5(3), pp. 219-231.
Dubas, J. M. and Toledo, S. A. (2016) Taking higher order thinking seriously: Using
Marzano’s taxonomy in the economics classroom. International Review of Economics
Education, Vol. 21, pp. 12-20. doi: 10.1016/j.iree.2015.10.005.
Ebadi, S. and Shahbazian, F. (2015) Exploring the cognitive level of final exams in
Iranian high schools: Focusing on Bloom’s taxonomy. Journal of Applied Linguistics
and Language Research, Vol. 2(4), pp. 1-11.
Edler, J., Eberman, L., Kahanov, L., Roman, C. and Mata, H. (2015) Athletic trainers’
knowledge regarding airway adjuncts. Athletic Training Education Journal, Vol.10(2),
pp. 164-169. doi: 10.4085/1002164.
Edwards, N. (2010) An analysis of the alignment of the grade 12 physical sciences
examination and the core curriculum in South Africa. South African Journal of
Education, Vol.30, pp. 571-590.
Page 23
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 25
ISSN: 1758-2199
Faragher, L. and Huijser, H. (2014) Exploring evidence of higher order thinking skills
in the writing of first year undergraduates. The International Journal of the First Year
in Higher Education, Vol. 5(2), pp. 33-44. doi: 10.5204/intjfyhe.v5i2.230.
FitzPatrick, B. and Schulz, H. (2015) Do curriculum outcomes and assessment
activities in Science encourage higher order thinking? Canadian Journal of Science,
Mathematics and Technology Education, Vol.15(2), pp. 136-154. doi:
10.1080/14926156.2015.1014074.
Greatorex, J., Shaw, S., Hodson, P. and Ireland, J. (2013) Using scales of cognitive
demand in a validation study of Cambridge International A and AS Level Economics
Research Matters: A Cambridge Assessment Publication (15), pp.29-37.
Greatorex, J. and Suto, W. M. I. (2016) Extending educational taxonomies from
general to applied education: can they be used to write and review assessment
criteria? (Roundtable). Paper presented at the European Association for Research on
Learning and Instruction, Ludwig-Maximilians-Universität Munich, Germany.
Haolader, F. A., Ali, M. R. and Foysol, K. M. (2015) The taxonomy for learning,
teaching and assessing: Current practices at polytechnics in Bangladesh and its
effects in developing students' competences. International Journal for Research in
Vocational Education and Training, Vol.2(2), pp. 99-118. doi:
10.13152/IJRVET.2.2.2.
Harrow, A. J. (1972) A taxonomy for the Psychomotor domain: A guide for developing
behavioural objectives. New York: David McKay.
Hauenstein, A. D. (1998) A conceptual framework for educational objectives: A holistic
approach to traditional taxonomies. Lanham, MD: University Press of America.
Herman, J. L., Webb, N. M. and Zuniga, S. A. (2007) Measurement issues in the
alignment of standards and assessments. Applied Measurement in Education &
Training, Vol. 20(1), pp. 101-126. doi: 10.1080/08957340709336732.
Hiong, L. C. and Osman, K. (2013) A conceptual framework for the integration of 21st
Century Skills in biology education. Research Journal of Applied Sciences,
Engineering and Technology, Vol.6(16), pp. 2976-2983.
Page 24
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 26
ISSN: 1758-2199
Hutchins, S., McDermott, P., Carolan, T., Gronowski, M., Fisher, A. and DeMay, M.
(2013) Interpersonal skills summary report Research Note 2013-03. Fort Belvoir: Alion
Science and Technology, Inc.
İlgüy, M., İlgüy, D., Fişekçioğlu, E. and Oktay, I. (2014) Comparison of case-based
and lecture-based learning in dental education using the SOLO taxonomy. Journal of
Dental Education, Vol.78(11), pp. 1521-1527.
Johnson, C. G. and Fuller, U. (2006) Is Bloom’s taxonomy appropriate for computer
science? Paper presented at the 6th Baltic Sea conference on computing education
research: Koli Calling 2006.
Johnson, M. and Mehta, S. (2011) Evaluating the CRAS framework: Development and
recommendations. Research Matters: A Cambridge Assessment Publication (12),
pp.27-33.
Kendall, J. S., Ryan, S., Weeks, S., Alpert, A., Schwols, A. and Moore, L. (2008)
Thinking and learning skills: What do we expect of students. Denver: Mid-continent
research for education and learning.
Klein, C., DeRouin, R. E. and Salas, E. (2006) Uncovering workplace interpersonal
skills: A review, framework, and research agenda. In: G. P. Hodgkinson and J. K. Ford
(Eds.), International Review of Industrial and Organizational Psychology 2006, Vol.
21. West Sussex: John Wiley & Sons.
Laxmisan, A., Hakimzadaa, F., Sayan, O. R., Green, R. A., Zhang, J. and Patel, V. L.
(2007) The multitasking clinician: Decision-making and cognitive demand during and
after team handoffs in emergency care. International Journal of Medical Informatics,
Vol. 76, pp. 801-811.
Luebke, S. and Lorié, J. (2013) Use of Bloom’s taxonomy in developing reading
comprehension specifications. Journal of Applied Testing Technology, Vol. 14.
Marzano, R. J. and Kendall, J. S. (2007) The new taxonomy of educational objectives.
2nd ed. USA: Corwin Press Inc.
Marzano, R. J. and Kendall, J. S. (2008) Designing and assessing educational
objectives: Applying the new taxonomy. United States of America: Corwin Press Inc.
Page 25
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 27
ISSN: 1758-2199
Mayer, R. (2002) A taxonomy for computer-based assessment of problem solving.
Computers in Human Behavior, Vol.18(18), pp. 9. doi: 10.1016/S0747-5632(02)00020-1.
Nair, P. J. (2012) Is talent management accentuated by competency mapping? With
special reference to educational sector. International Journal of Social Science &
Interdisciplinary Research, Vol.1(11), pp. 132-147.
Nakyam, J., Kwangsawad, T. and Sriampai, P. (2013) The development of foreign
language substance group curriculum based on Marzano’s taxonomy. Educational
Research and Reviews, Vol. 8(14), pp. 1109-1116.
Näsström, G. and Henriksson, W. (2008) Alignment of standards and assessment: A
theoretical and empirical study of methods for alignment. Electronic Journal of
Research in Educational Psychology, Vol.6 (3), pp. 667-690.
Palmer, E. J. and Devitt, P. G. (2007) Assessment of higher order cognitive skills in
undergraduate education: modified essay or multiple choice questions? BMC medical
education, Vol. 7(49), pp. 1-7. doi: 10.1186/1472-6920-7-4.
Pate, M. L. and Miller, G. (2011) Effects of think–aloud pair problem solving on
secondary–level students’ performance in career and technical education courses.
Journal of Agricultural Education, Vol.52(1), pp. 120-131. doi: 10.5032/jae.2011.01120
Pollitt, A., Ahmed, A. and Crisp, V. (2007) The demands of examination syllabuses
and question papers. In: P. Newton, J. Baird, H. Goldstein, and H. Patrick (Eds.),
Techniques for monitoring the comparaility of examination standards (pp. 166-206).
London: Qualifications and Curriculum Authority.
QCA. (2008) Inter-subject comparability studies. Study 1a: GCSE, AS and A Level
geography and history: QCA.
Rembach, L. and Dison, L. (2016) Transforming taxonomies into rubrics: Using SOLO
in social science and inclusive education. Perspectives in Education, Vol.34(1), pp.
68-83. doi: 10.18820/2519593X/pie.v34i1.6.
Savic, M. and Kashef, M. (2013) Learning outcomes in affective domain within
contemporary architectural curricula. International Journal of Technology and Design
Education, Vol.23(4), pp. 987-1004. doi: 10.1007/s10798-013-9238-8.
Page 26
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 28
ISSN: 1758-2199
Shaw, S. D. and Crisp, V. (2012) An approach to validation: Developing and applying
an approach for the validation of general qualifications. Research Matters: A
Cambridge Assessment Publication, Special Issue 3.
Toledo, S. and Dubas, J. M. (2016) Encouraging higher-order thinking in general
chemistry by scaffolding student learning using Marzano’s taxonomy. Journal of
Chemical Education, Vol.93, pp. 64−69. doi: 10.1021/acs.jchemed.5b00184.
Webb, N., M. Herman, J., L. and Webb, N., L. (2007) Alignment of mathematics state-
level standards and assessments: The role of reviewer agreement. Educational
Measurement: Issues and Practice, Summer, Vol.17 (2).
Yang, D. (2013) Comparing assessments within junior geography textbooks used in
mainland China. Journal of Geography, Vol.112(2), pp. 58-67. doi:
10.1080/00221341.2011.648211.
Page 27
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 29
ISSN: 1758-2199
Appendices
Appendix 1: The Five Example Questions for the Exam Assessment
1. Complete the following sentence.
‘Global warming is…..’
a. the world getting hotter
b. the world getting colder
c. ice caps melting.
2. Here is a diagram in Draw (an Open Office computer programme). Save the file.
(Diagram Given)
3. Generate and test hypotheses about which is the best mode(s) of communication
(written, oral, face to face etc) for training overseas teachers to use a coursework mark
scheme.
4. Your task is to park and secure two cherry pickers on a lorry for transportation.
Set a goal for your performance. As you perform the task, identify and describe how
effectively you think you are performing.
5. Here are the price lists for running a conference at each of three Cambridge Colleges.
i. Select the best option.
ii. Explain how the information helped you select the best option.
Appendix 2: Summary
Responses to the question: ‘Which of the following were accessible? The description of…’
Number of positive responses
The Information domain 6
The Mental procedures domain 5
The Psychomotor procedures domain 5
The Interpersonal domain 6
The Thinking systems 5
Page 28
Educationalfutures Greatorex, Ireland and Coleman
Vol.10(1) July 2019 Two taxonomies are better than one __________________________________________________________________________________________
e-journal of the British Education Studies Association 30
ISSN: 1758-2199
‘Which of the following were appropriate for analysing the knowledge and thinking in [the
examination questions]1?’
Number of positive responses
The Information domain 6
The Mental procedures domain 6
The Psychomotor procedures domain 2
The Interpersonal domain 6
The Thinking systems 6
‘How easy was it to analyse the domains and thinking systems involved in the
assessment tasks?’
Easy Neutral Difficult
Written examination 1 3 1
Assignment 0 5 0
Presentation and Reflection 1 4 0
1 The original question gave the subject and qualification, which were anonymised for this paper.