Is Character Development Quantifiable? Creation and Validation of the Character Growth Index Mark A. Liston and Marvin W. Berkowitz University of Missouri-St. Louis Authors Note Mark A. Liston, College of Education, University of Missouri-St. Louis; Marvin W. Berkowitz, College of Education, University of Missouri-St. Louis. Mark A. Liston is now at the Liston Group, Joplin, MO. Data for this study was gathered for Mark A. Liston’s doctoral dissertation. Research for this study was supported in part by the Center for Character and Citizenship, University of Missouri-St. Louis. Correspondence concerning this article should be addressed to Mark Liston. E-mail: [email protected]Commented [1]: Should I add the foundation that gave me scholarships while doing the PhD? The Center? Liston Group?
54
Embed
Is Character Development Quantifiable? Mark A. Liston … · · 2017-04-26Is Character Development Quantifiable? ... (e.g., The Big Five personality construct; John & Naumann, 2010;
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Is Character Development Quantifiable?
Creation and Validation of the Character Growth Index
Mark A. Liston and Marvin W. Berkowitz
University of Missouri-St. Louis
Authors Note
Mark A. Liston, College of Education, University of Missouri-St. Louis; Marvin W.
Berkowitz, College of Education, University of Missouri-St. Louis.
Mark A. Liston is now at the Liston Group, Joplin, MO.
Data for this study was gathered for Mark A. Liston’s doctoral dissertation. Research for
this study was supported in part by the Center for Character and Citizenship, University of
Missouri-St. Louis.
Correspondence concerning this article should be addressed to Mark Liston. E-mail:
& Malone, 2009), in large part due the absence of both a consensual conceptualization and a
psychometrically sound measure. In 2009, the US House Committee on Education defunded
character education (USHR, 2012) stating,
“The Partnerships in Character Education program has not received funding since FY
2009. The program has an extremely limited impact (italics added) … The [USDE]
conducted a review of more than 40 Character Education programs (Person, Moiduddin,
Hague-Angus, & Malone, 2009) … and found only two with positive results” (USDE,
2012b, footnote 17).
The Person et al. report focuses on the problems the Character Education programs
encountered in measuring their effects and outcomes. Logically the evaluation of a character
training program should include measuring participants’ character in its multiple aspects. Of the
36 evaluated programs in the Person et al. report (2009), only one used a multidimensional
character assessment. Presumably this is due to the lack of such a valid, reliable instrument. The
report’s recommendation was that Character Education needed 1) a conceptual basis including a
unified character taxonomy and 2) quantitative assessment tools to measure both character
strength and growth.
Measuring a multidimensional concept (character) by identifying its multidimensional
components (virtues or strengths) in order to construct a valid multidimensional test is both
necessary and challenging (Diener et al., 2010; Duckworth & Quinn, 2009; Lopez & Snyder,
2003). As Park and Peterson (2006b) said, “Character strengths are complex constructs that
Commented [3]: The quote suggests that the reason was no effects, no the inability to identify measures. Need to spin this differently. Perhaps say the decision to defund was due to very limited effects in existing research. THEN say that perhaps having a valid, reliable measure may be sensitive enough to reveal effects.
Commented [4]: Is this good?
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 5
require comprehensive measures” (p. 902).
At this time, multi-dimensional character growth cannot be quantified (Hanson, Dietsch,
& Zheng, 2012). This article first reviews the research fields that emphasize character, their
conceptualization of character, and character assessments. Second, a unified taxonomy is
created and a multidimensional character measure is developed and validated that hold potential
to assess character growth.
Conceptualizing character. Character theory has focused on defining character by its
components or strengths (Peterson & Seligman, 2004). Though Aristotle seemed to believe the
task of creating the definitive taxonomy of character strengths was impossible, many have
attempted to do so over the centuries (Peterson & Seligman, 2004). While other areas of
psychology have broad acceptance of a taxonomy (e.g., The Big Five personality construct; John
& Naumann, 2010; Peterson & Seligman, 2004), no such construct is widely accepted for
character.
Three fields that emphasize and study character have been prominent in the past 30 years:
Youth Development (PYD; Leffert et al., 1998; Scales & Leffert, 2004; Search Institute, 1997);
and Positive Psychology (PP; Peterson & Seligman, 2004). Though researchers in each are
aware of the others, collaboration seems uncommon (Liston, 2014).
Character education (CE). CE has been shown in repeated studies to have positive
outcomes in student behavior and academic achievement and school culture (Tatman,
Edmonson, & Slate, 2009). As a field of research, certain efforts to conceptualize character are
notable (Berkowitz & Bier, 2006). The John Templeton Foundation has funded many scholarly
Commented [5]: Here you can begin by reflecting on the taxonomies that drove the measures you have just reviewed. Then say a common denominator taxonomy would help the field. Then describe how you did that.
Commented [6]: Done
Commented [7]: I still think you are better off just calling this Developmental Assets. Unless you want to mean a broader field including that. Then you need to cite more than the Search Institute. This is likely what got you in hot water with Rich originally.
Commented [8]: See edits and note below.
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 6
studies of 16 character strengths identified in their charter (JTF, 2013). JTF does not claim these
to be a definitive construct though it is largely congruent with PP’s 24 strengths.
In 1992, the Josephson Institute (JI) sponsored a conference “to formulate a nonpartisan,
non-sectarian framework for character development. The result was the Aspen Declaration,
which created a list of shared ethical values...” (Josephson, 2009). Thirty scholars and advocates
crafted by consensus six primary character traits “...that transcend cultural, religious, and
socioeconomic differences” (Josephson, 2011). JI launched Character Counts to promote these
“Six Pillars of Character.” The pillars serve as meta-traits that encompass 24 character strengths
(Josephson, 2011) in marked parallel to the VIA model (see below; Peterson & Seligman, 2004).
Lickona and Davidson developed CE’s most extensive and nuanced conceptualization
with three constructs. They state that these “represent a conceptual progression”: Ten Essential
Virtues (Lickona, 2004); Eight Strengths of Character that “operationalize” the virtues (Lickona
& Davidson, 2005); and 65 comprehensive strengths Character = Values In Action (Davidson &
Lickona, 2009).
Positive Youth Development (PYD). PYD was begun in the 1980’s by youth workers,
organizations, and researchers who combined sociological, educational, and psychological
[vocabulary’s] definition focuses on some combination of (and the interactions among) five core
constructs,” one of which includes “the child’s developmental strengths (attributes including
skills, competencies, values, and dispositions)” (p. 2). This core construct is termed “character”
by one of the four vocabularies (p.2). Another vocabulary, the Search Institute’s Developmental
Assets, places in this construct 20 “internal developmental assets” as “personal characteristics
and behaviors” (p.2) that could be considered either character strengths or indications of the
presence of strengths (Leffert et al., 1998; Scales, Benson, Leffert, & Blyth, 2000; Search
Institute, 1997, n.p.).
Positive Psychology (PP). Arguably the best construct of character in the past 15 years
emerged from Positive Psychology (Linley et al., 2007; Toner, Haslam, Robinson, & Williams,
2012). PP has the most extensive theory of character with an explicit, detailed conceptualization
and system of classification. In Character Strengths and Virtues (2004) Peterson and Seligman
analyzed the best existing character concepts and lists of strengths gathered from antiquity and
recent psychology. They reviewed philosophers, religions, and various cultures world-wide to
gather universally-acknowledged virtues. Peterson and Seligman defined character strengths as
“the psychological ingredients – processes or mechanisms – that define the virtues” (p. 13).
With their team of scholars, they created 11 criteria by which they determined what qualified as
a character strength. Twenty-four such strengths were selected and grouped into six classic
virtues: courage, wisdom, humanity, justice, temperance, and transcendence. Though they admit
the construct is nascent, it is the most thoroughly developed and broadly researched character
taxonomy available (Linley et al., 2007; Toner, Haslam, Robinson, & Williams, 2012).
[MAY NEED MORE ON PP CONCEPTUALIZATION]
Commented [9]: Here is where this confusion is most apparent. You define PYD widely and then reduce it to one (somewhat anomalous) model, 40 developmental assets.
Commented [10]: Is this good?
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 8
Multidimensional character assessments. Valid measures of certain single character
strengths are available (Heppner, Witty, & Dixon, 2004; Lopez & Snyder, 2003; Sackett, &
Wanek, 1996). Many were not developed from a central character construct and have no
established norms. Measuring one character strength in isolation from other strengths and apart
from a construct limits the measures’ practicality and validity (Lopez & Snyder, 2003; Park &
Peterson 2006b). Most character strengths have high degrees of correlation and their definitions
often overlap (McGrath, Rashid, Park, & Peterson, 2010). Simply combining existing measures
of singular strengths to comprehensively assess character is therefore not likely to produce a
valid and reliable instrument although it is possible (Theokas et al., 2005). The following is a
brief review of available character measures.
Character Development Survey. In reviewing the 32 scales from published research
listed in the Person et al. study, only one specifically measured character: The Character
Development Survey (CDS, Johns, 1997). The measure proved reliable in its initial use by its
author but had no validation, the “Gold Standard of test evaluation” (Haynes, Richard, &
Kubany, 1995, p. 239).
Character Traits. Dr. Clete Bulach developed one of only two measures on the
Character.org website (formerly Character Education Partnership) under their “Individual
Assessment” list that specifically says it is a measure of multidimensional character
(http://www.character.org/more-resources/assessment-tools/individual/). To develop this
instrument, Dr. Bulach asked 130 teachers what they would see or hear if one of the 16 character
traits were present. He then used the behaviors they identified to form the survey's items.
Called simply Character Traits, the scale “consists of 96 behaviors used to measure students’
perceptions of their peers’ behavior on 16 character dimensions” (Bulach, 1996, 2002). Rather than
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 9
assessing the student’s own character, items ask the student’s perceptions of peers’ behavior.
The instrument tested 462 students grades 3, 4, 7, and 10, achieved .96 reliability,
In an unpublished manuscript, Bulach (2000) reported his factor analysis revealed three
factors he entitled school community relations' behaviors, curriculum related behaviors, and
internal relations' behaviors. Bulach stated, “There is little data on the construct validity of this
instrument” (p.4).
Profiles of Student Life: Attitudes and Behavior Scale. Leffert et al. (1998) used the
Search Institute’s 20 internal assets and 20 external assets to create the Profiles of Student Life:
Attitudes and Behavior Scale (PSL-AB; Leffert et al., 1998; Scales et al., 2000). This measure
fostered research and advancement in conceptualizing Positive Youth Development. PSL-AB is
broad, including concepts regarding community, family relations, academics, and activities and
thus could not be considered a character assessment.
Theokas et al. (2009) state that PSL-AB has metric challenges: “Limitations associated
with single-item asset measures and the limited reliability of some multi-item asset measures
included in the survey have prevented the organization of the asset items into empirically derived
superordinate categories or domains” (p.5).
VIA Measures. From the construct detailed in Character Strengths and Virtues, Peterson
and Park developed the Values In Action Inventory of Strengths (VIA-IS; 2003). The purpose of
VIA-IS is to help individuals identify their “signature character strengths” (Park & Peterson,
2009). VIA-IS and its adolescent version (VIA Youth Survey; Park, 2005) became the first
measure of multidimensional character based on a strong construct. Over a million people have
taken these two assessments online (Linley et al., 2007).
The VIA surveys have some metric shortcomings that have limited their validity. These
Commented [11]: I want to keep PYD as the field but, as with Character Education, use the conceptualization closest to character strengths for the list. Does this work?
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 10
include their length (224 and 198 items respectively), design as an ordinal (vs. numeric)
measure, reading level, content and discriminant validity, and prototypicality ratings (Liston,
2014; McGrath, 2015; Park & Peterson, 2006b). A specific concern regarding the latter is that
exploratory factor analysis of VIA-IS indicates that many items’ structural coefficients were
deficient, double-factored, or did not have their highest score in the intended factor (Brdr &
Kashdan, 2010; Linley, et al., 2007; Liston, 2014; Park & Peterson, 2006b).
Despite these limitations, the VIA measures arguably are the best available character
assessment tools. Their items and subscales can inform character research in many ways. First,
they are based on the best conceptualization in psychology and education (Snyder & Lopez,
2007). Second, they are the most comprehensive and widely-used items and subscales available
(Brdr & Kashdan, 2011; McGrath, Rashid, Park, & Peterson, 2010). Third, factor analysis is
possible with an ordinal measure and exploratory factor analysis of these measures indicated
factor strength through adequate to good coefficient alphas (Brdr & Kashdan, 2010; Linley et al.,
2007; McGrath, 2010; Park & Peterson, 2006a, 2007, 2009).
The VIA measures’ purpose is to reveal one’s greatest or “signature strengths” at the time
the test is taken. They have not yet been studied to determine their ability to measure character
development. In a personal conversation with co-creator of both VIA instruments Nansook Park
(July 26, 2011), she disclosed that VIA focused more on signature strengths that are consistent
throughout one’s lifetime than on character growth that develops with effort and time. She
specified that some items assess more trait-like or personality-based features of character
strengths and would probably not be likely to show development. VIA Education Director Dr.
Ryan Niemiec stated that the VIA Youth Survey is “not optimal for measuring character growth
due to: (a) Insensitivity to variations in character trait use; (b) A ‘ceiling effect’ when one scores
Commented [12]: I thought McGrath could only get 3 factors out of these? Shouldn’t that be discussed as a limitation? Also sets up your finding that it has a solid factor structure.
Commented [13]: It’s not a limitation as the purpose of most factor analysis is factor reduction.
Commented [14]: Need to say something about the difference between measuring character and measuring character development. May not be obvious to the reader.
Commented [15]: DON’T YOU THINK THAT IS COVERED IN THE FOLLOWING DISCUSSION?
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 11
high in a trait initially; [and] (c) The VIA surveys’ … design for ordinal results, i.e., the rank-
ordering of strengths rather than measuring the level of each strength for later comparison to
measure growth” (personal correspondence, February 27, 2013).
Primary dimensions of character. This study reviewed the most widely accepted
models in order to generate an integrative taxonomy to determine the primary character
strengths. This was accomplished in two steps. First, expert opinion was considered from the
three fields of character and five lists were selected. PP was represented by Peterson and
Learning Love lrn’g Life learner Engage.*, Read* 3
Optimism Hope Pos. Attitude Positive Future 3
Wisdom x x Plan'g/ decsnmkg 3
Gratitude x # x 2.5
Leadership x #Initiative x 2.5
Peace #Patience ^^ Pers Pwr*, Cnfl Rs* 2
Curiosity x ^^ School Engage* 2
Confidence ^^ Self-esteem 1.5
Creativity x ^^ 1.5
Zest x ^^ 1.5
Wonder/Exce
l x 1
Humor x 1
Coding Explanation: Some cells contain similar words the expert used to represent that strength.
x means the expert’s strength name is similar or identical to the collective name (1 point). If the
cell is blank, the expert did not include the strength. a # means the strength is taught as an aspect of one of the Six Pillars (½ point).
b ^^ means the strength is on Lickona and Davidson's larger list of over 65 strengths (½ point). c * means the strength is an aspect of one of the 20 Assets (½ point).
If strengths are too similar, items created to measure each could factor together. The 29
strengths were evaluated for such synonymy. Additional expert opinions were considered (A.
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 13
Duckworth, personal communication, February 23, 2013; Lee & Ashton, 2006; T. Lickona,
personal communication, March 7, 2012; Worthington & Scherer, 2004). These processes
resulted in the following decisions:
● Thirteen strengths were retained as listed in the grid: Love, Honesty, Self-Control,
● Wonder/Excellence and Humor were removed because they were only listed by Positive
Psychology.
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 14
This procedure produced 18 strengths that were hypothesized to cover the primary
dimensions of character. Each was defined by integrating the experts’ concepts of each strength.
The strengths are: Cooperation, Courage, Creativity, Forgiveness, Gratitude, Honesty, Humility,
Kindness, Love, Optimism, Peace, Perseverance, Respect, Responsibility, Self-Control, Social
Intelligence, Spirituality, and Wisdom (These strengths will be capitalized to denote proper
names in this article). [GOOD OR BAD IDEA?]
Assessing multidimensional character. The second initial step to measure character
growth is to create an assessment that operationalizes and tests the conceptualization. Roth-
Herbst, Borbely, and Brooks-Gunn (2007) have concluded that “little work… has attempted to
create a reliable and valid scale to measure the many components implied by the term
‘character’” (p. 175). No developmental norms or trajectories for levels of character strength
have been established (Lopez & Snyder, 2003). In fact, the concept of character growth has not
been psychometrically established (Hanson, Dietsch, & Zheng, 2012; Lerner & Callina, 2014).
An essential consideration in creating this assessment was to enable character
development or growth to be measured. This requires care in establishing the definitions of the
strengths and the methods by which they are quantitatively assessed (Hanson, Dietsch, & Zheng,
2012; Leffert et al., 1998; Lerner & Callina, 2014). Peterson and Seligman’s (2004)
conceptualization of character strengths seems to include their development over time. Their
stance “…recognizes individual differences that are stable and general but also shaped by the
individual’s setting and thus capable of change” (p. 10). They insist that “positive traits need to
be placed in context…” (p. 11). They locate strengths “…within people and people within their
settings…” (p. 11). They define “trait-like” is “being tonic (constant) versus phasic (waxing and
waning depending on their 'use'). A tonic characteristic (e.g., kindness or humor) shows itself
Commented [16]: This needs to be reworked to reflect the difference between “trait-like” and “state-like”. This is a significant departure of
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 15
steadily in a variety of settings...” (p. 11). This definition of trait-like differs from McCrae and
Costa’s concept of traits as fixed and biologically set (McCrae et al., 2000).
This conceptualization of PP character strengths allows some degree of development. It
may be argued that it is akin to current personality trait theory that recognizes research showing
personality develops gradually over long periods of time (Roberts, Wood, & Smith, 2005). The
trait-like concept of character strengths does not adequately describe rapid change in
performance or moral character that may be observed (Luthans, Avolio, Avey, & Norman,
2007). This is especially true during childhood and adolescence, known as stages of rapid
development (Roberts, Wood, & Smith, 2005).
The trait-like concept also does not consider the effects of circumstance or situation that
affects character expression (Hamaker, Nesselroade, & Molenaar, 2007). Theoretically,
character development may be gradual or rapid, often influenced by one’s environment (Lerner
& Callina, 2014). What is required to conceptualize such progress is a theory containing both
qualities required for development: (a) a degree of stability required for a strength to be sustained
over time yet also (b) a degree of variability to allow progress, change, and growth in character
(Baumeister, 2012). Without stability, attempts to measure character traits would be like
measuring one’s mood or energy level, known in trait-state theory as states of being with high
variability. That is, mood might circumstantially rise and fall without a developmental
progression. Without variability, such as in McCrae and Costa’s concept of traits, no
development can be expressed (Baumeister, 2012; Linley et al., 2007; Macdonald, Miles, &
Munro, 2008).
In contrast, the concept of character development as state-like contains both stability and
variability.
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 16
Theoretically, character strengths as defined by Peterson and Seligman (2004) should
develop over time and that development could be measured. Due to its gradual nature, this may
make such assessment difficult.
If character development is more state-like than trait-like, A character measure is needed
that: (a) is based on expert opinion and the best available constructs to determine the primary
character strengths; (b) is multidimensional, assessing those strengths with metric validity; and
(c) uses items designed to indicate both stability and variability in order to assess development.
This study endeavored to create such a measure.
This study’s research question is: Can a valid, reliable measure of multi-dimensional
adolescent character be developed? Adolescents were chosen due to (a) the rapid development
occurring during these years; (b) schools’ need for such a measure; and (c) the linguistic and
cognitive capacities of adolescents to understand such an instrument and engage in self-reflection
and self-assessment.
The project’s goals were:
1. To use the conceptualized primary character strengths to construct the Character Growth
Index (CGI) as a multidimensional character measure; and
2. To validate CGI.
Method
Two studies were designed to meet these goals.
Study 1:
Creating and field testing a measure. The first task involved an iterative process of
Commented [17]: And the linguistic and cognitive capacities of adolescents to understand such an instrument and to engage in self-reflection and self-assessment.
Commented [18]: Done
Commented [19]: Probably shouldn’t treat the taxonomy as a “study.” Perhaps present this piece of the project in the intro section as a conceptual analysis. Then create the study questions as what you will do with the taxonomy; e.g. create items and a reliable measure and then validate it. Hence, two studies, not three. Then write up the method as almost two method sections: Study 1 and Study 2.
Commented [20]: Is this good?
CREATION AND VALIDATION OF THE CHARACTER GROWTH INDEX 17
creating items to measure the 18 primary dimensions, field testing those items, analyzing the
results, adjusting definitions of the primary dimensions, improving the items, and repeat (Clark
& Watson, 1995). Items were created: (a) based on definitions provided by integrating expert
opinion; (b) to assess expression of the strength in current cognition, mood, and behavior; and (c)
that differentiate developmental progress rather than identify unchanging traits or fluctuating
emotional states (Clark & Watson, 1995; Cook & Campbell, 1979). Three to six items were
developed for each of the 18 primary dimensions of character with answers in a 5-point Likert
scale from “Very much like me” to “Not at all like me.”
Criteria required to determine a measure’s factorial structure vary among metric experts
but a common path can be found (Clark & Watson, 1995; Comrey & Lee, 1992; Costello &