8/21/2019 CMU - Teaching Portfolio Guide
1/175
Eberly Center for Teaching Excellence | 5000 Forbes Avenue, Cyert Hall 125 | Pittsburgh, Pennsylvania 15213
www.cmu.edu/teaching | 412.268.2896
www.cmu.edu/teaching
Guidelines for Teaching Portfolios
C
R
E
A
T
I
N
G
A
C
O
M
M
U
N
I
T
Y
O
F
E
D
U
C
A
T
O
R
S
What is a Teaching Portfolio? ..............................Page 1
Why do a Teaching Portfolio? ..............................Page 1
What goes in a Teaching Portfolio? ..................Page 1-2Reflective statement
Statement of teaching responsibilities
Supporting documents
Statement of teaching goals for the future
What kinds of supporting documentsshould I include? ............................................... Page 2-5
Your own teaching
Your students
Your colleagues, department, and institution
Your discipline or teaching in general
What kind of support is available to meas I compile my Teaching Portfolio? .....................Page 5
Examples of documents for theTeaching Portfolio ............................................Page 6-8
C O N T E N T S
http://www.cmu.edu/teaching/http://www.cmu.edu/teaching/8/21/2019 CMU - Teaching Portfolio Guide
2/175
Eberly Center for Teaching Excellence | 5000 Forbes Avenue, Cyert Hall 125 | Pittsburgh, Pennsylvania 15213
www.cmu.edu/teaching | 412.268.2896
www.cmu.edu/teaching
C
R
E
A
T
I
N
G
A
C
O
M
M
U
N
I
T
Y
O
F
E
D
U
C
A
T
O
R
S
Guidelines for Teaching Portfolios
Page 1
What is a Teaching Portfolio?T e teac ing port o io is a escription o
a professors major strengths and teaching achieve-
ments. It describes documents and materials which
collectively suggest the scope and quality of a profes-
sors teaching performance. (Seldin 1997). Because
teaching efforts and accomplishments are difficult to
capture solely by means of numerical summaries, the
concept o a port o io ecomes use u in t is arena. A
teac ing port o io goes eyon a ist to inc u e e a o-
rations an re ections. In t is sense it is simi ar to an
artists portfolio, because it showcases your best work
and conveys a sense of yourself and your vision.
Why do a Teaching Portfolio?The goal for a teaching portfolio is twofold.
At its core, compiling a teaching portfolio is essen-
tially a reflective activity. The very process of writ-
ing down your teaching philosophy and corroborat-
ing it with the appropriate evidence automatically
causes you to reflect on your teaching. It can help
clarify your goals, underscore your development asan e ucator, an ig ig t areas or urt er growt .
However, port o ios are a so use y t e a minis-
tration for the assessment of your teaching perfor-
mance. Therefore, you are writing both for yourself
and for an external audience, for the purposes of
self-reflection and summative evaluation.
What goes in a Teaching Portfolio?Teaching portfolios generally exhibit a simi-
ar structure, ut t e in ivi ua ocuments inc u ecan vary signi icant y. You wi want to p ay to your
strengths as you showcase the breadth of your accom-
plishments, so you should think about what they are
and how to best represent them. The structure of a
teaching portfolio consists of four broad categories:
{ Reflective statement} Also called the teaching or
ducation philosophy, this document frames the
whole portfolio. It communicates who you are as a
teac er: ow you e ine e ucation, ow you concep-
tua ize your ro e an t e earners ro e in an out o
the classroom, and how you translate your beliefs
nto action (typically 1 or 2 pages, but you shold
heck with your Department Head).
{ Statement of teaching responsibilities } This doc-
ument should communicate at a glance the breadth
f the instructors teaching activities. It should be
presented in a tabular format, and possibly support-
d by a narrative in the appendix. The table should
nclude every teaching assignment over the past 3
ears, inc u ing:
O icia teac ing assignments
Ot er teac ing activities e.g., An rew s Leap
Supervisory activities e.g., in epen ent stu -
es, SURG grants)
For each teaching assignment, the information that
s typically included consists of:
type of course (e.g., studio, lab, large lecture)
umber of students enrolled
breakdown of majors/non-majors or under-
raduate/graduate if relevant
numerical summary of FCE ratings
whether the course has been developed or
u stantia y revampe y t e instructor.
The other 2 categories should include information
appropriate to eac activity. T e ta es rom yourannua reports are a goo p ace to start to create t is
ocument. In addition, check with your Department
Head for departmental guidelines (up to 3 pages).
{ Supporting documents } This section will provide
vidence for the claims in the philosophy and is
http://www.cmu.edu/teaching/http://www.cmu.edu/teaching/8/21/2019 CMU - Teaching Portfolio Guide
3/175
Eberly Center for Teaching Excellence | 5000 Forbes Avenue, Cyert Hall 125 | Pittsburgh, Pennsylvania 15213
www.cmu.edu/teaching | 412.268.2896
www.cmu.edu/teaching
C
R
E
A
T
I
N
G
A
C
O
M
M
U
N
I
T
Y
O
F
E
D
U
C
A
T
O
R
S
Guidelines for Teaching Portfolios
Page 2
really the portfolios body. This is the section most
similar to the artists portfolio. For guidelines on
how to organize the documents in this category, see
t e Supporting Documents section e ow.
{ Statement of teaching goals for the future } T e
port o io s ou e orwar - oo ing, wit i eas
or uture action ue e y re ections on t e past
and the present. Just like a researcher needs to
have a research agenda, an educator should be able
to articulate how she wants her teaching to grow.
For instance, you might talk about working on a
specific challenge in the classroom, or the wish to
create a summer program to reach out to talented
high-school students, and so on. Some instructors
pre er to integrate t is statement into t e teac ing
p i osop y. In t is case, t e goa s s ou e c ear y
highlighted within that document (1 page).
What kinds of supporting documentsshould I include?
ust like with an artists portfolio, the aim is
not to be comprehensive. A novice instructor might
not have copious material to showcase, whereas
some o y w o s een teac ing or a ong time mig t
c oose to e more se ective in er c oices. Some
people might opt to showcase the most significant
contributions thorough the years, others might wish
to emphasize their most recent work. Others still,
might choose to emphasize development, starting
from a challenge in their own teaching, steps they
took to overcome it, and finally evidence of having
successfully navigated the challenge. Whatever theirection c osen, t e port o io is a ynamic ocu-
ment an it wi evo ve over t e course o your ca-
reer. U timate y, it s ou emonstrate commitment
to and/or impact on:
. Your own teaching. This section aims to demon
trate that the instructor is engaged in an ongoing
reflection about her teaching.
. Your students. T ere is no teac ing wit ou
earning. The best teachers inspire, motivate, and
reate an environment w ere earning can occur
T is section aims to emonstrate a t is.
. Your colleagues, department and institution
Teaching takes place in a broader context than the
lassroom. Therefore, it should be informed by and
nform the departments curricular goals and by thenstitutions pedagogical mission. This section aims
o demonstrate all this.
4. Your discipline or teaching in general. The bes
eaching is also informed by a broader scholarship
whether disciplinary (e.g., Science Education), cross
isciplinary (e.g., First Year Programs) or specific ped
agogies e.g., Service Learning . T is section aims to
emonstrate your participation in t ose ia ogues.
In a t ese imensions t ere is a trajectory
rom commitment to impact, and instructors wil
be at different points along the four continua de
ending on where they are in their career. For in
tance, beginning instructors (years 1 to 3) would
be expected to show commitment to reflecting on
heir own teaching, to their students, and to learn
ng about educational goals and practices of the in
titution and the discipline. At a later stage (years
4 to 6 , t e process o re ection s ou resu t in
oncrete c anges imp emente in t e course, suc
as the development of new materials, or the integra
ion of new technologies and pedagogies. Instructors
hould have some preliminary measures of the im
act of such changes. Advanced instructors should
be able to document a demonstrated impact on
tudent learning and motivation, as well as broader
http://www.cmu.edu/teaching/http://www.cmu.edu/teaching/8/21/2019 CMU - Teaching Portfolio Guide
4/175
Eberly Center for Teaching Excellence | 5000 Forbes Avenue, Cyert Hall 125 | Pittsburgh, Pennsylvania 15213
www.cmu.edu/teaching | 412.268.2896
www.cmu.edu/teaching
C
R
E
A
T
I
N
G
A
C
O
M
M
U
N
I
T
Y
O
F
E
D
U
C
A
T
O
R
S
Guidelines for Teaching Portfolios
Page 3
participation in pedagogical activities outside their
own classroom, possibly up to the national level
(e.g., taking active roles in the education section
o t eir iscip inary association . Some exceptiona
instructors mig t a so e a e to emonstrate recog-
nition of their impact. This could take the form of
awards, internal or national, being invited to speak
on teac ing topics, an so on. The figure below
synthesizes this model.
Different types of evidence are appropriate
at each stage. For instance, a record of attendance
to teaching development seminars might be enough
to demonstrate commitment to your own teaching,
but in order to demonstrate impact you will need toarticu ate ow t e now e ge rom t ose seminars
trans ates into pe agogica practices in your courses.
The type of evidence presented can vary even with-
in the same stage, depending on what is appropriate
to the type of course and discipline. For instance,
if you are trying to document student learning and
performance, you might have indirect measures
(e.g., the words of an external observer saying that
the students appeared to be learning) and more di-
rect ones (e.g., samples of student work, or pre-posttests in appropriate omains .
So how do you select documents for the
portfolio?As a genera ru e, as you gat er more evi-
dence of impact, this evidence gradually replaces the
evidence of commitment. Likewise, more direct mea-
sures should be preferred to indirect ones when avail-
able and appropriate. Following is a list of suggested
indicators for the four dimensions. Sometimes t e
way to ocument t ose in icators is se -evi ent e.g.,
stu ents scores on tests ; or cases where it is not im-
ediately clear how to document them, possible evidence
ollows in italics.
. Your Own Teaching
Participation in an association con-
erned with the improvement of
eaching and learning (e.g., AAC&U,AERA, SENCER, Education section of
isciplinary association)
Taking/auditing courses relevant to
eaching (e.g., cognitive psychology,
group ynamics, pu ic spea ing,
u tura iversity
See ing ee ac rom co eagues or E er y
enter
tatement from colleagues or Eberly Center records
Observing other classes Evidence of reflection on your own teaching
(Course logs)
Participation in seminars and workshops to
mprove teac ing
Records from Eberly Center or other relevant
rgan zat on
. Your Students
a. Student Learning and Performance
Stu ents scores on tests, i e pre-post tests Stu ents a oratory wor oo s an ogs
Student essays, creative work and projects or
ieldworks reports
Demonstration of change in student per-
ormance (e.g., successive drafts in light of
rading criteria or rubrics)
Ones Own
Teach ng tudents
olleagues |Department |
Inst tut on
D sc pl ne oreach ng n
eneralommitment
Impact
http://www.cmu.edu/teaching/http://www.cmu.edu/teaching/8/21/2019 CMU - Teaching Portfolio Guide
5/175
Eberly Center for Teaching Excellence | 5000 Forbes Avenue, Cyert Hall 125 | Pittsburgh, Pennsylvania 15213
www.cmu.edu/teaching | 412.268.2896
www.cmu.edu/teaching
C
R
E
A
T
I
N
G
A
C
O
M
M
U
N
I
T
Y
O
F
E
D
U
C
A
T
O
R
S
Guidelines for Teaching Portfolios
Page 4
Research on the impact of changes to thecourse
Reports or published papers
Instructional and assessment materials
developed (e.g., handouts, projects, grading
rubrics, pre-post tests)
Student ratings of own learning (From FCEs)
Student comments from FCEs related to
their own learning
Unsolicited letters from students about their
own learning
Interview data collected from students, suchas SGIDs or focus groups
Made available to instructor confidentially by
Eberly Center; her choice whether to submit them
or not
Statements from colleagues who have ob-
served the instructor teach or who have
observed the quality of student work
Statements from teaching consultants who
observed the instructor teaching
Made available to instructor confidentially by
Eberly Center; her choice whether to submit them
or not
Statements on the preparedness of instructors
students by instructors who teach courses
downstream
Letters from alumni about impact
Reports on students preparedness by their
employers
b. Student Inspiration/Motivation
Record of students who elect another course
with same instructor (e.g. letters from students)
Evidence of the effect of courses on student
major/career choices
From student letters
Letters from alumni about impact Student ratings of satisfaction with course/
instructor (from FCEs)
Student comments from FCEs
Unsolicited letters form students
Interview data collected from students, such
as SGIDs or focus groups
Made available to instructor confidentially by
Eberly Center; her choice whether to submit them
or not
Statements from colleagues who have ob-
served the instructor teach Statements from teaching consultants who
have observed the instructors teaching
Made available to instructor confidentially by
Eberly Center; her choice whether to submit them
or not
Early evaluations and evidence of responsive-
ness to student feedback
Evidence of effective supervision of honors,
masters or PhD theses, independent studies,
5th year scholars, SURG grants (from student
letters)
Information on instructor availability to
students (above and beyond office hours, in
person or online) (Self-report, or from students
letters and comments on FCEs)
3. Your Colleagues/Department/Institution
Internal education awards
Letters from Dean praising the instructors
teaching
Adoption of instructors innovation bybroader curriculum (e.g. template syllabus for
all sections of intro course, labs developed by
instructor still used in the course when other
people teach it)
http://www.cmu.edu/teaching/http://www.cmu.edu/teaching/8/21/2019 CMU - Teaching Portfolio Guide
6/175
Eberly Center for Teaching Excellence | 5000 Forbes Avenue, Cyert Hall 125 | Pittsburgh, Pennsylvania 15213
www.cmu.edu/teaching | 412.268.2896
www.cmu.edu/teaching
C
R
E
A
T
I
N
G
A
C
O
M
M
U
N
I
T
Y
O
F
E
D
U
C
A
T
O
R
S
Guidelines for Teaching Portfolios
Page 5
Instructional or assessment materials devel-
ped (e.g., syllabi, educational software)
Developing new courses or revamping old
nes (self-report, syllabi)
Deve oping course port o ios to aci itate t e
task of future instructors teaching the course
Evidence of effectiveness of help given to
olleagues or graduate students on teaching
mprovement, informally or through Eberly
enter or other workshops (from faculty and
graduate student letters, Eberly Center database)
Serve on educational or curriculum develop-ment revision committees
Organizing or ea ing teac ing eve opment
pportunities TA training or orientation,
seminars in instruction, teaching teas) (self-
report, letters from participants)
Informal help given to colleagues or graduate
students on teaching improvement (self-report
of time and task involved, colleague and graduate
student letters)
Evidence of success of internship, co-op, sum-mer aca emy programs set up y instructor
Setting up interns ip program, or co-op, sum-
mer academy (self-report or program documentation)
4. Your Discipline or Teaching in General
Awar s rom externa institutions
Invitations to teach for outside agencies
Instructional or assessment materials (e.g.,
textbooks and instructors manuals)
Publications on teaching (research articles, op-
d columns) (published articles, internal reports
for work not yet accepted for publication, or
nvitations to contribute for work not done yet)
Other kinds of invitations based on ones
reputation as a teac er (e.g., a magazine or radio
interview)
Recor s o a option o own text oo y
t er peop e
Editing or contributing to a professional
ournal on teaching ones subject
Evidence of success of internship, co-op sum-
mer program with external students (number
f applicants, student evaluation of satisfaction,
uture course selection if tracked)
What kind of support is available to me as Icompile my Teaching Portfolio?
ec wit your Department Head or epart-
mental or college guidelines. Some departments have
written documents. Your entorsmight be another
possible source of support. Finally, the Eberly Center
s available for consultations as well.
http://www.cmu.edu/teaching/http://www.cmu.edu/teaching/8/21/2019 CMU - Teaching Portfolio Guide
7/175
Eberly Center for Teaching Excellence | 5000 Forbes Avenue, Cyert Hall 125 | Pittsburgh, Pennsylvania 15213
www.cmu.edu/teaching | 412.268.2896
www.cmu.edu/teaching
C
R
E
A
T
I
N
G
A
C
O
M
M
U
N
I
T
Y
O
F
E
D
U
C
A
T
O
R
S
Guidelines for Teaching Portfolios
Page 6
Examples of documents for the TeachingPortfolio
{ Reflective statements/teaching philosophies }
The following pages include three examples of re-
flective statements, selected from both tenure and
teaching tracks. They are from multiple disciplines
and all look different, but they all convey a sense of
the authors teaching philosophy.
Steven Rudich, Computer Science
This statement is very concise, but it clearly
communicates what the instructor values in
teaching, and how he translates his values into
classroom pedagogy.
Pamela Lewis, Heinz School
This statement blends text and diagrams to pro-
vide a sense of the evolution of the instructors
self-concept through the years. It also com-
municates that she is familiar with some of the
teaching literature. This statement includes the
statement of teaching goals as well.
Peggy Knapp, English
This statement demonstrates development
through a different model, an initial philosophy
(dated 1999) plus an addendum to reflect her
recent accomplishments(dated 2003).
{ Supporting documents }
The following documents highlight some points
across the continuum from commitment to impact.
The list is not comprehensive, but is meant to get
you started thinking about the possibilities. Not all
documents have actually appeared in a packet yet,
but all could be included. Again, the selection draws
from both tenure and teaching track faculty.
Instructional and assessment materials
developed
Handout: How to write a philosophy
paper by Maralee Harrell, Philosophy
This handout reveals an understanding of
the level of writing and argumentation skills
first-year students possess, and a commitment
to proactively helping them develop further.
Grading rubric by Shelley Evenson, Designand HCII
The significance of the rubric is to clarify
expectations and grading criteria for the
students in order to facilitate their study and
their learning.
Grading According to a Rubric, Maralee
Harrell, Philosophy (published in Teaching
Philosophy)
Publishing a paper on your own grading
rubric is evidence of impact on the disciplin-ary discourse.
http://www.cmu.edu/teaching/http://www.cmu.edu/teaching/8/21/2019 CMU - Teaching Portfolio Guide
8/175
Eberly Center for Teaching Excellence | 5000 Forbes Avenue, Cyert Hall 125 | Pittsburgh, Pennsylvania 15213
www.cmu.edu/teaching | 412.268.2896
www.cmu.edu/teaching
C
R
E
A
T
I
N
G
A
C
O
M
M
U
N
I
T
Y
O
F
E
D
U
C
A
T
O
R
S
Guidelines for Teaching Portfolios
Page 7
Examples of documents for the TeachingPortfolio
Research on the impact of changes to the
course
Using Argument Diagrams to Improve
Critical Thinking Skills in Introductory
Philosophy, Maralee Harrell, Philosophy,
technical report
This report documents the positive impact of
a specific innovation the instructor intro-
duced in her class by means of a statistical
data analysis of student performance.
Implementing a Computerized Tutor in a
Statistical Reasoning Course: Getting the
Big Picture, Oded Meyer, statistics, Paper
presented at the American Association for
Educational Research conference.
In this case, the claim of impact is validated
by acceptance for presentation and by the
educational/scientific community.
Adoption of instructors innovation by broader
curriculum
A Successful Peer Writing Assistant
Program, Bonnie Youngs and Anne Green,
Modern Languages, paper published in the
Foreign Language Annals
This article documents the key features of a
program the two faculty members designed,
which has been very successful and is now
used by instructors in all seven languages
taught in the Modern Languages department.
Developing New Courses or Revamping Old
Ones
Tutoring for Community Outreach: A
Course Model for Language Learning and
Bridge Building between Universities and
Public Schools, Susan Polanski, Modern
Languages, paper published in the Foreign
Language Annals
This article describes a new course marry-
ing disciplinary content and service-learning
pedagogy.
Organizing or leading teaching development
opportunities
Have your cake and eat it too, lecture
on how to give lectures by Steven Rudich,
Computer science
This seminar, given to faculty and graduate
students in Computer Science, documentsthe instructors commitment to the depart-
ment. Evaluations of the seminar (not shown
here) serve as evidence how of well received
the lecture was by participants.
TA Handbook 2005-2006, Maralee
Harrell, Philosophy
The handbook documents the instructors
effort to clarify roles and responsibilities for
TAs in the department, as well as to equip
them with some valuable pedagogical knowl-
edge.
http://www.cmu.edu/teaching/http://www.cmu.edu/teaching/8/21/2019 CMU - Teaching Portfolio Guide
9/175
Eberly Center for Teaching Excellence | 5000 Forbes Avenue, Cyert Hall 125 | Pittsburgh, Pennsylvania 15213
www.cmu.edu/teaching | 412.268.2896
www.cmu.edu/teaching
C
R
E
A
T
I
N
G
A
C
O
M
M
U
N
I
T
Y
O
F
E
D
U
C
A
T
O
R
S
Guidelines for Teaching Portfolios
Page 8
Examples of documents for the TeachingPortfolio
Setting up internship, co-op, summer academy
etc
Andrews Leap, Steven Rudich, Computer
Science
This is a summer program for talented high-
school students to explore the frontiers of
computer science. The materials presented
document the instructors commitment to
outreach in Computer science education.
Records of advanced studies of participants
(not shown here) give evidence of impact on
students motivation and learning.
Allegheny Intermediate Unit Outreach
Program, Scott Matthews, Engineering and
Public Policy
This is a campus-based outreach program for
high-school students to familiarize them with
environmental principles and generate inter-est in related careers. The materials presented
document some of the impact in terms of
students increased environmental awareness.
Publications on teaching
A Statics Concept Inventory: Development
and Psychometric Analysis, Paul Steif et
al., Mechanical Engineering
This paper, published in the Journal of
Engineering Education, documents a test
developed by the author and used at many
universities to record conceptual progress in
the learning of statics.
Humorous Engineering 101, Larry
Cartwright, Civil and Environmental
Engineering (award-winning paper at the
ASEE conference 2001)
This paper is evidence not only of national
impact but also national recognition.
What We Know about Learning, Herb
Simon, Psychology (1997 Frontiers inEducation conference keynote address)
This paper too is evidence of national impact
and recognition, across fields.
Other kinds of invitations based on ones
reputation as a teacher
Laura Lee: Bridging the Gap between
School and Practice, in Direct Connection,
volume 3, issue 2, December 2000
This interview demonstrates the instructors
influence on architecture education nationally.
http://www.cmu.edu/teaching/http://www.cmu.edu/teaching/8/21/2019 CMU - Teaching Portfolio Guide
10/175
8/21/2019 CMU - Teaching Portfolio Guide
11/175
8/21/2019 CMU - Teaching Portfolio Guide
12/175
8/21/2019 CMU - Teaching Portfolio Guide
13/175
8/21/2019 CMU - Teaching Portfolio Guide
14/175
8/21/2019 CMU - Teaching Portfolio Guide
15/175
8/21/2019 CMU - Teaching Portfolio Guide
16/175
8/21/2019 CMU - Teaching Portfolio Guide
17/175
8/21/2019 CMU - Teaching Portfolio Guide
18/175
8/21/2019 CMU - Teaching Portfolio Guide
19/175
8/21/2019 CMU - Teaching Portfolio Guide
20/175
8/21/2019 CMU - Teaching Portfolio Guide
21/175
8/21/2019 CMU - Teaching Portfolio Guide
22/175
8/21/2019 CMU - Teaching Portfolio Guide
23/175
8/21/2019 CMU - Teaching Portfolio Guide
24/175
8/21/2019 CMU - Teaching Portfolio Guide
25/175
8/21/2019 CMU - Teaching Portfolio Guide
26/175
8/21/2019 CMU - Teaching Portfolio Guide
27/175
1
How to Write a Philosophy PaperBy Mara Harrell
How do I know what I think until I see what I say? EM Forster
Writing is important, especially to a liberal arts education, because in many ways the act ofwriting is the embodiment of the process of critical thinking. Consider what Douglas Soccio
1has
said about the subject:
Critical thinking is the conscious, deliberate, rational assessment of claims according to clearlydefined standards of proof (Soccio 2001, 39).
One of the signs of good critical thinking is the willingness to accept the best evidence, even
when it requires modifying or rejecting a cherished belief of highly desired conclusion (Soccio2001, 41).
Just as we could say that philosophy is the pursuit of wisdom (for philosophers are, literally,lovers of wisdom), we can say that philosophy is a deep and comprehensive inquiry into humanexperience. In order to perform this inquiry, philosophy relies on knowledge from other
disciplines, but also may go further in its investigation by analyzing the assumptions made byother disciplines, like what it means to be human, what it means to have knowledge, etc.
As such, philosophical inquiry is a special brand of critical thinking. As philosophers we are
intensely interested in what we should believe, what we should do, and how we are to know whatwe should believe and what we should do. The process of creating, analyzing, and evaluating
arguments helps us to determine what claims are most likely to be true, and thus helps us achieveour philosophical goals.
The goal of this handout is to help you to become comfortable writing philosophy papers. Many
of the ideas, directives, and admonitions are idiosyncratically my own, and may not be shared byall philosophy teachers. However, I do think that much of it will be useful to you not only in my
class, but in other philosophy courses you may take, and indeed in other college courses you maytake.
I. Vocabulary and Logic
In philosophy, we take precise definitions of words very seriously. Many philosophers havespent a good chunk of their lives arguing about what a particular word or phrase means. In this
spirit, I am going to introduce several technical terms that have particular meaning in logicalphilosophical discourse. These words may have different colloquial uses, so be conscientious
about using them properly in your philosophy papers.
Statement: A statementis a sentence that can either be true or false. Example: It will raintomorrow. We say that this sentence has a truth-value (either true or false), and perhaps we can
1Soccio, Douglas J. (2001)How to Get the Most Out of Philosophy, 4th
Edition. Belmont, CA: Wadsworth.
8/21/2019 CMU - Teaching Portfolio Guide
28/175
2
even know what the truth-value is. Not all sentences in English are statementsquestions,commands, and propositions, for example. Some non-statement sentences can, however, be
transformed into statements with some re-wording. Example: Be a doctor! can be usefullytransformed into You should be a doctor if the context permits.
Conditional Statement: A conditional statementis also a sentence that can be either true orfalse. They are special, though, because they occur so often in arguments. A conditionalstatement has two parts: the antecedentand the consequent. A conditional statement generally
has the form of an if, then statement, in which we have If [the antecedent], then [theconsequent]. Consider the following example:
Ifyou earned an A on the final paper, thenyou earn an A in the class.The entire statement is the conditional, you earned an A on the final paper is the antecedent,
and you earn an A in the class is the consequent.
Argument: An argumentis a set of statements, one of which is the conclusion, and the othersarepremises. The premises provide support for the conclusion. In other words, the conclusion
asserted to be true on the basis of the premises.Example: Premise: Either it will rain tomorrow, or it will be sunny tomorrow.
Premise: It will not rain tomorrow.Conclusion: It will be sunny tomorrow.
An argument can be good or bad based on (1) how well the premises support the conclusion, and(2) whether the premises are actually true.
Validity: A validargument is one in which it is not possible for the conclusion to be false if the
premises are true. This is a very bold statement, not about what is actually the case, but aboutwhat could possibly be the case. It is helpful, when considering validity, to consider the notion of
possible worlds. I can imagine a possible world in which grass is blue, and I can imagine apossible world in which trees are blue. But consider the following argument:
Premise: If grass is blue, then trees are blue.Premise: The grass is blue.
Conclusion: Trees are blue.There is no possible world in which the premises are true, but the conclusion false. Thus, this is a
valid argument.
Conversely, an invalidargument is one in which it is possible for the premises to be true and theconclusion false. Consider the following argument:
Premise: If you earned an A on the final paper, then you earn an A in the class.Premise: You earn an A in the class.
Conclusion: You earned an A on the final paper.It is possible for the premises to be true, but the conclusion false, if there are other ways to get an
A in the class, and you achieved one of them. For example, it may also be a policy in the classthat if your homework average is an A, then you earn an A in the class. In this case you can earn
an A in the class without earning an A on the final paper.On the other hand, the following argument is valid:
Premise: If you earned an A on the final paper, then you earn an A in the class.Premise: You earned an A on the final paper.
8/21/2019 CMU - Teaching Portfolio Guide
29/175
3
Conclusion: You earn an A in the class.Here, if the premises are true, then the conclusion must be true.
*It is important to note that, given our definitions, a statement cannot be valid or invalid, and an
argument can neither be true nor false.
Soundness: A sound argument is a valid argument in which all the premises are actually true inour world. This means that any argument that is either invalid, or valid with at least one false
premise, is unsound. Consider this example again:Premise: If grass is blue, then trees are blue.
Premise: The grass is blue.Conclusion: Trees are blue.
This is a valid but unsound argument because at least one of the premises is not actually true.Consider again the invalid argument from above:
Premise: If you earned an A on the final paper, then you earn an A in the class.Premise: You earn an A in the class.
Conclusion: You earned an A on the final paper.This argument is unsound because it is invalid, regardless of whether the premises are actually
true.
Formal Fallacy: An invalid argument may be a bad argument. Consider again the followingargument:
Premise: If you earned an A on the final paper, then you earn an A in the class.Premise: You earn an A in the class.
Conclusion: You earned an A on the final paper.This is considered to be a bad argument because it exemplifies the formal fallacy ofAffirming
the Consequent. No consider a similar argument:Premise: If you earned an A on the final paper, then you earn an A in the class.
Premise: You did not earn an A on the final paper.Conclusion: You did not earn an A in the class.
This is also an invalid argument, because it is possible that the premises could be true while theconclusion false. Recall the example above: it may also be a policy in the class that if your
homework average is an A, then you earn an A in the class. In this case you can earn an A in theclass without earning an A on the final paper. This is also considered to be a bad argument
because it exemplifies the formal fallacy isDenying the Antecedent.
Strength: An invalid argument may be a good argument. Consider the following argument:Premise: 90% of Americans are afraid of snakes.
Premise: Jane is an American.Conclusion: Jane is afraid of snakes.
This argument is invalid because it is certainly possible that Jane is part of the 10% of Americanswho are not afraid of snakes. Thus, it is possible for the premises to be true and the conclusion to
be false. However, it is unlikely that Jane is a part of the 10% rather than the 90%, so it isunlikely that the conclusion would be false if the premises are true.
8/21/2019 CMU - Teaching Portfolio Guide
30/175
4
A strongargument, then, is an invalid argument in which is likely that the conclusion istrue, given that the premises are true. Unlike validity, strength can come in degrees. Consider a
similar argument:Premise: 99% of Americans are afraid of snakes.
Premise: Jane is an American.
Conclusion: Jane is afraid of snakes.Here, it is even more likely that the conclusion is true given that the premises are true. And sinceit is more likely, we say that this argument is stronger than the first, although they are both
considered strong.
Conversely, a weakargument is an invalid argument in which it is not likely that the conclusionis true, given the truth of the premises. Consider the following argument:
Premise: 30% of Americans speak French.Premise: Jane is an American.
Conclusion: Jane speaks French.While it is possible that Jane is part of the 30% of Americans who speak French, it is more likely
that she is part of the 70% who do not. Thus, this is a weak argument.
Strong arguments are not always those with premises that assert percentages. Many (if not all)scientific laws are actually the conclusions of strong arguments, the premises of which are
assertions about what we have experienced so far, and the conclusions of which are assertionsabout what we will continue to experience in the future. These are often referred to as arguments
by induction.
In addition, many arguments by analogyare strong arguments. This type of argument is one wecome across quite frequently in philosophy. The premises generally are (1) that two situations
are analogous (or alike in some important respects), and (2) that certain things are true of onesituation. The conclusion is then that those same things will be true of the second situation. The
strength of arguments by analogy depend on how good the analogy is for the purposes of theargument.
Cogency: Just as we can evaluate valid arguments in terms of the actual truth or falsity of their
premises, we can evaluate invalid arguments. A cogentargument is a strong argument in whichall the premises are actually true in our world. This means that any argument that is either weak,
or strong with at least one false premise, is uncogent. Consider the following argument:Premise: All swans observed so far are white.
Conclusion: All swans are white.This is quite a strong argument, but, unfortunately, black swans have now been observed in
Australia. Thus the premise is false, and the argument is uncogent.
Informal Fallacy: Not all valid arguments are good arguments. Those with clearly falsepremises are bad, of course. However, there are some particular arguments that seem to have a
valid form and possibly true premises, but, upon examination of the content of the argument,clearly exemplify errors is reasoning.
8/21/2019 CMU - Teaching Portfolio Guide
31/175
5
Straw Man:This is an argument attacking another persons argument. Often, when debating apoint with another, it is wise to reconstruct the other persons argument in your own words. This
ensures that you understand the argument, and can help you evaluate it. We often do this in classwith the readings for the day; after all you cannot criticize someones argument until you
understand exactly what it is. The straw man fallacy occurs when you misrepresent the other
persons argument, or a statement made by the other person, in a way that makes it much easierto criticize his or her argument.
Argument against the person(Ad hominemfallacy): This is also an argument against theconclusion of another persons argument. The conclusion of the argument is that the other person
is wrong, but the premises of the argument, instead of giving good reasons not to believe theclaim itself, attack the other persons character, intelligence, or motivation. In general statements
about a person have nothing to do with whether that persons argument is valid, sound, strong,etc.
Appeal to the People: This is an argument in which the basis for the conclusion is the desire or
need to be accepted or valued by a larger group. Unless there are specific reasons given to thecontrary (as in scientific inquiry, for example), the fact that some group of people believe a
statement (or supposedly believe a statement), is not an indication that the statement is true.
Appeal to Ignorance: This is an argument in which the basis for believing the truth of a statementis that its falsity has not been proven. Unfortunately, the converse argument seems just as
persuasive: believing a statement is false merely because its truth has not been proven. Again,unless there are specific reasons given to the contrary, lack of proof for (or against) a statement
does not constitute proof that the statement is false (or true).
Equivocation: This fallacy is made when an argument has a valid form, but in fact two or moremeanings of a single word or phrase are used when validity depends on using just one meaning.
Begging the Question: This fallacy is also made when an argument has a valid form, but in fact
the conclusion is merely a restatement of one or all of the premises. That is, begging the questionoccurs when one is assuming (using as a premise) that which is to be proven (the conclusion).
False Dilemma: This is another fallacy made when the argument has a valid form, but one of the
premises is unfairly narrowing the possibilities to be considered. It usually occurs when someoneasserts a dilemma (either A or B, with the implicit assumption that nothing other than A or B
could be the case), but the fallacy can also occur for an assertion of three, four or more choices(as in: either A, B, or C, with the implicit assumption that nothing other than A, B, or C could be
the case).
Appeal to Unreliable Authority: Generally, an appeal to authority can make a very strong (in thetechnical sense) argument for a particular conclusion. However, this is only true when the
authority appealed to is actually a reliable source of information about the subject. This fallacyoccurs when the source cited as a reason to believe a claim is either (1) not actually a reliable
authority, (2) a reliable authority on some subjects, but not the one in question, or (3) is assertinga claim that is quite controversial even among reliable authorities in the field.
8/21/2019 CMU - Teaching Portfolio Guide
32/175
6
False Cause: This fallacy occurs when the claim is made that one possible cause of a
phenomenon is a (or the) cause of another phenomenon without evidence excluding otherpossible causes. This fallacy comes in two common forms. The first ispost hoc ergo propter hoc
(after this, therefore because of this). This fallacy occurs when A is asserted to be the cause of
B solely because A occurred before B. The second form is the slippery slope. This fallacy occurswhen A is asserted to inevitably cause B, through some sort of chain reaction, despite lack ofevidence that each link in the chain will inevitably lead to the next link.
II. Structure
The structure of you paper is very important. I should be able to follow your line of reasoning
from the first sentence to the last without getting lost, confused, or sidetracked. Wonderful ideasarent worth much if it is not clear how they all fit together.
Argument. In its most basic form, your paper should be an extended argument, the conclusion ofwhich is a statement you are asserting to be true. An extended argument is one that incorporates
one or more sub-argument into the main argument. For example, your main argument may havefour premises which lead to your conclusion. If, however, your premises are slightly
controversial, not obviously true, or not common knowledge, you should provide sub-argumentsto support your premises. Moreover, the premises which support these sub-conclusions may need
arguments of their own. Thus, an extended argument is a series of embedded arguments,ultimately leading to your main conclusion.
Thesis. The thesis is a statement that is the conclusion of the main argument of your paper.
Every paper should have a thesis. Your thesis should be clearly stated at the beginning of yourpaper in the introduction.
Introduction: The introduction to your paper has at least two purposes. First, you need to tell
your reader what your conclusion will be. This is your thesis statement. Second, you need to tellyour reader roughly how you will get to this conclusion. This is not a detailed outline of your
paper, but rather a concise summary of the steps in argument.
Body of the paper: The body of the paper comprises the argument for your thesis. Eachparagraph should be a step in the process of supporting your final claim. This process has several
parts:
(1)
A demonstration of how the truth of a set of premises leads makes the truth of theconclusion (or sub-conclusion) either necessary or probable. This step may be skipped ifthe argument or sub-argument follows a known valid form.
(2)Assertions of factual claims, obtained from a variety of sources, that are used as premisesis an argument or sub-argument.
(3)Considerations of counter-arguments. This follows the recognition that there may bepeople who would disagree with you about either (1) or (2). You should consider the
objections they might raise, and respond.
8/21/2019 CMU - Teaching Portfolio Guide
33/175
7
Conclusion. Contrary to what you may have learned in High School, your conclusion should not
be a restatement of your introduction. Rather, the conclusion is a handy place to tie up looseends. Here is where, for example, you can consider objections to your argument to which you do
not have the space or expertise to respond. This is also a good place to briefly consider the
implications of the acceptance of your conclusion. Or this may be a good place to explain whatfurther work may need to be done in this area.
III. Content
The content of your paper will mostly be dependent on the structure, for the content is the filling-
in of the outline dictated by the structure. However, the particular kinds of arguments andevidence you use will depend on the topic you are considering, and the particular claims you are
making.
Arguments. While all philosophy papers should advance an argument, there are many differenttypes of arguments that one can make. Thus, writing in philosophy can take many forms:
Advancing your own original argument
Exploring the consequences of a particular hypothesis or position Reconstructing someone elses position, theory, or argument
Evaluating someone elses argument Some combination of the above
Advancing your own original argumentis the basic task of providing a set of premises and
demonstrating how these premises lead to your conclusion. The premises can be asserted on thebasis of a number of different kinds of sources that constitute evidence that they are true.
Exploring the consequences of a particular hypothesis or positionis the task of assuming that the
hypothesis, or the set of statements comprising the position, is true, and showing whatconclusions can be drawn from them, either alone, or in addition to some other set of premises
you assert.
Reconstructing someone elses position, theory, or argument is the task of putting the otherpersons position, theory, or argument into your own words with as little excess verbiage as
possible. In so doing, you should always abide by the principles offairness and charity. Theprinciple of fairness says that you should always paraphrase someone else in a way that is as
close to his or her intentions as possible. This prevents the possibility of committing the Straw
Man fallacy. The principle of charity says that, when you may be confused about the authorsintentions, you should always interpret him or her in the best possible light. This means, for anargument for example, interpreting the premises and conclusion in such a way as to make the
argument valid instead of invalid, or strong instead of weak. This kind of reconstruction is itselfa kind of argument because you will need textual support to provide the evidence that your
reconstruction is as fair as possible.
8/21/2019 CMU - Teaching Portfolio Guide
34/175
8
Evaluating someone elses argumentconsists, generally, in showing that his or her argument isgood or bad. The goodness of an argument may depend on many things, but as a rule, sound
arguments and cogent arguments are considered good, while valid but unsound, strong butuncogent, and weak arguments are all considered bad. In any case, you need to provide your own
argument to show that the other persons argument fits some established criteria of what
constitutes a good or bad argument. In general, if you disagree with the conclusion of someoneelses argument, you have two options for evaluating it negatively: you can either show that theargument form is bad (i.e. show that it is a weak argument), or you can show that the premises
are false (or you an do both).
Evidence. One thing that differentiates philosophy papers from other kinds of papers (Englishpapers, scientific papers) is the wide variety of the kinds of evidence that are routinely used and
accepted to support various different kinds of claims: Derivations from first philosophical principles (e.g., I think, therefore I am)
Scientific or factual claims Logical principles
Appropriate personal experience
Linguistic analysis (definitions, colloquial uses of terms and phrases)
Expert testimony Textual evidence
The kinds of evidence you use to support your thesis may vary according to the assignment and
the topic, but some basic guidelines should be followed.
Relevance. The evidence you use should be directly and obviously related to the claim it issupposed to support. If it is either indirectly or not obviously related, you need to provide
additional arguments why this evidence does indeed support your conclusion. We have alreadyseen some fallacies you may commit if you use irrelevant evidence, likeAd hominem, appeal to
the people, or appeal to ignorance.
Reasonableness. The evidence you use to support your claim should be such that any reasonableperson would accept it, based not on the information you have, but on the information you have
given in your paper. This means that, in general, if your evidence is controversial or notobviously true given the information you have offered, you will have to provide additional
reasons to accept it.
Sufficiency. The evidence you use to support your claim must be sufficient to support it. That is,you must have enough relevant, reasonable evidence to persuade your reader. This means that,
for example, you may not make sweeping generalizations about what all philosophers have beenconcerned about throughout the course of history if you have only read a handful of philosophy
texts.
Definitions. As I said above, philosophers are generally very concerned with definitions, andespecially concerned with being clear and consistent. This means that if you are going to use or
consider important philosophical terms or concepts, you need to define them as clearly as
8/21/2019 CMU - Teaching Portfolio Guide
35/175
9
possible. However, a conventional dictionary is usually not up to the task of providing adequatedefinitions of technical philosophical terms. There are many reasons for this:
(1) A conventional dictionary is usually concerned with giving colloquial uses of words, nottechnical definitions that may vary by field or author (consider the difference between the
colloquial use of the word force, and the technical definition this word has in physics).
(2)
Conventional dictionaries do not generally contain philosophical phrases as singleentries, and the meaning of a phrase often cannot be captured by a conjunction of themeanings of the individual words.
(3) Philosophers writing in different settings may use words and phrases in slightly differentways. In addition, philosophers often spend good portions of articles, or entire articles, on
determining the precise definition of a term or phrase, exactly because there may be nogenerally accepted definition of it. In this case, a dictionary will be of no use at all.
Citations: Different sources may be used for different purposes, but all should be documented
clearly and correctly. Many of the kinds of evidence you will use to support a claim involvesources that you will need to document.
Scientific or factual claims unless you are stating a fact that is obviously commonknowledge and generally uncontroversial (e.g. the earth is round), you will need to cite
the sources of your scientific and/or factual claims (like how many people visit thePittsburgh Zoo every year, or what a Pandas natural habitat is).
Expert testimony if you are going to state that an expert in the field has made someclaim, you need to cite the source in which the claim was advanced, be it a scholarly
article, a newspaper story, or personal communication. Textual evidence in making a case for a particular interpretation of an authors words or
reconstruction of an authors argument, you will need to cite textual evidence to showthat you are following the principles of fairness and charity.
8/21/2019 CMU - Teaching Portfolio Guide
36/175
10
IV. General Miscellaneous Advice
The thesis should something sufficiently interesting, about which reasonable people could
differ, and which can be defended. In this paper, I will show that two plus two is four isobviously true and not worth writing about. In this paper, I will prove that Colorado
Springs has a tropical climate is impossible to prove. Dont set your sights either too highor too low.
Not everything you assume to be true is actually self-evidently true. A lot of what happens inphilosophy is learning about ones own assumptions and how to defend those, or to get rid of
the ones that are indefensible. You might think its impossible for anyone to disagree withthe statement that There are no true liberals left in American politics; even Clinton and Gore
are basically Republicans, but not everyone will be like-minded. If, halfway through, you cant figure out what the sentence youre writing has to do with the
thesis, stop and try to figure out how it links. If it doesnt at all, it probably doesnt need tobe there. If it does but you need to explain why, add the explanation.
Transitions are important; they lead the reader through your argument. Transitional words,
though, should only be used for actual transitions. Obvious problems are using words liketherefore and thus followed by a statement which has nothing to do with what camebefore it.
Texts have to be interpreted, which means you need to quote passages to support yourinterpretation. The following are not appropriate uses of text as they stand. (Nor are the
appropriate citations.)o The Bible says be nice to people.
o Kierkegaard says that Abraham is important (Kierkegaard).o Nietzsche says Kierkegaard is tiresome (Nietzsche, 20-150).
o Marx loves capitalism. As he says, it involves amazing wealth (Marx, 241).o Platos dialogue is about love (class notes).
o
Plato believes in a theory of forms (Plato Introduction, 2).o Descartes believes in dualism (www.randomhack.com/descartes/bs.html).
Evidence for a claim can be of varying types. A lot of the work in evaluating arguments rests
in deciding how to evaluate evidence.
o A reference to an authority is not necessarily the best evidence. One must consider heauthoritys credentials, and whether there is reasonable disagreement among experts.
Arguments from authority are not valid, but they can be good arguments nonetheless.o Ad hominems are logical fallacies: Marx said it, and hes stupid, therefore its false
is not a good argument. There are many other logical fallacies you should avoid as
well; you should become familiar with them.o I think is not evidence in itself. I think it is going to rain tomorrow is not a better
reason to bring out an umbrella than it is going to rain tomorrow. Generally, since itis yourpaper, I will assume that statements you assert are things you think are true.
Avoid useless generalities: In todays modern complex society, throughout time peoplehave struggled with political questions, etc. They waste space and make you sound like
someone who doesnt have enough to say. Youre also probably not qualified to say thingslike philosophers have always struggled with the question of How many philosophers
have you read?
8/21/2019 CMU - Teaching Portfolio Guide
37/175
8/21/2019 CMU - Teaching Portfolio Guide
38/175
11
Avoid making absolutist, overly general claims. Any argument whose conclusion istherefore Plato is totally wrong is probably a bad argument. Remember that most issues
worth talking about are not simple. Most issues worth talking about also arent cut and dried: the other side probably does
have something worth taking into consideration. The best reasoning is that which
understands and anticipates opposing arguments, and takes them into account. With respect to vocabulary, there are a number of points:
o Keep your writing simple, but not simplistic. Only use big words and jargon if you
must; otherwise use simple, easy-to-understand language.o Dont use a variety of different words to refer to the same idea or problem just because
you dont want to sound repetitive. You should worry more about precision and lessabout beauty.
o Philosophers often give words precise, technical meanings that are not used in thatprecise or technical way in ordinary conversation. You should know if a word is being
used in this way in anything you read; and you should only use these words in yourwriting if you fully understand them and can use them correctly.
o
In addition, you may want to give a term a precise meaning that is different from itscolloquial meaning in your paper. It is fine to do this as long as you carefully explain
the definition youre using, and use the term consistently with this definitionthroughout your paper.
o Some philosophers also have a habit of giving the same technical word a slightlydifferent meaning than others do. Thus, while there is no need to explain terms like
valid or necessary in your paper, you do need to explain what you mean byphysicalism or sense data.
Always follow the guidelines for quotations and citations. A coherent argument is difficult to sustain over many pages. I recommend having someone
else read your paper for clarity, cohesiveness, etc., before you hand it in. Proofreading your paper is not the same as running spell-check. Spell-check wont catch
common mistakes like:o Using their for there or vice versa
o Using its for its or vice versao Writing based off when the appropriate clause is based on
o Writing centered around when the appropriate clause is centered ono Mismatching the subject of the sentence with the possessives: One should do their
own work should be One should do his or her own work.o Misspelling an authors name
Do not use rhetorical questions or slang in your writing. They diminish the clarity of yourpaper by making the reader guess at your meaning.
8/21/2019 CMU - Teaching Portfolio Guide
39/175
8/21/2019 CMU - Teaching Portfolio Guide
40/175
8/21/2019 CMU - Teaching Portfolio Guide
41/175
8/21/2019 CMU - Teaching Portfolio Guide
42/175
8/21/2019 CMU - Teaching Portfolio Guide
43/175
8/21/2019 CMU - Teaching Portfolio Guide
44/175
8/21/2019 CMU - Teaching Portfolio Guide
45/175
8/21/2019 CMU - Teaching Portfolio Guide
46/175
8/21/2019 CMU - Teaching Portfolio Guide
47/175
8/21/2019 CMU - Teaching Portfolio Guide
48/175
8/21/2019 CMU - Teaching Portfolio Guide
49/175
8/21/2019 CMU - Teaching Portfolio Guide
50/175
8/21/2019 CMU - Teaching Portfolio Guide
51/175
8/21/2019 CMU - Teaching Portfolio Guide
52/175
8/21/2019 CMU - Teaching Portfolio Guide
53/175
Using Argument Diagrams 1
Using Argument Diagrams to Improve
Critical Thinking Skillsin Introductory Philosophy
Maralee Harrell
Carnegie Mellon University
Department of Philosophy
135 Baker HallPittsburgh, PA, 15213
(412) 268-8152
8/21/2019 CMU - Teaching Portfolio Guide
54/175
Using Argument Diagrams 2
Abstract
In an experiment involving 139 students in an introductory philosophy course we tested whetherstudents were improving their ability to think critically about arguments and whether using
argument diagramming as an analysis aid contributed to this improvement. We determined that
the students did develop this skill over the course of the semester. We also determined that the
students in one section of the course gained significantly more than the students in the othersections, and that this was due almost entirely to their ability to use argument diagrams. We
conclude that learning how to construct argument diagrams significantly improves a students
ability to analyze, comprehend, and evaluate arguments.
8/21/2019 CMU - Teaching Portfolio Guide
55/175
Using Argument Diagrams 3
Using Argument Diagrams to Improve Critical Thinking Skills
in Introductory Philosophy
In the introductory philosophy class at Carnegie Mellon University (80-100 What Philosophy Is)
one important learning goal is the development of general critical thinking skills. Even though
there are a few generally accepted measures of these skills (e.g. the California Critical ThinkingSkills Test and the Watson Glaser Critical Thinking Appraisal, but see also Halpern, 1989 and
Paul, Binker, Jensen, & Kreklau, 1990), there is surprisingly little research on the sophistication
of, or on effective methods for improving, the critical thinking skills of college students. Theresearch that has been done shows that the population in general has very poor skills (Perkins,
Allen, & Hafner, 1983; Kuhn, 1991; Means & Voss, 1996), and that very few courses actually
improvement these skills (Annis & Annis, 1979; Pascarella, 1989; Stenning, Cox, & Oberlander,1995).
Critical thinking involves the ability to analyze, understand, and evaluate an argument.
Our first hypothesis is that students improved on these tasks after taking the introductoryphilosophy course. However, we wanted to determine not only whether they improved, but how
much improvement could be attributed to alternative teaching methods.One candidate method is the use of argument diagrams as an aid to overall argument
comprehension, since we believe that they significantly facilitate understanding, analysis, andevaluation. An argument is a series of statements in which one is the conclusion, and the others
are premises supporting this conclusion; and an argument diagram is a visual representation of
these statements and the inferential connections between them.For example, at the end ofMeno, Plato (1976) argues through the character of Socrates
that virtue is a gift from the gods (89d-100b). While the English translations of Platos works are
among the more readable philosophical texts, it is still the case not only that the text containsmany more sentences than just the propositions that are part of the argument, but also that,
proceeding necessarily linearly, the prose obscures the inferential structure of the argument. Thusanyone who wishes to understand and evaluate the argument may reasonably be confused. If, on
the other hand, we are able to extract just the statements Plato uses to support his conclusion, and
visually represent the connections between these statements (as shown in Figure 1), the structureof the argument is immediately clear, as are the places where we may critique or applaud it.
Recent interest in argument visualization (particularly computer-supported argument
visualization) has shown that the use of software programs specifically designed to help students
construct argument diagrams can significantly improve students critical thinking abilities overthe course of a semester-long college-level course (Kirschner, et al. 2003; Twardy, 2004; van
Gelder, 2001, 2003). But, of course, one need not have computer software to construct an
argument diagram; one needs only a pencil and paper. However, to our knowledge there hasbeen no research done to determine whether it is the mere ability to construct argument
diagrams, or the aid of a computer platform and tutor (or possibly both) that is the crucial factor.
Our second hypothesis is that the crucial factor in the improvement of critical thinkingskills is the ability to construct argument diagrams. This hypothesis posits that students who
construct correct diagrams during argument analysis tasks should perform better on these tasks
than students who do not.
8/21/2019 CMU - Teaching Portfolio Guide
56/175
Using Argument Diagrams 4
FIGURE 1 An argument diagram representing one of the arguments in Platos Meno.
We typically teach several sections of Carnegie Mellon Universitys introduction to
philosophy course (80-100 What Philosophy Is) each semester, with a different instructor foreach section. While the general curriculum of the course is set, each instructor is given a great
deal of freedom in executing this curriculum. For example, each section is a topics based coursein which epistemology, metaphysics, and ethics are introduced with both historical andcontemporary primary-source readings. Each instructor, however, chooses a text, the order of the
topics, and the assignments for his or her section. The students who take this course are a mix of
classes and majors from all over the University.
In the Spring of 2004, students in Section 1 were explicitly taught how to constructargument diagrams to represent a selection of text. In contrast, students in Sections 2, 3, and 4
were not explicitly taught the use of argument diagrams, but ratherif they were taught to
analyze arguments at allwere taught to use more traditional kinds of representations (e.g. listsof statements).
In this study, we test the first hypothesis by comparing the pretest and posttest scores of
all the students in 80-100 in the Spring semester of 2004. We test the second hypothesis in threeways: (1) by comparing the pretest and posttest scores of students in Section 1 to students in
Sections 2, 3, and 4, (2) by comparing the pretest and posttest scores of students who constructed
correct argument diagrams on the posttest to those students who did not, and (3) by comparing
total scores on individual questions on the posttest of students who constructed the correctargument diagrams for that question to those students who did not.
Virtue is not knowledge
Virtue is a true belief
Virtue is either knowledge or true belief
True belief is a gift from the gods
Virtue cannot betaught
Something canbe taught if andonly if it isknowledge
Virtue guidescorrect actions
Virtue is a gift from the gods
Only knowledgeand true beliefguide correctactions
There are noteachers ofvirtue
Something can betaught if and only if ithas teachers
8/21/2019 CMU - Teaching Portfolio Guide
57/175
Using Argument Diagrams 5
Method
Participants
139 students (46 women, 93 men) in each of 4 sections of introductory philosophy (80-100 What
Philosophy Is) at Carnegie Mellon University in the Spring of 2004 were studied. Each section of
the course had a different instructor and teaching assistant, and the students chose their section.
There were 35 students (13 women, 22 men) in Section 1, 37 students (18 women, 19 men) inSection 2, 32 students (10 women, 22 men) in Section 3, and 35 students (5 women, 30 men) in
Section 4. The students in Section 1 were taught the use of argument diagrams to analyze the
arguments in the course reading, while the students in the other three sections were taught moretraditional methods of analyzing arguments.
Materials
Prior to the semester, the four instructors of 80-100 in the Spring of 2004 met to determine the
learning goals of this course, and designed an exam to test the students on relevant skills. The
identified skills were to be able to, when reading an argument, (i) identify the conclusion and thepremises; (ii) determine how the premises are supposed to support the conclusion; and (iii)
evaluate the argument based on the truth of the premises and how well they support theconclusion.
We used this exam as the pretest (given in Appendix A) and created a companionposttest (given in Appendix B). For each question on the pretest, there was a structurally
(nearly) identical question with different content on the posttest. The tests each consisted of 6
questions, each of which asked the student to analyze a short argument. In questions 1 and 2, thestudent was only asked to state the conclusion (thesis) of the argument. Questions 3-6 each had
five parts: (a) state the conclusion (thesis) of the argument; (b) state the premises (reasons) of the
argument; (c) indicate (via multiple choice) how the premises are related; (d) provide a visual,graphical, schematic, or outlined representation of the argument; and (e) decide whether the
argument is good or bad, and explain this decision.
Procedure
Each of the four sections of 80-100 was a Monday/Wednesday/Friday class. The pretest wasgiven to all students during the second day of class. The students in sections 2 and 3 were given
the posttest on the last day of classes, while the students in sections 1 and 4 were given the
posttest as one part of their final exam, during exam week.
Results and Discussion
Test Coding
Pre- and posttests were paired by studentsingle-test students were excluded from the sample
so that there were 139 pairs of tests in the study. Tests which did not have pairs were used for
coder-calibration, prior to the coding of the 139 pairs of tests.Two graduate students independently coded all 278 tests (139 pairs). Each pre-/posttest
pair was assigned a unique ID, and the original tests were photocopied (twice, one for each
coder) with the identifying information replaced by the ID. We had an initial grader-calibrationsession in which the author and the two coders coded several of the unpaired tests, discussed our
codes, and came to a consensus about each code. After this, each coder was given the two keys
(one for the pretest and one for the posttest) and the tests to be coded in a unique random order.
8/21/2019 CMU - Teaching Portfolio Guide
58/175
Using Argument Diagrams 6
The codes assigned to each question (or part of a question, except for part (d)) were
binary: a code of 1 for a correct answer, and a code of 0 for an incorrect answer. Part (e) of eachquestion was assigned a code of correct if the student gave as reasons claims about the truth of
the premises and/or the support of premises for the conclusion. For part (d) of each question,
answers were coded according to the type of representation used: Correct argument diagram,
Incorrect or incomplete argument diagram, List, Translated into logical symbols like a proof,Venn diagram, Concept map, Schematic (e.g., P1 + P2/Conclusion (C)), Other or blank.
To determine inter-coder reliability, the Percentage Agreement (PA), Cohens Kappa ()
and Krippendorffs Alpha () were calculated for each test (given in Table 1).
TABLE 1Inter-coder Reliability: Percentage Agreement (PA),
Cohens Kappa (), and Krippendorffs Alpha () for each test
PA
Pretest .85 .68 .68Posttest .85 .55 .54
The inter-coder reliability was fairly good, however, upon closer examination it wasdetermined that one coder had systematically higher standards than the other coder on the
questions in which the assignment was open to some interpretation (questions 1 & 2, and parts
(a), (b), and (e) of questions 3-6). Specifically, on the pretest, out of 385 question-parts on which
the coders differed, 292 (75%) were cases in which Coder 1 coded the answer as correct whileCoder 2 coded the answer as incorrect; and on the posttest, out of 371 question-parts on which
the coders differed, 333 (90%) were cases in which Coder 1 coded the answer as correct while
Coder 2 coded the answer as incorrect. In light of this, the codes from each coder on thesequestions were averaged, allowing for a more nuanced scoring of each question than either coder
alone could give.
Since we were interested in how the use of argument diagramming aided the student in
answering each part of each question correctly, the code a student received for part (d) ofquestions 3-6 were preliminarily set aside, while the addition of the codes received on questions
1 and 2, as well as parts (a), (b), (c), and (e) of questions 3-6 determined the raw score a student
received on the test.
TABLE 2The variables and their descriptions recorded for each student
Variable Name Variable Description
Pre Fractional score on the pretestPost Fractional score on the posttest
A* Averaged score (or code) on the pretest for question *B* Averaged score (or code) on the posttest for question *Section Enrolled sectionSex Students sexHonors Enrollment in Honors courseGrade Final Grade in the courseYear Year in school
The primary variables of interest were the fractional pretest and posttest scores (the raw
score converted into a percentage), and the individual average scores for each question on the
pretest and the posttest. In addition, the following data was recorded for each student: which
8/21/2019 CMU - Teaching Portfolio Guide
59/175
Using Argument Diagrams 7
section the student was enrolled in, the students final grade in the course, the students year in
school, the students sex, and whether the student had taken the concurrent honors courseassociated with the introductory course. Table 2 gives summary descriptions of these variables.
Average Gain from Pretest to Posttest for All Students
The first hypothesis was that the students critical thinking skills improved over the course of thesemester. This hypothesis was tested by determining whether the average gain of the students
from pretest to posttest was significantly positive. The straight gain, however, may not be fully
informative if many students had fractional scores of close to 1 on the pretest. Thus, thehypothesis was also tested by determining the standardized gain: each students gain as a fraction
of what that student could have possibly gained. The mean scores on the pretest and the posttest,
as well as the mean gain and standardized gain for the whole population of students is given inTable 3.
TABLE 3Mean fractional score (standard deviation) for the pretest and the posttest,
mean gain (standard deviation), and mean standardized gain (standard deviation)
Pre Post Gain GainSt.WholePopulation
0.59 (0.14) 0.78 (0.12) 0.19 (0.01) 0.43 (0.03)
The difference in the means of the pretest and posttest scores was significant (paired t-test;p< .001). In addition, the mean gain was significantly different from zero (1-sample t-test;
p< .001) and the mean standardized gain was significantly different from zero (1-sample t-test;p
< .001). From these results we can see that our first hypothesis is confirmed: overall the studentsdid have significant gains and standardized gains from pretest to posttest.
Comparison of Gains of Students by Section and by Argument Diagram Use
Our second hypothesis was that the students who were able to construct correct argumentdiagrams would gain the most from pretest to posttest. Since the use of argument diagrams was
only explicitly taught in Section 1, we first tested this hypothesis by determining whether the
average gain of the students in Section 1 was significantly different from the average gain of thestudents in each of the other sections. Again, though, the straight gain may not be fully
informative if the mean on the pretest was not the same for each section, and if many students
had fractional scores close to 1 on the pretest. Thus, we also tested this hypothesis using thestandardized gain. The mean scores on the pretest and the posttest, as well as the mean gain and
standardized gain for the sub-populations of students in each section is given in Table 4.
TABLE 4Mean fractional score (standard deviation) for the pretest and the posttest,
mean gain (standard deviation), and mean standardized gain (standard deviation)
Pre Post Gain GainSt.
Section 1 0.64 (0.14) 0.85 (0.10) 0.21 (0.02) 0.51 (0.07)
Section 2 0.53 (0.16) 0.70 (0.14) 0.17 (0.03) 0.32 (0.05)
Section 3 0.58 (0.14) 0.79 (0.08) 0.21 (0.02) 0.48 (0.04)
Section 4 0.63 (0.10) 0.80 (0.09) 0.17 (0.02) 0.42 (0.05)
8/21/2019 CMU - Teaching Portfolio Guide
60/175
Using Argument Diagrams 8
Since there was such variability in the scores on the pretest among the different sections,
we ran an ANCOVA on the each of the variables Post, Gain, and GainSt, with the variable Preused as the covariate. This analysis indicates that the differences in the pretest scores was
significant for predicting the posttest scores (df = 1, F= 24.36,p< .001), the gain (df = 1, F=
125.50,p< .001), and the standardized gain (df = 1, F= 29.14,p< .001). In addition, this
analysis indicates that, even accounting for differences in pretest score, the differences in theposttest scores among the sections were significant (df = 3, F= 8.71,p< .001), as were the
differences in the gains (df= 3, F= 8.71,p< .001) and the standardized gains (df= 3, F= 6.84,p
< .001).This analysis shows that a students section is a significant predictor of posttest score,
gain, and standardized gain, but it does not tell us how they are different. The hypothesis is that
the posttest score, gain and standardized gain for students in Section 1 is significantly higher thanall the other sections. Thus, we did a planned comparison of the variables Post, Gain, and GainSt
for Section 1 with the other sections combined, again using the variable Pre as a covariate. This
analysis again indicates that the differences in the pretest scores was significant for predicting theposttest scores (df = 1, F= 32.28,p< .001), the gain (df = 1, F= 107.37,p< .001), and the
standardized gain (df = 1, F= 21.42,p< .001). In addition, this analysis indicates that, evenaccounting for differences in pretest score, the differences in the posttest scores between Section
1 and the other sections were significant (df = 1, F=11.89,p= .001), as were the differences inthe gains (df= 1, F= 11.89,p= .001) and the standardized gains (df= 1, F= 8.07,p= .005),
with the average posttest score, gain, and standardized gain being higher in Section 1 than in the
other three sections.Although these differences between sections (at least with standardized gain scores)
obtained, they do not provide a direct test of whether students who (regardless of section)
constructed correct argument diagrams have better skills. The explanation is that, although thestudents in Section 1 were the only students to be explicitly taught how to construct argument
diagrams, a substantial number of students from other sections constructed correct argumentdiagrams on their posttests. In addition, a substantial number of the students in Section 1
constructed incorrect argument diagrams on their posttests. Thus, to test whether it was actually
the construction of these diagrams that contributed to the higher scores of the students in Section1, or whether is was the other teaching methods of the instructor for Section 1, we introduced a
new variable into our model.
Recall that the type of answer given on part (d) of questions 3-6 was the data recorded
from the test. From this data, a new variable was defined that indicates how many correctargument diagrams a student had constructed on the posttest. This variable is PostAD (value = 0,
1, 2, 3, 4).
The second hypothesis implies that the number of correct argument diagrams a studentconstructed on the posttest was correlated to the students pretest score, posttest score, gain and
standardized gain. Since there were very few students who constructed exactly 2 correct
argument diagrams on the posttest, and still fewer who constructed exactly 4, we grouped thestudents by whether they had constructed No correct argument diagrams (PostAD = 0), Few
correct argument diagrams (PostAD = 1 or 2), or Many correct argument diagrams (PostAD = 3
or 4) on the posttest. The results are given in Table 5.
8/21/2019 CMU - Teaching Portfolio Guide
61/175
Using Argument Diagrams 9
TABLE 5Mean fractional score (standard deviation) for the pretest and the posttest,
mean gain (standard deviation), and mean standardized gain (standard deviation)
Pre Post Gain GainSt.
No Correct 0.56 (0.16) 0.74 (0.12) 0.18 (0.02) 0.39 (0.03)
Few Correct 0.57 (0.13) 0.75 (0.12) 0.17 (0.02) 0.37 (0.04)
Many Correct 0.66 (0.13) 0.88 (0.06) 0.22 (0.02) 0.56 (0.06)
Again, since there was such variability in the scores on the pretest among the different
sections, we ran an ANCOVA on the each of the variables Post, Gain, and GainSt, with the
variable Pre used as the covariate. This analysis indicates that the differences in the pretest scoreswas significant for predicting the posttest scores (df = 1, F= 24.68,p< .001), the gain (df = 1, F
= 132.81,p< .001), and the standardized gain (df = 1, F= 30.97,p< .001). This analysis also
indicates that, even accounting for differences in pretest score, the differences among the
students who constructed 0, Few or Many correct argument diagrams on the posttest aresignificant (df= 2, F= 14.66,p< .001), as are the differences in gains (df= 2, F= 14.66,p