Using Institutional Learning Outcomes to Foster Meaningful Dialogue and Decision-Making
Giovanni Sosa, Ph.D.Chaffey College
RP Conference 2013
The Standards (pg.1)Instructional programs, student supportservices, and library and learning supportservices facilitate the achievement of theinstitution's stated student learning outcomes
ACCJC Standards
Improving Institutional Effectiveness (IB)The institution demonstrates a conscious effort to produce and support student learning, measures that learning, assesses how well learning is occurring, and makes changes to improve student learning…
ACCJC Standards
Improving Institutional Effectiveness (IB)…[it] demonstrates its effectiveness by
providing 1) evidence of the achievement of student learning outcomes…
ACCJC Standards
SLO Assessmentis a continual
process!Criteria
Means ofAssessment
Summary ofEvidence
LearningOutcomesStatement
Use ofResults
Adopted Institutional SLOs◦ Communication◦ Critical Thinking & Information Competency◦ Community/Global Awareness & Responsibility◦ Personal, Academic, and & Career Development
http://www.chaffey.edu/general_info/competencies.shtml
Core Competencies
Facione (1990) 46 experts were convened to discuss role of
CT in educational assessment and instruction
Identified Core CT skills and Sub-Skills
Critical Thinking
1) Interpretation – To comprehend/express meaning of wide variety of experiences
2) Analysis – Identify inferential relationships among statements/concepts
3) Evaluation – Assess the credibility of statements; assess logical strength of inferential statements/concepts
4) Inference – Form hypotheses
Critical Thinking: Core CT SKills
5) Explanation – To state results of one’s reasoning;
6) Self-Regulation – Self-reflection of one’s views to question or confirm reasoning
Critical Thinking: Core CT SKills
MSLQ (Pintrich, McKeachie, & Lin, 1987)◦ Developed in process of studying how to make
students more efficient learners
What does the CT assessment specifically measure?
Critical Thinking Assessment
CT assessment found to be statistically associated to course performance (r = .15)
Critical Thinking Assessment
SuccessNot
Successful Total
High CT 57 42 100
Low CT 42 57 100
Total 100 100 200
Pertains to issues of cultural diversity, human rights, and prejudice reduction – both within national borders and across national borders
Knowledge that individual possesses of other cultures (Cognitive)
The extent to which individual empathizes with values of other cultures (Affective)
Willingness to take stand on cultural issues (Participatory)
Global Awareness (Clarke, 2004)
Inspired by Global Perspective Institute (Braskamp et al., 2011)◦ Holistic Human Development:
Cognitive domain (“Whatdo I know?”) Intrapersonal domain (“”Who am I?”) Interpersonal domain (“How do I relate to others?”)
What does our Global Awareness assessment specifically measure?
Global Awareness Assessment
1907 Total Responses 78 Courses 33 Departments
◦ Chemistry (N = 199)◦ English (N = 195)◦ Theatre (N =159)◦ Cinema (N = 154)◦ CIS (N = 137)◦ Economics (N = 102)
Voluntary Participation
Results: Course Characteristics
SampleN = 1,907
DistrictN = 56,216
Transferable to UC/CSU 69.8% 54.8%
Transferable to CSU 13.2% 14.5%
Not Transferable 16.9% 30.7%
Results: Course CharacteristicsComparison with District Data (SP12)
Transfer Status
SampleN = 1,907
DistrictN = 52,216
Face-to-face 96.5% 94.0%Hybrid 3.5% 2.2%Online 0.0% 3.8%
Results: Course CharacteristicsComparison with District Data (SP12)
Distance Learning
SampleN = 1,907
DistrictN = 52,216
Basic Skills 1.8% 16.0%Not Basic Skills 98.2% 84.0%
Results: Course CharacteristicsComparison with District Data (SP12)
Basic Skills
SampleN = 1,791
DistrictN = 18,434
Cum GPA 2.92 2.52Units Attempted 42.78 34.42Units Completed 40.80 31.32
Results: Student CharacteristicsComparison with District Data (SP12)
Performance Indicators
SampleN = 1,791
DistrictN = 51,526
Success Rate 84.4 70.53
Withdrawal Rate 3.3 11.1
Results: Student CharacteristicsComparison with District Data (SP12)
Performance Indicators
SampleN = 1,673
DistrictN = 43,767
GPA Converted Grade 2.88 2.61
Critical Thinking
CT Level Range Percentile
Low 0 – 19 ≤24th Percentile
Medium 20 – 28 25-74th Percentile
High 29 - 35 ≥75th Percentile
Critical Thinking: Levels
Variables Examined:◦ First Generation Status◦ Gender◦ Age Range◦ Parents’ Education◦ Ethnicity◦ UC/CSU Transfer vs. Non-Transferable
CT Differences by Demographics/Course Transfer Status
Variables Examined:◦ Success Rate◦ Withdrawal Rate◦ GPA Converted Grades◦ Units Attempted◦ Units Completed◦ Cum. GPA◦ Assessment Tests (Reading Comp/Sentence Skills/
Math)
CT Differences by Performance Indicators
CT Level Number of Students
Mean Units Attempted
Low 446 44.29
Medium 962 41.26
High 496 44.79
*d = .11 - Medium level vs. Low level d = .13 – Medium level vs. High level
CT Differences by Performance Indicators: Units Attempted
CT Level Number of Students
Mean Units Earned
Low 446 42.76
Medium 962 39.06
High 496 42.66
*d = .13 for comparisons of Medium level with either Low or High
CT Differences by Performance Indicators: Units Earned
CT Level Number of Students
Mean Cum. GPA
Low 446 2.98
Medium 962 2.87
High 496 2.94
*d = .13 - Medium level vs. Low level d = .09 - Medium level vs. Low level
CT Differences by Performance Indicators: Cum. GPA
Global Awareness
GA Level Range Percentile
Low 0 – 25 ≤24th Percentile
Medium 26 – 28 25-74th Percentile
High 29 - 30 ≥75th Percentile
Global Awareness: Levels
Variables Examined:◦ First Generation Status◦ Gender◦ Age Range◦ Parents’ Education◦ Ethnicity◦ UC/CSU Transfer vs. Non-Transferable
GA Differences by Demographics/Course Transfer Status
GA Differences by Age Range
GA Level19 or
Younger 20 to 24 25 or Older Total
Low 149 209 96 454
Medium 249 384 204 837
High 133 265 215 613
Total 531 858 515 1904
*r = .14
Variables Examined:◦ Success Rate◦ Withdrawal Rate◦ GPA Converted Grades◦ Units Attempted◦ Units Completed◦ Cum. GPA◦ Assessment Tests (Reading Comp/Sentence Skills/
Math)
GA Differences by Performance Indicators
GA Level Number of Students
Mean Units Attempted
Low 454 39.34
Medium 837 41.85
High 613 46.92
*d = .09 – Low vs. Medium d = .27 – Low vs. High d = .17 – Medium vs. High
GA Differences by Performance Indicators: Units Attempted
GA Level Number of Students
Mean Units Earned
Low 454 37.37
Medium 837 39.76
High 613 44.95
*d = .09 – Low vs. Medium d = .28 – Low vs. High d = .19 – Medium vs. High
GA Differences by Performance Indicators: Units Earned
GA Level Number of Students
Mean Sentence Skills
Low 384 81.73
Medium 698 83.98
High 497 86.40
*d = .11 – Low vs. Medium d = .24 – Low vs. High d = .13 – Medium vs. High
GA Differences by Performance Indicators: Sentence Skills
Relationship between CT and GA?
Low GA High GA Total
High CT 120 217 334
Low CT 117 66 186
Total 237 283 520
r = .15
Enhancing institutional effectiveness◦ Self-reflective dialogue
Utilized assessments serve as reliable tools for measuring CT and GA◦ But may not adequately measure all aspects of
corresponding competency More assessment of foundation/online
courses Implications of units attempted/earned &
GPA
Conclusions
Random sampling for Spring 2013 Inclusion of two additional core
competencies for Spring 2013 Division-wide assessment during Fall 2012 Application to specific course level
assessment First Annual Student Services Poster Session
Institutional Response to Findings
Braskamp, L. A., D. C. Braskamp, K. C. Merrill, & M. E. Engberg (2011). Global Perspective Inventory. Global Perspective Institute, Inc., http://gpi.central.edu
Clark, V. (2004). Students’ global awareness and attitudes to internationalism in a world of cultural convergence. Journal of Research in International Education, 3, 51-70.
Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. American Philosophical Association (pgs. 13 -19). Newark, DE.
Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A Manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). National Center for Research to Improve Postsecondary Teaching and Learning. Washington, DC.
References