Development and Validation of the Public Affairs Scale:

10/29/20101 | |

Development and Validation of the Public Affairs Scale:Implications for Research on Service Learning and Community Engagement

10/29/20102 | |

International Association for Research on Service Learning and Community EngagementAnnual Conference – 2010

Chantal Levesque-BristolElizabeth Carmichael StrongJeffrey Cornelius-WhiteYogini DhopadeNicole Howland

10/29/20103 | |

Public Affairs Mission at Missouri State• In 2007-2008 – At the beginning of the Public Affairs

Scale Development

• Re-definition and Focus of the Mission• Search for an Associate Provost for Student Development and

Public Affairs• Operationalization of the public affairs mission at the department

and college level unit of analysis

10/29/20104 | |

Pillars of the Public Affairs Mission• Community Engagement• Cultural Competence• Ethical Leadership

10/29/20105 | |

Community Engagement• Community Engagement is recognizing needs in the

communities within which one belongs, then contributing knowledge and working with the community to meet those needs.  Community engagement requires extending beyond one's self for the betterment of the community - a process that fosters greater awareness and personal growth. 

10/29/20106 | |

Cultural Competence• Cultural Competence begins with cultural self-

awareness and expands to knowledge of, respect for, and skills to engage with those of other cultures.  Culturally competent individuals respect multiple perspectives and are able to successfully negotiate cross-cultural differences.

10/29/20107 | |

Ethical Leadership• Ethical Leadership is striving for excellence and

integrity as one continually develops ethical and moral reasoning while contributing to the common good.  Ethical leaders have the courage to live by their principles in all parts of their personal and professional lives (Center for Ethical Leadership, 2007).

10/29/20108 | |

Indicators of the Public Affairs Mission• Leader in the American Democracy Project

• e-Citizenship initiative

• Annual Public Affairs Conference• Internationally known speakers

• Annual Public Affairs Week• Organized by students

• Common Reader Programs and Themed Year-Long Activities• 2008-2009 Sustainability• 2009-2010 Evolving Economic Realities• 2010-2011 Leading in a Global Society

• Citizenship and Service-Learning Office

10/29/20109 | |

Why develop the Public Affairs Scale?• Operationalization of the Public Affairs Mission –

develop Public Affairs scale• Assessment of the development of the Public Affairs

pillars in students-validity of scale• Outcomes of civically engaged students from

perspectives of community partners and students

10/29/201010 | |

Sources of information examinedDevelopment procedureParticipantsAnalytical Strategy

Method

10/29/201011 | |

Sources of Information examined• Center for Ethical Leadership• American Speech-Language-Hearing Association• Alameda County Public Health Department• Association of American Colleges and Universities• Carnegie Foundation for the Advancement of Teaching• Center for Disease Control• Transcultural Nursing,• US Department of Health and Human Services: Health

Resources and Services Administration.

10/29/201012 | |

Public Affairs Scale Development Procedure• Review of information and sources described above• Construction of 81 items to assess the three pillars of

the mission• 30 items were created to assess Community Engagement• 22 items were created to assess Cultural Competence• 29 items were created to assess Ethical Leadership

• Construct validity assessed with Socially Responsible Leadership Scale – Revised (Tyree, 1998)• previously validated instrument derived from the Social Change

Model (Higher Education Research Institute, 1996),

10/29/201013 | |

Socially Responsible Leadership Scale• Consciousness of Self (individual)• Congruence (individual)• Commitment (individual)• Collaboration (group)• Common Purpose (group)• Controversy with Civility (group)• Citizenship (community/society)• Change (community/society)

10/29/201014 | |

Socially Responsible Leadership Scale• Change – believing in the importance of making a better

world and a better society through some activity• Citizenship – believing in a process whereby an

individual and/or a group become responsibly connected to the community and to society through some activity

• Collaboration – working with others in a common effort• Commitment – having the energy that motivates an

individual to serve and that drives the collective effort

10/29/201015 | |

Socially Responsible Leadership Scale• Common Purpose – having shared goals and values when

working with others• Congruence – thinking, feeling, and behaving with consistency,

genuineness, authenticity and honesty toward others• Consciousness of Self – being aware of the beliefs, values,

attitudes and emotions that motivate a person to take action• Controversy with Civility – believing in two fundamental

realities of any creative effort: (1) that differences in viewpoint are inevitable, and (2) that such differences must be aired openly but with civility

10/29/201016 | |

Participants• Mostly women (70.6%) • Age ranged from 18 to 59 with a mean age of 23• Mostly Caucasians (80%)• Approximately 69% of them were undergraduate students• Mostly freshmen (35.7%)• Approximately 10% each of sophomores, juniors, and seniors.• The students came from all colleges with the College of

Health and Human Services (28.6%) and the College of Education (23.1%) having the most participants.

10/29/201017 | |

Analytical Strategy• Original 81 items were subjected to an Exploratory

Factor Analysis with 3 factors specified• Retained items that maintained conceptual breadth

within the three areas.• Item reduction

• Eliminating items not significantly loading (belonging) on their predicted factor

• Eliminating items loading on the wrong factor• Eliminating items loading on two or three of the factors to a

relatively equal degree (cross-loading).

10/29/201018 | |

Scale Structure and ReliabilityScale Means and Standard DeviationsScale Means by Class RankCorrelations between the Pillars of the Public Affairs and Class RankCorrelations Between the Pillars of the Public Affairs and Social Responsibility

Results

10/29/201019 | |

Scale Structure and Reliability• Scale is presented in packet of handouts

• http://www.missouristate.edu/assets/fctl/Public_Affairs_Self-Report_Scale_Validation_11-16-09.pdf

• Stable 3-factor Public Affairs Scale explaining 45% of the variance• Community Engagement (15 items)

• a = .90• Cultural Competence (10 items)

• a = .75• Ethical Leadership (15 items)

• a = .92

10/29/201020 | |

For the three pillars of the Public Affairs Scale

Scale Means and Standard Deviations

10/29/201021 | |

Mean Std Dev Skew Min Max

Community

Engagement3.93 .58 -.61 1.00 5.00

Ethical

Leadership4.19 .47 -1.24 1.00 5.00

Cultural

Competence3.55 .53 -.45 1.40 5.00

10/29/201022 | |

For the three pillars of the Public Affairs Scale

Scale Means by Class Rank

10/29/201023 | |

Class

Rank

Cultural

Competence

Community

Engagement

Ethical

Leadership

Fresh. 3.45 3.80 4.16

Soph. 3.53 3.90 4.12

Juniors 3.53 4.11 4.17

Seniors 3.56 4.03 4.13

Graduate 3.69 3.99 4.28

p < .01 p < .05 p = .05

10/29/201024 | |

Correlations between the Pillars of the Public Affairs and Class Rank

• Community Engagement (r = .14, p < .05)• Cultural Competence (r = .17, p < .01).

10/29/201025 | |

Correlations Between the Pillars of the Public Affairs and Social Responsibility

• All correlations significant at the p < .001 level.• Highest correlations were generally found with Ethical

Leadership• Correlations were highest between the subscales of the Public

Affairs and the Socially Responsible Leadership Scale that are conceptually most similar• Provides empirical validation for the Public Affairs Scale and

Conceptualization• Correlations were high and significant without being too high to signify that

the constructs were simply replicated in another scale.

10/29/201026 | |

Correlations Between the Pillars of the Public Affairs and Social Responsibility

• Community Engagement and Citizenship (r = .706)• Ethical Leadership and Collaboration (r = .688)• Ethical Leadership and Congruence (r = .681)• Cultural Competence and Change (r = .570).• Cultural Competence and Controversy with Civility (r =

.529).

10/29/201027 | |

Use of the Public Affairs Scale to Evaluate a First Year Service-Learning Program

10/29/201028 | |

Presentation Outline

• Theoretical framework• Measurement strategy• Data analysis• Interpretation• Contributions to the field• 2010 research• Questions for future research

10/29/201029 | |

Theoretical framework• Service-learning intervention in the first-

year experience course contributes positively to self-efficacy, leadership, interpersonal skills and engagement in the campus and community

• Knapp, T., Fisher, B., & Levesque-Bristol, C. (2010) Service-Learning’s Impact on College Students’ Commitment to Future Civic Engagement, Self-Efficacy, and Social Empowerment, Journal of Community Practice, 18: 2, 233-251

10/29/201030 | |

Measurement Strategy• Introduction to University Life

• One credit class• 107 sections • 4 service-learning sections• 203 students completed the Time 1 survey, and 173 completed

Time 2 survey• 87 in the service-learning sections• 116 In the non service-learning sections

• 135 completed both assessments

10/29/201031 | |

Measurement ToolCollaboration with the Faculty Center for Teaching and Learning

• Public Affairs Self-Report Scale• Self efficacy• Five facets of Cognitive Communication Competence

(planning, modeling, presence, reflection, and consequence)Survey includes 3 sections

• Identification number• General information• 76 questions rated in Likert Scale

Strongly Disagree to Strongly AgreeAlways True of Me to Never True to Me

10/29/201032 | |

Sample questions• I know I can make a difference in my community (self

efficacy)• I am able to communicate effectively with people from

different cultures (communication)• I am dependable and reliable (leadership)• I feel insecure about my ability to do things

(contributing to campus and community)

10/29/201033 | |

Survey Administration• CASL staff members administered pre- and post-

surveys during the first and last week of classes• 4 SL sections and 5 control sections• Staff members read a one page survey explanation• Surveys were gathered and data was entered into

SPSS

10/29/201034 | |

Data Analysis• Mean comparisons between service-learning sections

(N = 80) and non service-learning sections (N = 93) at time 2.

• Mean comparisons for service-learning (N = 66) and non service-learning (N = 69) sections to evaluate change over time.

10/29/201035 | |

Mean Comparison SL Vs. Non SLScale SL No SL

Mean Mean Sig.Community Engagement 4.19 3.86 .001**Cultural Competence 3.62 3.42 .018*Ethical Leadership 4.00 4.18 .006**Self-Efficacy 3.88 3.70 .016*Planning Comp. 3.59 3.35 .075†

Modeling Comp. 3.91 3.81 .417Presence Comp. 4.20 4.19 .987Reflection Comp. 4.04 3.82 .076†

Consequence Comp. 4.04 3.89 .219

10/29/201036 | |

Change Over Time SL and Non SLMean Non SL T1

Mean Non SL T2

Sign. Mean SL T1

Mean SL T2

Sign.

Public Affairs -Comm. Engagement

3.74 3.90 .031** 3.93 4.19 .000**

-Cultural Comp. 3.45 3.51 .244 3.41 3.58 .011**-Ethical Leadership

4.28 4.23 .325 4.30 4.33 .648

Self Efficacy 3.78 3.76 .708 3.78 3.84 .300Cognitive Comm. Comp.

-Planning 3.31 3.30 .841 3.50 3.66 .095†

-Modeling 3.72 3.86 .066† 3.75 3.83 .361 -Presence 4.25 4.19 .423 4.32 4.23 .223 -Reflection 3.89 3.81 .393 3.96 4.07 .209 -Consequence 3.90 3.84 .520 4.05 4.07 .841

10/29/201037 | |

Interpretation of Findings• At the end of the semester, there were significant

differences on all the public affairs subscales and self-efficacy• Students in the service learning sections reported greater levels

of cultural competence, community engagement, ethical leadership, and self-efficacy

• Marginally significant differences were observed for planning and reflection

10/29/201038 | |

Interpretation of Findings• Throughout the semester, change was

more pronounced for students in the service learning sections- Cultural competence, community engagement

and planning

10/29/201039 | |

Contributions to the field• Missouri State’s research underscores findings of

other service-learning studies, i.e., Astin, & Sax, Eyler & Giles,

Astin, Volelgesang, Ikeada & Yee, Bringle & Hatcher, in regard to self-efficacy, leadership and engagement

• Students in service-learning sections reported greater levels of cultural competence, ethical leadership, and community engagement

• Service-learning impacted the increase in cultural competence, one pillar of the Public Affairs Mission

10/29/201040 | |

Continued Research• Introduction to University Life (IDS 110) moved

from a 1 hour course to a 2 hour course (GEP 101)

• Nine SL sections available• 97 sections overall• Research continues this semester

10/29/201041 | |

Questions

• How do service-learning experiences in the first year shape the rest of a student’s college career?

• How do service-learning experiences in the first year impact a student’s commitment to service beyond college?

10/29/201042 | |

Overall Implications• Public Affairs Scale Development

• Operationalization of the Public Affairs Mission – develop Public Affairs scale

• Assessment of the development of the Public Affairs pillars in students-validity of scale

• Outcomes of civically engaged students from perspectives of community partners and students

• This last point is the goal of some future research

• Emphasize continued reciprocity

10/29/201043 | |

We would love tohear from you!

• Chantal Levesque-Bristol• [email protected]

• Elizabeth Carmichael Strong• [email protected]

10/29/201044 | |

BibliographyAstin, A.W.(1993). What matters in college? Four critical years revisited. San Francisco: Jossey-Bass.Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. Journal of

College Student Development, 39(3), 251-263.Astin,A.W., Volelgesang, L.J., Ikeda, E.K., & Yee, J.A. (2000). How service-learning affects students.

Higher Education Research Institute, University of California, Los Angeles.Bringle, R.G., & Hatcher, J.A. (1995). A service-learning curriculum for faculty. Michigan Journal of

Community Service Learning, 2, 112-122.Eyler, J., & Giles, D.E. (1999). Where’s the learning in service-learning? San Francisco: Jossey Bass.Eyler, J., Giles, D. Stenson, C. & Gray, C. (2001). What We Know about The Effects of Service-

Learning on College Students, Faculty, Institutions and Communities, 1993-2000. Third Edition. Scotts Valley, CA: Learn and Serve America National Service—Learning Clearinghouse.

Tyree, T. M. (1998). Designing and instrument to measure socially responsible leadership using the social change model of leadership development. Unpublished doctoral dissertation, University of Maryland, College Park, MD.

Zlotkowski, Edward. (2002). Service-Learning and The First-Year Experience: Preparing Students for Personal Success and Civic Responsibility (Monograph No. 34). Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition.