Development and Evaluation of Communities of Practice (CoPs) in Hong Kong Eva WONG & Theresa KWONG (King CHONG, Dimple THADANI & Wing Leung WONG) Centre for Holistic Teaching and Learning Hong Kong Baptist University Lilly International Spring Conference COLLEGE AND UNIVERSITY TEACHING AND LEARNING May 28-31, 2015
35
Embed
Development and Evaluation of Communities of Practice (CoPs) …chtl.hkbu.edu.hk/te/presentation/lilly15-4.pdf · 2015-07-14 · Development and Evaluation of Communities of Practice
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Development and Evaluation of Communities of Practice (CoPs) in Hong Kong
Eva WONG & Theresa KWONG
(King CHONG, Dimple THADANI & Wing Leung WONG)
Centre for Holistic Teaching and Learning Hong Kong Baptist University
Lilly International Spring Conference COLLEGE AND UNIVERSITY TEACHING AND LEARNING
May 28-31, 2015
Outline • Introduction to Hong Kong Baptist University
(HKBU)
• Development of CoPs at HKBU
• The Master Assessment Plan for CoPs at HKBU
• Preliminary findings
• Summary and Implications
Where is Hong Kong Baptist University?
Hong Kong Baptist University
• Situated in an expensive area of town
• Easy access
• A “long” campus consisting of “old” and “new” buildings
4
We are here!
• Established in 1956 with a Christian heritage • 3 Faculties, 4 Schools & an Academy ‒ Academy of Visual Arts ‒ Faculty of Arts ‒ Faculty of Science ‒ Faculty of Social Sciences ‒ School of Business ‒ School of Chinese Medicine ‒ School of Communication ‒ School of Continuing Education
• Programmes from Associate Degree to PhD • ~ 10,000 students • ~ 2,400 staff
5
Development of CoPs in Hong Kong • The “Aspirations for the Higher Education System in Hong Kong”
report published in December 2010. – enhance teaching and learning – a network of outstanding higher-education educators in Hong Kong to lead
Communities of Practice (CoPs) both within and across institutions
• CoPs can significantly help faculties
– improve faculty teaching – benefit student learning (Handzic and Lagumdzija, 2010) – relatively new to Hong Kong’s higher education
• Faculty Community of Practice (CoP)
– a cross-disciplinary group of faculty and staff members engaging in collaborative activities for enhancing teaching and learning (Cox, 2004)
• HKBU received start-up funding from the University Grants Committee (UGC), matched with its own funds to establish Communities of Practice (CoPs).
• Whole Person Education (WPE) - overarching principle of CoPs: Help HKBU students to think creatively, critically and positively
• 5 themes: 1) Development of the new 4-year degree - Academic Advising 2) Assessing learning outcomes - programme ILOs and University’s Graduate
Attributes 3) Enhancing Teaching & Learning - online resources and eTools 4) Enabling students to achieve excellence – showcasing/ publishing quality
student work 5) Establishing a community of scholars for interdisciplinarity
Development of CoPs at HKBU
Establishment of CoPs
Milestones of CoPs Development at HKBU
UGC Supported Retreat on CoPs: learn about CoPs
Successfully secured the start-up funding from the UGC
University-wide Briefing on establishing CoPs
Sharing of CoPs experience
Workshop Series conducted by CoP consultant on how to establish CoPs
Conducting evaluation to provide feedback about CoPs’ effectiveness
Selection Criteria of CoPs project
More details at http://chtl.hkbu.edu.hk/main/cops-funding/
Community of Practice on Whole-person Education in Medical Services
Development of Teaching Portfolio Framework though a Multidisciplinary Community of Practice
Enhancing students’ Graduate Attribute (GAs) though Problem-based Learning and Service Learning in Formal Academic Courses
Establishing a Community of Multidisciplinary Scholars to Support Education in Environmental Protection and Conservation in Hong Kong
A community of Practice to Create a Model for Student e-portfolio as a Tool for Life-long Learning and Assessment
Enhancing Student Learning through Holistic Mentoring Programme and Comprehensive Proficiency Test in Analytical Science
The Importance of Assessment to CoPs
Hubball et al (2004) envisage that evaluation represents a
key to the long-term success and sustainability of CoPs
Development of a robust and assessment plan is crucial,
to gather sound evidence for evaluation and comparison.
Master Assessment Plan for the CoPs at HKBU
• Central theme: – Determinate whether the CoPs are successful, and how – Detect changes of faculty participants in their teaching – Ascertain influences on student learning
• Goals and outcomes are:
– Provide evidence – Ascertain students’ achievement of leaning outcomes -
the Graduate Attributes (GAs) of HKBU – Compare with CoPs established in the US – Identify the strengths and weaknesses, and experience
sharing
Assessment Framework
Evidence Collection
Direct
Faculties (Changes in teaching & learning
practices)
Students (Representative assignments)
Indirect
Faculties (Questionnaire + Focus Group Interviews)
Students (Survey + interviews)
Study Methods Questionnaire
• By Beach & Cox (2009) , well established;
• Investigate how faculties’ participation in CoPs has enhanced their teaching, and help improve student learning;
• 6 categories with a total of 94 items;
• 5-point Likert scale (5: highest; 1 lowest);
• questionnaire survey conducted online;
• Focus group interviews - enrich understanding.
Focus Group Interview
• Conducted with faculty members who had completed the questionnaire;
• Answered 6 open-ended questions developed;
• One hour interview – each group;
• Audio-taped and transcribed.
Main Structure of the Questionnaire Categories Number of
Items Measurement Scale
(5= highest; 1 = lowest) 1 Your changes as a result of
the CoP participation 13 Degree of Impact:
5 = a very substantial impact 1 = no impact
2 Changes in personal attitude that affected student learning
3 Teaching and learning approaches implemented that resulted in changed student learning
19 Degree of Implementation: 5 = a very substantial amount 1 = not at all
4 Assessments used to judge student learning changes
21 Degree of Adoption: 5 = a very substantial amount 1 = not at all
5 Changes in student learning outcomes as a result of your CoP Project’s activities
31 Degree of Changes: 5 = a very substantial amount 1 = not at all
6 Overall 2 Degree of Changes: 5= a very substantial amount 1 = not at all
94
Open-ended Questions Focus Group Interview
Questions
1. What overall changes have you experienced after your participation in the CoP?
2. What are the change(s) in your personal attitude that affected student learning, after your participation in the CoP?
3. What teaching and learning approach (es) have you implemented as the result of your participation in the CoP?
4. What assessment method(s) have you used to judge the changes in student learning?
5. What change(s) have you observed in student learning outcomes as the result of your participation in the CoP?
6. What is your overall experience from your participation in the CoP? Do you have any suggestions for better implementing your CoP project (e.g. operation and arrangements etc)?
Preliminary Findings
No. of Members
No. of Responses
Response Rate (%)^
CoP 1 10 10 100.0
CoP 2 11 1 9.1
CoP 3 9 9 100.0
CoP 4 8 6 75.0
CoP 5 12 5 41.7
CoP 6 10 6 60.0
^ As advised by our CoPs Consultant: 1. Size for each CoP is between 8 and 12 members; 2. Only CoPs with response rate >= 60.0% will be included in
the analysis; therefore, only CoPs1, 3, 4 & 6 are included.
* Focus group interview for CoP 4 was still on going at the time of preparing for this presentation.
Profile of Focus Group Interview
Activities that your CoP Project has Undertaken (Can provide more than 1 answer)
8.0%
5 Designed/employed technology in a course AND Incorporated connections with professional opportunities
Total number of count = 125
4
10.4%
Designed guidelines for engaging in a process (e.g. discussion)
3
11.2%
Made class/ curriculum more inclusive
2
13.6%
Surveyed students to obtain information to incorporate into your teaching
1 Learned about and incorporated approaches to reach different learning styles
15.2%
Your Changes as a Result of the CoP Participation (Questionnaire)
*P-value in BLUE signifies that there is a significant difference at α=0.05
HKBU
(CoP 1, 3, 4 and 6)
Beach & Cox (2009) 6 universities in US
(Table5)
Mean S.D. N Mean S.D. N P-value 01. Perspective on teaching and learning /other aspects of higher education beyond discipline 3.58 0.76 26 3.93 1.11 369 0.0273
02. Interest in teaching process 3.73 0.78 26 3.86 1.08 361 0.4276 03. Understanding and interest in scholarship of teaching 3.67 1.09 24 3.80 1.14 368 0.5630
04. View of teaching as an intellectual pursuit 3.81 0.75 26 3.74 1.16 364 0.6706 05. Comfort level as a member of the university community 3.75 0.84 28 3.55 1.18 374 0.2424
06. Total effectiveness as a teacher 3.59 0.75 27 3.55 1.14 354 0.7851 07. Awareness of ways to integrate teaching/research experience 3.73 1.00 26 3.41 1.27 365 0.1230
08. Technical skill as a teacher 3.33 0.92 24 3.38 1.46 341 0.8185 09. Awareness of how to serve student learning needs 3.79 0.98 29 3.33 1.19 354 0.0165 10. Confidence in addressing student needs in/out of class 3.72 0.84 29 3.29 1.20 352 0.0105
11. Understanding of your role at the university 3.55 0.87 29 3.26 1.23 358 0.0946 12. Awareness of how diversity influences/enhances teaching and learning 4.11 0.85 27 3.25 1.30 351 0.0000
13. Research and scholarly interest with respect to discipline. 3.40 0.82 25 3.18 1.27 366 0.2128
Your Changes as a Result of the CoP Participation (Focus group interview)
Quotations
“Via sharing with my students, I found necessary to help students foster their academic, career, and life development……”
“I designed service learning in one of my courses as the result of participating in CoP, I am happy to know that my students like the course contents, they think what they have learned inside/outside classroom is useful ……”
“I found that my students come from different regions with different learning, working, and life experience…… so I am implementing several teaching approaches to enhance their interactions……”
HKBU
(CoP 1, 3, 4 & 6)
Mean S.D. N
09. Awareness of how to serve student learning needs
3.79 0.98 29
10. Confidence in addressing student needs in/out of class
3.72 0.84 29
12. Awareness of how diversity influences/ enhances teaching and learning
4.11 0.85 27
Changes in Personal Attitude that Affected Student Learning (Questionnaire)
*P-value in BLUE signifies that there is a significant difference at α=0.05
Assessments Used to Judge Student Learning Changes (Focus group interview)
Quotations
“Statistics show that lending of CoP books is much higher than other books.”
“Student have developed abilities to think deeply and independently on clinical practices…… such as better writing of patient records.”
“Students have applied what they had learnt not only inside but also outside classroom, they help their family members by what they have learnt, this is an outcome that I did not expect, it is a surprise!”
“Students, who were resistant to us in the past, now become more positive……”
HKBU
(4 CoPs) Mean S.D. n
04. Students more interested 3.67 0.86 21
07. Students more motivated 3.71 1.01 21
06. Deeper learning 3.85 1.04 20
09. Your successful achievement of new/more learning objectives 3.65 0.99 20
11. Increased beneficial connections with students outside the classroom
3.90 1.00 21
Changes in Student Learning Outcomes as a Result of your CoP Project’s Activities (Questionnaire)
HKBU
(4 CoPs)
Beach & Cox (2009) 6 universities in US
(Table 7)
Mean S.D. n Mean S.D. n P-value 01. Development of an ability to work productively with others 3.57 1.16 23 3.50 1.17 233 0.7975
02. Development of an openness to new ideas 3.65 1.07 23 3.46 1.20 235 0.4174 03. Development of the capacity to think for oneself 3.65 1.23 23 3.44 1.21 237 0.4293 04. Ability to think holistically – to see the whole as well as the parts 3.70 1.15 23 3.39 1.23 241 0.2256
05. Understanding of perspectives and values of course or discipline 3.64 1.09 22 3.39 1.21 228 0.3184
06. Ability to think creatively 3.68 1.09 22 3.38 1.22 242 0.2181 07. Ability to synthesize and integrate information and ideas 3.65 1.11 23 3.37 1.19 244 0.2488 08. Improved learning of concepts and theories 3.74 1.05 19 3.36 1.17 241 0.1352 09. Ability to apply principles and generalizations already learned to new problems and situations 3.63 1.12 19 3.35 1.15 236 0.2922
10. Problem-solving skills 3.68 1.09 22 3.35 1.17 240 0.1743 11. Analytical skills 3.70 1.02 23 3.32 1.13 234 0.0963 12. Skill in use of techniques and methods used to gain new knowledge 3.65 0.99 20 3.31 1.13 231 0.1460
13. Skill in using materials and tools central to course or discipline 3.63 1.01 19 3.30 1.23 234 0.1782
14. Ability to ask good questions 3.48 1.12 21 3.28 1.18 242 0.4452 15. Development of respect for others 3.52 1.24 23 3.26 1.26 220 0.3366 16. Ability to evaluate methods and materials in a course or discipline 3.53 1.02 19 3.24 1.20 221 0.2487
*P-value in BLUE signifies that there is a significant difference at α=0.05
HKBU
(4 CoPs)
Beach & Cox (2009) 6 universities in US
(Table 7)
Mean S.D. n Mean S.D. n P-value 17. Ability to draw reasonable inferences from observations 3.68 0.95 19 3.22 1.18 233 0.0450 18. Development of a multidisciplinary perspective 3.70 1.08 20 3.15 1.28 211 0.0336 19. Development of a lifelong love of learning 3.45 1.06 22 3.14 1.23 219 0.1915 20. Skill in using technology 3.23 0.97 22 3.14 1.32 214 0.6999 21. An increased rate of intellectual development 3.45 0.96 22 3.12 1.26 198 0.1366 22. Development of an informed concern about contemporary social issues 3.56 1.31 16 3.10 1.31 210 0.1763
23. Improved writing skills 3.05 1.13 19 3.00 1.27 220 0.8472 24. Development of an aesthetic appreciation in a course or discipline 3.42 1.17 19 2.98 1.33 202 0.1221
25. Development of an informed appreciation of other cultures 3.38 1.20 16 2.98 1.35 210 0.2113
26. Development of a capacity to make informed ethical choices 3.39 1.24 18 2.97 1.40 205 0.1765
27. Ability to develop appropriate study skills, strategies, habits 3.45 1.15 20 2.96 1.21 211 0.0703
28. Improved learning of terms and facts 3.63 1.12 19 2.91 1.24 217 0.0079 29. Development of an informed historical perspective 3.28 1.18 18 2.82 1.32 197 0.1200 30. Development of a commitment to exercise the rights and responsibilities of citizenship 3.53 1.31 19 2.72 1.36 194 0.0112
*P-value in BLUE signifies that there is a significant difference at α=0.05
Changes in Student Learning Outcomes as a Result of your CoP Project’s Activities (Questionnaire) cont’d
Changes in Student Learning Outcomes as a Result of your CoP Project’s Activities (Focus group interview)
Quotations
“…… students have improved their thinking pattern and become more mature when solving problems.”
“Some students have read extra references and asked me questions…… I discussed with them about these questions, which I believe also beneficial for me as a teacher.”
“…… the internship project can help students enhance knowledge and improve communication skills…… I think that it is an approach to whole person education.”
“Student have applied the health-related knowledge which they had learnt to serve elders……”
“To implement service learning, we have invited some elders (i.e. patients) to offer students internship opportunities…… our students’ services were appreciated by these elders……”
HKBU
(4 CoPs)
Mean S.D. n
10. Problem-solving skills 3.68 1.09 22
14. Ability to ask good questions 3.48 1.12 21
28. Improved learning of terms and facts 3.63 1.12 19
30. Development of a commitment to exercise the rights and responsibilities of citizenship
3.53 1.31 19
31. Utilization of internship experience 3.36 1.29 11
Overall (Questionnaire and Focus Group Interview)
HKBU
(4 CoPs)
Beach & Cox (2009)
6 universities in US
Mean S.D. n Mean 1. The degree to which you believe that student learning
changed as a result of your participation in CoP 3.47 1.01 30 3.29
2. The extent to which you believe that student learning changed as a result of changes in your personal attitude about teaching
3.43 1.00 28 3.10
*P-value cannot be compiled since there are no corresponding S.D. and number of responses provided from the paper.
Quotations
“Adoption of service learning made our teaching vivid, it is good for training students’ critical thinking…… the learning outcome(s) can be achieved.”
“Through participating in CoP, I further transformed my requirements into more detailed steps and explicitly communicate them to students……. my students’ responses were positive because they knew how to achieve.”
“…… students were offered with activities to cooperate, such as organisation and allocation of tasks ……”
Quotations “Previously I thought that students should be self-motivated; after participating in CoP, I found that students preferred teachers to stimulate their learning, they are more willing to improve themselves……”
“After joining CoP, I have become more sensitive…… I have unpacked my assessment components…… students’ responses were positive and they improved.”
“Before CoP implemented, our mentoring approach was just to have meals with students; after joining this CoP, we spent more time in designing activities to help students enhance teamwork skills.”
Summary and Implications • Participating faculty members reported benefitting from
the CoPs: – General changes – Teaching attitudes – Teaching and learning approaches – Assessment methods
• Faculty members’ CoP participation led to enhanced student learning
• A well-grounded conclusion will be made after integrating with the direct evidence from students’ assignments and faculties’ pedagogies and teaching practices.
• Implementation of CoPs at HKBU is now being deepened by continuing support to selected established CoPs and inviting further applications.
Questions?
Thank you!
Selected References 1. Barr, R. B. & Tagg, J. (1995). From teaching to learning – a new paradigm for undergraduate education. Change
(November/December), 13-25. 2. Beach, A. L., & Cox, M. D. (2009). The impact of faculty learning communities on teaching and learning.
Learning Communities Journal, 1 (1), 7-27. 3. Cox, M. D. (1999) Peer consultation and faculty learning communities. In C. Knapper & Piccinin (Eds.), Using
Consultation to Improve Teaching: New Directions for teaching and Learning (Vol. 79, pp. 39-49). San Francisco: Jossey-Bass.
4. Cox, M. D. (2001) Faculty learning communities: Change agent institutions into learning organizations. In D. Lieberman and C. Wehlberg (Eds.), To Improve the Academy (Vo;. 19, pp. 69-93). Bolton, MA: Anker Publishing.
5. Cox, M. D. (2004). Introduction to faculty learning communities. In M.D. Cox & L. Richlin (Eds.), Building Faculty Learning Communities: New Directions for Teaching and Learning (Vol. 97, pp. 5-23). San Francisco: Jossey-Bass.
6. Dees, D. M., Zavota, G., Emens, S., Harper, M., Kan, K. H., Niesz, T., Tu, T-H., Devine, M. A., & Hovhannisyan, G. (2009) Shift professional identities: reflections on a faculty learning community experience. Leaning Communities Journal, 1 (2), 49-73.
7. Handzic, M & lagumdzija, A. (2010). Global faculty learning community: as case study. Knowledge Management Research & Practice, 8, 89-98.
8. HKBU. (2012). Proposal for the communities of Practice (CoPs) Initiative. 9. Hubbal, H., Clarke, A., & Beach, A. L. (2004). Assessing faculty learning communities, in M. D. Cox & L. Richlin
(Eds.), Building faculty learning communities: New Directions for Teaching and Learning (Vol 97, pp. 87-100). San Francisco: Jossey-Bass.
10. Hubball, H.T., & Albon, S. (2007) Developing a faculty learning community: enhancing the scholarship of teaching, learning and curriculum practice, Journal on Excellence in College Teaching, 18 (2), 119-142.
11. Richlin, L., & Cox, M.D. (2004). Developing scholarly teaching and the scholarship of teaching and learning through faculty learning communities. In M.D. Cox & L. Richlin (Eds.), Building Faculty Learning Communities: New direction for teaching and learning (Vol 97, pp. 127-135). San Francisco: Jossey-Bass.
12. Taylor, D. (Ed.). (1997). Many Families, Many literacies: an International Declaration of principles. Portsmonth, NH: Heinemann.