Academic Assessment Committee€¦ · Travis Crone (FAC) (Conflicting Meeting) Present ☐ Michael Duncan (FAC) Present ☒ Shannon Fowler (FAC) Present ☒ Bryce Hantla Present ☒

1

Academic Assessment Committee Date: 9-26-17 Time: 2:30-3:30pm Room: A701 Note taker: Dr. Bryce Hantla

Committee Members: Linda Becerra (FAC) Present

☒ Utpal Bose (FAC) Present

☒ Kevin Buckler

Present

☒ Lea Campbell (Chair)

Present

☒

Travis Crone (FAC) (Conflicting Meeting)

Present

☐ Michael Duncan (FAC)

Present

☒ Shannon Fowler (FAC)

Present

☒ Bryce Hantla Present

☒

Michelle Hernandez-Perez

Present

☒ Melissa Hovsepian (FAC) (Conflicting Meeting) (

Present

☐ John Kelly (FAC) Present

☒ Faiza Khoja Present

☐

Lucas Logan (FAC) Present

☒ Ami Möller Present

☒ Sharon Noel Present

☒ Mike Tobin (FAC) Present

☒ Joe Westfall (FAC)

Present

☒ Present

☐ Present

☐ Present

☐

I. Approve 9/12/17 Minutes: Committee members reviewed and voted to approve the

minutes from 9/12/17 with one change recommended. Lea Campbell with request that

the approved minutes be posted to the committee’s website.

II. Finalize Assessment Due Dates for 2016/2017 Assessment Reports:

Current due dates for the 2016/2017 assessment reports are:

o Due from faculty to chair: October 30, 2017 o Due from chair to dean: November 15, 2017 o Due from dean to the Office of Academic Assessment: December 15, 2017

Faculty Assessment Coordinators and Assistant Directors of Assessment felt that the December 15th was workable.

Ms. Moller asked about October 31, 2017, as the website listed October 30, 2017.

Dr. Campbell affirmed that October 30 is the official date as noted in the Policy Statement 03A31 but October 31, 2017, would be sufficient as this is an internal due date. Dr. Campbell committed to updating the website to reflect the October 30th date as noted in policy.

III. AY2018 To-do List: The Academic Assessment Committee identifies tasks that need to

be completed during each academic year. Recommendations from the Office of

Assessment include:

a. Update the Assessment of Education Programs Policy (PS03.A.31)

Justification: The policy was last revised in May 2013 and needs to be updated

to reflect changes which have been made to the university’s assessment

processes and staffing. Assessment Resources Web Pages

2

b. Establish an Assessment Resource webpage for faculty and staff.

Justification: There has been a need for this resource page for some time but

the Office of Assessment has lacked the resource to develop the page. UHD

has recently hired additional assessment staff and Dr. Campbell feels that that

the office have the resources to now create a curated list of assessment

resources for academic programs. The Academic Assessment Committee

would be responsible for reviewing resources to ensure each section of the

page provided faculty with easily located, usable information.

c. Evaluate 2016-17 Assessment Reports

Justification: Review of academic assessment reports is a standing

responsibility of the Academic Assessment Committee.

The Academic Assessment Committee did not identify any additional tasks however,

members are able to add additional items to the list if a need arises.

IV. Begin Review/revision of the Academic Assessment Policy

a. 2.1: Definition of Educational Programs

i. Developmental Education Program

1. Move to Vote: Remove “The Developmental Education Program” (p.

1) aspect of the policy because it is not strictly an “academic

program.” (no move or second)

2. Vote: Faculty agreed that assessment of the Developmental

Education Program should be moved to a more appropriate area

(e.g., Student Support Services). The location for this move was not

determined and was established to need to be discussed “above” this

committee.

ii. General Education Program

1. Recommendation: Dr. Fowler recommended to remove “General

Education program” from this policy but make a mention of the

General Education Policy of the University in the definitions section

(perhaps a hyperlink).

2. FACs agreed that this is the clearest course of action for this policy.

iii. Stand-alone Minors

1. Dr. Campbell recommended that this bullet point be reserved for a

future discussion.

2. The FACs agreed.

b. 2.2: Interdisciplinary Programs

i. Before determining if this was an appropriate definition, FAC’s asked Dr.

Campbell to put together a list of interdisciplinary programs for the group to

review.

c. 2.3 Developmental Programs:

https://www.uhd.edu/administration/employment-services-operations/resources/Documents/PS03A31.pdf

3

i. Remove section 2.3 since Developmental Education programs were being

removed from the policy.

d. 2.4 Program Faculty

i. Remove reference to General Education oversight committee

ii. Remove reference to Developmental Education

iii. Install stand-alone minors

e. 2.5 Program Learning Outcomes:

i. Section needs to be maintained but certificate programs are not “graduates,

so Dr. Campbell will agreed to research the inclusion of these programs at

this point of the definition.

f. Committee agreed to pick up next time with “2.6 Assessment”

V. Additional Business

a. None

VI. Adjourn

Action Items:

Item Date Due Person

Responsible

Status

As of 9/29/17

Send out Policy on Co-Curricular Programs 1 week before

next meeting

Dr. Lea Campbell Complete

Finalize 9/12/17 minutes and send out to

committee

Dr. Bryce Hantla Complete

Have approved 9/12/17 minutes posted to the

Academic Assessment Committee’s webpage.

1 week before

next meeting


Update Academic Assessment Policy per the

changes prescribed in these minutes


Research Interdisciplinary Degree Programs and

related policies to determine whether we can

reference these.

Send out to Committee.

Gather a list of all programs that fall under this

heading.

1 week before

next meeting


Pull together a curated list of assessment resources

for Academic Programs. What types of resources

for faculty and center/institute directors, etc.

Next meeting Everyone for

their respective

area

In Progress

Next Meeting

October 10, 2017

2:30-3:30PM

A701

4

Attachments:

PS03.A.31, Issue No. 2: Assessment of Educational Programs (DRAFT REVISIONS)

Approved Committee Minutes (9/12/2017)

PS03.A.31, Issue No. 2: Assessment of Educational Programs

VII. Approve 9/12/17 Minutes: Committee members reviewed and voted to approve the

minutes from 9/12/17 with one change recommended. Lea Campbell with request that

the approved minutes be posted to the committee’s website.

VIII. Finalize Assessment Due Dates for 2016/2017 Assessment Reports:

Current due dates for the 2016/2017 assessment reports are:

o Due from faculty to chair: October 30, 2017 o Due from chair to dean: November 15, 2017 o Due from dean to thde Office of Academic Assessment: December 15, 2017

Faculty Assessment Coordinators and Assistant Directors of Assessment felt that the December 15th was workable.

Ms. Moeller asked about October 31, 2017, as the website listed October 30, 2017.

Dr. Campbell affirmed that October 30 is the official date as noted in the Policy Statement 03A31 but October 31, 2017, would be sufficient as this is an internal due date. Dr. Campbell committed to updating the website to reflect the October 30th date as noted in policy.

IX. AY2018 To-do List: The Academic Assessment Committee identifies tasks that need to

be completed during each academic year. Recommendations from the Office of

Assessment include:

a. Update the Assessment of Education Programs Policy (PS03.A.31)

Justification: The policy was last revised in May 2013 and needs to be updated

to reflect changes which have been made to the university’s assessment

processes and staffing. Assessment Resources Web Pages

b. Establish an Assessment Resource webpage for faculty and staff.

Justification: There has been a need for this resource page for some time but

the Office of Assessment has lacked the resource to develop the page. UHD

has recently hired additional assessment staff and Dr. Campbell feels that that

the office have the resources to now create a curated list of assessment

resources for academic programs. The Academic Assessment Committee

would be responsible for reviewing resources to ensure each section of the

page provided faculty with easily located, usable information.

c. Evaluate 2016-17 Assessment Reports

5

Justification: Review of academic assessment reports is a standing

responsibility of the Academic Assessment Committee.

The Academic Assessment Committee did not identify any additional tasks however,

members are able to add additional items to the list if a need arises.

X. Begin Review/revision of the Academic Assessment Policy

a. 2.1: Definition of Educational Programs

i. Developmental Education Program

1. Move to Vote: Remove “The Developmental Education Program” (p.

1) aspect of the policy because it is not strictly an “academic

program.” (no move or second)

2. Vote: Faculty agreed that assessment of the Developmental

Education Program should be moved to a more appropriate area

(e.g., Student Support Services). The location for this move was not

determined and was established to need to be discussed “above” this

committee.

ii. General Education Program

1. Recommendation: Dr. Fowler recommended to remove “General

Education program” from this policy but make a mention of the

General Education Policy of the University in the definitions section

(perhaps a hyperlink).

2. FACs agreed that this is the clearest course of action for this policy.

iii. Stand-alone Minors

1. Dr. Campbell recommended that this bullet point be reserved for a

future discussion.

2. The FACs agreed.

b. 2.2: Interdisciplinary Programs

i. Before determining if this was an appropriate definition, FAC’s asked Dr.

Campbell to put together a list of interdisciplinary programs for the group to

review.

c. 2.3 Developmental Programs:

i. Remove section 2.3 since Developmental Education programs were being

removed from the policy.

d. 2.4 Program Faculty

i. Remove reference to General Education oversight committee

ii. Remove reference to Developmental Education

iii. Install stand-alone minors

e. 2.5 Program Learning Outcomes:

i. Section needs to be maintained but certificate programs are not “graduates,

so Dr. Campbell will agreed to research the inclusion of these programs at

this point of the definition.

https://www.uhd.edu/administration/employment-services-operations/resources/Documents/PS03A31.pdf

6

f. Committee agreed to pick up next time with “2.6 Assessment”

XI. Additional Business

a. None

XII. Adjourn Action Items:

Item Date Due Person

Responsible

Status

As of 9/29/17

Send out Policy on Co-Curricular Programs 1 week before

next meeting


Finalize 9/12/17 minutes and send out to

committee

Dr. Bryce Hantla Complete

Have approved 9/12/17 minutes posted to the

Academic Assessment Committee’s webpage.

1 week before

next meeting


Update Academic Assessment Policy per the

changes prescribed in these minutes


Research Interdisciplinary Degree Programs and

related policies to determine whether we can

reference these.

Send out to Committee.

Gather a list of all programs that fall under this

heading.

1 week before

next meeting


Pull together a curated list of assessment resources

for Academic Programs. What types of resources

for faculty and center/institute directors, etc.

Next meeting Everyone for

their respective

area

In Progress

Next Meeting

October 10, 2017

2:30-3:30PM

A701

Attachments:

Approved Committee Minutes (9/12/2017)

7

Approved Committee Minutes 9/12/2017

Academic Assessment Committee Date: [9-12-17] Time: 2:30-

3:30pm Room: A701 Note taker: Lea Campbell



☒ Kevin Buckler

Present

☒ Lea Campbell (Chair) Present

☒

Travis Crone (FAC) (Conflicting meeting)

Present


Present


Present


☒


Present

☒ Melissa Hovsepian (FAC)

Present

☒ John Kelly (FAC) Present


☒


☒ Ami Moller Present

☒ Noel, Sharon Present


☒ Westfall, Joe (Conflict) (FAC)

Present

☐ Present

☐ Present

☐ Present

☐

FAC Compensation: Faculty Assessment Coordinators (FAC) overload and stipend

requests have been submitted to the Provost’s office. FAC’s were asked to notify

the Office of Academic Assessment if stipends were not reflected on the next

paycheck.

TK20 Update:

Single-sign-on has been set up so staff and faculty can log in using network

credentials.

Blackboard integration is in development

Couse upload complete – this supports curriculum mapping and course-based

assessment in the MDCOB.

Outcomes are being entered

Assessment report template has been created

Assessment rubrics, etc. are being entered into TK20

Academic Assessment Processes for AY17 Assessment Reports: Since UHD is

between assessment management systems, faculty will be asked to submit their

16/17 assessment reports on the 4-column Word template (see attached) and the

assessment staff will load those into TK20. Current due dates are:

Due from faculty to chair: October 30, 2017

Due from chair to dean: November 15, 2017

Due from dean to the Office of Academic Assessment (OAA): December 15, 2017 Dr. Campbell requested that FACS and assistant assessment directors speak with their faculty to see if those dates are still viable given the disruption from hurricane Harvey.

General Education: UHD is in the last year of a three-year pilot of the new General

Education assessment process. This year Oral Communication, Visual

8

Communication, Social Responsibility rand Personal Responsibility will be assessed.

Dr. Hantla has been working with the Communication Studies faculty to identify a

process to capture oral presentations. While initially designed for Gen Ed, this

process could be of value to all faculty who wish to add an oral communications

component to their courses. Once the kinks are worked out, the process will be

shared with all faculty.

The 2015/2016 General Education assessment report is still in draft form and there

will be extensive efforts made to finally act on those data. The General Education

Committee will meet 9/27/17 and finalize dates for the Fall 2017 Assess-a-fest and

the January 2018 Core Evaluation meeting. Dr. Khoja requested that the General

Education Committee review the 2015/2016 General Education assessment report

and the Academic Committee Members asked to be included on emails reminding

faculty teaching in the Core to submit signature assignments. The Academic

Assessment Committee will also receive the 2016/2017 General Education

assessment report once it is drafted by the General Education Committee.

FAC Update: FAC’s and other members of the committee provided updates on

assessment within their areas.

New Business: None at this time.

Next Meeting: September 26, 2017, 2:30pm in A701

Attachments:

AY17 Academic Assessment Report Template

AY16 General Education Assessment Report (DRAFT)

9

Academic Program:_______________________

2016-2017 ASSESSMENT REPORT TEMPLATE Due Dates



Due from dean to the Office of Academic Assessment (OAA): December 15, 2017

Outcome Methods of Assessment and Related Success Criteria

Summary of Findings Use of Findings

Enter outcome assessed in 2016/2017

Enter assessment strategy and related success criterion

AM#1 1. N = ________ 2. Overview of process of evaluation: 3. Summary of findings: 4. Interpretation of findings: 5. Was success criteria met? (Yes; No, Partially Met, or Data not collected) Attach copies of data sets, longitudinal data, instruments used to collect data rubrics and meeting notes.

Strategies for Improving Learning MANDATORY if student performance fell below

success criteria Include timeline for implementation

Strategies for Improving Assessment

Optional Include timeline for implementation


AM#2: 1. N = ________ 2. Overview of process of evaluation: 3. Summary of findings: 4. Interpretation of findings: 5. Was success criteria met? (Yes; No, Partially Met, or Data not collected) Attach copies of data sets, longitudinal data, instruments used to collect data rubrics and meeting notes.





Add rows as necessary

10


Required: Update on prior year’s assessment: In the space below, provide an update on assessment activities from 2015/2016

LO Assessed in

2015/2016

Summary of Improvements Proposed in the 2015/2016 Assessment Report:

Provide a Status Update on Program Improvements Recommended in the Spring 2017 Assessment Report (one initiative per line). Indicate if the initiative is:

a) Completely implemented b) Currently being implemented c) Planning phase d) Not started e) Replaced by alternate strategy (please provide details on what that strategy is and what

prompted the decision to select a new improvement strategy.)

Enter outcome measured

in 2015/2016.

Enter strategies for improvement – one improvement per line.

Update.


Update.


OPTIONAL: Additional Improvement Strategies

STRATEGY TO IMPROVE WHICH FACULTY/STAFF ARE IMPLEMENTING: ISSUE, CONCERN OR PROBLEM FACULTY/STAFF ARE RESPONDING TO:

Have the academic program or unit revised the assessment plan? If “Yes,” please provide an updated assessment plan with changes highlighted in yellow. This is requested so we can update the assessment records in TK20. Signature of Chair_________________________________________________________ Date:______________________________ Signature of Dean _________________________________________________________ Date:______________________________

Yes No

11

AY2016 General Education Core Assessment AY 2016 Core Assessment Summary

1. Faculty assessed sophomores’ Written Communication and Teamwork skills using AAC&U rubrics in a two one-day workshop on January 14 and May 24, 2016. 2. Findings:

a. Direct measure: Rubric evaluation of student work. i. Success criterion of 70% of students will score at least 2.5 on a 4 pt. rubric was not met on either measure.

ii. Jan 14: Written Communication: 9 of 50 or 18% scored at least 2.5 May 24: Written Communication: 10 of 431 or 23% scored at least 2.5 TOTAL: Written Communication: 19 of 93 or 20% scored at least 2.5

iii. Jan 14: Teamwork Part 1 (Mult. Viewpoints): 12 of 49 or 24.5% scored at least 2.5. May 24: Teamwork Part 1 (Mult. Viewpoints): 14 of 58 or 24.1% scored at least 2.5

TOTAL: Teamwork Part 1 (Mult. Viewpoints): 26 of 107 or 24.2% scored at least 2.5 iv. Jan 14: Teamwork Part 2: 10 of 49 or 20% scored at least 2.5.

May 24: Teamwork Part 2: 15 of 58 or 26% scored at least 2.5 TOTAL: Teamwork Part 2: 25 of 107 or 23% scored at least 2.5

b. Indirect measures: Student Survey i. Success criterion of 70% of students confident or very confident

ii. Survey re: Written Com 1. N = 2830; Responses: 100; Response rate: 3.5%

a. Percent of students who were somewhat confident or very confident in their ability to develop ideas in writing: Sophomores: 78%

All respondents: 75% b. Percent of students who were somewhat confident or very confident in their ability to interpret ideas in writing.

Sophomores: 83% All respondents: 79%

iii. Survey re: Teamwork 1. N = 1645; Responses: 54; Response Rate: 3.3%

a. Percent of students who were somewhat or very confident in their ability to objectively consider viewpoints that differ from their own.

Sophomores: 62.5 % All respondents: 74.1%

b. Percent of students who were somewhat or very confident in their ability to work effectively with others to achieve a common goal:

Sophomores: 62.5% All respondents: 80%

1 Could not use 7 evaluations due to one reader not providing ratings for all dimensions.

12

3. Interpretation: Faculty will need to interpret findings 4. Recommended improvement strategies: 5. Strategies to improve learning: Faculty will need to provide improvements. 6. Recommended strategies to improve assessment:

i. Post copies of assignments well-aligned with rubrics on the Core Assessment webpage ii. Assist faculty in aligning assignments and rubrics, with special outreach to adjuncts.

iii. Revise “Context & Purpose” language in the Written Communication rubric. iv. Revise “Resolving Conflicts” language in the Teamwork rubric. v. Work with the NS faculty to determine which disciplines/courses will embed Teamwork Signature Assignments in labs and which will embed

the assignment in lectures. vi. Refine upload/download instructions.

vii. Increase response rate on student surveys by: 1. Distributing the survey in both the Fall and Spring; 2. Seeking assistance from faculty and Student Affairs to make students aware of the survey through other means besides Gatormail;

and 3. Sending multiple reminders.

Assessment Strategy Summary of Findings Use of Findings

Written Communication: Students who complete the core will be able to effectively develop, express and interpret ideas through written communication.

Direct Assessment Embedded Assessment: AAC&U rubric-based evaluation of standardized and embedded assignments randomly selected from several sections of multiple core courses Target Population: FTIC/FT sophomores Sample Size: During the pilot, 50 randomly selected artifacts drawn from an

N = 93 Fall =50 Spring = 43 (Originally 50 items were reviewed however, 7 samples were disqualified because one evaluator failed to enter scores.) Overview of collection and evaluation processes: Faculty teaching in the Written Communication, American History, and Language, Philosophy and Culture courses in the core assigned a Written Communication Signature Assignment during Fall 2015 and Spring 2016. Prior to grading, faculty archived students’ signature assignment submissions along with the assignment prompts in Blackboard 2. After the close of each semester, a random sample of 50 papers was drawn from sophomore-level students in the archive. Student, faculty and course identifiers were removed by Office of Institutional Effectiveness staff.

Recommended Strategies to Improve Learning: Recommendations to be provided by faculty. Strategy WC 1: As noted under findings, some assignments were not completely aligned with the rubric. The committee recommends working with faculty to better align assignments with the rubric and conducting a second round of assessment in AY2017.

Recommended Strategies to Improve Assessment: Strategy WC2: Student work was received from 79% of the 129 F2015 sections required to submit. A file corruption error prevented sampling from three sections in the archive. Student work was received from 85% of the 113 Spring 2016 sections required to submit. Both semesters,

13


archive of student work. Once fully implemented the number of selected artifacts will increase to 100. Success Criterion: At least 70% of student artifacts will score 2.5 or above.

Deans nominated 10 full-time faculty in the Fall and a second group of 8 faculty in the Spring to serve as evaluators of the artifacts. Faculty met on January 14, 2016 and on May 24, 2016, to evaluate student work. After a brief norming session, each student artifact was evaluated by at least 2 faculty using the AAC&U Written Communication rubric. A third readers was used when there was a lack of consensus among the first 2 readers. Summary of findings: Fall: 9 of the 50 (18%) student artifacts received a mean score of at least 2.5 on a 4-point scale. Spring: 10 of 432 or 23% of student artifacts received a mean score of at least 2.5 on a 4-point scale. Overall: 19 of 93 or 20% of student artifacts received a mean score of at least 2.5 on a 4-point scale. Mean score by dimension was: Context: Fall 2015: 2 Spring 2016: 2.2 AY2016: 2.1 Content Development: Fall 2015: 1.7 Spring 2016: 1.9 AY2016: 1.7 Genre: Fall 2015: 1.6 Spring 2016: 1.8 AY2016: 1.7 Sources/Evidence: Fall 2015: 1.5

Spanish courses submitted assignments in Spanish and could not be evaluated since the Written Communication assessment is an evaluation of students’ ability to writing in English. Some assignments were not well-aligned with the rubric and some assignment prompts were not provided. Professional development will be provided to help faculty;

a) Align writing prompts with the rubric; and b) Use technology to efficiently archive student

work and assignment prompts. The committee will make a special attempt to work with adjuncts. Strategy WC3: The committee will post 3 well-constructed writing prompts on the Core Assessment Website as exemplars. Faculty will receive copies of these assignments as part of the professional development training noted in Strategy 1. Strategy WC4: The committee recommends that faculty avoid using essay questions from tests as the Written Communication Signature Assignment since such responses are not usually long enough for students to demonstrate all the dimensions of the rubric. Also test responses do not allow students the time needed to polish their responses which would better represent the full range of their ability. Strategy WC5: The least amount of consensus among faculty evaluators occurred on the Context/purpose dimension. The Core Assessment Committee will have a group of faculty review the rubric and make recommendations on how best to clarify that dimension.


14


Spring 2016: 1.9 AY2016: 1.7 Mechanics: Fall 2015: 2 Spring 2016: 2 AY2016: 2 Overall: Fall 2015: 1.7 Spring 2015: 1.9 AY2016: 1.8 Faculty did not reach consensus 26% of the time on the Context/Purpose dimension, followed by Genre and Sources (14% respectively). Interpretation of findings: Additional analysis to be provided by faculty: This assessment marks the first evaluation of Written Communication under the revised General Education Core which was implemented Fall 2014. Not all assignments were well-aligned with the rubric, which contributed to the difficulty faculty had in reaching consensus on some papers and possibly the low scores overall. The success criterion was not met. Please see attached for:

1. AY2016_Core_Assesment_Faculty_Evaluators 2. AY2016_Core_Assesment_AACU_Written_Comm_Rubric 3. AY2016_Core_Assesment_Written_Comm_Data_Summary 4. AY2016_Core_Assesment_REVISED_Submission_Instructions

Strategy WC6: Fall 2015 was the first semester UHD used an archival process to collect student work. The TTLC assisted the faculty in developing a written set of instructions however some of the information needs to be revised and clarified. UPDATE-Strategy WC6: The Professional Writing faculty reviewed the instructions and provided edits. The instruction been revised and distributed to faculty. A copy of the revised instructions can be found in the attachments.

Indirect Assessment UHD Core Objectives Student Survey Target Population: FTIC/FT sophomores

N = 2830 Responses: 100 Response Rate: 3.5% Overview of collection and evaluation processes: Students in courses in the Written Communication, Language, Philosophy & Culture courses in the Core curriculum were asked to complete a survey in May 2016. A total of 100

Additional recommendations to be provided by faculty. Recommended Strategies to Improve Assessment Strategy WC7: The Office of Academic Assessment will assist faculty in improving the response rate on the survey to include:

15


Target Response Rate: At least 50% of all FTIC/FT sophomores Success criterion: At least 70% of respondents will agree or strongly agree they are confident in their ability to develop and interpretation of ideas in writing.

students responded but since these courses contain students at multiple levels, responses were received from Freshmen and Sophomores as well as Juniors and Seniors. Results are presented for the aggregate group as well as for Sophomores alone.

Freshman 23 23% Sophomore 18 18% Junior 34 34% Senior 25 25% Post-bac 0 0% Other 0 0% Total 100 Summary of findings: Percent of fulltime (4+classes) Sophomores: 67% All respondents: 67% Percent of students who were somewhat confident or very confident in their ability to interpret ideas in writing. Sophomores: 83% All respondents: 79% Percent of students who were somewhat confident or very confident in their ability to develop ideas in writing: Sophomores: 78% All respondents: 75% These two survey items form the basis of the indirect assessment of the Written Communication outcome. In both instances Sophomores as well as the group as a whole met the success criterion. Additional questions were included in the survey to better understand how students experience writing instruction at UHD. Percent that felt their writing had improved somewhat or a great deal over the semester: Sophomores: 89%

Distributing the survey in both the Fall and Spring;

Seeking assistance from faculty and Student Affairs to make students aware of the survey through other means besides Gatormail; and

Sending multiple reminders.

16


All respondents: 80% Percent of students who sought help with writing 3 or more times: Sophomores: 46% from a classmate/UHD student 31% from a professors 85% indicated they had not used UHDs Writing Lab. All respondents: 27% from a classmate/UHD student 31% from a professors 69% indicated they had not used UHD’s Writing Lab. Percent of students, by classification, who used the Writing Lab at least once during the spring semester: Freshmen: 29% Sophomores: 15% Juniors: 31% Seniors: 41% Freshmen most often approached a professor for help: 91% reported asking for help from a professor at least once. Sophomores sought help from friends (67%), classmates (62%) and professors (63%) at roughly equal levels. Sixty-one percent of juniors sought help from a professor at least once. Professors were the most often used source among seniors (89% visited at least 1 time). Approximate total number of pages students reported writing in S2016: Full-time Freshmen: 41.3 pages (min 12 pages/max 51.5 pages) Part-time Freshmen: 43.6% (min 11 pages/max 58.5 pages) Full-time Sophomores: 30.6 pages (min 12 pages/max 52.5 pages) Part-time Sophomores: 39 pages (min 31.5 pages/max 40.5 pages) Full-time Juniors: 31.9 pages (min 12 pages/max 62.5 pages) Part-time Juniors: 31.9 pages (min 12 pages/max pages 43.5 pages)

17


Full-time Seniors: 48 pages (min 16 pages/max: 63.5 pages) Part-time Seniors: 29 pages (min 0 pages/max 44.5 pages) Students were also asked to what extent their courses had helped them improve either “quite a bit” or “very much” in the following areas: Use evidence and sources to advance a thesis or argument Sophomores: 77.9% All respondents: 71.7% Use correct grammar and writing mechanics Sophomores: 66.7% All respondents: 66.7% Logically organize writing Sophomores: 61.11% All respondents: 66%: Develop a thesis statement Sophomores: 64.7% All respondents: 59.2 Consider audience need Sophomores: 66.7% All respondents: 56% Observe discipline-specific writing conventions Sophomores: 55.6% All respondents: 55.6% Utilize faculty/peer feedback to improve writing Sophomores: 50% All respondents: 48.5% Students were asked how they would recommend improving the writing program and 39 provided feedback. Of these, 18 % recommended increased support such as online videos, weekend writing workshops, etc.)

18


17% recommended either increasing the number of writing assignments or increasing the number of required courses focused on writing. 10% felt the program was adequate as is. A complete transcript of student comments can be found in AY2016_Written_Comm_Survey_Analysis. Interpretation of findings: Analysis to be provided by faculty: The success criterion was met among the small number of students responding however efforts will be made to increase student response. Please see attached for:

1. AY 2016_Written_Comm_Survey 2. AY2016_Written_Comm_Survey_Analysis

Teamwork Part 1: Students who complete the core will be able to consider different points of view. Direct Assessment:

Embedded

Assessment:

AAC&U rubric-based

evaluation of

standardized and

embedded assignments

randomly selected from

several sections of

multiple core courses.

Target Population:

During the pilot, 50 randomly selected artifacts drawn from an archive of student work. Once fully implemented the number of selected artifacts will increase to 100.

N = 107 F2015 = 49 F2015 Data (Note: 1 sample was discarded because it was not read during the evaluation session.) S2016 = 58 S2016 Data Overview of collection and evaluation processes: Faculty teaching in the Oral Communication and NS courses in the core assigned a Teamwork Signature Assignment during Fall 2015 and Spring 2016. Prior to grading, faculty archived students’ signature assignment submissions along with the assignment prompts in Blackboard 2. After the close of each semester, a random samples of artifacts created by sophomore-level students was drawn from the archives. Student, faculty and course identifiers were removed from student work by Office of Institutional Effectiveness staff. Deans nominated 10 full-time faculty to serve as evaluators for the Fall 2015 samples and an additional 10 for the Spring 2016 samples. On January 14, 2016, the faculty gathered for a norming session and to evaluate the F2015 samples. A second group of faculty met on May 29, 2016 for a norming session and to evaluate the S2016 samples. Each student artifact was evaluated by at

Recommended Strategies to Improve Learning: Strategies to improve learning to be provided by the faculty.

19


At least 70% of student artifacts will score 2.5 or above.

least 2 faculty using the “Considers multiple viewpoints” dimension on the AAC&U Teamwork rubric. A third reader was used when there was a lack of consensus among the first 2 readers. Summary of findings: Percent of students who received a score of 2.5 or higher (4 pt. scale) on the viewpoints dimension: F2015: 12 (24.5%) of 49 students S2016: 14 (24.1%) of 58 students Total: 26 (24.2%) of 107 students Mean score: Soliciting multiple viewpoints F2015: 2.2 S2016: 2.2 AY2016: 2.2 Faculty evaluators were unable to reach a consensus in their evaluation of student work on 9 samples in the fall and on 10 samples in the spring. Interpretation of findings: Additional analysis to be provided by faculty: This assessment marks the first evaluation of Teamwork under the revised General Education Core which was implemented Fall 2014. Not all assignments were well-aligned with the rubric, which most likely contributed to the difficulty faculty had in reaching consensus on some papers and possibly the low scores overall. The success criterion was not met. Please see attached for:

1. AY2016_Core_Assessment_Faculty_Evaluators 2. AY2016_Core_Assessment_Teamwork_AAC&U_Rubric 3. AY2016_Core_Assessment_Teamwork_Data_Summary. 4. AY2016_Core_Assessment_NS-Emp and Quant Signature

Assignment-F2016.xlsx 5. AY2016_Core_Assesment_REVISED_Submission_Instructions

20


Indirect Assessment UHD Core Objectives Student Survey Target Population: FTIC/FT sophomores Target Response Rate: At least 50% of all FTIC/FT sophomores Success Criterion: At least 70% of respondents will agree or strongly agree they are confident in their ability to consider different points of view.

N = 1645 Responses: 54 Response Rate: 3.3% Overview of collection and evaluation processes: Students in courses in the Natural Sciences and Communication Studies courses in the Core curriculum were asked to complete a survey in May 2016. A total of 54 students responded but since these courses contain students at multiple levels, responses were received from Freshmen and Sophomores as well as Juniors and Seniors. Results are presented for the aggregate group as well as for Sophomores alone.

Freshman 8 15% Sophomore 8 15% Junior 24 44% Senior 11 20% Post-bac 3 6% Other 0 0% Total 54 Summary of findings: Percent of students who were somewhat or very confident in their ability to objectively consider viewpoints that differ from their own. Sophomores: 62.5 % All respondents: 74.1% The survey results can be found in the attached AY2016_Core_Assessment_Teamwork_Survey_Analysis. Emphasis on Teamwork Skills in Classes Students were also asked the extent to which their course work helped them learn key teamwork skills. Half sophomores felt that their courses helped them “quite a bit” or “very much” to learn to consider other perspectives or viewpoints when working in a team. Sophomores: 50 % (quite a bit/very much) All respondents: 61.1% (quite a bit/very much) Please see attached AY2016_Core_Assessment_Teamwork_Crosstab for the data related to this.


21


Interpretation of findings: Analysis to be provided by faculty: The success criterion was not met. Please see attached for:

1. AY 2016_Teamwork_Survey 2. AY2016 _Teamwork_Survey_Analysis 3. AY2016_Core_Assessment_Teamwork_Crosstab

Teamwork Part 2: Students who complete the core will be able to work effectively with others to support a shared purpose or goal.

Direct Assessment Embedded Assessment: AAC&U rubric-based evaluation of standardized and embedded assignments randomly selected from several sections of multiple core courses Target Population: FTIC/FT sophomores During the pilot, 50 randomly selected artifacts drawn from an archive of student work. Once fully implemented the number of selected artifacts will increase to 100.

N = 49 F2015 Data (Note: 1 sample was discarded because it was not read during the evaluation session.) N = 58 S2016 Data Overview of collection and evaluation processes: Faculty teaching in the Oral Communication and NS courses in the core assigned a Teamwork Signature Assignment during Fall 2015 and Spring 2016. Prior to grading, faculty archived students’ signature assignment submissions along with the assignment prompts in Blackboard 2. After the close of each semester, a random samples of artifacts created by sophomore-level students was drawn from the archives. Student, faculty and course identifiers were removed from student work by Office of Institutional Effectiveness staff. Deans nominated 10 full-time faculty to serve as evaluators for the Fall 2015 samples and an additional 10 for the Spring 2016 samples. On January 14, 2016, the faculty gathered for a norming session and to evaluate the F2015 samples. A second group of faculty met on May 29, 2016 for a norming session and to evaluate the S2016 samples. Each student artifact was evaluated by at least 2 faculty using the AAC&U Teamwork rubric. A third reader was used when there was a lack of consensus among the first 2 readers. Summary of findings: F2015 Sample: 10 (20%) of 49 student artifacts received a mean score of at least 2.5 on a 4-point scale.

Recommended Strategies to Improve Learning: Additional recommendations to be provided by faculty. As noted under findings, some assignments were not aligned with the Teamwork rubric. The Core Assessment Committee recommends first working with faculty to align assignments and conduct a second round of assessment at the end AY2017. Strategies for improving learning will be identified after the Spring assessment.

Recommended Strategies to Improve Assessment: Strategy TW1: Student work was received from 69% of the 109 sections required to submit. A file corruption error prevented sampling from one section in the archive. Some assignment prompts were not provided. Professional development will be provided to help faculty;

a) Align writing prompts with the rubric; and b) Use technology to archive student work and

prompts. Special efforts will be made to contact adjuncts. UPDATE -Strategy TW1: The committee hosted 7 sessions in Spring 2016 to assist faculty with the assignment development. One-on-one sessions were also available. A participant list and session dates can be found in the attachments.

22



S2016 Sample: 15 (26%) of the 58 student artifacts received a mean score of at least 2.5 on a 4-point scale. Mean score by rubric dimension was: Contributions to team meetings: F2015: 2.0 S2016: 1.8 AY2016: 1.8 Facilitate participation of other team members: F2015: 1.8 S2016: 1.9 AY2016: 1.9 Foster positive team climate: F2015: 2.2 S2016: 2.1 AY2016: 2.2 Individual contributions outside of meetings: F2015: 1.7 S2016: 1.9 AY2016: 1.8 Resolving conflict (restated as resolving disagreements for the S2016 sample): F2015: 2.0 S2016: 2.3 AY2016: 2.2 Overall: F2015: 2.0 S2016: 2.0 AY2016: 2.0 In F2015, evaluators did not reach consensus most often on dimensions relating to: Resolving conflict 33% of samples

Strategy TW2: The committee will post 3 well-constructed teamwork prompts on the Core Assessment Website as exemplars. Strategy TW3: The committee recommends that assignments require that students provide specific examples which demonstrate their mastery of the rubric dimensions as opposed to one-sentence responses. Strategy TW4: As noted in the Findings, the least amount of consensus among faculty evaluators occurred on the Resolving Conflict dimension. Evaluators felt students were interpreting the word “conflict” as strong aggression between team members, which rarely occurred. Faculty evaluators recommended revising the dimension to describe a lower level of disagreement that does tend to occur in teams. UPDATE:-Strategy TW4: Faculty met during the Spring semester to address the wording of the Resolving Conflict dimension. A copy of the revised rubric can be found in the attachments (AY2016_Core_Assessment_Teamwork_Rubric REVISED ( 5 17 16)) Strategy TW5: In the Life and Physical Sciences band of the Core, some courses have labs separated from the lectures, taught by different faculty. There was confusion about whether labs or lecture sections would submit student work. The committee has asked faculty to identify which lecture courses adopted team-based learning as an instructional strategy. In those classes, the teamwork assignments will be drawn from the lecture courses. In the remaining courses with a separate lab, the lab sections will contribute the teamwork exercise.

23


Foster positive climate: 24% Individual contributions outside of meeting: 20% Facilitates contributions of others: 20% In S2016, a more extensive norming process was used and the instances where there was a lack of consensus declined to under 20% of samples with the exception of Facilitates Participation (21% of samples). The “No Evidence” classification was used when evaluators could not find evidence of the student practicing that particular dimension. The issue may have been that the writing prompt did not reference that dimension or the student was prompted but he/she elected not to write about that dimension. In F2015, evaluators noted No Evidence on the Resolving Conflict dimension in 16% of the samples. During the Spring, faculty revised the wording on that dimension to reference Resolving Disagreement and while instances of No Consensus dropped from 33% to 16%, instances of No Evidence increased from 16% in Fall to 22% in Spring. Difficulty with the “Resolve Conflict/Disagreements” dimension may be due to students’ reluctance to present teammates in a negative light. As a way to solicit more authentic responses, a faculty recommended adding a prompt to the Teamwork assignment similar to: “A team contract is used to help work groups spell out the responsibilities of team members as they work on a project. Would a team contract have helped your team? If so, how would a team contract supported the group’s work?” Interpretation of findings: Additional analysis to be provided by faculty: This assessment marks the first evaluation of Teamwork under the revised General Education Core which was implemented Fall 2014. Not all assignments were well-aligned with the rubric, which most likely contributed to the difficulty faculty had in reaching consensus on some papers and possibly the low scores overall. The success criterion was not met. Please see attached for:

1. AY2016_Core_Assesment_Faculty_Evaluators 2. AY2016_Core_Assessment_Teamwork_AAC&U_Rubric

UPDATE-Strategy TW5: The NS department has identified the lecture and lab sections where Teamwork artifacts will be collected. Please see AY2016_Core_Assessment_NS-Emp and Quant Signature Assignment-F2016.xlsx for the complete overview. Status: Complete

24


3. AY2016_Core_Assessment_Teamwork_Data_Summary. 4. AY2016_Core_Assessment_NS-Emp and Quant _Signature

Assignment-F2016.xlsx 5. AY2016_Core_Assesment_REVISED_Submission_Instructions 6. AY2016_Core_Assessment_Teamwork_Rubric REVISED ( 5 17 16)

Indirect Assessment UHD Core Objectives Student Survey Target Population: FTIC/FT sophomores Target Response Rate: At least 50% of all FTIC/FT sophomores Success Criterion: At least 70% of respondents will agree or strongly agree they are confident in their ability to work with others to achieve a common purpose or goal.


Freshman 8 15% Sophomore 8 15% Junior 24 44% Senior 11 20% Post-bac 3 6% Other 0 0% Total 54 Summary of findings: Percent fulltime (4+classes) Sophomores: 63% All respondents: 76% Percent of students who were somewhat or very confident in their ability to work effectively with others to achieve a common goal: Sophomores: 62.5% All respondents: 80% Percent of students who were somewhat or very confident in their ability to successfully complete an activity in a group setting:

Additional recommendations to be provided by faculty. Recommended Strategies to Improve Assessment Strategy TW6: The Office of Academic Assessment will assist faculty in improving the response rate on the survey to include:




25


Sophomores: 75% All respondents: 81.5% While students felt confident in their ability to work effectively in teams, responses to other items in the survey indicate that students do not see teamwork as an effective mechanism for organizing work. Percent of students agreed or strongly agreed that they could learn as much working in a group as they would working alone. Sophomores: 50% All responses: 51.9% Percent who agreed or strongly agreed that the work product produced in a group was as good as what they could produce alone. Sophomores: 25% All responses: 44.5% Percent who agreed or strongly agreed that working in a group was an efficient way of completing large projects. Sophomores: 25% All responses: 55.8% Percent who greed or strongly agreed that they got the grade they deserved when they worked in a group. Sophomores: 37.5% All responses: 51.9% While employers highly value employees’ ability to work in teams and recommend that HE place more emphasis on teamwork (see AAC&U survey in attachments), only 41% of the respondents agreed or strongly agreed that much of the work done in their chosen profession would be done in groups. Among the Sophomores responding to the survey, 62.5% agreed or strongly agreed that much of the work done in their chosen profession would be done in groups. Thirty one of the 54 students offered a comment when asked how UHD could help students improve their teamwork skills and roughly 40% of those were negative in tone.

26


All those new professors they had no idea what they are doing but following dictated pastern and teaching style which it is all team work not learning style.

I’m not like teams work because people are lazy they don't want to do the work until last minute or don't participate in any activities.

Don't. Group projects are useless and unfair to people without strong personalities.

We shouldn't be forced to work in teams. Some people do better working alone!

Group projects are the worst, so I'd recommend eliminating them. All too often, the workload is not evenly distributed, nor are the abilities of all members of a group on a similar level. The grading of such projects is therefore not equitable nor fair to the member who does the most work. In one class this semester, we were chastised for taking control of a project! Unimaginable!

Survey results analysis and students’ comments can be found in the attached AY2016 Teamwork Survey Analysis however names of specific faculty, when referenced, have been redacted. Emphasis on Teamwork Skills in Classes Students were asked the extent to which their course work helped them learn key teamwork skills. The percent of students who felt their classes contributed “quite a bit” or “very much” to helping them develop key skills are given below Make substantive contributions to team meetings Freshmen & Sophomores: 7 of 16 (43.8%) Juniors & Seniors: 17 of 35 (48.6%) Facilitate the contributions of others within the team Freshmen & Sophomores: 6 of 16 (37.5%) Juniors & Seniors: 18 of 35 (51.4%) Communicate effectively within the team; Freshmen & Sophomores: 10 of 16 (62.5%)

27


Juniors & Seniors: 22 of 35 (62.9%) Help the team resolve differences of opinion to reach consensus Freshmen & Sophomores: 6 of 16 (31.5%) Juniors & Seniors: 22 of 35 (62.9%) Address problem behaviors within the team: Freshmen & Sophomores: 7 of 16 (43.8%) Juniors & Seniors: 17 of 35 (49.8%) Foster a positive team climate: Freshmen & Sophomores: 10 of 16 (62.5%) Juniors & Seniors: 20 of 35 (57.1%) Crosstab for this section can be find in the file, AY2016_Core_Assessment_Teamwork_Crosstab Interpretation of findings: Analysis to be provided by faculty: While the success criterion was met among all respondents, it was not met among the Sophomores responding to the survey. The small number of students responding is problematic and efforts will be made to increase student response. Please see attached for:


General Strategies to improve the Norming and Evaluation Sessions: Note: These recommendations are independent of the learning outcomes however do impact the processes used to conduct the assessment. A group of full-time faculty participated in a norming and evaluation session on January 14, 2016 and evaluated Written Communication and Teamwork student artifacts. Each paper was read at least twice and a third reader was employed when faculty did not reach consensus on a particular artifact. At the close of the session, faculty were asked to provide their general impressions and recommendations which are captured above. Faculty evaluators also completed a survey at the close of the session (see attached). All responding evaluators indicated the session was valuable they would be likely or very likely to participate in the core evaluation process again.

28


Recommendation 1: At the next session, it was recommended that laptops be provided and faculty report their findings electronically. Follow-up: Unable to accomplish. The Office of Institutional Effectiveness approached IT about making laptops available during the evaluation sessions. Per IT, UHD’s wireless system would not accommodate that many wireless devices. Therefore paper and pencil was used again. Recommendation 2: Some committee members provided assistance to the session facilitators rather than serving as evaluators. This was extremely helpful and it is recommended that this practice be continued in the future. Follow-up: Completed. The Assistant Director of Assessment in CHSS assisted the facilitators during the May assessment. This practice will be continued into the Fall. Recommendation 3: Provide bottled water. Follow-up: Completed. Bottled water was provided during the Mays assessment session. The practice will be continued into the Fall.

2

Approved Committee Minutes 9/12/2017

Academic Assessment Committee Date: [9-12-17] Time: 2:30-

3:30pm Room: A701 Note taker: Lea Campbell



☒ Kevin Buckler

Present

☒ Lea Campbell (Chair) Present

☒

Travis Crone (FAC) (Conflicting meeting)

Present


Present


Present


☒


Present

☒ Melissa Hovsepian (FAC)

Present

☒ John Kelly (FAC) Present


☒


☒ Ami Moller Present

☒ Noel, Sharon Present


☒ Westfall, Joe (Conflict) (FAC)

Present

☐ Present

☐ Present

☐ Present

☐

FAC Compensation: Faculty Assessment Coordinators (FAC) overload and stipend

requests have been submitted to the Provost’s office. FAC’s were asked to notify

the Office of Academic Assessment if stipends were not reflected on the next

paycheck.

TK20 Update:

Single-sign-on has been set up so staff and faculty can log in using network

credentials.

Blackboard integration is in development

Couse upload complete – this supports curriculum mapping and course-based

assessment in the MDCOB.

Outcomes are being entered

Assessment report template has been created

Assessment rubrics, etc. are being entered into TK20

Academic Assessment Processes for AY17 Assessment Reports: Since UHD is

between assessment management systems, faculty will be asked to submit their

16/17 assessment reports on the 4-column Word template (see attached) and the

assessment staff will load those into TK20. Current due dates are:



Due from dean to the Office of Academic Assessment (OAA): December 15, 2017 Dr. Campbell requested that FACS and assistant assessment directors speak with their faculty to see if those dates are still viable given the disruption from hurricane Harvey.

General Education: UHD is in the last year of a three-year pilot of the new General

3

Education assessment process. This year Oral Communication, Visual

Communication, Social Responsibility rand Personal Responsibility will be assessed.

Dr. Hantla has been working with the Communication Studies faculty to identify a

process to capture oral presentations. While initially designed for Gen Ed, this

process could be of value to all faculty who wish to add an oral communications

component to their courses. Once the kinks are worked out, the process will be

shared with all faculty.

The 2015/2016 General Education assessment report is still in draft form and there

will be extensive efforts made to finally act on those data. The General Education

Committee will meet 9/27/17 and finalize dates for the Fall 2017 Assess-a-fest and

the January 2018 Core Evaluation meeting. Dr. Khoja requested that the General

Education Committee review the 2015/2016 General Education assessment report

and the Academic Committee Members asked to be included on emails reminding

faculty teaching in the Core to submit signature assignments. The Academic

Assessment Committee will also receive the 2016/2017 General Education

assessment report once it is drafted by the General Education Committee.

FAC Update: FAC’s and other members of the committee provided updates on

assessment within their areas.

New Business: None at this time.

Next Meeting: September 26, 2017, 2:30pm in A701

Attachments:

AY17 Academic Assessment Report Template

AY16 General Education Assessment Report (DRAFT)

4


2016-2017 ASSESSMENT REPORT TEMPLATE Due Dates



Due from dean to the Office of Academic Assessment (OAA): December 15, 2017

Outcome Methods of Assessment and Related Success Criteria

Summary of Findings Use of Findings

Enter outcome assessed in 2016/2017


AM#1 1. N = ________ 2. Overview of process of evaluation: 3. Summary of findings: 4. Interpretation of findings: 5. Was success criteria met? (Yes; No, Partially Met, or Data not collected) Attach copies of data sets, longitudinal data, instruments used to collect data rubrics and meeting notes.






AM#2: 1. N = ________ 2. Overview of process of evaluation: 3. Summary of findings: 4. Interpretation of findings: 5. Was success criteria met? (Yes; No, Partially Met, or Data not collected) Attach copies of data sets, longitudinal data, instruments used to collect data rubrics and meeting notes.






5


Required: Update on prior year’s assessment: In the space below, provide an update on assessment activities from 2015/2016

LO Assessed in

2015/2016

Summary of Improvements Proposed in the 2015/2016 Assessment Report:

Provide a Status Update on Program Improvements Recommended in the Spring 2017 Assessment Report (one initiative per line). Indicate if the initiative is:

f) Completely implemented g) Currently being implemented h) Planning phase i) Not started j) Replaced by alternate strategy (please provide details on what that strategy is and what

prompted the decision to select a new improvement strategy.)

Enter outcome measured

in 2015/2016.


Update.


Update.


OPTIONAL: Additional Improvement Strategies

STRATEGY TO IMPROVE WHICH FACULTY/STAFF ARE IMPLEMENTING: ISSUE, CONCERN OR PROBLEM FACULTY/STAFF ARE RESPONDING TO:

Have the academic program or unit revised the assessment plan? If “Yes,” please provide an updated assessment plan with changes highlighted in yellow. This is requested so we can update the assessment records in TK20. Signature of Chair_________________________________________________________ Date:______________________________ Signature of Dean _________________________________________________________ Date:______________________________

Yes No

6

AY2016 General Education Core Assessment AY 2016 Core Assessment Summary

7. Faculty assessed sophomores’ Written Communication and Teamwork skills using AAC&U rubrics in a two one-day workshop on January 14 and May 24, 2016. 8. Findings:

a. Direct measure: Rubric evaluation of student work. i. Success criterion of 70% of students will score at least 2.5 on a 4 pt. rubric was not met on either measure.

ii. Jan 14: Written Communication: 9 of 50 or 18% scored at least 2.5 May 24: Written Communication: 10 of 433 or 23% scored at least 2.5 TOTAL: Written Communication: 19 of 93 or 20% scored at least 2.5

iii. Jan 14: Teamwork Part 1 (Mult. Viewpoints): 12 of 49 or 24.5% scored at least 2.5. May 24: Teamwork Part 1 (Mult. Viewpoints): 14 of 58 or 24.1% scored at least 2.5

TOTAL: Teamwork Part 1 (Mult. Viewpoints): 26 of 107 or 24.2% scored at least 2.5 iv. Jan 14: Teamwork Part 2: 10 of 49 or 20% scored at least 2.5.

May 24: Teamwork Part 2: 15 of 58 or 26% scored at least 2.5 TOTAL: Teamwork Part 2: 25 of 107 or 23% scored at least 2.5

b. Indirect measures: Student Survey i. Success criterion of 70% of students confident or very confident

ii. Survey re: Written Com 1. N = 2830; Responses: 100; Response rate: 3.5%

a. Percent of students who were somewhat confident or very confident in their ability to develop ideas in writing: Sophomores: 78%

All respondents: 75% b. Percent of students who were somewhat confident or very confident in their ability to interpret ideas in writing.

Sophomores: 83% All respondents: 79%

iii. Survey re: Teamwork 1. N = 1645; Responses: 54; Response Rate: 3.3%

a. Percent of students who were somewhat or very confident in their ability to objectively consider viewpoints that differ from their own.

Sophomores: 62.5 % All respondents: 74.1%

b. Percent of students who were somewhat or very confident in their ability to work effectively with others to achieve a common goal:

Sophomores: 62.5% All respondents: 80%


7

9. Interpretation: Faculty will need to interpret findings 10. Recommended improvement strategies: 11. Strategies to improve learning: Faculty will need to provide improvements. 12. Recommended strategies to improve assessment:

i. Post copies of assignments well-aligned with rubrics on the Core Assessment webpage ii. Assist faculty in aligning assignments and rubrics, with special outreach to adjuncts.

iii. Revise “Context & Purpose” language in the Written Communication rubric. iv. Revise “Resolving Conflicts” language in the Teamwork rubric. v. Work with the NS faculty to determine which disciplines/courses will embed Teamwork Signature Assignments in labs and which will embed

the assignment in lectures. vi. Refine upload/download instructions.

vii. Increase response rate on student surveys by: 1. Distributing the survey in both the Fall and Spring; 2. Seeking assistance from faculty and Student Affairs to make students aware of the survey through other means besides Gatormail;

and 3. Sending multiple reminders.


Written Communication: Students who complete the core will be able to effectively develop, express and interpret ideas through written communication.

Direct Assessment Embedded Assessment: AAC&U rubric-based evaluation of standardized and embedded assignments randomly selected from several sections of multiple core courses Target Population: FTIC/FT sophomores Sample Size: During the pilot, 50 randomly selected artifacts drawn from an

N = 93 Fall =50 Spring = 43 (Originally 50 items were reviewed however, 7 samples were disqualified because one evaluator failed to enter scores.) Overview of collection and evaluation processes: Faculty teaching in the Written Communication, American History, and Language, Philosophy and Culture courses in the core assigned a Written Communication Signature Assignment during Fall 2015 and Spring 2016. Prior to grading, faculty archived students’ signature assignment submissions along with the assignment prompts in Blackboard 2. After the close of each semester, a random sample of 50 papers was drawn from sophomore-level students in the archive. Student, faculty and course identifiers were removed by Office of Institutional Effectiveness staff.

Recommended Strategies to Improve Learning: Recommendations to be provided by faculty. Strategy WC 1: As noted under findings, some assignments were not completely aligned with the rubric. The committee recommends working with faculty to better align assignments with the rubric and conducting a second round of assessment in AY2017.

Recommended Strategies to Improve Assessment: Strategy WC2: Student work was received from 79% of the 129 F2015 sections required to submit. A file corruption error prevented sampling from three sections in the archive. Student work was received from 85% of the 113 Spring 2016 sections required to submit. Both semesters,

8


archive of student work. Once fully implemented the number of selected artifacts will increase to 100. Success Criterion: At least 70% of student artifacts will score 2.5 or above.

Deans nominated 10 full-time faculty in the Fall and a second group of 8 faculty in the Spring to serve as evaluators of the artifacts. Faculty met on January 14, 2016 and on May 24, 2016, to evaluate student work. After a brief norming session, each student artifact was evaluated by at least 2 faculty using the AAC&U Written Communication rubric. A third readers was used when there was a lack of consensus among the first 2 readers. Summary of findings: Fall: 9 of the 50 (18%) student artifacts received a mean score of at least 2.5 on a 4-point scale. Spring: 10 of 434 or 23% of student artifacts received a mean score of at least 2.5 on a 4-point scale. Overall: 19 of 93 or 20% of student artifacts received a mean score of at least 2.5 on a 4-point scale. Mean score by dimension was: Context: Fall 2015: 2 Spring 2016: 2.2 AY2016: 2.1 Content Development: Fall 2015: 1.7 Spring 2016: 1.9 AY2016: 1.7 Genre: Fall 2015: 1.6 Spring 2016: 1.8 AY2016: 1.7 Sources/Evidence: Fall 2015: 1.5

Spanish courses submitted assignments in Spanish and could not be evaluated since the Written Communication assessment is an evaluation of students’ ability to writing in English. Some assignments were not well-aligned with the rubric and some assignment prompts were not provided. Professional development will be provided to help faculty;

c) Align writing prompts with the rubric; and d) Use technology to efficiently archive student

work and assignment prompts. The committee will make a special attempt to work with adjuncts. Strategy WC3: The committee will post 3 well-constructed writing prompts on the Core Assessment Website as exemplars. Faculty will receive copies of these assignments as part of the professional development training noted in Strategy 1. Strategy WC4: The committee recommends that faculty avoid using essay questions from tests as the Written Communication Signature Assignment since such responses are not usually long enough for students to demonstrate all the dimensions of the rubric. Also test responses do not allow students the time needed to polish their responses which would better represent the full range of their ability. Strategy WC5: The least amount of consensus among faculty evaluators occurred on the Context/purpose dimension. The Core Assessment Committee will have a group of faculty review the rubric and make recommendations on how best to clarify that dimension.


9


Spring 2016: 1.9 AY2016: 1.7 Mechanics: Fall 2015: 2 Spring 2016: 2 AY2016: 2 Overall: Fall 2015: 1.7 Spring 2015: 1.9 AY2016: 1.8 Faculty did not reach consensus 26% of the time on the Context/Purpose dimension, followed by Genre and Sources (14% respectively). Interpretation of findings: Additional analysis to be provided by faculty: This assessment marks the first evaluation of Written Communication under the revised General Education Core which was implemented Fall 2014. Not all assignments were well-aligned with the rubric, which contributed to the difficulty faculty had in reaching consensus on some papers and possibly the low scores overall. The success criterion was not met. Please see attached for:

5. AY2016_Core_Assesment_Faculty_Evaluators 6. AY2016_Core_Assesment_AACU_Written_Comm_Rubric 7. AY2016_Core_Assesment_Written_Comm_Data_Summary 8. AY2016_Core_Assesment_REVISED_Submission_Instructions

Strategy WC6: Fall 2015 was the first semester UHD used an archival process to collect student work. The TTLC assisted the faculty in developing a written set of instructions however some of the information needs to be revised and clarified. UPDATE-Strategy WC6: The Professional Writing faculty reviewed the instructions and provided edits. The instruction been revised and distributed to faculty. A copy of the revised instructions can be found in the attachments.

Indirect Assessment UHD Core Objectives Student Survey Target Population: FTIC/FT sophomores

N = 2830 Responses: 100 Response Rate: 3.5% Overview of collection and evaluation processes: Students in courses in the Written Communication, Language, Philosophy & Culture courses in the Core curriculum were asked to complete a survey in May 2016. A total of 100

Additional recommendations to be provided by faculty. Recommended Strategies to Improve Assessment Strategy WC7: The Office of Academic Assessment will assist faculty in improving the response rate on the survey to include:

10


Target Response Rate: At least 50% of all FTIC/FT sophomores Success criterion: At least 70% of respondents will agree or strongly agree they are confident in their ability to develop and interpretation of ideas in writing.

students responded but since these courses contain students at multiple levels, responses were received from Freshmen and Sophomores as well as Juniors and Seniors. Results are presented for the aggregate group as well as for Sophomores alone.

Freshman 23 23% Sophomore 18 18% Junior 34 34% Senior 25 25% Post-bac 0 0% Other 0 0% Total 100 Summary of findings: Percent of fulltime (4+classes) Sophomores: 67% All respondents: 67% Percent of students who were somewhat confident or very confident in their ability to interpret ideas in writing. Sophomores: 83% All respondents: 79% Percent of students who were somewhat confident or very confident in their ability to develop ideas in writing: Sophomores: 78% All respondents: 75% These two survey items form the basis of the indirect assessment of the Written Communication outcome. In both instances Sophomores as well as the group as a whole met the success criterion. Additional questions were included in the survey to better understand how students experience writing instruction at UHD. Percent that felt their writing had improved somewhat or a great deal over the semester: Sophomores: 89%




11


All respondents: 80% Percent of students who sought help with writing 3 or more times: Sophomores: 46% from a classmate/UHD student 31% from a professors 85% indicated they had not used UHDs Writing Lab. All respondents: 27% from a classmate/UHD student 31% from a professors 69% indicated they had not used UHD’s Writing Lab. Percent of students, by classification, who used the Writing Lab at least once during the spring semester: Freshmen: 29% Sophomores: 15% Juniors: 31% Seniors: 41% Freshmen most often approached a professor for help: 91% reported asking for help from a professor at least once. Sophomores sought help from friends (67%), classmates (62%) and professors (63%) at roughly equal levels. Sixty-one percent of juniors sought help from a professor at least once. Professors were the most often used source among seniors (89% visited at least 1 time). Approximate total number of pages students reported writing in S2016: Full-time Freshmen: 41.3 pages (min 12 pages/max 51.5 pages) Part-time Freshmen: 43.6% (min 11 pages/max 58.5 pages) Full-time Sophomores: 30.6 pages (min 12 pages/max 52.5 pages) Part-time Sophomores: 39 pages (min 31.5 pages/max 40.5 pages) Full-time Juniors: 31.9 pages (min 12 pages/max 62.5 pages) Part-time Juniors: 31.9 pages (min 12 pages/max pages 43.5 pages)

12


Full-time Seniors: 48 pages (min 16 pages/max: 63.5 pages) Part-time Seniors: 29 pages (min 0 pages/max 44.5 pages) Students were also asked to what extent their courses had helped them improve either “quite a bit” or “very much” in the following areas: Use evidence and sources to advance a thesis or argument Sophomores: 77.9% All respondents: 71.7% Use correct grammar and writing mechanics Sophomores: 66.7% All respondents: 66.7% Logically organize writing Sophomores: 61.11% All respondents: 66%: Develop a thesis statement Sophomores: 64.7% All respondents: 59.2 Consider audience need Sophomores: 66.7% All respondents: 56% Observe discipline-specific writing conventions Sophomores: 55.6% All respondents: 55.6% Utilize faculty/peer feedback to improve writing Sophomores: 50% All respondents: 48.5% Students were asked how they would recommend improving the writing program and 39 provided feedback. Of these, 18 % recommended increased support such as online videos, weekend writing workshops, etc.)

13


17% recommended either increasing the number of writing assignments or increasing the number of required courses focused on writing. 10% felt the program was adequate as is. A complete transcript of student comments can be found in AY2016_Written_Comm_Survey_Analysis. Interpretation of findings: Analysis to be provided by faculty: The success criterion was met among the small number of students responding however efforts will be made to increase student response. Please see attached for:

3. AY 2016_Written_Comm_Survey 4. AY2016_Written_Comm_Survey_Analysis

Teamwork Part 1: Students who complete the core will be able to consider different points of view. Direct Assessment:

Embedded

Assessment:

AAC&U rubric-based

evaluation of

standardized and

embedded assignments

randomly selected from

several sections of

multiple core courses.

Target Population:

During the pilot, 50 randomly selected artifacts drawn from an archive of student work. Once fully implemented the number of selected artifacts will increase to 100.

N = 107 F2015 = 49 F2015 Data (Note: 1 sample was discarded because it was not read during the evaluation session.) S2016 = 58 S2016 Data Overview of collection and evaluation processes: Faculty teaching in the Oral Communication and NS courses in the core assigned a Teamwork Signature Assignment during Fall 2015 and Spring 2016. Prior to grading, faculty archived students’ signature assignment submissions along with the assignment prompts in Blackboard 2. After the close of each semester, a random samples of artifacts created by sophomore-level students was drawn from the archives. Student, faculty and course identifiers were removed from student work by Office of Institutional Effectiveness staff. Deans nominated 10 full-time faculty to serve as evaluators for the Fall 2015 samples and an additional 10 for the Spring 2016 samples. On January 14, 2016, the faculty gathered for a norming session and to evaluate the F2015 samples. A second group of faculty met on May 29, 2016 for a norming session and to evaluate the S2016 samples. Each student artifact was evaluated by at


14



least 2 faculty using the “Considers multiple viewpoints” dimension on the AAC&U Teamwork rubric. A third reader was used when there was a lack of consensus among the first 2 readers. Summary of findings: Percent of students who received a score of 2.5 or higher (4 pt. scale) on the viewpoints dimension: F2015: 12 (24.5%) of 49 students S2016: 14 (24.1%) of 58 students Total: 26 (24.2%) of 107 students Mean score: Soliciting multiple viewpoints F2015: 2.2 S2016: 2.2 AY2016: 2.2 Faculty evaluators were unable to reach a consensus in their evaluation of student work on 9 samples in the fall and on 10 samples in the spring. Interpretation of findings: Additional analysis to be provided by faculty: This assessment marks the first evaluation of Teamwork under the revised General Education Core which was implemented Fall 2014. Not all assignments were well-aligned with the rubric, which most likely contributed to the difficulty faculty had in reaching consensus on some papers and possibly the low scores overall. The success criterion was not met. Please see attached for:

6. AY2016_Core_Assessment_Faculty_Evaluators 7. AY2016_Core_Assessment_Teamwork_AAC&U_Rubric 8. AY2016_Core_Assessment_Teamwork_Data_Summary. 9. AY2016_Core_Assessment_NS-Emp and Quant Signature

Assignment-F2016.xlsx 10. AY2016_Core_Assesment_REVISED_Submission_Instructions

15


Indirect Assessment UHD Core Objectives Student Survey Target Population: FTIC/FT sophomores Target Response Rate: At least 50% of all FTIC/FT sophomores Success Criterion: At least 70% of respondents will agree or strongly agree they are confident in their ability to consider different points of view.


Freshman 8 15% Sophomore 8 15% Junior 24 44% Senior 11 20% Post-bac 3 6% Other 0 0% Total 54 Summary of findings: Percent of students who were somewhat or very confident in their ability to objectively consider viewpoints that differ from their own. Sophomores: 62.5 % All respondents: 74.1% The survey results can be found in the attached AY2016_Core_Assessment_Teamwork_Survey_Analysis. Emphasis on Teamwork Skills in Classes Students were also asked the extent to which their course work helped them learn key teamwork skills. Half sophomores felt that their courses helped them “quite a bit” or “very much” to learn to consider other perspectives or viewpoints when working in a team. Sophomores: 50 % (quite a bit/very much) All respondents: 61.1% (quite a bit/very much) Please see attached AY2016_Core_Assessment_Teamwork_Crosstab for the data related to this.


16


Interpretation of findings: Analysis to be provided by faculty: The success criterion was not met. Please see attached for:


Teamwork Part 2: Students who complete the core will be able to work effectively with others to support a shared purpose or goal.

Direct Assessment Embedded Assessment: AAC&U rubric-based evaluation of standardized and embedded assignments randomly selected from several sections of multiple core courses Target Population: FTIC/FT sophomores During the pilot, 50 randomly selected artifacts drawn from an archive of student work. Once fully implemented the number of selected artifacts will increase to 100.

N = 49 F2015 Data (Note: 1 sample was discarded because it was not read during the evaluation session.) N = 58 S2016 Data Overview of collection and evaluation processes: Faculty teaching in the Oral Communication and NS courses in the core assigned a Teamwork Signature Assignment during Fall 2015 and Spring 2016. Prior to grading, faculty archived students’ signature assignment submissions along with the assignment prompts in Blackboard 2. After the close of each semester, a random samples of artifacts created by sophomore-level students was drawn from the archives. Student, faculty and course identifiers were removed from student work by Office of Institutional Effectiveness staff. Deans nominated 10 full-time faculty to serve as evaluators for the Fall 2015 samples and an additional 10 for the Spring 2016 samples. On January 14, 2016, the faculty gathered for a norming session and to evaluate the F2015 samples. A second group of faculty met on May 29, 2016 for a norming session and to evaluate the S2016 samples. Each student artifact was evaluated by at least 2 faculty using the AAC&U Teamwork rubric. A third reader was used when there was a lack of consensus among the first 2 readers. Summary of findings: F2015 Sample: 10 (20%) of 49 student artifacts received a mean score of at least 2.5 on a 4-point scale.

Recommended Strategies to Improve Learning: Additional recommendations to be provided by faculty. As noted under findings, some assignments were not aligned with the Teamwork rubric. The Core Assessment Committee recommends first working with faculty to align assignments and conduct a second round of assessment at the end AY2017. Strategies for improving learning will be identified after the Spring assessment.

Recommended Strategies to Improve Assessment: Strategy TW1: Student work was received from 69% of the 109 sections required to submit. A file corruption error prevented sampling from one section in the archive. Some assignment prompts were not provided. Professional development will be provided to help faculty;

c) Align writing prompts with the rubric; and d) Use technology to archive student work and

prompts. Special efforts will be made to contact adjuncts. UPDATE -Strategy TW1: The committee hosted 7 sessions in Spring 2016 to assist faculty with the assignment development. One-on-one sessions were also available. A participant list and session dates can be found in the attachments.

17



S2016 Sample: 15 (26%) of the 58 student artifacts received a mean score of at least 2.5 on a 4-point scale. Mean score by rubric dimension was: Contributions to team meetings: F2015: 2.0 S2016: 1.8 AY2016: 1.8 Facilitate participation of other team members: F2015: 1.8 S2016: 1.9 AY2016: 1.9 Foster positive team climate: F2015: 2.2 S2016: 2.1 AY2016: 2.2 Individual contributions outside of meetings: F2015: 1.7 S2016: 1.9 AY2016: 1.8 Resolving conflict (restated as resolving disagreements for the S2016 sample): F2015: 2.0 S2016: 2.3 AY2016: 2.2 Overall: F2015: 2.0 S2016: 2.0 AY2016: 2.0 In F2015, evaluators did not reach consensus most often on dimensions relating to: Resolving conflict 33% of samples

Strategy TW2: The committee will post 3 well-constructed teamwork prompts on the Core Assessment Website as exemplars. Strategy TW3: The committee recommends that assignments require that students provide specific examples which demonstrate their mastery of the rubric dimensions as opposed to one-sentence responses. Strategy TW4: As noted in the Findings, the least amount of consensus among faculty evaluators occurred on the Resolving Conflict dimension. Evaluators felt students were interpreting the word “conflict” as strong aggression between team members, which rarely occurred. Faculty evaluators recommended revising the dimension to describe a lower level of disagreement that does tend to occur in teams. UPDATE:-Strategy TW4: Faculty met during the Spring semester to address the wording of the Resolving Conflict dimension. A copy of the revised rubric can be found in the attachments (AY2016_Core_Assessment_Teamwork_Rubric REVISED ( 5 17 16)) Strategy TW5: In the Life and Physical Sciences band of the Core, some courses have labs separated from the lectures, taught by different faculty. There was confusion about whether labs or lecture sections would submit student work. The committee has asked faculty to identify which lecture courses adopted team-based learning as an instructional strategy. In those classes, the teamwork assignments will be drawn from the lecture courses. In the remaining courses with a separate lab, the lab sections will contribute the teamwork exercise.

18


Foster positive climate: 24% Individual contributions outside of meeting: 20% Facilitates contributions of others: 20% In S2016, a more extensive norming process was used and the instances where there was a lack of consensus declined to under 20% of samples with the exception of Facilitates Participation (21% of samples). The “No Evidence” classification was used when evaluators could not find evidence of the student practicing that particular dimension. The issue may have been that the writing prompt did not reference that dimension or the student was prompted but he/she elected not to write about that dimension. In F2015, evaluators noted No Evidence on the Resolving Conflict dimension in 16% of the samples. During the Spring, faculty revised the wording on that dimension to reference Resolving Disagreement and while instances of No Consensus dropped from 33% to 16%, instances of No Evidence increased from 16% in Fall to 22% in Spring. Difficulty with the “Resolve Conflict/Disagreements” dimension may be due to students’ reluctance to present teammates in a negative light. As a way to solicit more authentic responses, a faculty recommended adding a prompt to the Teamwork assignment similar to: “A team contract is used to help work groups spell out the responsibilities of team members as they work on a project. Would a team contract have helped your team? If so, how would a team contract supported the group’s work?” Interpretation of findings: Additional analysis to be provided by faculty: This assessment marks the first evaluation of Teamwork under the revised General Education Core which was implemented Fall 2014. Not all assignments were well-aligned with the rubric, which most likely contributed to the difficulty faculty had in reaching consensus on some papers and possibly the low scores overall. The success criterion was not met. Please see attached for:

7. AY2016_Core_Assesment_Faculty_Evaluators 8. AY2016_Core_Assessment_Teamwork_AAC&U_Rubric

UPDATE-Strategy TW5: The NS department has identified the lecture and lab sections where Teamwork artifacts will be collected. Please see AY2016_Core_Assessment_NS-Emp and Quant Signature Assignment-F2016.xlsx for the complete overview. Status: Complete

19


9. AY2016_Core_Assessment_Teamwork_Data_Summary. 10. AY2016_Core_Assessment_NS-Emp and Quant _Signature

Assignment-F2016.xlsx 11. AY2016_Core_Assesment_REVISED_Submission_Instructions 12. AY2016_Core_Assessment_Teamwork_Rubric REVISED ( 5 17 16)

Indirect Assessment UHD Core Objectives Student Survey Target Population: FTIC/FT sophomores Target Response Rate: At least 50% of all FTIC/FT sophomores Success Criterion: At least 70% of respondents will agree or strongly agree they are confident in their ability to work with others to achieve a common purpose or goal.


Freshman 8 15% Sophomore 8 15% Junior 24 44% Senior 11 20% Post-bac 3 6% Other 0 0% Total 54 Summary of findings: Percent fulltime (4+classes) Sophomores: 63% All respondents: 76% Percent of students who were somewhat or very confident in their ability to work effectively with others to achieve a common goal: Sophomores: 62.5% All respondents: 80% Percent of students who were somewhat or very confident in their ability to successfully complete an activity in a group setting:

Additional recommendations to be provided by faculty. Recommended Strategies to Improve Assessment Strategy TW6: The Office of Academic Assessment will assist faculty in improving the response rate on the survey to include:




20


Sophomores: 75% All respondents: 81.5% While students felt confident in their ability to work effectively in teams, responses to other items in the survey indicate that students do not see teamwork as an effective mechanism for organizing work. Percent of students agreed or strongly agreed that they could learn as much working in a group as they would working alone. Sophomores: 50% All responses: 51.9% Percent who agreed or strongly agreed that the work product produced in a group was as good as what they could produce alone. Sophomores: 25% All responses: 44.5% Percent who agreed or strongly agreed that working in a group was an efficient way of completing large projects. Sophomores: 25% All responses: 55.8% Percent who greed or strongly agreed that they got the grade they deserved when they worked in a group. Sophomores: 37.5% All responses: 51.9% While employers highly value employees’ ability to work in teams and recommend that HE place more emphasis on teamwork (see AAC&U survey in attachments), only 41% of the respondents agreed or strongly agreed that much of the work done in their chosen profession would be done in groups. Among the Sophomores responding to the survey, 62.5% agreed or strongly agreed that much of the work done in their chosen profession would be done in groups. Thirty one of the 54 students offered a comment when asked how UHD could help students improve their teamwork skills and roughly 40% of those were negative in tone.

21


All those new professors they had no idea what they are doing but following dictated pastern and teaching style which it is all team work not learning style.

I’m not like teams work because people are lazy they don't want to do the work until last minute or don't participate in any activities.

Don't. Group projects are useless and unfair to people without strong personalities.

We shouldn't be forced to work in teams. Some people do better working alone!

Group projects are the worst, so I'd recommend eliminating them. All too often, the workload is not evenly distributed, nor are the abilities of all members of a group on a similar level. The grading of such projects is therefore not equitable nor fair to the member who does the most work. In one class this semester, we were chastised for taking control of a project! Unimaginable!

Survey results analysis and students’ comments can be found in the attached AY2016 Teamwork Survey Analysis however names of specific faculty, when referenced, have been redacted. Emphasis on Teamwork Skills in Classes Students were asked the extent to which their course work helped them learn key teamwork skills. The percent of students who felt their classes contributed “quite a bit” or “very much” to helping them develop key skills are given below Make substantive contributions to team meetings Freshmen & Sophomores: 7 of 16 (43.8%) Juniors & Seniors: 17 of 35 (48.6%) Facilitate the contributions of others within the team Freshmen & Sophomores: 6 of 16 (37.5%) Juniors & Seniors: 18 of 35 (51.4%) Communicate effectively within the team; Freshmen & Sophomores: 10 of 16 (62.5%)

22


Juniors & Seniors: 22 of 35 (62.9%) Help the team resolve differences of opinion to reach consensus Freshmen & Sophomores: 6 of 16 (31.5%) Juniors & Seniors: 22 of 35 (62.9%) Address problem behaviors within the team: Freshmen & Sophomores: 7 of 16 (43.8%) Juniors & Seniors: 17 of 35 (49.8%) Foster a positive team climate: Freshmen & Sophomores: 10 of 16 (62.5%) Juniors & Seniors: 20 of 35 (57.1%) Crosstab for this section can be find in the file, AY2016_Core_Assessment_Teamwork_Crosstab Interpretation of findings: Analysis to be provided by faculty: While the success criterion was met among all respondents, it was not met among the Sophomores responding to the survey. The small number of students responding is problematic and efforts will be made to increase student response. Please see attached for:


General Strategies to improve the Norming and Evaluation Sessions: Note: These recommendations are independent of the learning outcomes however do impact the processes used to conduct the assessment. A group of full-time faculty participated in a norming and evaluation session on January 14, 2016 and evaluated Written Communication and Teamwork student artifacts. Each paper was read at least twice and a third reader was employed when faculty did not reach consensus on a particular artifact. At the close of the session, faculty were asked to provide their general impressions and recommendations which are captured above. Faculty evaluators also completed a survey at the close of the session (see attached). All responding evaluators indicated the session was valuable they would be likely or very likely to participate in the core evaluation process again.

23


Recommendation 1: At the next session, it was recommended that laptops be provided and faculty report their findings electronically. Follow-up: Unable to accomplish. The Office of Institutional Effectiveness approached IT about making laptops available during the evaluation sessions. Per IT, UHD’s wireless system would not accommodate that many wireless devices. Therefore paper and pencil was used again. Recommendation 2: Some committee members provided assistance to the session facilitators rather than serving as evaluators. This was extremely helpful and it is recommended that this practice be continued in the future. Follow-up: Completed. The Assistant Director of Assessment in CHSS assisted the facilitators during the May assessment. This practice will be continued into the Fall. Recommendation 3: Provide bottled water. Follow-up: Completed. Bottled water was provided during the Mays assessment session. The practice will be continued into the Fall.

2 | P a g e