California State University Bakersfield Canvas Pilot ... Report to AS-ASI.pdfthe Canvas Pilot by October 1,2019. The report is complete except for three items that are still outstanding.

California State University Bakersfield Canvas Pilot Report

10/4/2019

Report to Academic Senate and Associated Students Inc. The following report is provided based on a request by Academic Senate to provide a report on the Canvas Pilot by October 1,2019. The report is complete except for three items that are still outstanding. We continue to complete the details required by the Canvas Pilot Committee.

1) The results from Fall 2019 for the Course Instruction Survey 2) The results from Fall 2019 for the Students Learning Survey 3) The completion of the Course Migration Survey – slated for Oct. 11

Overview In Spring 2016, three LMSs were evaluated as part of a CSU system-wide evaluation conducted by the Chancellor’s Office, campus members of the Directors of Academic Technology, and faculty. Results from the evaluation (brief overview in Attachment A) showed that for students, Canvas offered an enhanced, contemporary learning experience. For faculty, Canvas offered superior performance in terms of ease-of-use, analytics, grading, course migrations and openness. In Spring of 2017, in preparation for the upcoming end of the Chancellors Office Master Enabling Agreement for Blackboard on 6/2021, CSUB’s Information Technology Committee added a Canvas Pilot to the list of recommendations provided to the Information Technology Advisory Council. This recommendation was approved in May of 2018. Since that time, CSUB has participated in 4 terms worth of pilots (Winter 2019, Spring 2019, Summer 2019, and now Fall 2019). Our focus is to provide the best possible systems that enable student success. For additional information on “Why Canvas” please see Attachment B. Facts Blackboard

• CSUB currently uses Blackboard as its Learning Management System • In total for the 4 terms below we have had 617 individual instructors using Blackboard

to deliver 2058 courses. § Winter 2019 – 18 individual faculty offered 22 courses to 175 individual

students. § Spring 2019 - 409 individual faculty offered 859 courses to 9528

individual students

§ Summer 2019 – 77 individual faculty offered 91 courses to 1257 individual students

§ Fall 2019 - 481 individual faculty offered 1086 courses to 11061 individual students

• CSUB pays $85,630 for Blackboard. We do not have any support for Blackboard. • CSUB is using the current Chancellor Office (CO) contract which expires on 6/2021. • The Chancellor’s Office is working with Blackboard to renew a CO contract extending for

2 more years. Price will stay consistent. Canvas

• CSUB is currently piloting Canvas • In total – 85 unique faculty actively evaluated or are evaluating 215 courses enrolling

5551 unique students § Winter 2019 - 3 individual faculty offered 2 courses enrolling 14

students. § Spring 2019 - 28 individual faculty offered 50 courses enrolling 1382

individual students. § Summer 2019 - 14 individual faculty offered 16 courses enrolling 283

individual students. § Fall 2019 - 70 individual faculty offered 147 courses enrolling 4275

individual students. • Canvas would cost CSUB $109,019 annually.

Canvas Engagement • CSUB had two canvas demos for faculty in Spring 2019. • CSUB had nine faculty focus sessions

o 3 in Spring 2019 o 6 in Fall 2019

• CSUB had migrated 2,020 Blackboard courses for 551 instructors to address course migration concerns.

o 63 faculty provided feedback on our migration survey o 156 courses were reviewed by these 63 faculty o 41 faculty (65%) were satisfied with the course content migration

• In response to request by AS&SS and the larger Academic Senate, three Faculty Canvas Liaisons (Kim Mishkind, Tim Burke, and Tracey Salisbury) were selected to enhance the support for the pilot program for Fall 2019 semester. They will start assessing the needs for support, offer office hours, individual consultations and workshops to their colleagues.

• Faculty Teaching and Learning Center provides an outstanding support to faculty and staff. From July 2019 to September 2019, FTLC provided nearly 200 individual

consultations sessions, 12 workshops (additional 22 various workshops are scheduled for the Fall 2019 semester). The team has resolved over 200 Canvas service tickets since the pilot program started.

Support

• Blackboard Support provided by CSUB o There were 124 requests for Blackboard support in the form of an Incident or Request

since July 1st, 2019 (to date). The majority were from students accounting for 62.9% while faculty requests for Blackboard support accounted for 32.2%.

o Most of the requests for support were handled by the CSUB Helpdesk staff accounting for nearly 87.1% of those requests. Calculated by counting requests assigned directly to the Helpdesk and those that were never assigned (resolved by HD prior to assignment).

o Further, of the 124 requests for Blackboard support, 48 (38.7%) were related to students and faculty having problems accessing courses. Another 42 (33.8%) were directly related to students and faculty having trouble logging into Blackboard.

• Canvas Support • Canvas support portal (students and faculty contacting Instructure directly) shows 107 student

and faculty requests for support from July 1st, 2019 (to date). There were 27 (25%) requests for login support, and 44 (41%) requests for support related to courses, assignments, and quizzes. All other support requests did not fall into specific trends.

• During the same period, there were only 15 requests for support from the Helpdesk. 11 of the 15 (73.3%) related to students and faculty having problems logging into Canvas. Three requests were related to the ability to view a course. All instances were referred to Canvas support.

• Interesting Support Facts: o Helpdesk Hours – Blackboard Support

§ Mon-Thurs 7am – 10pm § Fri – 7am – 5pm § Sat – Sun – Closed

o Canvas Support § 24/7

o Timing for requests for support provided by Canvas: § 3 out of 107 – support was received after 10pm and before 7am. § 26 out of 107 – support was received on Saturday or Sunday. § 70 out of 107 – support was received via Live Chat.

Issue (Why have we written this report?) To support student success, CSUB should be using an intuitive and adaptable learning management system that allows students and faculty accessibility through not only their computers, but mobile phones and tablets. This ease of use will enable a higher level of adoptability and engagement by all stakeholders with an easier, more connected learning

experience. Finally, the system should be able to provide actionable data for faculty, advisors and campus leadership on the progress and success of our students. Relevant Information

• All California Community Colleges use Canvas. o Transfer students will already be familiar with Canvas.

• Kern High School District selected Canvas for their online program • 4 years ago, 13 CSU’s used Blackboard, today only 8. • Blackboard Campuses

• Bakersfield • Chico • Dominguez Hills • East Bay • Pomona • San Bernardino • San Diego • Stanislaus State

• Canvas Campuses • SJSU- 6 years on Canvas • Humboldt • Northridge • Channel Islands • Sacramento • Fresno • CSU LA (recently completed pilot and moved to Canvas) • Sonoma State (recently completed pilot and moved to Canvas)

• The Chancellor’s Office is in discussions with Blackboard to develop a new contract o Campuses would need to go to the Software as a Service version of Blackboard

which would create some impact to faculty. o Furthermore, the Ultra version of Blackboard that provides responsiveness and a

more modern design of Blackboard will require faculty to perform content migrations with some redesign of courses.

• Those CSUs that have selected Canvas, have gone to Canvas after one Pilot. • Migration of remaining Blackboard faculty to Canvas should be done in 6 months

(feedback from Sac State, Channel Island) once the decision is made. Feedback from faculty is to use a longer time period for migration until 6/2021.

Analysis

• CSUB has conducted 4 surveys over the last 4 terms (Winter 2019, Spring 2019, Summer 2019, and now Fall 2019.)

o Faculty Course Creation Survey [Appendix D]

§ This survey is administered at the beginning of a term before week three § Response data from the survey includes all 4 terms:

• 159 instructors surveyed, 83 responses. • 66% of instructors had previous experience with Canvas • 85% of instructors had previous experience with Blackboard • 88% of instructors chose Canvas over Blackboard

o Faculty Instruction Survey [Appendix E] § This survey is administered at the end of a term, starting 3 weeks prior to

final-exams. § Response data from the survey includes just 3 terms:

• 47 instructors surveyed, 37 responses. • 76% of instructors had previous experience with Canvas • 92% had previous experience with Blackboard • 94% of instructors chose Canvas over Blackboard

o Student Learning Survey [Appendix F] • This survey is administered at the end of a term just like the Faculty

Instruction Survey, starting 3-weeks prior to final-exams. • Response data from the survey includes just 3 terms:

• 1733 students surveyed, 374 responded. • 41% of students had previous experience with Canvas • 88% of students had previous experience with Blackboard • 66% of students chose Canvas over Blackboard

o Faculty Course Migration Survey [Appendix G} • Survey sent to all 551 faculty, 101 responded reviewing 258 courses • 71% of instructors did not have any Canvas experience. • 69 of the 101 instructors were somewhat to very satisfied with the

course migration. • 18 of the 101 instructors were somewhat or very dissatisfied with the

automatic conversion. • 59% of instructors are interested in participating in Canvas for

upcoming terms. • Functionality

o Some science faculty have reported issues using the tools. In review, this is mostly a training and time issue. However, we will look at all more science editors.

o Many faculty have reported no issues, commented on the enhanced benefits of being on Canvas and expressed no desire to return to Blackboard

o Survey results show Canvas to have better functionality than Blackboard • Conversion cost

o There will be a onetime conversion cost for our faculty, from in-person discussions, average shows about 10 hours per course, however, certain courses may be extensive.

• Pros of Canvas o Meeting students where they are [Transfer students are already familiar

with Canvas] o Simple to use o Web 2.0 platform (responsive) o Mobile app is strong o Modern integration layer allowing for data and analytics o Larger open community of contributors developing content into commons

reducing time needed for faculty to build course content and reducing the cost of material for students

o Built-in term-based Analytics which helps faculty with course instruction, as well as facilitates a conversation with student(s) for early intervention

• Cons of Canvas o A small group of faculty may experience possible difficulties in converting

certain courses over to a new LMS and will require additional assistance. o Cost compared to Blackboard.

• Pros of Blackboard o Faculty familiar with the platform o Current students familiar with the platform o Strong testing/quizzing tool o Grading tool offers more flexibility

• Cons of Blackboard o Incoming California Community College transfer students need to learn a

new LMS o Complex user interface o Is not responsive for mobile, tablets, or laptops o Mobile app has not been updated in several years. o Innovation by Blackboard into platform non-existent o Blackboard as a company has sold off assets to support continued viability o Upgrades are complex and will require faculty to re-work their content

• Technical analysis o Canvas offers a modern integration platform allowing us to pull data from

Canvas into other systems o For Blackboard we will need to go through two upgrades – from Managed

Hosting to SaaS and from Blackboard 9 to Ultra. • Cost

o While our decision-making process should not be driven by cost, it is a factor. The key is to have an LMS that promotes student success (Appendix C) by offering actionable data, full LTI integration, and API integration capabilities.

Furthermore, it should be simple to use for faculty to reduce workload once course migration is completed. Here are the costs and additional options we can take.

Blackboard Learn Instructure Canvas Base license** $ 118,755 discounted to

$ 85,300 $ 109,019

24x7 Support Included until 2021, then $ 9000.

$ 21,803

Helpdesk Implementation, onetime

$15,000 $ 500

Tier 1 faculty support - $ 22,677 ** Comparing apples to apples, since we do not have support today for Blackboard would be to just take into account the Base license costs.

• Other considerations o Although minimal, upgrading Blackboard will cause faculty to do content

migration work • Support Recommendation

o Due to having a new Learning Management System a more robust support model would be preferred. That said, Faculty have voiced concern that they need more Blackboard support than what CSUB offers now and that Canvas support would be useful

Conclusion Based on results from our Canvas Pilots, CSUB should adopt Canvas as its next LMS. We should sign a five-year deal locking in the lowest rate possible and providing continuity for faculty and students. This would cost us $110,000 annually. We should fully migrate from Blackboard to our new Canvas environment by the end of Spring semester 2021. Timeline If we are not moving forward with Canvas we need to inform all current pilot participants that this decision has been made and why. This should be done by Nov. 22, 2019, to give us time to move all our faculty back over to Blackboard. If we decide to move forward or no decision is made by Nov. 22, 2019, we will need to enter into a contract with Canvas and will be running two systems simultaneous. As mentioned in the relevant info, we are the only campus that is requiring additional time in making the decision and migrating our campus over to Canvas.

Current Rubric for Decision Making 40% faculty voice – Survey responses, discussion with Academic Senate 40% student voice – Survey responses, discussion with Associated Students Inc. 20% ecosystem – technology, mobile capabilities, integration-apis, analytics, cost

Appendix A: CSU Learning Platform Summary – Chancellor’s Office

Overall Summary Based on scoring from the CSU LPS Evaluation Committee and feedback from CSU faculty and students, the CSU found that three of the four proposals for Learning Platforms and Services should be recommended for system-wide MEAs as the products and companies are acceptable options for Enterprise LMS Systems and Services for campuses. The CSU plans to announce an Intent to Award to Blackboard Learn, D2L BrightSpace and Instructure Canvas.

Background During the Spring of 2016, the CSU published RFP # 150431 for Learning Platforms and Services. Four proposals were submitted: Blackboard Learn, Blackboard Moodlerooms, D2L BrightSpace and Instructure Canvas. Blackboard MoodleRooms did not advance past the first evaluation stage and is not included in this summary. Scoring was based on the documented responses, product demonstrations, sandboxes, company interviews and pricing. Minimum requirements included information security standards, accessibility and migrations. More information is available at http://csulps.com (this site is no longer available).

Overall Impressions All three LMS products performed well in this evaluation process, and the evaluation committee found that all three would be acceptable options for CSU campuses. In general, it was the consensus of the committee that Blackboard is having difficulty developing a cloud-based product while keeping active two other non-cloud based legacy products. Canvas gave the firm impression of a native cloud-based product with the kind of interface and flow that seems to be in line with today’s web applications. While there is lots of interest about Canvas, its more pricey proposal had a cooling effect. Although D2L presents a lot of promise, it fails to generate the excitement of Canvas. We should note that the D2L LEAP product still generated fair amount of curiosity on the part of the evaluation committee because of the focus on student success in the CSU, but there was concern regarding the amount of work which would be required to set it up with faculty. Migration Criterion: All vendors were provided two CSU courses to migrate, one from a Moodle instance and one from a Blackboard instance. These two courses were chosen because they represent the LMSs used in 21 campuses in the CSU. The courses selected utilize many features (discussion board, quizzes, mathematical formulations) allowing us to test the migration ability of each product. All products were successfully able to migrate these two courses. Furthermore, each vendor offers a variety of migration services and they are all quite familiar with migrating from the other vendors’ products.

Attachment B: Why Canvas Why Canvas? We are living in a time that complex and accelerated changes are happening. As an institution, we need to keep up with these progressive changes. We are currently using Bb and have been using Bb for a few years. During this time, a number of changes have occurred in LMS features as well as student expectations. What are the drivers as to why we should switch? At the top of the list – Student Success.

1. Web 2.0 platform: We need a platform that enhances how our students learn and how they want to access relevant course material. Canvas is a web 2.0 platform which means all its tools are built to work with mobile devices. Our students are mobile driven. The data we’ve been collecting over the last year shows that our students, more and more are relying on mobile phones, laptops and other hand held devices to connect to our systems. The most important factor is that the platform is open, allows integration, and will enable us to have a more powerful learning platform meeting the needs of our students. Over 65% of the 1300 pilot users prefer Canvas.

2. Ease of use: Discussions with colleagues on campuses using Canvas indicate that the learning curve for instructors and students is short. They adapt quickly. Many of our faculty teach at the community college and are already familiar with Canvas. Some faculty who are currently using Canvas have requested the change, as they believe that Canvass is more intuitive and easier to use. Students participating in the Canvas pilot reported that they like using it – especially the modern, clean interface. Ease of use is essential. Canvas’ great strength is that it is built to be easier to understand and use. Many faculty found Blackboard difficult to use, and most of our pilot faculty found Canvas an improvement. This will save faculty time and make it easier to experiment with new teaching approaches and tools. We also expect that more teaching tools will be available to our faculty as Canvas is constantly improving. Once we have completed the transition, the easier tool will be simpler to support for both faculty, students, and the support organizations.

3. The push of Canvas into the Community College system and high schools: The entire CA Community College system has adopted Canvas as their LMS. This means that an increasing number of our students will be familiar with how the tool works when they arrive at CSUB. All of our main feeder transfer schools use Canvas. Further, Kern High School District is also a Canvas user – https://kernhigh.instructure.com

4. A community of institutions: Many of our peer institutions have moved or considered to move to Canvas: this strong community of peer institutions will keep us at the forefront of developing teaching practices and tools. Canvas works closely with the extended Canvas community in developing new features. Furthermore, content that is created and shared into Commons will be material that can help strengthen courses and reduce the costs to our students. We understand there are all kinds of questions about whether people will be able to do what they currently can in Blackboard after we move to Canvas. Many of these questions will be answered in the materials that we will be sharing over the coming months. If you have a more specific question we would be happy to discuss individual concerns. In the meantime, you can find information on Canvas by visiting their website: Canvas community: https://community.canvaslms.com

5. Finally, cost while a factor is not a driving factor. Canvas in the end may be 5-10% more expensive than what we pay under our current Blackboard Contract but the benefits of the platform are clear.

CSU Canvas adoption:

1. CSU & Canvas campuses ( 8 campuses)

1. SJSU- 6 years on Canvas 2. Humboldt- Signed client 2 yrs 3. Northridge- Signed client 18 months 4. Channel Islands- Signed client 1 yr 5. Sacramento- Signed client 1 yr 6. Fresno (Signed client 5 yr) 7. CSU LA (recently completed pilot and moved to Canvas) 8. Sonoma State (recently completed pilot and moved to Canvas)

2. Canvas Current Pilots (2 campuses)

1. Bakersfield 2. SDSU

UC Canvas adoption:

1. UC Berkeley 2. UC Davis 3. UC Irvine 4. UC Merced

5. UC San Diego 6. UC Santa Barbara 7. UC Santa Cruz

Appendix C: Analytics support that we can use to Promote Student Success

We can see on a daily basis those students that are participating and those that aren’t and provide alert information as an early warning.

Furthermore, we are able to see how students are doing in terms of their grade distribution. For example below. .8% of our students are receiving a 50% in their courses. That means 35 students of those 4,395. This data can be given to advisors or others in the Student Care Network for early warning or action.

Another example of the Grade Distibution of those students receiving between a 30 and 49 on their assignments. So below is 2.7% or about 100 students. Acting on this data would yield positive results to our Student Success.