Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)

8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)

http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 1/48

THE ROAD TO

ANALYTICS:LAYING THEFOUNDATION

Educause Annual Conference9/29/2014

Linda Gilbert

Juliana Lancaster

Georgia Gwinnett College





ABOUT OUR INSTITUTION

Georgia Gwinnett CollegeBegan in 2006

Massive growth since inception

Enrollment in 2006 = 118

Enrollment in 2014 = 11,000

“Campus of Tomorrow”



ABOUT YOU….



ROAD TO ANALYTICS

Learning Outcomes

Learn about the current status of analytics in higher education andcontributing factors

Explore general issues, challenges, and potential strategies related to

implementing analytics in higher education

Identify next steps to apply information from this session to yourinstitutional context



SESSION OVERVIEW

Initial Presentation: Analytics Overview

Activity: Resources and background

Activity: Identify productive working areas

Presentation: GGC Case StudyFinal Activity: Next Steps (using working areas)

Wrap-Up



ANALYTICS:

THE HOPE AND THE HYPE

Linda Gilbert



THE TECHNOLOGY HYPE CYCLE

Technology Trigger

Peak of Inflated Expectations

Trough of Disillusionment

Slope of Enlightenment

Plateau of Productivity

Visibility/Expectations

TimeSource: Gartner Researchhttp://www.gartner.com/technology/research/methodologies/hype-cycle.jsp



VISIBILITY/EXPECTATIONS

Horizon Report 2014 (One year out)

Horizon Report 2013 and 2013 (Two to three years out)

“Positioned in the same two-to-three year adoption

timeframe as it was last year, learning analyticscontinues to be an emerging field, one that is growing

quickly, but is still just out of reach for most

educators.”

Horizon Report 2013



VISIBILITY/EXPECTATIONS

Horizon Report 2014 (One year out)

Horizon Report 2013 and 2012 (Two to three years out)

MOOC – Learning Analytics and Knowledge (LAK) 2013

SoLAR – Society for Learning Analytics Research

Educause – Featured Topic in Library

http://www.educause.edu/library/analytics

Educause Review Special Issue July/Aug 2012

Educause Analytic Sprint, July 2012

ECAR Study of Analytics in HE, August 2012



TROUGH OF DISAPPOINTMENT

Google Flu Study

Purdue Course Signals

Next Gen Learning Analytics Reports

Chronicle of Higher Education ArticleRecent Big Data Struggles are ‘Birthing Pangs,’ Researchers Say (March 2014)



THE SLOPE OF ENLIGHTENMENT

Sharing lessons learned

Building on effective strategies

Identifying common issues and challenges

Developing useful modelsCustomizing to context

Building expertise!



INFLUENCES AND DEFINITIONS



(Learning) Analytics

BusinessIntelligence

Big Data

EducationalData Mining

StatisticalmethodsIntelligent

Tutors

Personalization

Adaptivelearning

INFLUENCES

Source: http://www.slideshare.net/gsiemens/iadis-shanghai



INFLUENCES

“Big Data”

Defined by the 3V’s - Volume, Variety, Velocity

Getting to “the unknowns”

How to leverage the benefits of “big data” while“avoiding paralysis by analysis”.



ANALYTICS IN HIGHER ED

Academic Analytics

Learning Analytics

Predictive Analyticshttp://net.educause.edu/ir/library/pdf/ELI3026.pdf



ACADEMIC ANALYTICS

“Academic analytics, although a relatively new concept,refers to the analysis of data to help educational institutions monitor

progress on key institutional goals, such as student retention,faculty productivity, and the impact of outreach and engagement.”

http://www.educause.edu/library/academic-analytics

“A process for providing higher education institutions with the data

necessary to support operational and financial decisionmaking.”

http://net.educause.edu/ir/library/pdf/ELI3026.pdf

Focus: Institutional, regional/national/international







LEARNING ANALYTICS

Learning analytics is the measurement, collection, analysis, and reporting of data

about learners and their contexts, for purposes of understanding

and optimizing learning and the environments in which it occurs.

Society of Learning Analytics Research, cited by

“Field associated with deciphering trends and patterns from educational big data, or

huge sets of student-related data, to further the advancement of apersonalized, supportive system of higher education

Horizon Report 2013

“The use of analytic techniques to help target instructional, curricular, and support

resources to support the achievement of specific learning goals.” http://net.educause.edu/ir/library/pdf/ELI3026.pdf

Focus: Learners, Faculty, (Departments)

http://www.nmc.org/publications/2013-horizon-report-higher-ed



http://www.nmc.org/publications/2013-horizon-report-higher-ed



PREDICTIVE ANALYTICS

“An area of statistical analysis that deals with extractinginformation using various technologies to uncover relationships and

patterns within large volumes of data that can be used to predictbehavior and events.


“Educational Data Mining is an emerging discipline, concerned withdeveloping methods for exploring the unique types of data that comefrom educational settings…, it often has multiple levels of meaningfulhierarchy, which often need to be determined by properties in the dataitself , rather than in advance.”

LAK2013 Course Materials Focus: All levels





MODELFocus

(Unit of Analysis)

Learner (Learning Analytics)

Institution (Academic Analytics)

Forecasting (Predictive Analytics)

Type of Analysis(Level of Inference)

Reporting (Descriptive Analytics)

Gilbert 2013



MODELS

Focus Type of analytics

1 What happened? Standard reports in real time

2 How many, how often, where? Ad hoc reports (real time)

3 Where exactly is the problem? Query/drill down (real time)

4 What actions are needed? Alerts (real time)

5 Why is this happening? Statistical analysis

6 What if these trends continue? Forecasting/extrapolation

7 What will happen next? Predictive modeling

8 What’s the best thing that could happen? Optimization

Adapted from the Davenport/Harris FrameworkSource: http://www.educause.edu/library/resources/building-organizational-capacity-analytics



APPLICATIONS IN HE

1. Extracting and analyzing data from learning management systems

2. Building an analytics matrix that incorporates data from multiple sources (socialmedia, LMS, student information systems, etc).

3. Profile or model development of individual learners (across the analytics matrix)

4. Predictive analytics: determining at-risk learner

5. Automated intervention and adaptive analytics: i.e. the learner model should beupdated rapidly to reflect near real-time learner success and activity so thatdecisions are not made on outdated model

6. Development of "intelligent curriculum" where learning content is semanticallydefined

7. Personalization and adaptation of learning based on intelligent curriculum wherecontent, activities, and social connections can be presented to each learner based

on her profile or existing knowledge8. Advanced assessment: comparing learner profile with architecture of knowledge in

a domain for grading or assessment Source: LAK2013 (MOOC)



APPLICATIONS IN HE



INTRIGUING DIRECTIONS

Trend analysis and prediction Early warning/intervention

Enrollment/Admission decisions

Personalization and adaptive learning

Online class content Mentoring support

Modeling: Learners, knowledge domains, relationships

Development of policies, norms, and technology OLA: Open Learning Analytics Architecture (SoLAR)



CONSIDERATIONS AND CONCERNS

Messy Data

Availability

Data choices

Human Biases

WYSIATI Limits to Prediction

“Black Swans”

Autonomous Agents

Feedback Loops – Self-fulfilling prophecies

“The problem starts when smart people innice suits and lab jackets proclaim that “the

data says…” In truth, the data never saysanything.

We interpret it in one way or another and

there are lots of ways to interpret itincorrectly.” - Greg Satell

http://www.digitaltonto.com/2011/the-pitfalls-of-prediction/



CONSIDERATIONS AND CONCERNS

Data Tools

Many current systems are stand alone, single functionality

Visualization Tools Needed

Data Use/Culture

Good tools =/= Good models =/= Good decisions

Privacy and Ethics:

Just because we can… should we?



MOVING INTO ANALYTICS

10/26/2012

Source: http://www.educause.edu/library/resources/building-organizational-capacity-analytics



Governance

Policies

Culture

MODELS

Tools andTechnology

Culture

Expertise



ACTIVITY:RESOURCES AND BACKGROUND

Form groups of THREE at each table.

Individually read pages 29(bottom)-33 fromBuilding Organizational Capacity for Analytics

Five minutes – we’ll time for you

As you read, identify passages (main one and a couple ofback-ups) that you feel have important implications for yourwork. Be prepared to share it in a few minutes.



ACTIVITY:RESOURCES AND BACKGROUND

“Three Levels of Text”

Rounds (5 minutes total per person) LEVEL 1: Read aloud the passage you selected

LEVEL 2: Say what you think about the passage (interpretation, connectionto past experiences, etc.)

LEVEL 3: Say what you see as the implications for your work.

The group responds to what has been said.

Credit: schoolreforminitiative.org



ACTIVITY:IDENTIFY WORKING AREAS

On your own, take five minutes to review your responses to thematurity index that you did as you came in.

Feel free to change responses based on any shift in your thinking duringthe session.

Identify the category on the index that you see as yourinstitution’s strongest area. Provide a short ‘why’ statement onthe green post it.

Identify the category on the index that you see as yourinstitution’s weakest area. Provide a short ‘why’ statement onthe yellow post it.

Post your notes on the appropriate flipchart. Take a break!



BREAK TIME



ACTIVITY: REVIEW

Review the flip charts and look for interestingpatterns.

Move to the table flagged for the area you want tofocus on for the next activity Ok to sit with colleagues now



GGC: DESIRE, CHALLENGE, AND NEED

Juliana Lancaster



THE OVERALL GOAL

Broad and deep use of data for improvement and decisionmaking at all levels

Straightforward access to data needed for specific purposes

Strategic use of analytics for large-scale and long-termplanning



THEN AND NOW

2006-07

Completely new institution

118 students

3 academic majors

11 faculty/ 30 staff No policies or processes

Technology infrastructureinherited from USG multi-institution Center

2014-15

10,900 students

14 academic majors

500 faculty/ 500staff

Policies and processes in place fordata security, management, andsharing

Technology infrastructuresupports ??????



BIGGEST CHALLENGES

Rapid Growth

10 faculty over 500 (FT and PT)

118 students over 10000 students

30 administrators/staffapprox. 500 (FT and PT)

Budget Limitations “Great Recession” in 2008

Reduced state funding overall

Lag in funding relative to growth



BIGGEST CHALLENGES

“Building the plane while it is in the air”

“Doing the urgent instead of the important”

Establishing processes for the right now while trying to envision what will

be needed later Ad hoc and idiosyncratic uses of various resources by different people

Little time for careful planning

Duplication of requests and effort rampant



THE ROAD WE TRAVELED: GETTINGSTARTED

Identifying the need… and “whose problem is it?”

Managing the massive growth and complexity.

Getting permission from TPTB

Getting the right people in the roomFiguring out where to start!

Locating resources USG

EDUCAUSE

Policy Dimensions of Analytics inHigher Education. Educause

Review July 2012

Data Governance,Data Classification,Roles and

Responsibilities,Data Policies,Legal Requirements



THE ROAD WE TRAVELED:WHAT WE DID AND WHAT WE LEARNED

Data/Reporting/Tools

Home-grown internal system first

Used experience to identify longer-term needs

Then investigated larger solutions

Governance/Infrastructure

Secured commitment of senior administration first

Focused early on policy needs

Managed up

Investment

Extensive discussion (aka time)

Communication!



THE ROAD WE TRAVELED:WHAT WE DID AND WHAT WE LEARNED

Expertise

Gathered the right people to tackle the needs Culture

Changed behavior by changing policy

Insisted on use of new process

Culture

Built on existing commitment to use of data

Kept discussions going until everyone agreed

Changed behavior by changing policy

Insisted on use of new process

Process

Established and regularized a common process Built relationships and communication through regular meeting



THE EARLY WIN…DATA REQUEST PROCESS

Building culture of data awareness

Need to ask

Need to secure/protect

Building expectation of accessibility

Easy to make a request, spawns automatic email notifications

Manage approval and fulfillment process within on-line form

Data provider is easier to identify

Building repository of information

Data requests capture work done



WHERE ARE WE NOW?

MaturityIndex

Data/Reporting/

Tools

3

Governance/Infrastructure

4

Investment

2Expertise

2

Culture

4

Process

4





ACTIVITY: YOUR NEXT STEPS

Take a little time to review resources related to your issuearea.

Together with the group, brainstorm “what can be done” inyour area of focus. Be ready to share with other groups.

List three “next steps” YOU can take.



CLOSING QUESTIONS,

COMMENTS, AND DISCUSSION

10/26/2012



RESOURCES AND CREDITS

Materials will be posted under the session topic at Educause

Presenters:

Linda Gilbert ([email protected])

Juliana Lancaster ( [email protected])

Credits: We would like to acknowledge the other charter members of the Data Managersand Stewards Working Group at GGC:

Nancy Grattan

Lily Hwang

Laura Ledford

Kris Nagel

10/26/2012

mailto:[email protected]






Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)

Documents