Mehrnoosh vahdat workshop-data sharing 2014

Learning Analytics with DEEDS simulatorBenefits and Challenges of Data Sharing

Mehrnoosh Vahdat, ICE PhD student at UNIGE & TU/eMember of LACE Project, Infinity Technology Solutions

September 16, 2014

Outline

• Research plan

• Learning Analytics (LA) with DEEDS▫ Goal▫ DEEDS –a shared resource▫ Student’s learning process with

DEEDS ▫ LA role

• What to share?▫ Data▫ Features

• Who benefits from data sharing?▫ Challenges

• References

Research plan

LA/ EDM

Higher Education

Electronic Engineering Courses: Logic Networks

Simulator

Data collection and pre-

processing

Prediction of learning outcome from

interaction data

IndustrySchools

Analyzing learner’s behavior

Goal

• To extract the profiles of students activities, performed during the training sessions of a course of logic networks

• To relate such activities with the students’ performance at intermediate verification tests

• To explore students learning behavior on system while using Deeds simulator

• To extract non-trivial patterns from students’ interaction

• To assist instructors to be aware of students learning process

DEEDS

• Stands for: Digital Electronics Education and Design Suite

• Is an interactive simulation environment for e-learning in digital electronics

• Provides learning materials

• Asks to solve varied-level problems

DEEDS – A freeware and a shared resource

• Deeds is free to use for academics

• It has been and it is used now in several European universities and project assignments have been shared among European schools (within the European Union LeonardoDaVinci NetPro project)

• Deeds educational materials have been translated and published in English, Italian, Turkish, Spanish, Catalan.

Students’ Learning Process with DEEDS Simulator

Components

LA role

• Prediction of students’ learning outcome from their activities in each session

• To understand which learning behavior is effective in the outcome

• To distinguish the students who need more attention in early sessions of the course

• To understand which course content/ exercise is critical

• To provide help in-time

What to share?

• What data level to share concerning the data anonymity?

• Which features are critical in prediction?

• Which prediction method (to predict students’ grades) is effective for these data?

Data

• Activity logs from system

• Data from the questionnaires:

▫ Demographic data: consent, general, motivation, background knowledge, ICT literacy, learning style.

▫ Data from students feedback

• Data from observation and semi-structured interview

• Group assessment per session

• Final grade at the end of the semester

Feature Extraction

Which features are critical in prediction?

• Samples of students’ activities :▫ Text-editor: The time students spend on writing their answers.▫ Image: Students work with images of simulation imported from the

tool.▫ Circuit-simulator : Students work on an exercise with the circuit

simulator.▫ Timing-diagram: Students run the circuit simulator.▫ FSM-Simulator: Students work on Finite State Machine Simulator.▫ Browser-exercise code: Students study the exercise.▫ Warning: they might have taken a wrong action.

Who benefits from data sharing?

• Researchers: ▫ To benefit from simulation-based critical features, and prediction methods,

to develop recommendation engines integrated in digital electronics simulators

• Teachers: ▫ To plan their lessons based on students’ needs and their effective activities,

to help students in-need in early sessions of the course, and help them avoid most frequently mistakes.

• Students: ▫ To get recommendations about activities and resources, receive more

personalized help.• DEEDS/ digital electronics simulator developers:

▫ To improve simulators and adapt it to the students’ needs

Challenges?

• Cost:

▫ Of applications, methods to obtain meaningful data.

• Data

▫ Interoperability: To bring all data levels together

▫ Reliability: user’s role in activity data, trial and error or decision making?

▫ Context and time: needs lots of work to make sense of unorganized information

• Ethical obligations:

▫ Privacy and anonymity(Bienkowski, Feng, & Means, 2012; del Blanco et al., 2013; Gyllstrom, 2009)

These slides are provided under the Creative Commons Attribution Licence: http://creativecommons.org/licenses/by/4.0/. Some images used may have different licence terms.

Learning Analytics with Deeds simulator: Benefits and Challenges of Data Sharing

by Mehrnoosh Vahdat

was presented at Learning Analytics Data Sharing – LADS14 Workshop at EC-TEL.Graz - 16th September 2014

[email protected]

http://goo.gl/ouywVU

@MehrnooshV

References

• Baker, R.S., Corbett, A.T., Koedinger, K.R. (2004). Detecting student misuse of intelligent tutoring systems• Baker, S.J.d., Yacef, K. (2009) The State of Educational Data Mining in 2009: A Review and Future Visions.• Bienkowski, M., Feng, M., Means, B. (2012). Enhancing Teaching and Learning through Educational Data Mining and Learning

Analytics: An Issue Brief • Chatti, M.A., Dyckhoff, A.L., Schroeder, U. and Thüs, H. (2012) ‘A reference model for learning analytics’, Int. J. Technology

Enhanced Learning, Vol. 4, Nos. 5/6, pp.318–331.• del Blanco, A. et al. (2013). E-Learning Standards and Learning Analytics: Can Data Collection Be Improved by Using Standard Data

Models?• Donzellini G., Ponta D. (2007) A Simulation Environment for e-Learning in Digital Design. Trans. on Industrial Electronics, vol. 54,

no. 6: 3078—3085.• Glahn, C., Specht, M., Koper, R. (2007) Smart indicators on learning interactions. In: Duval, E., Klamma, R., Wolpers, M. (eds.) EC-

TEL 2007. LNCS, vol. 4753, pp. 56–70. Springer, Heidelberg.• Gyllstrom, K. (2009). Enriching Personal Information Management with Document Interaction Histories: A Thesis• Gyllstrom, K. (2009) Passages through time: chronicling users' information interaction history by recording when and what they

read, Proceedings of the 14th international conference on Intelligent user interfaces, February 08-11, 2009, Sanibel Island, Florida, USA [doi>10.1145/1502650.1502673]

• Romero, C., Ventura, S. (2007). Educational Data Mining: A Survey from 1995 to 2005.• Romero, C., Ventura, S. (2010) Educational Data Mining: A Review of the State-of-the-Art.• Siemens, G., Baker, S.J.d. (2010). Learning Analytics and Educational Data Mining: Towards Communication and Collaboration