Methods to test an e-learning Web application.

Methods to test an e-learning Web application.

gerardo.morales@it-sudparis.eu

Problematic and Motivations

LMS are rapidly increasing its complexity Learning standards Security policies Suggestions of users

2Jtel Summer school 09

Terchova, Slovakia 2 June 2009

Problematic and Motivations

The real problem of an Open Source LMS It is a hard task to assure the stability

because Daily changes (new features) Hundred of different developers

Different thinking Working separately

3Jtel Summer school 09

Terchova, Slovakia 2 June 2009

Conformance testing

4

Model

Implementation inJava

Implementation inC++

Conforms?

standard ISO 9646

Jtel Summer school 09Terchova, Slovakia 2 June 2009

5

Regression testing

File a.tcl (version 0.1)

File b.tcl (version 0.1)

File c.tcl (version 0.1)

File d.tcl (version 0.1)

Execute conformancetests

File c.tcl (version 0.2)

Feature 1Feature 2Feature 3Feature 4

Jtel Summer school 09Terchova, Slovakia 2 June 2009

Regression testing

File a.tcl (version 0.1)

File b.tcl (version 0.1)

File c.tcl (version 0.1)

File d.tcl (version 0.1)

Feature 1Feature 2Feature 3Feature 4

Execute conformancetests

File c.tcl (version 0.2)

6Jtel Summer school 09

Terchova, Slovakia 2 June 2009

Feature 5 ?

7

Regression testing

Feature 1Feature 2Feature 3Feature 4

Version 0.1

Feature 1Feature 2Feature 3Feature 4Feature 5

Version 0.2

?

Regression bugs detected in Features 2 and 3

Jtel Summer school 09Terchova, Slovakia 2 June 2009

Automatic Model Based Testing

Verdict

System

?

Jtel Summer school 09Terchova, Slovakia 2 June 2009

Automatic Model Based Testing

Automatic testing

Verdict

System

Jtel Summer school 09Terchova, Slovakia 2 June 2009

Automatic Model Based Testing

Automatic testing

Verdict

Test cases

System

??

Jtel Summer school 09Terchova, Slovakia 2 June 2009

Automatic Model Based Testing

Automatic testing

Verdict

Test cases

FORMALModel of system

System

Jtel Summer school 09Terchova, Slovakia 2 June 2009

Automatic Model Based Testing

Automatic testing

Verdict

Test cases

FORMALModel of system

System

????

Jtel Summer school 09Terchova, Slovakia 2 June 2009

Model

Model

An abstraction of the real thing We abstract away all the irrelevant details We abstract away all the confusing details We make a Simplification of a real system

Visibility of a system to be understood evaluated and criticized

13Jtel Summer school 09

Terchova, Slovakia 2 June 2009

Model

Jtel Summer school 09Terchova, Slovakia 2 June 2009

Finit States Machine

Ready to model?

Prolearn Summer school 09Slovakia 2 June 2009

F: (usr,pass)

L:register

Signup model

Jtel Summer school 09Terchova, Slovakia 2 June 2009

q1 Our Login Page

Signup model

Jtel Summer school 09Terchova, Slovakia 2 June 2009

q1

login(usr,pass) signup

Usr_id=pass_id

Message: « Unknown

User, pass »

q1

q2

q3

Signup model

Jtel Summer school 09Terchova, Slovakia 2 June 2009

q1

login(usr,pass) signup

Usr_id=pass_id

Message: « Unknown

User, pass »

q1

q2

q3

http://mymoodleinstance.com

http://mymoodleinstance.com/signup

http://mymoodleinstance.com/home

Signup model

Jtel Summer school 09Terchova, Slovakia 2 June 2009

q1

login(usr,pass) signup

Usr_id=pass_id

q1q2

q3

Task 1

Build a model of…

Welcome

User & Password

Fill this form

Jtel Summer school 09Terchova, Slovakia 2 June 2009

User

Password

OK

SignoutWelcome Harry Potter

My Frequently asked Questions

My Faq

1 Q: Where is Terchova?A: In Slovakia2 Q Where is Gerardo?A: In Terchova

Add an Q&A

Signout

Howards School LMS

Add an Q&A

Question:

Answer:

OK

Signout

Signup model

Jtel Summer school 09Terchova, Slovakia 2 June 2009

q1

login(usr,pass) signup

Usr_id=pass_id

Message: « Unknown

User, pass »

q1

q2

q3

q1

q1q2 q3

Signup model

Jtel Summer school 09Terchova, Slovakia 2 June 2009

q1

login(usr,pass) signup

Usr_id=pass_id

Message: « Unknown

User, pass »

q1

q2

q3

q1

q1q2 q3

q1q2 q3

q5q7

q8

PRESENTATION OF METHODOLOGIES

23Jtel Summer school 09

Terchova, Slovakia 2 June 2009

Simulator of the model

24Jtel Summer school 09

Terchova, Slovakia 2 June 2009

s0

s1 s2 s3

s4 s0 s5 s6 s7

TP: Action=input w State= s5 ck1= d

x/ya/b

c/d

a/dd/e w/y

a/bc/d f/g

Simulation of the model

25Jtel Summer school 09

Terchova, Slovakia 2 June 2009

s0

s1 s2 s3

s4 s0 s5 s6 s7

TP: Action=input w State= s5 ck1= d

x/ya/b

c/d

a/dd/e w/y

a/bc/d f/g

Trace1: [s0, x, y,s1]2: [s1, w,y,s5]

Inputsx,w

Expected outputsy,y

Expected final states5

Output of the test generator

26Jtel Summer school 09

Terchova, Slovakia 2 June 2009

?login(user1, password1) / ! Text (Welcome)?frequently_asked_questions() / ! Text (My Faq)

.

.

.

.

?login(user1,user2) Twt:: form find « login »Twt:: field fill « user1 »Twt:: fiend find « password »Twt:: field fill « password1 »Twt:: form submit

Translation to tclwebtest

Conformance test case (Automatic Model Based testing)

27Jtel Summer school 09

Terchova, Slovakia 2 June 2009

SystemSpecs

FormalModel

(Functional+Security)

AbstractTest Suites

TestScripts

TestExecution

TraceReport

Ed

itor

Test Generator Test Transformer

Execu

tor

Analyzer

1. M

odelin

g

2. Generation 3. Translation

5. Analysis

4. E

xecu

tion

TestPurposes

USE CASE

28Jtel Summer school 09

Terchova, Slovakia 2 June 2009

Test Preamble Test

29

Conformance test case

Jtel Summer school 09Terchova, Slovakia 2 June 2009

In the example of FAQ edition If the page we got after the action is the corret one

if {[string match "*admin/one-faq*" $response_url] } { if { [catch {tclwebtest::form find ~n "faq_add_edit"} errmsg] || [catch\\ {tclwebtest::field find ~v "$faq_new_name"} errmsg] } { aa_error "faq::twt::edit_one failed $errmsg : Dont't Edit a Faq" } else { aa_log "Faq Edited“}

30

Verdict assignation

Jtel Summer school 09Terchova, Slovakia 2 June 2009

In the example of FAQ edition If the page we where redirect is the correct one If the new name of the faq is founded in the faq

page

if {[string match "*admin/one-faq*" $response_url] } { if { [catch {tclwebtest::form find ~n "faq_add_edit"} errmsg] || [catch\\ {tclwebtest::field find ~v "$faq_new_name"} errmsg] } { aa_error "faq::twt::edit_one failed $errmsg : Dont't Edit a Faq" } else { aa_log "Faq Edited“}

31

Verdict assignation

Jtel Summer school 09Terchova, Slovakia 2 June 2009

32

Acs-automated testing

Jtel Summer school 09Terchova, Slovakia 2 June 2009

Conformance test case

Extract the semantic of the LMS Generating a static trace while interacting

with the Web application

33Jtel Summer school 09

Terchova, Slovakia 2 June 2009

Conformance test case

Extract the semantic of dotLRN Generating a static trace while interacting

with the Web application

34Jtel Summer school 09

Terchova, Slovakia 2 June 2009