Dynamic Analysis and Testing of Ajax User Interfaces

10-04-23 @ IPA Herfstdagen

Software Engineering Research Group

DelftUniversity ofTechnology

Dynamic Analysis and Testing of Ajax User Interfaces

Ali Mesbah, Arie van Deursen

Software Engineering Research GroupDelft University of Technology, The Netherlands

2Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Classical Web Applications

Shortcomings of the Multi-page style:• Low lever of user interactivity• Redundant data transfer• High user-perceived latency• Client is passive

ServerServerClientBrowserClient

Browser

HTTP

GET

HTML

3Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Ajax

Asynchronous JavaScript and XML• Asynchronous server communication

through XMLHttpRequest• Web standards-based (HTML, CSS, XML)• Dynamic display through DOM• JavaScript binds everything together

Features:• No full page refresh!• Single-page web interface• Google Suggest, Gmail, Yahoo! Mail

4Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Ajax-based Web Applications

• Client-side execution• State changes & navigation• Dynamic DOM• Delta-communication• Clickables: Elements having

event-listeners attached to them

<a href="javascript:OpenNewsPage();"><a href="#" onClick="OpenNewsPage();"><div onClick="OpenNewsPage();"><a href="news.html" class="news"><input type="submit" class="news"/><div class="news">

$(".news").click(function() { $("#content").load("news.php");});

5Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Research Question

How can we automatically analyze (test) Ajax web applications?

6Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Current Approaches

• General Approach: extract link, send a request to the server and analyze the response. i.e., HTML (based on the Multi-page style)

• Capture/Replay: demands a substantial amount of manual effort

• Static Analysis: misses the complex run-time behavior

• Dynamic Analysis: promising but detecting the various doorways to different dynamic states and providing proper interface mechanisms for input values is challenging.

7Dynamic Analysis and Testing of Ajax User Interfaces

| 26

A Method for Deriving Ajax States

• Conduct dynamic analysis of the user interface to find Clickable elements and state changes

• Reverse-engineer a State-flow Graph to abstract the navigation paths and state changes

8Dynamic Analysis and Testing of Ajax User Interfaces

| 26

User Interface State Changes

• In classic web applications, each UI state is represented by a URL and the corresponding web page

• In Ajax, it is the internal change of the DOM tree on the (single-page) user interface that represents a UI state change

• To adopt a generic solution, we define a UI state change as: a change on the DOM tree caused either by server-

side state changes propagated to the client, or client-side events handled by the Ajax engine

9Dynamic Analysis and Testing of Ajax User Interfaces

| 26

State-flow Graph

10Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Inferring the State Machine (Crawljax)

11Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Detecting Clickables

• Find a set of Candidate Elements (CC)• Expose each to en event (onclick, onmouseover, …)• Analyze the effect on the DOM

We distinguish between 3 ways of obtaining CCs:1. Full Auto Scan: HTML tagname (div, span, a, …)

with attribute constraints, e.g., div:{class=“menuitem”}

2. Annotation: element attribute crawljax=“true”, or “false”

3. A DSL: Crawling Ajax Specification Language (CASL)

12Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Detecting States

Input: DOM Trees before and after the event:

• Compare DOM Trees:• As is• Textual: extract textual content and compare• Structural: extract structural content and compare• Meta-model: reverse-engineer the Schema for each

DOM, then compare the schemas

• Comparison method: Edit Distance, Levenshtein method with a similarity threshold

13Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Processing Document Tree Deltas

Upon every new state entry:• Compute the differences between the previous document

tree and the current one • We use an enhanced Diff algorithm to find the “delta

updates” • Delta updates may be due, for example, to a server request

call that injects new elements into the DOM• Find new candidate elements in the delta updates, and

examine them in a recursive depth-first manner

14Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Navigating the States

• Problem: going ‘Back’ in the browser

• Solutions: 1. Register the state explicitly and use Back2. Click through from the initial state

• Use element IDs: not always present, not always persistent

• Use XPath Expressions: much more reliable, not 100% water-

proof

15Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Data Entry Points

Detecting DOM forms:• For each state change, extract all the forms• For each form, calculate a hashcode based on the form

properties• Check if form is already present in the database• If not, store the form completely in the database with the

hashcode as its ID• If present, use the corresponding custom values in the

database (if present) to fill the form in the browser and submit

• Upon submission, the resulting state change is recursively analyzed.

16Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Full Auto Scan

For each state:1. Retrieve all candidate clickables2. Fire event on each candidate element3. Compare DOM trees: Edit Distance. Is DOM changed?4. Make a new State and add to the State-flow graph5. Find the XPath Expression of the Clickable Element6. Add the clickable as an edge to the SFG between state

before and after the click7. Analyze forms8. Do all this recursively

17Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Testing Ajax States

• With access to different dynamic DOM states we can check the user interface against different constraints

• We use invariants as oracle• Invariants on the DOM tree• Invariants between DOM states, and• Application-specific invariants

18Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Generic DOM Invariants

Generic DOM Invariants:• Validated DOM• No error messages in DOM• Accessible State• Secure State• Discoverable links• No Dead Clickables• Consistent Back-Button

19Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Application-specific Invariants

Examples:• //case one: warn about collapsable divs within expandable

items String xpathCase1 = "//LI[contains(@class

,’expandable’)]/DIV[contains(@class ,’collapsable’)]";

• //case two: warn about collapsable items within expandable items

String xpathCase2 = "//LI[contains(@class ,’expandable’)]/UL/LI[contains(@class ,’collapsable’)]";

20Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Testing Ajax Paths

• While running the crawler to derive the state machine can be considered as a first full test pass, the state machine itself can be further used for testing purposes.

We generate test cases from the state machine:• K shortest paths from Index to sinks (nodes with no

outgoing edges)• Cycles included only once, All-transitions coverage• Transform each path found into a JUnit test case

21Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Testing Ajax Paths@Testpublic void testcase1 () {browser.goToUrl (url);/*Element-info: SPAN class=expandable-hitarea */browser .fireEvent(new Eventable (new Identification("xpath", "//DIV[1]/SPAN[4]"), "onclick"));

Comp.assertEquals(oracle.getState ("S_1"). getDom(), browser.getDom ());

/*Element-info: DIV class=hitarea id=menuitem2 */browser.fireEvent(new Eventable(new Identification("xpath","//P[2]/ DIV[2]"), "onmouseover"));…}

22Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Testing Ajax Paths

Generated test suite can be used in several ways:• It can be run as is on the current version of the Ajax

application, but for instance with a different browser to find browser incompatibilities

• It can be applied to altered versions of the Ajax application to support regression testing (against a Gold-standard)

• The typical use of the derived test suite will be to take apart specific generated test cases, and augment them with application-specific assertions, capturing specific fault-sensitive click trails.

23Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Tool Implementation

• Crawljax: crawling through dynamic states• ATUSA: plugin-based framework on top of Crawljax:

• preCrawling• inCrawling• postCrawling

• Java 1.6• Watij (IE), XULRunner (Mozilla)• Numerous libraries• Java Plugin Framework• Open source

24Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Tool Implementation

25Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Empirical Evaluation

• 4 case studies (3 open source), 1 industrial• TUDU, FreakTask, PetStore, CoachJeZelf

• Instrumented the server side code with Clover• Instrumented the client-side JavaScript code with

JSCoverage• Fault seeding and code coverage

• Code Coverage: 73% server-side, 70% client-side• More than 180 Violations per case• 80% fault detection

26Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Concluding Remarks

We have proposed a method for testing Ajax applications automatically through

1. A crawler that can detect clickables and forms and analyze state changes

2. A testing framework that can check the application behavior through generic invariants as well as application-specific invariants

3. A test suite generation from the inferred state machine.

• Tools are available for download: http://spci.st.ewi.tudelft.nl/content/software

27Dynamic Analysis and Testing of Ajax User Interfaces

| 26

Publications

• Ali Mesbah, Engin Bozdag, and Arie van Deursen (2008). Crawling Ajax by Inferring User Interface State Changes. In Proceedings of the 8th International Conference on Web Engineering (ICWE'08), IEEE Computer Society.

• Ali Mesbah and Arie van Deursen (2009). Invariant-Based Automatic Testing of Ajax User Interfaces. In Proceedings of the 31st International Conference on Software Engineering (ICSE'09), IEEE Computer Society.