Usability Assessment Framework for ACGs using AHP USABILITY ASSESSMENT FRAMEWORK FOR CODE GENERATORS USING ANALYTICAL HIERARCHY PROCESS Joshua R. Dolecal.

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

USABILITY ASSESSMENT FRAMEWORK FOR CODE GENERATORS USING

ANALYTICAL HIERARCHY PROCESS

Joshua R. Dolecal

Dahai Liu

Remzi Seker

Andrew Kornecki

Department of Human Factors and Systems

Embry-Riddle Aeronautical University

Daytona Beach, Florida

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

2

Outline

• Usability

• Code Generation

• Rationale

• Framework

• AHP

• Results

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

3

Usability

• Also called: Usability Engineering, Human Factors, Ergonomics, and User Centered Design.

• UCD is defined as “the practice of designing products so that users can perform required use, operation, service, and supportive tasks with a minimum of stress and maximum of efficiency. (Woodson,1981)

• Nielson (1993) defines Usability as the overall acceptability of a system.

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

4

Usability Criteria

• Learnability

– How easy something is to learn

• Efficiency

– Of the product (how well it aids users in performing tasks)

• Memorability

– How easy it is to remember skills learned/used

• Errors

– Error prevention, correction, and identification

• Satisfaction

– How well users are satisfied with the product

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

5

Process of Generating Code

• Select template

• Create a model/chart/diagram

• Select output language (C/C++, Java, Ada…)

• Generate Code

(Stephens, 2002)

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

6

Rationale

• ACG is relatively new

• ACGs will change the way companies work and SW is developed

• Usability need to be addressed, for special users– Experts (Computer Programmers)

• Little research done

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

7

Framework

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

8

Framework

• General Usability Criteria– (Nielson, 1993) Basic well accepted usability

• Specific Usability Criteria– (Stephens, 2002 & Whalen, and Heimdahl, 1998 &

Maclay, 2000, etc.) – Elements that are unique to ACGs according to the

current literature

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

9

Method

• AHP framework constructed– Usability criteria and– ACG functional analysis

• 8 Graduate students gave pairwise comparison. (~6 months exposure with ACGs)

• AHP weight

• Consistency

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

10

Analytical Hierarchy Process

• 3 Main steps

– Establish a hierarchy– Calculate pair-wise comparison weights– Check for consistency– Apply weights to gathered data (objective/subjective)

• Process can be Automated using Expert Choice– http://www.expertchoice.com/

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

11

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

12

AHP Values

Thomas Saaty (1982)

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

13• Thomas Saaty (1982)

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

14

4 Step Consistency Process

• Thomas Saaty (1982)

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

15

Consistency

• When dealing with many criteria/variables, people are not going to be perfectly consistent. It is up to you to review their decisions with them such that the “decision maker” understand transitivity.– E.g. If A > B and B > C then A > C

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

16

Results

AHP Weights

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

Frequency

Length of use

Content

Complexity

Time spent

Feedback

Time to modify

Log Maintenance

Expertise required

Time to compile

Semantic consistency

Flexibility

Code size

Code range

Meaningfulness

Percentages %

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

17

Results

AHP Weights

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35

Support

Efficiency

Satisfaction

Source Code Modification

Translator

Output

Percentage %

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

18

Results

AHP Weights

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

General Usability Criteria

Specific Usability Criteria

Percentage %

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

19

Results

• Semantic consistency and User satisfaction received the highest weights, but traversing up the hierarchy we see the Translator and then Specific Usability Criteria received the highest weight

• A customized answer was produced– Best product for your situation– Does not label products as better than others– Allows a combination of subjective & objective data

Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP

20

• Questions, Comments?