Single VS. Multiple Textbox Event Creation Experiment

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

1/26

Single VS. Multiple

Textbox EventCreation Experiment

Filip Mares TIM Carleton University

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

2/26

Outline

Background

Experiment Method

Analysis

Future Experiments Conclusion

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

3/26

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

4/26

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

5/26

Twegather

Started in Summer 2009

Joint project for TeamCampmembers

Team of 5 active members

Close to a Minimum ViableProduct (MVP)

Create anevent with3 mouseclicks

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

6/26

Difference from competition

Simple event creation process

Registration not required for creating

RSVP via Social Networks

TwitterFacebook (soon)

Free

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

7/26

Experiment

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

8/26

Experiment

Natural Process

Increase event creation

Reduce creation process difficulty

Re-design front page

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

9/26

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

10/26

Hypotheses

H1: Participants will create events quicker onaverage using the proposed page designopposed to the current page.

H2: Participants will make fewer mistakes whencreating events using the current designopposed to the proposed design.

H3:Participants will prefer the proposedTwegather event creation page to the existingevent creation page design.

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

11/26

Method

A/B testDesign A Current

Design B Proposed

8 participants Recorded screen with software

Pre-thought event

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

12/26

Metrics

Completion Time

Errors Encountered

QuestionnaireLIKERT

Designs Preference

Designs Difficulty

Comments

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

13/26

Participants will create events quicker onaverage using the proposed page designopposed to the current page.

Hypothesis 1

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

14/26

Completion Time

Mean: 41.5

seconds SD: 13.6 seconds

Mean: 22.75

seconds SD: 7.28 seconds

Design A Design B

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

15/26

Completion Time

P Value: 0.0091

Group comparison:

Design A pval: 0.779Design B pval: 0.7648

Hypothesis VerifiedDesign A is faster to

complete

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

16/26

Participants will make fewer mistakeswhen creating events using the currentdesign opposed to the proposed design.

Hypothesis 2

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

17/26

Errors Encountered

Mean: 1.38 errors

SD: 0.52 errors

Mean: 0.625

errors SD: 0.625 errors

Design A Design B

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

18/26

Errors Encountered

P Value: 0.0199

Group Comparison:

Group A pval:0.6376

Group B pval: 0.39

Hypothesisrejected

Design B had lesserrors

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

19/26

Participants will prefer the proposed Twegather event

creation page to the existing event creation pagedesign.

Hypothesis 3

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

20/26

Design Preference

Mean: 3.5/5

rating SD: 0.93

Mean: 3.63/5

Rating SD: 1.19

Design A Design B

Note: (really disliked) 1 5 (really liked)

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

21/26

Design Preference

P value: 0.8565

Group Comparison:Group A pval: 1

Group B pval: 0.788 Hypothesis

rejectedThere were no

obvious preferreddesign

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

22/26

Threats to Validity

Typing skill level

Internet connection

Test Environment

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

23/26

Participant Feedback

Single textbox hard at 1st, easierafterwards

Multiple textbox requires less thinking

Multiple textbox followed order of otherservices

Good parsing of regular phrases in single

textbox

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

24/26

Future Experiments

Event syntax auto-completion vs.preview

Event creation workflow

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

25/26

Conclusion

Event creation completed faster withsingle textbox

Less errors encountered in single

textbox design Participants did not have a strong

favorite design

8/8/2019 Single VS. Multiple Textbox Event Creation Experiment

26/26

Questions?

Thank You