User involvement in living lab research: experiences from an interdisciplinary study on future mobile applications De Moor, K. Berte,K, De Marez, L., Joseph,

User involvement in living lab research: experiences from an interdisciplinary study

on future mobile applicationsDe Moor, K. Berte ,K, De Marez, L., Joseph, W., Deryckere, T. & Martens, L.

MICT - WiCa - IBBT Ghent University

Third International Seville Conference onFuture-Oriented Technology Analysis (FTA):

Impacts and implications for policy and decision-making

16th- 17th October 2008

User involvement in living lab research

Context and introduction

ICT sector

- Changing user roles

• growing pressure to innovate, to impress, ...

• shorter product life cycles

• innovation as commodity

• implications for research and product development

• active and dynamic (co-)production

• ‘push’ versus ‘pull’ approaches

• user as innovator

• 'user-driven and user-generated innovation

User involvement in living lab research

Theoretical perspectives

Technology and society

Traditional tension: user vs. technology

User involvement in living lab research

Paradigm shift

User-driven innovation

• more systematic + direct user involvement• specific type of knowledge• methodological reorientation (e.g. living labs)• focus on future technologies, users and experiences • interdisciplinary process • complexity

(Source: Sleeswijk Visser, Stappers et al., 2005: 123).

User involvement in living lab research

Integration challenges and objectives

Gap user- and technology-oriented approaches

1. continuous and adequate involvement of the user

2. integration and translation of knowledge from multidisciplinary process (bridging ‘the gap’)

Objectives:• illustrate how challenges might be tackled

• share results and experiences from own empirical research

• focus on 3 moments of ‘user involvement’ prior-to-launch

PRIOR-TO-LAUNCH

OPPORTUNITY IDENTIFICATION

R&D

TEST MARKET & PILOTING

CONCEPT DESIGN

CONCEPT DEVELOPMENT &

EVALUATION

INNOVATION DEVELOPMENT &

PRODUCTION

User involvement in living lab research

General methodology: ROMAS project

Research on Mobile Applications and Services

• goal: user-oriented assessment of (future) wireless city applications & services

• living lab setting of i-City Hasselt (www.i-city.be)

• panel of >1000 test users

• wireless application services (PDA, laptop, smart phones, ...)

• interdisciplinary approach for testing technological applications

• supported by Flemish Government and industry partners:

User involvement in living lab research

Results phase 1: opportunity identification

Goal: identification of current and future mobile opportunities

• challenge: user involvement in early stage

• users’ limited imaginary capabilities

• desk research + focus groups

• focus on time spending framework and archetypes

• e.g. Archetype Patricia and some of her daily activities

User involvement in living lab research

Results phase 1: opportunity identification

List of 80 (future) mobile applications

User involvement in living lab research

Results phase 1: opportunity identification

Integration of research results for archetype Patricia

• mapping of new ideas x daily activities of the archetype

• indication of origin and status of the mobile application

User involvement in living lab research

Results phase 2: concept evaluation

Goal: creation of workable concepts + evaluation

• based on wild ideas

• adoption potential was evaluated by large audience (N:312)

• two steps: 1. application clustering + ranking 13 clusters

2. user clustering 6 clusters

- e.g. mobile news: 3,11/5- not very appealing

- e.g. indication of parking spaces and availability: 4,23/5- very appealing

User involvement in living lab research

Results phase 3A: test market

Mobile news: assessment of adoption potential

• 5 working applications + 1 idea

• only accessed 1-2 times by majority i-City panel

• illogical choice (not appealing) but influencing factors

• PSAP-Scale technology specific adoption segmentation

• comparison with theoretical adoption segments (Rogers)

N: 269

User involvement in living lab research

Phase 3B: QoS optimisation vs. QoE

Challenge: ‘bridging the gap’ integration of knowledge

• context: importance of good ‘user experience’ (QoE) (e.g. iPhone)

• QoS: technical and performance parameters

• linking/translating subjective (social, contextual, ...) dimensions to technical QoS-parameters

• creation of new, interdisciplinary methodology

• Wapedia-application: case-study (N=10)

• controlled research setting

User involvement in living lab research

Phase 3B: 5-step interdisciplinary methodology

1. Pre-usage user research

- detection of relevant user experience dimensions and expectations: e.g. price, navigation, speed, display size, …

- multi-method approach (e.g. free listing, prioritizing, conjoint analysis, QoE-dimensions questionnaire,...)

2. Pre-usage translation workshops

- find optimal match between ‘user-indicated’ QoE dimensions and ‘measurable QoS parameters’ (e.g. Simulation exercises)

- social scientists + engineers

3. Monitoring during usage

- usage scenarios for test users

- different reception levels + monitoring of ‘signal strength’

- software probe model (cfr. Deryckere, Joseph et al, 2008)

User involvement in living lab research

Phase 3B: QoS optimisation vs. QoE

4. Post-usage questions on device

- after completion of usage scenario questions on device (general experience, frustration, speed, …)

5. Post-usage Comparison (expectations vs. experience)

- user experience gaps? Multi-method approach cfr. phase 1

reduction in speed (lower [dBm]

general experience drops

E.g. User 10 (male, 30)

User involvement in living lab research

Conclusion

User-driven involvement in living labs?

Discrepancy theory versus practice

• future-oriented technology research: role of the (future) user!

• continuous interaction (early phases)

• integrated and interdisciplinary approach

• methodological reorientation (e.g. more pull-driven living labs)

• push vs. pull debate

• different stakeholders different goals

• translation and interaction between disciplines as missing link

User involvement in living lab research

Questions and contact

[email protected]

[email protected]

Research Group for Media & ICT

IBBT / Ghent University

www.mict.be – www.ibbt.be

UGent