Top Banner
Tales from the Trenches - OR Replicating things that have already been said at this meeting Robert R. Hoffman, Ph.D. © 2011 Robert R. Hoffman All rights reserved “Data Coding, Analysis, Archiving, and Sharing for Open Collaboration: From OpenSHAPA to Open Data Sharing,”
24

Hoffman nsf presentation hoffman-25-aug11.ppt

Jan 28, 2015

Download

Documents

Jesse Lingeman

 
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Hoffman nsf presentation hoffman-25-aug11.ppt

Tales from the Trenches - OR –

Replicating things that have already been said at this meeting

Robert R. Hoffman, Ph.D.

© 2011 Robert R. Hoffman All rights reserved

“Data Coding, Analysis, Archiving, and Sharing for Open Collaboration: From OpenSHAPA to Open Data Sharing,”

Page 2: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

Experimental Psychology (Cognitive, Psycholinguistics)

U of Cinti, U of Minn

Expertise Studies Cognitive Systems Engineering

Human Factors

Me

Page 3: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

Klein et al 2002

IEEE Intelligent Systems

Mental simulation Problem detection

Coordination Re-planning

Expertise development

(2 seconds ?)

Cacciabue & Hollnagel 1995

Macrocognition as a paradigm in

Cognitive Systems Engineering

Gunnar Johansson 1980

Page 4: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

Page 5: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

Communities of Practice

“Behavior” Versus Activity

Activity Theory

Work Analysis

Sociotechnics

Page 6: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

Ancient History

“How fast was the car going when it

(bumped, crashed) into the other car?”

Barbara Tversky, et al.

Page 7: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

That was then, this is….

then. . . . . .

Page 8: Hoffman nsf presentation hoffman-25-aug11.ppt

Myth of the Normal Curve

Examples: Sampling under a stopping rule Traffic delays (lots of brief ones, rare long ones) Achieving progressive criteria in pole vaulting Errors in motor coordination tasks

Patil, G. P. (1960). On the evaluation of the negative binomial distribution with examples. Technometrics, 2, 501-505.

Sichel, H. S. (1951). The estimation of parameters of a negative binomial distribution with special reference to psychological data. Psychometrika, 16, 107-127.

© 2011 Robert R. Hoffman All rights reserved

Page 9: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

"Call For Data"

•  Usability/Learnability analysis

•  Performance at the very first trials

of learning any task; any DVs

•  Exact modeling of discrete

non-Gaussian distributions

Page 10: Hoffman nsf presentation hoffman-25-aug11.ppt

Learning geometrical patterns Learning to use a cell phone by the elderly Learning to operate an automotive GPS (route-finding) Learning to recognize voices in auditory localization Learning to control an avatar in a virtual world Learning of the structure of biological categories Learning to fly a cockpit simulator

9 data sets on hand, 9 more pending

© 2011 Robert R. Hoffman All rights reserved

Page 11: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

?

Page 12: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

?

Page 13: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

?

Page 14: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

?

Page 15: Hoffman nsf presentation hoffman-25-aug11.ppt

Retabbing

Fonts

Column widths

Etc.

Rule #1

Clean-up is always necessary

© 2011 Robert R. Hoffman All rights reserved

Page 16: Hoffman nsf presentation hoffman-25-aug11.ppt

What does "No OT" mean?

What did you really do?

Did I fix the tab delineations correctly?

Rule #2

You always have to go back and talk to the researcher

Worse. . . .

People forget things about their own data, even short-term

© 2011 Robert R. Hoffman All rights reserved

Page 17: Hoffman nsf presentation hoffman-25-aug11.ppt

How do you cope with the consequences of these Rules?

Do you impose constraints?

- OR -

Do you acknowledge that clean-up will always be necessary, and figure out ways to make it easier.

The Control Challenge

© 2011 Robert R. Hoffman All rights reserved

Page 18: Hoffman nsf presentation hoffman-25-aug11.ppt

© Robert R. Hoffman All rights reserved

Requirements v. "Desirements"

("help" versus "impose")

Designing for kluges and work-arounds

Hoffman, R. R. & Elm, W. C. (2006, January/February). HCC implications for the procurement process. IEEE: Intelligent Systems, pp. 74-81.

Koopman, P. & Hoffman, R. R., (November/December 2003). Work-Arounds, Make-Work, and Kludges. IEEE: Intelligent Systems, pp. 70-75.

© 2011 Robert R. Hoffman All rights reserved

Page 19: Hoffman nsf presentation hoffman-25-aug11.ppt

Challenge #1

Finding data by data constraint (e.g., data type,

meaning)

Challenge #2

Finding data by design constraint (e.g., between v. within, etc.)

The Search Challenges

© 2011 Robert R. Hoffman All rights reserved

Page 20: Hoffman nsf presentation hoffman-25-aug11.ppt

Challenge #3

Frame problems (a priori categories v. search categories)

Challenge #4

Practicalities of representation - cryptograms, acronyms, abbreviations

The Representation Challenges

© 2011 Robert R. Hoffman All rights reserved

Page 21: Hoffman nsf presentation hoffman-25-aug11.ppt

Challenge #5

Chiding resarchers re: design limitations, methodological questions

Challenge #6

Topical research data v. Statistics itself as an area of research

The Purpose Challenges

© 2011 Robert R. Hoffman All rights reserved

Page 22: Hoffman nsf presentation hoffman-25-aug11.ppt

Usefulness

Usability

Understandability

Learnability

Having seen all the Tools & Systems. . .

© 2011 Robert R. Hoffman All rights reserved

These features are measurable

e.g., Data on Researcher time, effort, resources

“labor intensive” “efficient data mining”

Page 23: Hoffman nsf presentation hoffman-25-aug11.ppt

© 2011 Robert R. Hoffman All rights reserved

Page 24: Hoffman nsf presentation hoffman-25-aug11.ppt

www.ihmc.us/users/rhoffman/main www.ihmc.us

Thank you!

© 2011 Robert R. Hoffman All rights reserved