Does a Programmer’s Activity Indicate Knowledge of Code?se.inf.ethz.ch/old/teaching/2010-S/0276/slides/rueegg.pdf · Introduction Motivation DOI: Degree-Of-Interest Study Subjects

1

Does a Programmer’s Activity IndicateKnowledge of Code?Software Engineering Seminar 2010

Who? Jan Rüegg

From? 2007 Paper By

Thomas Fritz, University of British Columbia

Gail C. Murphy, University of British Columbia

Emily Hill, University of Delaware

When? 16th March 2010

2

Imagine You See Something LikeThis. . .What?!?

if (fabs (text->letter_spacing) > 0.1)

{

PangoAttrList *attrs;

PangoAttribute *attr;

attrs = pango_layout_get_attributes(

layout->layout);

if (attrs)

pango_attr_list_ref (attrs);

else

attrs = pango_attr_list_new ();

attr = pango_attr_letter_spacing_new (

text->letter_spacing * PANGO_SCALE);

attr->start_index = 0;

attr->end_index = -1;

...

3

To The Rescue, Git Blame!. . . really?

(Martin Nordholts) if (fabs (text->letter_spacing) > 0.1)

(Martin Nordholts) {

(Michael Natterer) PangoAttrList *attrs;

(Martin Nordholts) PangoAttribute *attr;

(Martin Nordholts)(Michael Natterer) attrs = pango_layout_get_attributes(

(Michael Natterer) layout->layout);

(Michael Natterer) if (attrs)

(Michael Natterer) pango_attr_list_ref (attrs);

(Michael Natterer) else

(Michael Natterer) attrs = pango_attr_list_new ();

(Michael Natterer)(Martin Nordholts) attr = pango_attr_letter_spacing_new (

(Martin Nordholts) text->letter_spacing * PANGO_SCALE);

(Martin Nordholts)(Martin Nordholts) attr->start_index = 0;

(Martin Nordholts) attr->end_index = -1;

(Martin Nordholts) ...

4

The Basic Question

How can we measure a Programmers Knowledgeabout source Code?

HypothesisThe more Frequently and Recently a pro-grammer has interacted with an element X,the higher is his Knowledge about X.

4

The Basic Question

How can we measure a Programmers Knowledgeabout source Code?

HypothesisThe more Frequently and Recently a pro-grammer has interacted with an element X,the higher is his Knowledge about X.

5

Outline

Example, Hypothesis

IntroductionMotivationDOI: Degree-Of-Interest

StudySubjectsQuestionnaires

ResultsSignificance of DOIProblemsImprovements

5

Outline

Example, Hypothesis




5

Outline

Example, Hypothesis




6

MotivationWho knows What?

Important forI Expertise RecommendationsI IDE with Adaptive InterfaceI Recommending group members

Mylyn Degree-Of-Interest ModelI Introduced 2005 by Mik Kersten and Gail C. MurphyI Interest level in program elementsI Original goal: An Adaptive Eclipse Interface

6

MotivationWho knows What?

Important forI Expertise RecommendationsI IDE with Adaptive InterfaceI Recommending group members

Mylyn Degree-Of-Interest ModelI Introduced 2005 by Mik Kersten and Gail C. MurphyI Interest level in program elementsI Original goal: An Adaptive Eclipse Interface

7

Mylyn DOI ModelHow Mylyn improves Eclipse. . .

How does it work?I Each Java element has its DOI value thatI Increases with each select/edit/. . .

different operations are differently weightedI Decreases over time, when not accessed

Idea Elements with higher DOI:Higher up in Eclipse lists. . .

Effect Programmer spends less time searching forimportant items

7

Mylyn DOI ModelHow Mylyn improves Eclipse. . .

How does it work?I Each Java element has its DOI value thatI Increases with each select/edit/. . .

different operations are differently weightedI Decreases over time, when not accessed

Idea Elements with higher DOI:Higher up in Eclipse lists. . .

Effect Programmer spends less time searching forimportant items

8

Empirical Study

SubjectsI 19 Industry Java ProgrammersI Two IBM development laboratory locationsI Experience between one month to twenty years

Interactive Monitoring for five weeksI Eclipse Mylyn projectI Modified to the needs of the studyI Show about one questionnaire per week

EvaluationI Analysis of the collected dataI Interviews with programmers about the results

8

Empirical Study




8

Empirical Study




9

Questionnaires

Individual Questionnaires are generatedI After 20000, 40000 and 60000 interactionsI Random questions from each ofI 20% highest DOII 20% lowest DOII 20% in the middleI Automatically generated and verified

Typical Flow of Events1. Programmer works as usual with IDE2. IDE monitors elements for about about a week3. A questionnaire pops up, asking about high and low

DOI elements4. Programmer answers

9

Questionnaires

Individual Questionnaires are generatedI After 20000, 40000 and 60000 interactionsI Random questions from each ofI 20% highest DOII 20% lowest DOII 20% in the middleI Automatically generated and verified

Typical Flow of Events1. Programmer works as usual with IDE2. IDE monitors elements for about about a week3. A questionnaire pops up, asking about high and low

DOI elements4. Programmer answers

10

Typical QuestionsWhat the Subjects were asked. . .

1 public class Calculator2 {3 public Stack<StackItem> stack;4

5 public Calculator(double operand)6 {7 stack = new Stack<StackItem>();8 stack.push(new StackNumber(operand));9 }

10

11 public double popOperand()12 {13 double result = peekOperand();14 stack.pop();15 return result;16 }17 ...

11


Q1) Do you know the types of the parameters that arepassed to the invocation of constructor ’Calculator’?

Q2) Can you recall one method/constructor that callsmethod ’Calculator.popOperand()’?

12


1 public class Calculator2 {3 public Stack<StackItem> stack;4

5 public Calculator(double operand)6 {7 stack = new Stack<StackItem>();8 stack.push(new StackNumber(operand));9 }

10

11 public double popOperand()12 {13 double result = peekOperand();14 stack.pop();15 return result;16 }17 ...

13

GUIHow the Subjects answered. . .

13


13


14

ResultsSignificance of DOI

15

ResultsSignificance of Recency and Frequency

16


17


18

ProblemsThreads to Validity

I Number of Subjects

I Technical Problems with IDE

I Only Java/Eclipse watched

I "Nature" of Questions

I Only IDE monitored

19

ImprovementsWhat should be Added to the DOI

Authorship Leads to more knowledge about authored programelements

Code Stability Fewer code changes lead to longer knowledge aboutprogram elements

Activity Short-term kinds of activity, like debugging, have askewing effect

20

Summary

I DOI is a good Indicator for structural codeKnowledge

I Improvements are still possibleAuthorship, Activity, . . .

OutlookI Results can be used to create

better Expertise Recommendation tools

I Should be repeated withmore Programmers, Languages

20

Summary

I DOI is a good Indicator for structural codeKnowledge

I Improvements are still possibleAuthorship, Activity, . . .

OutlookI Results can be used to create

better Expertise Recommendation tools

I Should be repeated withmore Programmers, Languages

21


Q1) Can you recall the name of one class or interface that isdirectly extended, implemented by ’TYPE’, or can yourecall the name of one class that directly extends the type’TYPE’?

Q2) Do you know the types of the parameters that are passedto the invocation of method/constructor ’METHOD’?

Q3) Do you know two methods that are called bymethod/constructor ’METHOD’?

Q4) Can you recall one method/constructor that callsmethod/constructor ’METHOD’?

22

ImprovementsWhat should be Added to the DOI

Authorship More knowledge about authored program elements.

Authorship Duration More time spent creating an element ⇒ more knowledgeabout the element.

Code Stability Fewer code changes ⇒ longer knowledge about it

Code Patterns More code patterns ⇒ easier to infer knowledge

Role Important element ⇒ more knowledge

Task Locality More interaction with other programmers’ code ⇒ lessDOI-to-knowledge correspondence.

Activity Short-term kinds of activity, like debugging, have skewingeffect.

Work Experience More experience of programmer ⇒ betterDOI-to-knowledge correspondence.

23

MylynWhat it looks like. . .

http://www.eclipse.org/mylyn/

http://www.eclipse.org/mylyn/

Does a Programmer’s Activity Indicate Knowledge of Code?se.inf.ethz.ch/old/teaching/2010-S/0276/slides/rueegg.pdf · Introduction Motivation DOI: Degree-Of-Interest Study Subjects

Documents