Duplicate Bug Reports Considered Harmful ... Really?

Nicolas BettenburgSaarland UniversityQueenʼs University

Rahul PremrajSaarland UniversityVrije Uni. Amsterdam

Tom ZimmermannUniversity of CalgaryMicrosoft Research

Sunghun KimMIT CSAILHong Kong University

Duplicate Bug Reports Considered Harmful ...Really?!

2

# 2271

A

3

Bug Database

Duplicate Bug Reports

# 2271

A

3

Bug Database


BUG

# 2271

A

3

Bug Database


# 3219

B

# 2271

4

# 3219

Bug Database


A B

# 3219# 2271

4

Bug Database


A B

What are thereasons for duplicates?

5

Inexperienced Users6

Poor Search Feature7

Multiple Failures - One Defect8

Accidental Resubmission9

Intentional Resubmission

FIX THAT BUG!

10

ECLIPSE20% Duplicates

371 per month

11

Duplicate reports areusually ignored once identified!

12

But Wait!Is this really

the right thing to do?

13

“Duplicates [...] often add useful information.

[It is unfortunate that this information is filed

in a new report.]”

DeveloperWhat Makes a Good Bug Report?to appear in FSE 2008

14

15

Alan Page Director of Test Excellence, Microsoft

15

Alan Page Director of Test Excellence, Microsoft

Bug duplicates can provide valuable information [...]

Experiment 1 Do duplicate bug reports contain additional information?

Experiment 2 Can additional

information improve bug triaging?

16

2 EXPERIMENTS


17

SOURCE CODE

PATCHES

The infoZilla ToolDetects and Extracts

Structural Information:

STACK TRACES

SCREENSHOTS

Bug 137808

Summary: Exceptions from createFromString lock-up the editor

Product: [Modeling] EMF Reporter: Patrick Sodre <[email protected]>

Component: Core Assignee: Marcelo Paternostro <[email protected]>

Status: VERIFIED FIXED QA Contact:

Severity: normal

Priority: P3 CC: [email protected]

Version: 2.2

Target Milestone: ---

Hardware: PC

OS: Windows XP

Whiteboard:

Description: Opened: 2006-04-20 14:25 -0400

As discussed on the newsgroup under the Thread with the same name I am opening

this bug entry. Here is a history of the thread.

-- From Ed Merks

Patrick,

The value is checked before it's applied and can't be applied until it's valid.

But this BigDecimal cases behaves oddly because the exception thrown by

new BigDecimal("badvalue")

has a null message and the property editor relies on returning a non-null

message string to indicate there is an error.

Please open a bugzilla which I'll fix like this:

### Eclipse Workspace Patch 1.0

#P org.eclipse.emf.edit.ui

Index: src/org/eclipse/emf/edit/ui/provider/PropertyDescriptor.java

===================================================================

RCS file:

/cvsroot/tools/org.eclipse.emf/plugins/org.eclipse.emf.edit.ui/src/org/eclipse/emf/edit/ui/provider/PropertyDescriptor.java,v

retrieving revision 1.10

diff -u -r1.10 PropertyDescriptor.java

--- src/org/eclipse/emf/edit/ui/provider/PropertyDescriptor.java 21 Mar 2006

16:42:30 -0000 1.10

+++ src/org/eclipse/emf/edit/ui/provider/PropertyDescriptor.java 20 Apr 2006

11:59:10 -0000

@@ -162,7 +162,8 @@

}

catch (Exception exception)

{

- return exception.getMessage();

+ String message = exception.getMessage();

+ return message == null ? exception.toString() : message;

}

}

Diagnostic diagnostic =

Diagnostician.INSTANCE.validate(EDataTypeCellEditor.this.eDataType, value);

Patrick Sodre wrote:

Hi,

It seems that if the user inputs an invalid parameter that gets created from

"createFromString" the Editor locks-up until the user explicitly calls "restore

Default Value".

Is this the expected behavior or could something better be done? For

instance if an exception is thrown restore the value back to what it was before

after displaying a pop-up error message.

I understand that for DataTypes defined by the user he/she should take care

of catching the exceptions but for the default ones like BigInteger/BigDecimal

I think the EMF runtime could do some of the grunt work...

If you think this is something worth pursuing I could post an entry in

Bugzilla.

Regards,

Patrick Sodre

Below is the stack trace that I got from the Editor...

java.lang.NumberFormatException

at java.math.BigDecimal.<init>(BigDecimal.java:368)

at java.math.BigDecimal.<init>(BigDecimal.java:647)

at

org.eclipse.emf.ecore.impl.EcoreFactoryImpl.createEBigDecimalFromString(EcoreFactoryImpl.java:559)

at

org.eclipse.emf.ecore.impl.EcoreFactoryImpl.createFromString(EcoreFactoryImpl.java:116)

at

org.eclipse.emf.edit.ui.provider.PropertyDescriptor$EDataTypeCellEditor.doGetValue(PropertyDescriptor.java:183)

at org.eclipse.jface.viewers.CellEditor.getValue(CellEditor.java:449)

at

org.eclipse.ui.views.properties.PropertySheetEntry.applyEditorValue(PropertySheetEntry.java:135)

at

org.eclipse.ui.views.properties.PropertySheetViewer.applyEditorValue(PropertySheetViewer.java:249)

at

------- Comment #1 From Ed Merks 2006-04-20 15:09:23 -0400 -------

The fix has been committed to CVS. Thanks for reporting this problem.

Extracting Structural Information from Bug ReportsMSR 2008

Master Report

BUGthisasd

asdlknasdklnasdlk

askdnaklsdn

aksdnlaksdnlkasdkn

asd

sadddda

asdaddasd

aksdnlaskdnlkansd

Duplicate Report

BUGthisasd

asdlknasdklnasdlk

askdnaklsdn

aksdnlkasdkn

asdasdasdasdasd

a

s adddda

a

daddasd

asdasdasdasdasd

askdnlkansd

infoZilla

Experimental Setup

Elements

Elements

19

Master Report

BUGthisasd

asdlknasdklnasdlk

askdnaklsdn

aksdnlaksdnlkasdkn

asd

sadddda

asdaddasd

aksdnlaskdnlkansd

Elements

20

Extended Report

BUGthisasd

asdlknasdklnasdlk

askdnaklsdn

aksdnlaksdnlkasdkn

asd

sadddda

asdaddasd

aksdnlaskdnlkansdBUGthisasd

asdlknasdklnasdlk

askdnaklsdn

aksdnlkasdkn

asdasdasdasdasd

a

s

adddda

a

daddasd

asdasdasdasdasd

askdnlkansd

Elements

compare

Experimental Setup

ECLIPSE

21

16,511 Master ReportsECLIPSE

21

16,511 Master Reports

27,838 Duplicate ReportsECLIPSE

21



Master Extended21

Unique elements per report:



0

0.5

1.0

1.5

2.0

2.5

Patches Stacktraces Screenshots

0.29

1.42

1.94

0.14

0.50

1.83

Master Extended21

Unique elements per report:


22


22

They do!

Experiment 2 Can additional information improve bug triaging?

23

Developer

Bug Triage

24

DeveloperReport

BUG

Bug Triage

24

DeveloperReport

BUG

Fixed

BUG✓

Bug Triage

24

BUG

DeveloperReport

BUG

Fixed

BUG✓

Bug Triage

BUG

BUG

BUG

BUG

BUG

BUG

24

BUG

DeveloperReport

BUG

Fixed

BUG✓

Bug Triage

BUG

BUG

BUG

BUG

BUG

BUG Triager

24

25

Experimental Setup

•Machine learning to predict developers

•Train using master reports

•Train using extended reports

•10 Runs

Results for predicting Top-5 developers

43.75

52.50

61.25

70.00

1 2 3 4 5 6 7 8 9 10 All

56

65

6058

51

6061

525352

4751

5657

52

48

5755

484747

42

Master Extended

Run

26

Precision


27


27

They can!


28


28XMerge Reports

29

29

29

29

29

Duplicate Bug Reports Considered Harmful ... Really?

Education