A Tale of Two Browsers -- MSR 2011 challenge paper

A Tale of Two Browsers

MSR-11 Challenge21-22 May 2011, Honolulu

Olga Baysal, Ian J. Davis, and Michael W. GodfreyDavid R. Cheriton School of Computer ScienceUniversity of Waterloo, Waterloo, ON, Canada

{obaysal, ijdavis, migod}@uwaterloo.ca

Dimensions of study

• Release history, bugs, bug fixing• User adoption• Popularity and reliability

– We used the provided data, plus web traffic logs from the www.cs.uwaterloo.ca domain from Feb 2007 to Dec 2010

– Caveat lector, we play fast and furious in pursuit of the interesting

Release histories

Firefox•8 major releases

− MTBR: 10 months

Chrome•10 major releases

− MTBR: 2.5 months

Beta releases Official releases

Q1: Which browser is more defect prone?

Breakdown of bugs by severity for Firefox browser

Ignored bugs: new, available, assigned, started, expired, invalid, incomplete, unconfirmed, unknown, untriaged, re-opened

Q2: How quickly are bugs fixed?

severity #_bugs % median averageblocker 7,752 2% 2 39critical 39,367 10% 12 130major 44,385 11% 9 138normal 245,313 61% 13 157minor 24,231 6% 17 210trivial 10,725 3% 14 186enhancement 29,614 7% 21 299total 401,387 100% 12 165ignored 152,333 28%total reported 553,720 100%

Importantbugs

Breakdown of bugs by priority for Chrome browser

Q2: How quickly are bugs fixed?

priority #_bugs % median averagepriority 0 538 1% 20 26priority 1 5,957 13% 27 47priority 2 34,905 77% 28 72priority 3 1,902 4% 79 141none 2,043 5% 22 37total 45,349 100% 28 69ignored 24,743 35%total reported 70,092 100%

Importantbugs

Q2: How quickly are bugs fixed?

fix time # of Firefox bugs % # of Chrome bugs %

same day 100,050 25% 682 2%

3 days 155,875 39% 2,298 5%

a week 183,603 46% 4,864 11%

a month 239,846 60% 24,615 54%

> month 161,541 40% 20,734 46%

Bug resolution time for browsers

Q3: How do adoption trends differ? Adoption trends of Firefox browser

Q3: How do adoption trends differ? Adoption trends of Chrome browser

Q4: How stale is your browser?

… but Chrome performs automatic upgrades as soon as new versions are available

Firefox 0.8 0.9 1.0 1.5 2.0 3.0 3.5

Median staleness 1,512 1,241 1,362 1,036 529 232 126

Staleness % 100% 100% 100% 100% 75% 56% 39%

Chrome 0.2 0.3 0.4 1.0 2.0 3.0 4.0 5.0

Median staleness 350 305 378 217 153 120 63 13

Staleness % 37% 18% 4% 9% 10% 6% 3% 4%

Q5: Does browser relative popularity change over time?

UW 2010 w3schools.com 2010

Big difference in popularity of the IE browser!•Usage percentages also vary significantly in other usage tracking sites•Heavy reliance on Unix at CS, UW + pre-installation of Firefox

Conclusions

• “Mature”, ambitious in

scope, and in wide deployment

• fairly stable codebase, long release cycles

• short bugs fix cycles• slow adoption of newer

versions

• NKOTB, lean, mean, and

growing very quickly in popularity

• rapidly evolving codebase, short version cycles

• longer bugs fix cycles• users (forced to) quickly

adopt new versions as they become available