Comparing Market Efficiency with Traditional and Non-Traditional Ratings Systems in ATP Tennis

Dr Adrian Schembri Dr Anthony Bedford Bradley O’BreeNatalie Bressanutti

RMIT Sports Statistics Research GroupSchool of Mathematical andGeospatial SciencesRMIT UniversityMelbourne, Australia

www.rmit.edu.au/sportstats

Aims of the Presentation

Structure of ATP tennis, rankings, and tournaments;

Challenges associated with predicting outcomes of tennis matches;

Utilising the SPARKS and Elo ratings to predict ATP tennis;

Evaluate changes in market efficiency in tennis over the past eight years.

Background to ATP Tennis

ATP: Association of Tennis Professionals;

Consists of 65 individual tournaments each year for men playing at the highest level;

Additional:178 tournaments played in the Challenger Tour;

534 tournaments played in Futures tennis.

ATP Tennis Rankings

“Used to determine qualification for entry and seeding in all tournaments for both singles and doubles”;

The rankings period is always the past 52 weeks prior to the current week.

ATP Tennis Rankings – Sept 12th, 2011

How Predictive are Tennis Rankings? Case Study

Australian Hardcourt Titles January, 1998

Adelaide, Surface – Hardcourt

Lleyton Hewitt (AUS) Andre Agassi (USA)

Age 16 years 27 years

ATP Ranking 550 86 (6th in Jan, 1999)

How Predictive are Tennis Rankings? Case Study

Robby Ginepri Robin Soderling

6 - 4 7 - 5Age 16 years 27 years

Tourn Seed Unseeded 1

Aircel Chennai OpenJanuary 4 - 10, 2010

Chennai, Surface – Hardcourt

Challenges Associated with Predicting Outcomes in ATP Tennis

Individual sport and therefore natural variation due to individual differences prior to and during a match;

Constant variations in the quality of different players: Players climbing the rankings; Players dropping in the rankings; Players ranking remaining stagnant.

The importance of different tournaments varies for each individual players.

Recent Papers on Predicting ATP Tennis and Evaluating Market Efficiency

Forrest and McHale (2007) reviewed the potential for long-shot bias in men’s tennis;

Klaassen and Magnus (2003) developed a probability-based model to evaluate the likelihood of a player winning a match, whilst Easton and Uylangco (2010) extended this to a point-by-point model;

A range of probability-based models are available online, however these are typically volatile and reactive to events such as breaks in serve and each set result (e.g., www.strategicgames.com.au).

Aims of the Current Paper

Evaluate the efficiency of various tennis betting markets over the past eight years;

Compare the efficiency of these markets with traditional ratings systems such as Elo and a non-traditional ratings system such as SPARKS;

Identify where inefficiencies in the market lie and the degree to which this has varied over time.

Elo Ratings and the SPARKS

Introduction to Ratings Systems

Typically used to: Monitor the relative ranking of players with other players

in the same league; Identify the probability of each team or player winning

their next match.

Have been developed in the context of individual (chess, tennis) or group based sports (e.g., AFL football, NBA);

The initial ratings suggest which player is likely to win, with the difference between their old ratings being used to calculate a new rating after the match is played.

Introduction to SPARKS

Initially developed by Bedford and Clarke (2000) to provide an alternative to traditional ratings systems;

Differ from Elo-type ratings systems as SPARKS considers the margin of the result;

Has been recently utilised to evaluate other characteristics such as the travel effect in tennis (Bedford et al., 2011).

• where

oldoldppoldnew

PPSSPP

tswinsxf

processonOptimisati

.10000.1000

.100..

SPARKS: Case Study

Robin Soderling (SWE) Ryan Harrison (USA)6-2 6-4

Seeding 1 Qualifier

Pre-Match Rating 2986.3 978.4

Expected Outcome 20.1 -20.1

Observed Outcome Win Loss

SPARKS 24 6

SPARKS Difference 18 -18

Residuals -2.1 2.1

Post-Match Rating 2975.9 988.8

Longitudinal Examination of SPARKS

13000 18000 23000 28000 33000 38000 43000

Limitations of SPARKS: Case Study

Player Set 1 Set 2 Set 3 Set 4 Calculation SPARKS (Diff)Player 1 7 7 7 21 + (3*6) 39 (21)

Player 2 6 6 6 18 + (0*6) 18 (21)

Player Set 1 Set 2 Set 3 Set 4 Calculation SPARKSPlayer 1 6 3 6 6 21 + (3*6) 39 (21)

Player 2 2 6 2 2 12 + (1*6) 18 (21)

Player 2 6 6 6 18 + (0*6) 18 (21)

Player 2 2 6 2 2 12 + (1*6) 18 (21)

Player 2 competitive in all three sets.

Player 2 6 6 6 18 + (0*6) 18 (21)

Player 2 2 6 2 2 12 + (1*6) 18 (21)

Player 2 competitive in all three sets.

Player 2 competitive in 1 out of 4 sets.

Historical Results of the SPARKS Model

Year Win Prediction in all ATP Matches

2003 .64

2004 .64

2005 .69

2006 .67

2007 .66

2008 .67

2009 .69

2010 .72

The following table displays historical results of the raw SPARKS model over the past 8 years.

Historical Results of the SPARKS Model

Year Win Prediction in all ATP Matches

2003 .64

2004 .64

2005 .69

2006 .67

2007 .66

2008 .67

2009 .69

2010 .72

The following table displays historical results of the raw SPARKS model over the past 8 years.

Banding of Probabilities

Lower Band

Upper Band Midpoint

0.00 0.05 0.025

0.05 0.10 0.075

0.10 0.15 0.125

0.15 0.20 0.175

0.20 0.25 0.225

0.25 0.30 0.275

0.30 0.35 0.325

0.35 0.40 0.375

0.40 0.45 0.425

0.45 0.50 0.475

Probability banding is used primarily to determine whether a models predicted probability of a given result is accurate;

Enables an assessment of whether the probability attributed to a given result is appropriate based on reviewing all results within the band;

For example, if 200 matches within a given tennis season are within the .20 to .25 probability band, then between 20% and 25% (or approx 45 matches) of these matches should be won by the players in question.

Banding and the SPARKS Model

Lower Band

Upper Band Midpoint

0.00 0.05 0.025

0.05 0.10 0.075

0.10 0.15 0.125

0.15 0.20 0.175

0.20 0.25 0.225

0.25 0.30 0.275

0.30 0.35 0.325

0.35 0.40 0.375

0.40 0.45 0.425

0.45 0.50 0.475

Lower Band

Upper Band Midpoint

0.50 0.55 0.525

0.55 0.60 0.575

0.60 0.65 0.625

0.65 0.70 0.675

0.70 0.75 0.725

0.75 0.80 0.775

0.80 0.85 0.825

0.85 0.90 0.875

0.90 0.95 0.925

0.95 1.00 0.975

Lower Band

Upper Band Midpoint

0.00 0.05 0.025

0.05 0.10 0.075

0.10 0.15 0.125

0.15 0.20 0.175

0.20 0.25 0.225

0.25 0.30 0.275

0.30 0.35 0.325

0.35 0.40 0.375

0.40 0.45 0.425

0.45 0.50 0.475

Lower Band

Upper Band Midpoint

0.50 0.55 0.525

0.55 0.60 0.575

0.60 0.65 0.625

0.65 0.70 0.675

0.70 0.75 0.725

0.75 0.80 0.775

0.80 0.85 0.825

0.85 0.90 0.875

0.90 0.95 0.925

0.95 1.00 0.975

Represent the underdog.

Represent the favourite.

Banding and the SPARKS Model (2003-2010)

Under-estimates the probability of the under-dog winning.

Over-estimates the probability of the favorite winning.

Elo Ratings

Introduction to Elo Ratings

Elo ratings system developed by Árpád Élő to calculate relative skill levels of chess players

EOWRR ON

400/)(1011

BA RRE

Owhere: RN = New rating

RO = Old ratingO = Observed ScoreE = Expected ScoreW = Multiplier (16 for masters, 32 for lesser

players)

Probability Bands: Elo Ratings

Probability Bands: Elo Ratings (2003-2010)

High variability in the majority of probability bands during the burn-in period.

Advantages and Shortcomings of SPARKS and Elo Ratings

SPARKS considers the margin of the result, often a difficult task in the context of tennis;

Elo is only concerned with whether the player wins or loses, not the margin of victory in terms of the number of games or sets won;

Elo provides a more efficient model in terms of probability banding, suggesting that evaluating the margin of matches may be misleading at times.

Market Efficiency of ATP Tennis in Recent Years

ATP Betting Markets Used in the Current Analysis

Market AbbreviationBet 365 B365

Luxbet LB

Expekt EX

Stan James SJ

Pinnacle Sports PS

Elo ratings Elo

SPARKS SPARKS

Overall Efficiency of Each Market between 2003 and 2010

Market 2003 2004 2005 2006 2007 2008 2009 2010 OverallB365 .71 .67 .70 .71 .72 .71 .70 .70 .703

LB .70 .69 .68 .69 .70 .71 .70 .70 .697

PS .71 .65 .70 .68 .72 .70 .70 .70 .696

SJ .69 .69 .70 .67 .73 .69 .70 .71 .696

EX .72 .65 .72 .69 .73 .70 .70 .69 .698

Elo .59 .62 .66 .65 .70 .66 .68 .67 .654

SPARKS .63 .64 .69 .67 .66 .60 .69 .72 .667

Overall .68 .66 .69 .68 .71 .68 .70 .70 .69

Market 2003 2004 2005 2006 2007 2008 2009 2010 OverallB365 .71 .67 .70 .71 .72 .71 .70 .70 .703

LB .70 .69 .68 .69 .70 .71 .70 .70 .697

PS .71 .65 .70 .68 .72 .70 .70 .70 .696

SJ .69 .69 .70 .67 .73 .69 .70 .71 .696

EX .72 .65 .72 .69 .73 .70 .70 .69 .698

Elo .59 .62 .66 .65 .70 .66 .68 .67 .654

SPARKS .63 .64 .69 .67 .66 .60 .69 .72 .667

Overall .68 .66 .69 .68 .71 .68 .70 .70 .69

Heightened stability and efficiency across markets and seasons since 2008.

Converting Market Odds into a Probability

Novak Djokovic Rafael NadalMatch Odds $1.63 $2.25

Conversion 1/1.63 1/2.25

Probability of Winning .61 .44

2011 US Open Final

Accounting for the Over-Round

The sum of the probability-odds in any given sporting contest typically exceeds 1, to allow for the bookmaker to make a profit;

The amount that this probability exceeds 1 is referred to as the over-round;

For example, if the sum of probabilities for a given match is equal to 1.084, the over-round is equal to .084 or 8.4%

Accounting for the Over-Round

Conversion 1/1.63 1/2.25

Probability of Winning .61 .44

Sum of Probabilities 1.05

Over-Round 5%

2011 US Open Final

6 – 2 6 – 4 6 – 7 6 – 1

Comparison of Over-Round Across Markets

Kruskal-Wallis test with follow-up Mann-Whitney U tests:Significant difference between all betting markets aside from Pinnacle Sports and Stan James.

Over-Round for Bet 365 (2003-2010)

Accounting for the Over-Round: Normalised Probabilities and Equal Distribution

Raw Probability of Winning .61 .44

Over-round .05 .05

Normalisation .61/1.05 .44/1.05

Normalised Probability of Winning .58 .42

Equal Distribution .61 – (.05/2) .44 – (.05/2)

Equalised Probability of Winning .585 .415

Over-round .05 .05

Roger Federer Bernard TomicMatch Odds $1.07 $6.60

Over-round .08 .08

Roger Federer Bernard TomicMatch Odds $1.07 $6.60

Over-round .08 .08

Market Efficiency in ATP Tennis

SPARKS significantly less efficient when compared with the betting markets for all bands aside from .50 - .55.

Market Efficiency in ATP Tennis - Raw

General inefficiency across bands, likely due to no correction for the over-round.

Market Efficiency in ATP Tennis - Normalised

Market Efficiency in ATP Tennis – Equal Diff

Relative consistency in efficiency and variability within each band across markets.

Evidence of longshot bias for the .25 to .30 band.

Market Efficiency in ATP Tennis: Bet365

Longitudinal Changes in Market Efficiency

Few significant differences emerged when comparing efficiency across the bands over the past 8 years.

Homogeneity of variance tests revealed significantly less variability across markets in recent years.

Most Efficient Year: 2007

Least Efficient Year: 2004

Discussion of Findings

Psychological Player Considerations

Form of an individual player will affect the context and potential outcome of the entire match, as opposed to a team-based sport where individual players have less impact or can be substituted off if out of form.

Micro-events within a match, at times, have an impact on the outcome of the match. Examples: Rain delays Injury Time outs Code violations

Shortcomings of the Current Analysis

A set multiplier of ‘6’ was used for the SPARKS model based on the original SPARKS model published in 2000;

Only a limited number of betting markets were incorporated, and therefore it was not possible to utilise Betfair data into the analysis;

Differences in market efficiency and inefficiency were not evaluated at the surface level. This would be particularly interesting if evaluated for clay, given the volatility of player performance on clay when compared with other surfaces.

Future Work

Optimise the set multiplier of the SPARKS model;

Develop a model that combines SPARKS and Elo ratings;

Extend the current findings to incorporate women’s tennis given that evidence has shown greater volatility in the women’s game.

Incorporate data on other potential predictors of tennis outcomes. Examples include: The set sequence of the match Surface Importance of the tournament (e.g.,

Grand slams)

Conclusions

Whilst considerable variability was evident during the 2003 – 2007 seasons, an increase in consistency across markets since 2008.

Following a lengthy burn-in period of four years, the Elo model outperformed SPARKS and most betting markets across the majority of probability bands;

Whilst not efficient in terms of probability banding, the SPARKS model was able to predict an equivalent proportion of winners to the betting markets, and outperformed some markets in recent years;

A model that combines both Elo and SPARKS may yield the most efficient model.

Questions and Comments

Comparing Market Efficiency with Traditional and Non-Traditional Ratings Systems in ATP Tennis

predicting atp tennis

tennis klaassen

futures tennis

rankings players

individual tournaments

individual players

tournaments challenges

rankings period

Documents

ATP DDR3 DRAM Solutions - ATP Electronics

ApoTox-Glo Triplex Assay - Fisher Scientific„¢ Triplex...

The 2016 ATP® Official Rulebook - ATP World Tour

Watch - Novak Djokovic - atp dubai results 2015 - 2015 atp.....

ASA STANDARD DRAWINGS - Transport for NSW › system ›...

Energy source for contraction ATP ADP + Creatine phosphate =...

Critical role of ATP-induced ATP release for Ca2 ...

Recommender Systems TIETS43 -...

Sinteza ATP iz ADP in P, obnavljanje ATP HIDROLIZA ATP na...

Associations between online patient ratings and traditional....

The Structure Adenine and Hydrolysis of ATP - NSLC...

AdenineRibose3 Phosphate groups ATP Adenosine ATP Structure....

The pay-off phase Generates 4 ATP, 2...

Key Concepts ATP structure ATP – ADP cycle Caloric...

Fixing Army Doctrine - Army University Press · Fixing Army...

Comparing Market Efficiency with Traditional and...