Transport Demand Forecasting A Review of the Risks 21 June 2022
Transport Demand Forecasting
A Review of the Risks
2 May 2023
2 May 2023
“The Commanding General is well aware that the forecasts are no good. However, he needs them for planning purposes.”
Memo to Ken Arrow after his warnings on the unreliability of forecasts.
IntroductionForecasts are always wrong. Sometimes they’re right but only luckily. Nassim Nicholas Taleb
• Risks in Errors
– Model
– The Future
– Us
Forecast Horizon Existing Road New Road
Next Day ±7.5%
1 Year ±12.5% ±17.5%
5 Years ±20% ±27.5%
20 Years ±42.5% ±47.5%
Bain’s survey
Forecasting Without A Model1Parramatta
2Chatswood, North Sydney
3Wasteland, Cowboys
4CBD, Airports
8 lanes 4 lanes
• Population in 1985• Hh = Pop / Size• Trips = Rate * Hh• In cars = ms*trips• Cars = In Cars /Occ• Veh = Cars + Com• Split into Cells• Calc 2 to 4 + 4 to 2• Factor up to 2015• Share between Bridge and Tunnel
The Devil In the Detail“There is no such uncertainty as a sure thing.” Robert Burns, poet
This Link
This Intersection
•matrix estimation is a system of simultaneous equations•about 400,000 equations and at least 3.2 million unknowns •there are more than 2x10500 ways that this system of equations can be solved
VITM (Victorian Integrated Transport Model): • 2900 Zone
•Each trip matrix therefore has more than 8.4 million cells;
•More than 681 million cells in the model
• 30,000 nodes and 45,000 links = 300,000 turning movements
Trip Distribution
“Forecasting traffic patterns is a relatively simple exercise. Motorists will almost always take the most efficient route between two single points. ” Vesna Poljak, Australian Financial Review
Matrices
“The trouble with the future is that there are so many of them.” Niels Bohr, Physicist
Saving Money Wastes Money"An unsophisticated forecaster uses models as a drunken man uses lamp-posts - for support rather than for illumination.” after Andrew Lang
Mistake 3 – Saving Money Wastes Money
Original Refined % Difference
Crossing Point Inbound Outbound Inbound Outbound Inbound Outbound 1 11,800 12,300 12,800 15,500 8% 26% 2 2,800 3,500 2,100 2,800 -25% -20% 3 5,600 5,100 5,200 5,000 -7% -2% 4 10,300 10,500 10,400 10,800 1% 3% 5 6,500 2,200 4,400 2,200 -32% 0% 6 10,600 12,000 25,100 24,200 137% 102% 7 15,800 16,200 5,000 4,900 -68% -70% 8 18,900 19,700 31,400 30,800 66% 56% 9 10,200 11,300 11,400 12,800 12% 13% 10 17,600 20,200 17,700 20,400 1% 1% 11 22,700 22,800 30,500 32,000 34% 40% 12 7,400 8,100 7,200 6,500 -3% -20% 13 7,300 7,500 10,200 10,400 40% 39% 14 27,900 27,600 30,900 26,400 11% -4% 15 5,200 5,800 3,200 4,200 -38% -28% 16 5,200 6,600 4,900 5,700 -6% -14% 17 14,300 14,600 19,600 21,700 37% 49% 18 24,000 24,800 18,000 17,200 -25% -31% 19 26,100 23,900 34,600 33,700 33% 41% 20 14,400 13,500 34,900 33,800 142% 150% 21 16,900 20,800 11,500 11,900 -32% -43% 22 2,400 2,700 2,900 3,000 21% 11% 23 13,500 12,500 17,300 20,100 28% 61% 24 300 400 100 100 -67% -75% 25 8,900 7,900 5,500 4,600 -38% -42% 26 8,200 7,900 7,500 8,500 -9% 8% 27 14,500 13,400 25,200 27,100 74% 102%
Total 329,300 333,800 389,500 396,300 18% 19%
Errors Compounded“Any man whose errors take ten years to correct is quite a man” J Robert Oppenheimer, Physicist
The Future
• “I have seen the future and it is like the present, only longer.” Kehlogg Albran
• “I have seen the future and it doesn't work.” Robert Fulford
Searching for the PastForecasting is like driving along an unknown road, in the dark, looking out the back window. Peter de Vries, New Yorker
Searching for the Past
THE JOMET STUDY50km
70km
JOMET – The Johannesburg Metropolitan Planning for 2000
Most Likely Scenario Assumptions
JOMET - Forecast v Actual
Forecast (2005)
Estimated* (2005)
Error (%)
Population 4,695 5,520 -15% Employment 1,167 1,164 0%
* From Johannesburg City Council estimates
Forecasts of inputs for the JOMET area compared to the outcomes in thousands
Forecast (2005)
Actual (2002)
Estimated (2005)
Error (%)
Car 621,000 581,000 610,400 -2% Public Transport 558,000 510,900 552,300 -1%
Forecasts of morning peak trips for the JOMET area compared to the outcomes
Sydney Area Transport Study
Sydney Area Transport Study
Forecast (2000)
Actual (2001)
Error (%)
Population 4,286 3755 14% Employment 1,910 1,820 5%
Forecasts of inputs for the Sydney metropolitan area compared to the outcomes
Forecast (2000)
Actual (2001)
Error (%)
Home Based 9,878 9,000 10% Non Home Based 1,273 4,154 -69% Total 11,151 13,154 -15%
Forecasts of daily trips in the Sydney Area Transport Study compared to the outcomes in thousands
Sydney Area Transport Study
Forecast (2000)
Actual (2001)
Error (%)
Manufacture 550 208 164% Commerce/Finance/Property 458 470 -3%
Services/Public Service 615 843 -27%
Transport/Communication 134 145 -8% Building/Construction 115 139 -17% Other 38 15 153% Total 1910 1820 5%
Forecasts of employment by industry compared to the outcomes
Forecast (2000)
Actual (2001)
Error (%)
Office 892 735 21% Industrial 580 342 70%
Transportation and Communication 95 301 -68%
Sales and Service 305 348 -12% Other 38 94 -60% Total 1910 1820 5%
Forecasts of employment by industry compared to the outcomes
Sydney Area Transport Study
Melbourne Public Transport
Melbourne Public Transport
• Fuel prices were irregular, but increasing and interest rates were relatively high and climbing
Melbourne Public Transport
Flat Growth
Melbourne Public Transport
What they did right
• Forecast at an aggregate level
• Used models as a means to an end, not an end
• Asked the models the right questions
• Used sensitivities and/or multiple models
• Modern models are under greater expectations
Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise. John W Tukey
The Problem With US
Submission to Infrastructure Australia’s Symposium into Traffic Forecasting for Toll Roads “The real issue here is that it if a developer wants to take an optimistic view of the future and ask his traffic advisor to prepare forecasts on the basis of these optimistic assumptions, it is not the fault of the advisor that the forecasts are ‘high’”.
Flyvbjerg“… planners lie with numbers. Planners on the dark side are busy, not with getting forecasts right and following an ethical path, but with getting projects funded and built. The most effective planner is sometimes the one who can cloak advocacy in the guise of scientific or technical rationality.”
Bain“To knowingly inflate traffic and revenue projections is an act of deception – but it is not alone in that regard. Investors reviewing toll road studies should remain alert to two other potential acts of deceit.”
Introduction
• Patronage forecasting -- Why do it ?
• The risk of being wrong is high and the impact on reputation is bad
Is there a “right” forecast?
No! Forecasts need to be tailored to a purpose – high for impact assessment– low for CBA– high for owners to win a PPP deal– low for lenders
The Problem With Our Reputation
•Looking for a Practitioner’s Remedy
•Forecasts are not just reasonable but SEEN to be reasonable
•Provides alternative outcomes forecasting
•Understand the mechanics of the different models in order to forecast successfully
•A way to understand more deeply the complex interactions that contribute to transport demand in the future
A Solution
"You don't drown by falling in the water; you drown by staying there." Edwin Louis Cole
Reporting the Weather
The Aftermath
AFTERBEFORE
Multiple Models: ConfidenceStrategic Model
“If you have to forecast, forecast often.” Edgar Fiedler
Multiple Models: ConfidenceCapacity Based
Multiple Models: ConfidenceEquilibrium
Multiple Models: ConfidenceLogit
Multiple Models: Confidence
Summary• Models
– Trip Distribution– Asking too much– Using inappropriately
• Future– Not such a problem– Dealt with by Sensitivity testing– Avoid disasters
• Us– Lying cheating bastards– Misunderstand the models– Scared of uncertainty
The Computer Says “No!”
FROM TO
"Chicken guts are hard to read and invite flights of fancy or corruption.” Ian Hacking, Philosopher
Forecasting can make you look stupid
• It will be years – not in my time – before a woman will become Prime Minister. – Margaret Thatcher
• With over fifteen types of foreign cars already on sale here, the Japanese auto industry isn't likely to carve out a big share of the market for itself.
– Businessweek, August 2, 1968
• X-rays will prove to be a hoax. – Lord Kelvin, President of the Royal Society, 1883
• There's no chance that the iPhone is going to get any significant market share. – Steve Ballmer, USA Today, April 30, 2007
• Who the hell wants to hear actors talk? • H. M. Warner, Warner Brothers, 1927