How to obtain truth in forecasting …

In the comments, Yonah Freemark says
“But I don’t think the evidence is there right now to portray CAHSR’s estimates as “unreasonable.””

Now I don’t know what truly constitutes unreasonable, it depends on how you reason. By my reasoning, the forecasts are high compared to reference-class lines. Of course we do not know for certain what the actual ridership will be in 30 years, just as we don’t really know anything ‘for certain’ about the future. I would however lay money they miss their claimed forecast. This is something that is missing in travel demand forecasting, accountability.
To that end, I will bet $1000 that in 2030 CA HSR doesn’t meet current ridership projections as posted currently on their website: “88 – 117 million passengers annually by 2030 for the entire 800-mile high-speed train network”. Any takers? Maybe on http://www.longbets.org/
Contingent only on their completion of the project.
(I suppose we could have another bet on whether the line actually gets completed by 2030. I would bet no, but am less certain about this folly being avoided.)

My friend PB asked “can you think of some way to bet [as in to make money, not claim “I told you so”] the CA High Speed Rail (if built) will be a fiasco? Fiasco in the sense of waste of money/albatross on the state due to operational costs, cost overruns, level of subsidy per usage unit etc … so again not fiasco as in safety, operational downtime etc.”

Aside from one-on-one bets, about which PB notes ” I dont like making multi-year bets because of the mental overhead and some other weird asymmetries”, one would have to set up a betting pool on various operational characteristics, e.g. Policy Analysis Markets.
I have never done that, but am seriously considering setting something up for travel demand forecasts.
Unfortunately there is no obvious company to short (except maybe CAHSR bonds when they come out depending on whether the state is guarantor) (I assume you can short bonds like stocks, but I don’t know if the market exists for 20 years out). The consultants and contractors will all make out like bandits. If someone is fool enough to build it privately, short them.
My sense is forecasters should be required to post bonds about the accuracy of their forecast. Either that, or they should have to be paid in stock for the facility they are forecasting. This would greatly reward honesty and punish optimism bias and strategic misrepresentation.

2 thoughts on “How to obtain truth in forecasting …

  1. This is an interesting idea, and seems like it would likely have the effect of discouraging overly enthusiastic ridership forecasts. It seems like it also might have the unintended consequence of stymieing all transit projects, as few consultants would be willing to take risks when the future is so uncertain.
    I assume that in fairness you’d also like to set up a futures market for road projects (perhaps on a regional scale) to see whether expansions really reduce time spent in peak congestion and actual commute times over the long term. If not, traffic engineers and forecasters don’t get paid.

    Like

  2. In response to Brendon’s comment: This should apply to flows or travel times, depending on what the critical variable is. It is of course a multi-modal idea.
    Induced demand literature suggests elasticities below 1, so expansions should on average reduce travel time, though perhaps not by very much. Generally though they do serve more traffic, so from the point of view of maximizing travel demand (which a normal business would do), highways have success. If the aim were to reduce costs for users, then the success is less.
    Toll roads generally also have an optimism bias problem,un-tolled roads less so (see our paper
    http://nexus.umn.edu/Papers/ForecastAccuracy.pdf “Post-construction evaluation of traffic forecast accuracy” for an examination of how good/bad forecasts have been in the Twin Cities region. Overall freeway traffic was underestimated, while lower class road traffic was over-estimated.)

    Like

Comments are closed.