The Ensemble Approach to Forecasting: A Review and Synthesis

Recently published:

  • Wu, Hao, and Levinson, D. (2021) The Ensemble Approach to Forecasting: A Review and Synthesis. Transportation Research part C. Volume 132, 103357 [doi] [Author Link]
  • Highlights

    • Review and synthesize methods of ensemble forecasting with a unifying framework.
    • As decision support tools, ensemble models systematically account for uncertainties.
    • Ensemble methods can include combining models, data, and ensemble of ensembles.
    • Transport ensemble models have the potential for improving accuracy and reliability.


    Ensemble forecasting is a modeling approach that combines data sources, models of different types, with alternative assumptions, using distinct pattern recognition methods. The aim is to use all available information in predictions, without the limiting and arbitrary choices and dependencies resulting from a single statistical or machine learning approach or a single functional form, or results from a limited data source. Uncertainties are systematically accounted for. Outputs of ensemble models can be presented as a range of possibilities, to indicate the amount of uncertainty in modeling. We review methods and applications of ensemble models both within and outside of transport research. The review finds that ensemble forecasting generally improves forecast accuracy, robustness in many fields, particularly in weather forecasting where the method originated. We note that ensemble methods are highly siloed across different disciplines, and both the knowledge and application of ensemble forecasting are lacking in transport. In this paper we review and synthesize methods of ensemble forecasting with a unifying framework, categorizing ensemble methods into two broad and not mutually exclusive categories, namely combining models, and combining data; this framework further extends to ensembles of ensembles. We apply ensemble forecasting to transport related cases, which shows the potential of ensemble models in improving forecast accuracy and reliability. This paper sheds light on the apparatus of ensemble forecasting, which we hope contributes to the better understanding and wider adoption of ensemble models.

    Fig. 1. Methods of combining data and models.

    The Next Big Things Revisited

    On January 8, 2010, I was interviewed by the Jim Foti of the Minneapolis StarTribune for their beginning of the decade article The next big things

    My bit below, with comments, numbers added for tracking:

    New light-rail lines, many more MnPass lanes and cars that make driving decisions for you are in the commuting forecast for the next decade, says David Levinson, a civil engineering professor at the University of Minnesota.
    (1) Congestion levels won’t change much, he said.

    Mostly Correct. Obviously it depends on how you define “congestion” and “much”. MnDOT’s Congestion Report (not how I would define it, but it is a consistent time series), has percent of the urban freeway system that is congested going from 21.5% in 2010 to 24.2% in 2018   over a decade with a regional (MSA) population increasing from 3.3 to 3.6 million (8.4%) over the same period. (The CSA went from 3.7 to 4.0 million, 9%).  Mean commuting time to work in Hennepin County rose from 22.1 to 23.5 minutes. Some of that is due to congestion, some of that is due to longer distances as suburban growth continues.

    (2) The Twin Cities area will have more residents, (3) but the aging population will be working less, and (4)  increased telecommuting will mean that people won’t go into work as often.

    Correct, correct, correct. The Median age in Hennepin County when from 35.9 (2010) to 36.6 (2018)  according to the US Census. The Labor Force Participation Rate dropped from 71 to 70.3 over this period. Telecommuting increased, in 2018 5.7% of Minnesota residents worked at home, in 2010, 5.0% did.

    (5) The Southwest and (6) Central Corridor rail lines are scheduled to start mid-decade, and (7) one or two Minneapolis streetcar lines could be in the mix.

    Section B-B

    Incorrect (it was scheduled, it did not happen yet), Correct, ~ (could is vague). Central Corridor started in 2014. The Southwest LRT (Green Line Extension) is under construction, but not opened. The streetcars appear dead, but these things never die.

    (8) Levinson expects highway expansion to mainly take the form of new MnPass lanes, which are for carpools, buses, motorcycles and toll-paying solo drivers.

    50%. Some highway reconstruction has a MnPass element, but not as much as I expected.

    (9) He sees plug-in hybrids as the dominant car, meaning drivers will be buying less gas, so (10) a per-mile fee will be implemented to replace lost tax revenue.

    Incorrect, incorrect and too soon. Hybrids petered out, and EVs are still slow on the uptake. I still expect EVs to be dominant in new car sales by 2030, and some kind of distance fee to be implemented on EVs and AVs by then.

    (11) Cars will keep getting safer, he said, with (12) features such as automatic emergency braking and cruise control that adapts to the speed of surrounding traffic.


    Correct, correct. Advanced driver assist technologies have been added to newer vehicles. Fatality rates are not where they should be, (especially for pedestrians and bicyclists), but fatality rates are better falling from 2010 (0.72) to 2018 (0.63).

    A total of 12 numbered predictions basic predictions, 6 correct, 3 clearly incorrect, 5 too vague and half-right by my scoring.

    Review of ‘Forecasting Urban Travel: Past, Present and Future,’ by David Boyce and Huw Williams

    Book Review:

    BoyceWilliamsForecasting Urban Travel: Past, Present and Future, by David Boyce and Huw Williams. 2015. Cheltenham, U.K. and Northampton, Massachusetts: Edward Elgar. 650 + ix. ISBN: 9781848449602.

    David Boyce and Huw Williams have written the definitive history of travel demand modeling to date. The book really begins (chapter 2) with Douglass Carroll, whose innovations in Detroit, and then in Chicago, developed the foundational four-step transportation planning forecasting tool that is widely used and abused in metropolitan areas across the globe. The aim at the time was to develop forecasts of future traffic—how many trips, where are they going, how many would drive, and which routes would they use—that could be used to locate and size freeways being deployed with the upcoming Interstate Highway System. In one sense, it was enormously successful, as the model spread from the Midwest of the United States across the globe, and has been used to conduct analyses, inform, and justify projects. It also spurred enormous methodological advances, one of which earned Daniel McFadden a Nobel Prize in economics for his work on developing random utility choice models.

    After developing the framework, the book explores the formalization of spatial interaction models, especially the work of Alan Wilson (chapter 3), as well as the emergence of choice models in the 1960s, through development and generalization of the multinomial logit in its many, many forms (chapters 4 and 5), and the unification of the two. I have experience working as a travel demand modeler, so while it was familiar territory, it was also clarifying in many ways. Reading the book, I understood the linkages far more deeply than before. One of the merits of the book is the hiding of most of the mathematics in the end notes. (Note 1) The book also provides a very detailed history of developments in the United Kingdom and some discussion of developments in other European countries, in addition to the history from the United States’ perspective, providing a more global perspective on the issue. I was not aware the East Germans had developed their own modeling methods, and remain curious about practices elsewhere in the Eastern Bloc.

    The book turns to the present trend in microscopic models, including activity-based models (chapter 6), which model tours of individuals rather than trips of aggregate flows, and the Transims project (chapter 8). From a methodological perspective, this is clearly an improvement, and the direction most applied models seem to be moving toward. By recognizing time-space constraints as formulated by Ha ̈gerstrand, the models become more realistic. If it were presented clearly to the public, it would probably be more intuitive and comprehensible than traditional aggregate models. This has not improved their predictive value, however, and Transims was, as the book describes, a diversion of resources with little to show for it.

    After dealing with the first three steps of the travel demand model, the book then discusses network equilibrium and solution methods for route assignment or route choice (chapter 7). These evolved independently from the work of Beckmann, McGuire, and Winsten in the 1950s (coinci- dentally also in Chicago, though not interacting with Carroll’s group), were operationalized with steadily improving algorithms in the 1970s, and were mainstreamed into practice by the late 1980s. While equilibrium for route choice is now standard (whether or not traffic is in equilibrium, and whether or not route choice depends solely on travel cost), there is no standard “feedback” method to ensure that the travel time inputs to the travel demand process are consistent with the travel time outputs of route choice procedures, one of the many problems with actual transportation modeling practice, and a problem that as the authors note has received insufficient attention in the literature (the authors notably excepted).

    Travel demand models have their uses. Long-term forecasting—their putative rationale—is probably the weakest. Yet, this entire enterprise was nominally driven by the desire to forecast travel demand, not simply understand or model it. It is indeed in the title of the book. On that score, the field has, in my opinion, failed miserably. Boyce and Williams note that accuracy is poor, and not improving over time. While one can understand the naivety of early modelers in the 1950s and 1960s (who undoubtedly well understood the limitations), by the 1970s (and certainly by the 2010s), the futility of accurate forecasting should have become apparent to those both within and outside the field. The forecasts are driven by the assumption that behavior in the future, given identical characteristics, will be the same as today. Culture is outside the scope of models, with good reason, but if culture matters, or anything else that is also outside the model’s data, there will be misses. Modelers may claim data issues, or poor inputs, and those certainly matter, yet as the authors note, estimation of models across time is never done in practice. There are always reasons— incompatibility of surveys, time, budget, and so on. The excuse for using cross-sectional analysis in the 1950s was that there was no time series; only one survey (at most) had ever been done in any metropolitan area. The excuse today is what?

    In addition to behavior being static in these models, technology is as well. As the book notes, use of stated preference models to examine what would happen given a new technology attempts to push the boundaries of this, but it fails to say what technologies will actually be around, which will affect demand in ways we just have to admit that we cannot accurately foresee. This issue is increasingly important as new modes like shared autonomous vehicles are being considered, and autonomous vehicles (even if unshared) change the character of automobile travel. That a forecasting tool considering 30 years into the future cannot consider the possibility of such change in any reliable way suggests that it is probably not the right tool.

    For this reason, these travel demand forecasts, at one time the most sophisticated analyses done by humans with their early use of mainframes (described in chapter 10), fall into the same trap as much simpler forecasts: underestimating growth in the early years of a technology’s life cycle and overestimating in the late stages. Boyce and Williams note the many problems with forecasts toward the end of the book (chapter 11), but remain more optimistic than I would. I believe the field should take its tools and apply them where they might be useful: short-term analyses of minor changes, scenario analyses of alternatives, but most definitely not forecasts. This requires changing evaluation procedures and government regulations. However, there are enough problems today that remain unsolved, so that looking for problems 20–30 years down the road seems futile.

    In the end, this book serves as an excellent history of ideas about the research and its application in travel behavior and travel demand modeling forecasting, and should be widely read by researchers and practitioners in the field, and owned by their library. In my mind, however, it is a history of proceeding very deeply down the wrong rabbit hole, of systematic application of mathematical methods providing a veneer of science to cover, in the end, for political decisions.

    — David Levinson

    The Review appears in Journal of Regional Science June 2016 issue:


    1 As Alfred Marshall wrote: “But I know I had a growing feeling in the later years of my work at the subject that a good mathematical theorem dealing with economic hypotheses was very unlikely to be good economics: and I went more and more on the rules—(1) Use mathematics as a short-hand language, rather than as an engine of inquiry. (2) Keep to them till you have done. (3) Translate into English. (4) Then illustrate by examples that are important in real life. (5) Burn the mathematics. (6) If you can’t succeed in 4, burn 3. This last I did often.” pp. 427–428 of Memorials of Alfred Marshall, edited by A. C. Pigou. One edition is: New York, A. M. Kelley, 1966.

    New Book: The End of Traffic and the Future of Transport

    We are pleased to announce the publication of our latest book The End of Traffic and the Future of Transport on Kindle Editions and at the iBookstore. The price is $4.99.

    The End of Traffic and the Future of Transport, by David M. Levinson and Kevin J. Krizek
    The End of Traffic and the Future of Transport, by David M. Levinson and Kevin J. Krizek

    Table of Contents

    • Preface: The Lost Joy of Automobility
    • Climbing Mount Auto: The Rise of Cars in the 20th Century
    • Less Traffic is a Good Thing
    • What Killed America’s Traffic?
    • Pace of Change
    • Transitioning Toward Electric Vehicles
    • Autonomous Autos
    • MaaS Transport
    • Transit
    • Up and Out: The Future of Travel Demand and Where We Live
    • Adapting the Built Environment
    • Reduce, Reuse, Bicycle
    • Accelerating the End of Traffic via Pricing
    • Redeeming Transport
    • Post-script 1: What Happened to Traffic?
    • Post-script 2: Now extinct: the Traditional Transport Engineer

    In this book we propose the welcome notion that traffic—as most people have come to know it—is ending and why. We depict a transport context in most communities where new opportunities are created by the collision of slow, medium, and fast moving technologies. We then unfold a framework to think more broadly about concepts of transport and accessibility. In this framework, transport systems are being augmented with a range of information technologies; it invokes fresh flows of goods and information. We discuss large scale trends that are revolutionizing the transport landscape: electrification, automation, the sharing economy, and big data. Based on all of this, the final chapters offer strategies to shape the future of infrastructure needs and priorities.

    We aim for a quick read—and to encourage you and other readers to think outside your immediate realm. By the end of this book (today, if you so choose) you will appreciate the changing times in which you live. You will hopefully appreciate what is new about transport discussions and how definitions of accessibility are being reframed. You will be provided with new ways of thinking about the planning of transport infrastructure that coincide with this changing landscape. Even if transport is not your bailiwick, we like to think there is something interesting for you here. We aim to share new perspectives and reframe debates about the future of transport in cities.

    The road as an Ecosystem in the 21st Century

    Today in the Tech World, there is discussion of “platforms” and “ecosystems”. When we hear talk about Apple vs. Google, it is as much about the Apple ecosystem, particularly that around iOS, the operating system for the iPhone, vs. Android OS. The Operating Systems enable both device-based and cloud-based software services. I can buy apps that work in either eco-system, but not both (without purchasing twice). I can buy peripherals that work on one or the other, but generally not both. This mobile telephone ecosystem logic follows and is much larger than the previous decades’ PC operating system ecosystems.

    Roads are a different form of economic ecosystem, and perhaps the original one. There is the ecosystem for building roads, and there is an ecosystem for those using roads. Carriers as well as private vehicles are the users. But they have a set of roadside services (energy (hay, gasoline), shelter (inns (hello Jesus), motels, and hotels), and sustenance (food)) as well as many others that are less frequently used (tollbooths, money changing, black smithing, wheel wright, vehicle repair, and so on) that are configured a particular way for users of the road ecosystem.

    While the types of vehicles using roads, as well as the materials with which they are made has changed over time, the platform of the road as a place on which to hang a series of road-serving businesses is long-standing, and unlikely to disappear even as roads change with the next technological shift in vehicles.

    Without roads (dirt, gravel, block, rail, asphalt, or concrete), there would not be much economy. Certainly off-road vehicles and their passengers and drivers of various kinds would still require services, but the much higher cost of travel would significantly reduce the total economic impact. Secondary economic impacts on things like manufacturing, agriculture, and non-transportation services which do depend on transportation thus depend on this eco-system as well.

    There is a fascinating series of books by John Jakle and colleagues describing the emergence of the first order 20th Century Road Ecosystem: Fast Food, Motels, Gas Stations, and so on. What happens in the 21st Century with Vehicle Electrification and Automation?

    We can certainly speculate that charging stations ultimately replace gas stations. Even more, vehicles may be charged in motion from the roadway.

    Food production and delivery may also change in ways that are difficult to foresee. We can speculate that with automated vehicles, food may come to us in motion, rather than us stopping at the side of the road. While this synchronization, resembling the in-air refueling of Air Force One, seems far out, with full information and automated drivers, it may be quite trivial. This may or may not be a net improvement in food quality.

    Why stay at a hotel when your car can move you forward in space and time while you sleep?

    How else will the Road Ecosystem Change in the 21st Century?


    When forecasting the future of travel demand, we have two possible outcomes, compared to the present. People may either:

    • Travel more, or
    • Travel less

    (There is an infinitesimally small possibility they travel exactly the same amount, so we will ignore that). Of course they may travel more in some places or at some times or by some modes, and less for others, so we need to distinguish between micro and macro-outcomes. That depends on changes in a variety of inputs: In travel demand forecasting we have a variety of dimensions of travel we try to model:

    • Who:
    • When: Schedule
    • Why: Purpose
    • Where: Destination
    • How: Mode, Route

    If people in the future behave exactly the same as people today, and their demographics are similar, the built environment is similar, and policies are similar, and there are more of them, the travel demand model would produce roughly the same amount of travel per person, and more in the aggregate.

    But that is a big “If”.  This travel demand model does not capture behavioral changes, market changes, or technology changes, for starters.

    First, technology may change. A model built before a new class of modes became available cannot accurately forecast their use. For instance commercial ride sharing (“transportation network companies”, e.g. Lyft, Uber) might look like taxi, but in most US markets, taxi is so seldom used it is excluded from the model for lack of data. But it differs from taxi not only in fare (lower), but also response time (faster), whether you are sharing a ride with another party (perhaps: LyftLine, UberPool), quality of vehicle (nicer?), friendliness of driver (sociable?) and many other dimensions. That is an easy problem to identify, it is already here. Now, from a regional perspective, it may still be small enough to ignore, but who can say that will remain true in the 20 or 30 time frame of official forecasts.

    Car sharing and autonomous vehicles are among the other technology shifts that are both possible and unforecasted.

    Willingness to share vehicles is a behavioral question for which we have no answer about its likelihood. One of the purposes of money is to buy better services. Most people think riding separately is preferred to riding in a shared vehicle, but this is as much a social norm as a law of nature.

    Willingness to travel farther when the vehicle is autonomous is something we can only guess at (theory suggests if we don’t have to exert effort in driving, we may be willing to be mobile for longer times (and distances)).

    In theory we could model the supply side effects of an autonomous fleet in terms of road capacity, but we will have a mixed human-robot fleet for decades. In practice this seems quite difficult. Harder still is modeling the presence of vehicles, the response time of shared vehicles, the range of electric vehicles, distance to walk to car-sharing, and the like. Not that it cannot be done, but it cannot be done with confidence. These are a mix of technology, market, and behavioral problems which are likely to become relevant over the timeframe of forecasts.

    A plausible strategy given uncertainty is to look at scenarios in addition to expected values. Instead of making the deterministic forecast that in 20 years the line will have 40,000 passengers or 100,000 AADT, consider what it will be like under a variety of different assumptions. Bureaucrats don’t like uncertainty, which is why the rules are written the way they are. Politicians will choose whatever value in the range of numbers suits their rhetorical purpose. But the public should not be mislead by the false confidence associated with forecasts of expected values given without ranges and caveats — forecasts that have historically been quite poor.

    Sadly, people follow whoever expresses the most confidence, not whoever is the most accurate or honest or thoughtful. That is an evolutionary biological outcome to avoid  paralysis when the lion is actually chasing you: follow either whoever quickly decides to hold ground with spears or runs  the farthest the fastest, but don’t stand around contemplating the decision. But with new transportation investment, nothing is actually urgent.



    Roads to nowhere: The accuracy of travel demand forecasts for do-nothing alternatives

    Recently published

    “Roads to nowhere: The accuracy of travel demand forecasts for do-nothing alternatives” by Morten Skou Nicolaisen and Petter Næss in Journal of Transport Policy


    Impact appraisals of major transport infrastructure projects rely extensively on the accuracy of forecasts for the expected construction costs and aggregate travel time savings. The latter of these further depend on the accuracy of forecasts for the expected travel demand in both the do-something and do-nothing alternatives, in order to assess the impact of implementing new projects compared to doing nothing or postponing the decision. Previous research on the accuracy of travel demand forecasts has focused exclusively on the do-something alternatives, where inaccuracies have been revealed in the form of large imprecision as well as systematic biases. However, little or no attention has been given to the accuracy of demand forecasts for the do-nothing alternatives, which are equally important for impact appraisals. This paper presents the first ex-post evaluation of demand forecast accuracy for do-nothing alternatives, based on an empirical study of 35 road projects in Denmark and England. The results show a tendency for systematic overestimation of travel demand in the do-nothing alternatives, which is in contrast to the systematic underestimation of travel demand observed in previous studies of do-something alternatives. The main implication for planning practice is that the severity of future congestion problems is systematically overestimated. As a consequence, impact appraisals of road construction as a means of congestion relief appear overly beneficial.

    Extrapolations in Traffic vs. Reality

    There seems to be a global patterns among the Anglo-Saxon nations (and perhaps others). If you have more forecasting graphs like this for other places, please send them along via the Comments, and I will update this post.

    New Zealand:

    Figure 2: Historic New Zealand light vehicle traffic forecasts vs actual growth Source:
    Figure 2: Historic New Zealand light vehicle traffic forecasts vs actual growth Source:

    United States:

    U.S. DOT highway travel demand estimates continue to overshoot reality (SST By Eric SundquistI)
    U.S. DOT highway travel demand estimates continue to overshoot reality (SST By Eric SundquistI)

    United Kingdom:

    UK National Transport Model consistently overestimates future car use Source:’-according-figures-–-car-use-rise-still
    UK National Transport Model consistently overestimates future car use

    Some additional forecasts and/or trends from the comments and elsewhere


    Are Australians driving more (or a lot less?) via Alan Davies
    Are Australians driving more (or a lot less?) via Alan Davies


    Source: (Portuguese) via Marcos Paulo Schlickmann
    Source: (Portuguese) via Marcos Paulo Schlickmann

    Washington State:

    Washington State forecasts for SR-520. Source: Sightline Institute via
    Washington State forecasts for SR-520. Source: Sightline Institute

    British Columbia:

    Port Mann Bridge, British Columbia
    Port Mann Bridge, British Columbia  Source: Sightline Institute

    New Zealand:

    Traffic volumes across Waitemata Harbour
    Traffic volumes across Waitemata Harbour. Source:


    Intercounty Connector Source: Image by Claire Jaffe
    Intercounty Connector Source: Image by Claire Jaffe

    In other fields

    Forecasts of Treasury Yields vs. reality.
    Forecasts of Treasury Yields vs. reality.

    Delphi Energy Forecast
    Delphi Energy Forecast

    Jevons Coal Forecast
    Jevons Coal Forecast

    Evolution of Brent Oil Forecasts: Source:
    Evolution of Brent Oil Forecasts: Source:

    Past Economic Outlook projections of euro area GDP
    Past Economic Outlook projections of euro area GDP

    Demographic Changes: Revisions to Japan's Total Fertility Rate Forecasts
    Demographic Changes: Revisions to Japan’s Total Fertility Rate Forecasts

    Why have the IEA’s projections of renewables growth been so much lower than the out-turn?
    Why have the IEA’s projections of renewables growth been so much lower than the out-turn?

    “The energy world is undergoing massive transformation. Installations of renewable energy have skyrocketed around the world, exceeding most predictions from less than a decade ago. ” Source:

    State Revenue Forecasts are becoming increasingly wobbly
    State Revenue Forecasts are becoming increasingly wobbly. Source.

    Aurelija Augulyte (@auaurelija) 6/12/15, 03:44 but hey don't worry about Sweden. #Riksbank has it all under control.
    Aurelija Augulyte (@auaurelija)
    6/12/15, 03:44 writes:
    but hey don’t worry about Sweden. #Riksbank has it all under control.

    Gusher of Disappointment : Each month forecasters in the Wall Street Journal's survey of economists have predicted that oil prices would climb. Each forecast is the average of responses to that month's survey.
    Gusher of Disappointment : Each month forecasters in the Wall Street Journal’s survey of economists have predicted that oil prices would climb. Each forecast is the average of responses to that month’s survey.

    Falling inflation. Forecasters have expected inflation, as measured by the Consumer-price index, to rise more quickly toward the Fed's 2% larger. Each forecast is the average of responses of that month's survey.
    Falling inflation. Forecasters have expected inflation, as measured by the Consumer-price index, to rise more quickly toward the Fed’s 2% larger. Each forecast is the average of responses of that month’s survey.

    Effective fed funds rate and forward rates implied by futures contracts



    Projection of total deaths, United Kingdom, 1966-2030, Selected projections