I recently had an twitter and email conversation with Benjamin Ross about rail vs. bus benefit/cost analysis (BCA).

The problem is that conventional BCA in practice does not consider the quality differences of different modes, focusing primarily on travel time, monetary costs, and monetized externalities. Assuming everything else were analyzed correctly, this leads us to over-invest in low quality modes and under-invest in high quality modes, from a welfare-maximizing perspective.

Let’s start with a few premises

1. The value of time (value of travel time savings) of each user differs because of a variety of factors. Everyone is in a hurry sometimes, and so has a higher value of time (willingness to pay for saving time) when time-strapped than at other times. Some people have more money than others, and so find it easier to pay to save time. The related notion of value of travel time reliability (VTTR) is reviewed here.

2. We don’t actually know user value of time. (An alternative approach evaluates just based on travel time, and assumes everyone is equal, since time is just as fast for rich and poor people.  For instance Carlos Daganzo and his students (e.g. Gonzales) optimize in terms of time, and convert monetary and other costs into time, referring to value of time as a politically determined variable. E.g. section 2.3.2 here. developing a temporal value of money rather than a monetary value of time. This is not standard in transportation economics.)

3. We  assume the value of time of all users is the same in a Benefit/Cost Analysis because the alternative would bias investment toward users with a high value of time. E.g. wealthy people in the western suburbs would get more investment than poor people in the city because they have a higher value of time, which is politically unacceptable to admit, as they did not pay proportionate to their value of time (since transportation funding on major roads comes predominantly from gas taxes. In contrast for local roads it comes predominantly from property taxes, which of course are paid for more by the wealthy).  For a market good this is not a problem (rich people pay for and get better goods and services all the time, otherwise why be rich). We do BCA because transportation is a publicly provided good.

4. We have models which purport to know people’s value of time and do use that in forecasting travel demand. The ratio of coefficients to time costs and money costs is implicit in the mode choice model. The value of time is usually in practice estimated from revealed preference data, but values have a wide range depending on location and methodology.

5. Travel demand models are highly inaccurate, etc., for a variety of reasons.

6. If these models were correct, the log-sum of the denominator of the mode choice model multiplied by the value of time (determined by the coefficients on time and cost in the model), with a little math, gives you an estimate of Consumers Surplus. This estimate is not usually used in practice, as no one outside of economics and travel demand modeling believes in utility theory.

7. Benefit/Cost Analysis is much simpler (and more simplistic) than travel demand modeling, and uses travel time savings and monetary cost in estimating Consumers Surplus.

8. BCA doesn’t actually estimate CS, just change in CS, since we don’t know the shape of the demand curve, but can estimate small changes to the demand curve and assume the curve is linear. Those doing BCA often use the rule of 1/2 to find the area of the benefit trapezoid)

Area=benefit=(Tb-Ta)*(1/2)*(Qb+Qa).

Multiply the area by the Value of Time to monetize. This is shown in Figure 1.

9. This assumes the value of time experienced is the same independent of how it is experienced. Yet people clearly would pay more for a better experience. That doesn’t show up unless you have multiple demand curves (see below), and that is never done except by academics.

10. The travel demand model gives you an alternative specific constant (ASC), which says all else equal, mode X is preferred to mode Y, and will tell you how much additional demand there will be for X than Y under otherwise identical circumstances (namely price and time).

11. Empirical evidence suggests the ASC is positive for transit compared to car (all else equal, people like transit over car. Car mode shares are higher in most US markets because all else is not equal).

Usually the ASC is higher for new rail than new bus, since trains are a nicer experience. This is sometimes called the rail bias factor.

For instance Table 3 below reproduces values the FTA accepts for rail bias factors according to the linked report. The implication is that people would be willing to spend 15-20 minutes longer on a commuter rail than a local bus serving the same OD pair and otherwise with the same characteristics (except for the quality of the mode).

Much of this is just a question of modeling specification though, so e.g. the rationale includes things that (a) can be modeled and specified (but aren’t typically), and (b) may be improved for bus routes. Recent research says this number can be brought down a lot by better specification.

12. The Consumers Surplus from a mode choice model would reflect this with higher utility when rail is available than if bus were available.

13. The Consumers Surplus from BCA, using the rule of 1/2,  would be higher for a rail line (Figure 2) than a bus line (Figure 1) because the demand is higher.

14. The CS from BCA would not reflect fully the quality difference. It should be shown as moving the demand curve outward. The benefit from the red area (Figure 3) is missing.

15. The red area is impossible to estimate with any confidence, since the shape of the curves outside the known area (before and after) is unknown. I drew the total consumers surplus as a triangle (and the change in CS as a trapezoid) (Figure 3), but this is misleading. Certainly it is positive.

16. If it were a triangle, and the Demand curves were parallel, some geometry might reveal the shape, but we also don’t know the lines are parallel. In reality they surely aren’t. The high value of time folks (on the left) might be willing to pay a lot more for the improved quality than the low value of time folks on the right.

Ben Ross proposes to improve BCA and develop an adjustment factor to account for the differences in quality  between modes. He suggests we look at the number of minutes it takes to get a number of riders for each mode.

I have mathematized this. So R_q=C_rail,q – C_bus,q, where R is the travel time difference at some number of riders q, and C_m,q is the travel time (cost) at which you would get q riders on mode m. 

To illustrate:

If 1,000 people ride the bus at 10 minutes and 1,000 people ride the train at 12 minutes, Ben proposes the extra pleasure (or lessened pain) of taking rail is equal in value to a time savings of two minutes.

At a given margin, this is probably approximately correct. That is, the  marginal (the 1,000th) train rider is willing to take (pay) 12 minutes 12 minutes while the 1,000th bus rider insists on 10 minutes.

The problem we are trying to construct an area (the benefit). There is no guarantee that R is constant.

• The 2,000th rail rider might insist on 11 minutes, while the 2,000th bus rider requires 8 minutes. R₂₀₀₀= 11-8 =3 ≠ 12-10.
• The 10,000th rail rider might be willing to pay 3 minutes, while the 10,000th bus rider requires -3 minutes (you have to pay them 3 minutes to ride the bus). R₁₀₀₀₀=3–3 = 6.

Now we could try to find the “average” value of R, or the value of R for the average rider.  So let’s say you have forecast 30,000 riders for a line, then you try to find R for the 15,000th rider, and apply it over the whole range.

(What travel time do you need to get only 15,000 bus riders and 15,000 rail riders, this will be much different than the actual travel time you are modeling, and it will be a higher travel time, so the model will require some adjustment to obtain this number).

This again assumes distance between the curves is fixed. Unlike the rule of 1/2, which is meant to be applied over a small area, so the curvature doesn’t really matter, the assumption here is this applies over the whole demand curve, where differences in curvature might be quite significant.

If we used the model to trace out the demand curves, we could then integrate (find the red area), but this is data that is not generally obtained or reported to the economist doing the BCA. The modeler could compute this of course if they wanted to, with a bunch of model runs, but the modeler could just use the log sum, and no one believes the model or in utility or understands log sums. So the economists takes the forecast in its reduced form, and treats the method for getting it as a black box (or magic).

So is the approximation R reasonable? Is using this value better than using the implied R of 0 which is currently done?

As Ben notes,

All we really have is our one Alternative Specific Constant. It’s tough enough to draw a single value of that constant out of the available data, we surely can’t measure its dependence on income, walkability, etc.  What we actually know is the size of the rail preference under the conditions where the data was collected that the constant was calibrated against, not under the conditions that the model is simulating.

The hard part is scaling from measurement conditions to project conditions, but there are only a few simple alternatives (per trip, per mile, per minute) so if you don’t know which is right you could show results for all of them (and accept that reality may be in between).

I don’t see how this is different from the money value of time.  Doesn’t it involve the same kind of approximation?  And an assumed method of scaling?  Measured under one set of conditions, used under different conditions.

I don’t think I would trust using the model to trace out the demand curves.  The delta we’re looking at is ultimately derived from that Alternative Specific Constant.

When you only have one measured data point, drawing curves inevitably pulls in assumptions that tend to get insufficient examination and can easily introduce subtle (or not-so-subtle) errors.  The only robust conclusions are the ones that you can connect directly to your measured data point.  In my opinion (derived mostly from other kinds of models, but very strongly held) the best way to proceed is to treat your measured data point as a constant, multiply it by the relevant parameters, and go straight to an answer.  Then adjust it for whatever important factors that you can point to and explain in words why your measurement didn’t account for them and why your correction is appropriate.

You can certainly compare the calculation to a black-box model that solves partial differential equations (or in the transportation case a giant matrix), but you shouldn’t believe any model results whose cause you can’t explain convincingly after you get it.  (yes, the model sometimes detects your erroneous intuition, but most of the time it’s the model that is wrong).