Follow us on Twitter

Guide to economic evaluation of transport projects: Part 1 – Principles of cost-benefit analysis

Debates over major transport investments often get caught up in arguments over benefit-cost ratios, or BCRs. In recent years, projects such as the Transmission Gully and Puhoi to Warkworth motorways and the City Rail Link have been criticised for their low BCRs. These debates have often raised more questions than they resolve. So it’s necessary to ask: What is a BCR, how is it calculated, and what does it mean?

The good news is that there is a manual that explains it – New Zealand Transport Agency’s Economic Evaluation Manual (EEM). The bad news is that it’s tediously long and not written for a general audience. This series of posts aims to provide a guide for the perplexed. In it I will cover issues such as the principles of cost benefit analysis, the identification of economic benefits arising from transport projects, forecasting of transport outcomes, and accounting for the land-use impacts of transport projects.

Part one of this series provides a high-level overview of the principles of cost-benefit analysis.

Cost-benefit analysis, or CBA, is the general approach used to evaluate transport projects and many other public investments. It is commonly used by both local and central government for project evaluation, although it is not strictly required in all situations. (The Treasury’s Better Business Case guidelines, which apply to most public investment, provide an indication of which areas are covered by the standard approach.) The output from a CBA is a benefit-cost ratio that compares a project’s net benefits to society with its net costs to society and allows a funding agency to compare different projects in a consistent way.

CBA, which estimates the economic efficiency of a project, is not necessarily the only input to project evaluation. In recent years, NZTA has introduced several additional criteria to its assessment framework – “strategic fit” and “effectiveness”. In practice, these are more nebulous than CBA, and as a result I may leave them for a future post.

NZTA and other agencies such as Auckland Transport use CBA for two main purposes:

  • First, in order to choose a preferred solution to a particular transport problem. For example, AT’s 2012 City Centre Future Access Study (CCFAS) used CBA to compare six different options for adding capacity to city centre-based transport networks and concluded that the CRL was the best-performing option.
  • Second, in order to prioritise its investment portfolio. NZTA generally uses project BCRs to choose how to allocate funds to different projects within its funding categories. For example, when choosing how to allocate its walking and cycling budget, which totalled $53 million from 2012-2015, it would generally select the projects with the highest BCRs. Same deal for the budget for new and improved local road infrastructure – although these projects often have much lower BCRs than the walking and cycling ones, since NZTA is obliged to spend a greater share of its funding on them ($475 million over 2012-2015), regardless of the relative BCRs.

Cost-benefit analyses are generally conducted using a similar set of underlying principles. CBA seeks to:

  • Quantify all costs and benefits to society regardless of their incidence, or who benefits or pays
  • Identify the net benefits and costs of a project relative to a “do-minimum”, or what would have happened anyway
  • Consider benefits and costs over a longer forecast period, with future costs and benefits discounted to present values.

Worked example – a new bus line for Ruritania

In order to explain how CBA can be applied to transport evaluation, I’m going to present a simple worked example. In this example, a fictional transport agency in Ruritania, a small central European country, has noticed that transport demands are increasing in the capital city, Streslau. In particular, car congestion is worsening on the arterial road between two outlying neighbourhoods that are about 5 km apart from each other. We’ll call these neighbourhoods A and B.

Like many other European cities, Streslau has a mix of transport networks. Several subway lines were constructed during the 1920s, but work stopped during the Great Depression and never resumed after World War II. The post-war Communist government scrapped the damaged tramlines and replaced them with a mix of trolleybuses and diesel buses that mostly provide radial service to the city centre. In the early 2000s, a light rail line was built to connect the newly expanded airport to the city centre.

Unfortunately, none of these directly serve trips between A and B, and Streslau’s transport planners would like to add a new public transport service to meet demand on this route. Prior to the global financial crisis, the Ruritanian government had been considering borrowing money for a light rail line, but European austerity policies scuppered that plan. Now the transport agency must focus on finding a cheaper option that still provides good service between A and B. They’re investigating different options for a new bus line.

FANSHAWE ST red and green

Perhaps they were inspired by AT’s quick action on Fanshawe St bus lanes?

CBA principle: Identify all costs and benefits to society

Cost-benefit analysis aims to quantify all costs and benefits to society, regardless of who bears them or benefits. All of the effects of a project should be considered. When considering the benefits of a transport project, it would be incorrect to account for the benefits that a project would have to one group of transport users without considering any disbenefits or negative effects on other users.

In our Ruritanian example, let’s say that the transport agency was considering converting two traffic lanes to bus lanes to enable it to run a fast service between A and B. This would have benefits for bus users: they would get a quicker trip. But it may also inconvenience the remaining drivers, who might find it harder to make turns or pass turning cars. It’s also possible that motorists might benefit from a quicker trip, thanks to reduced congestion. CBA should account for all of these effects.

Similarly, when calculating the cost of a project, it is important to add up all costs. In our Ruritanian example, adding a new bus route will require both capital expenditure (capex) and operating expenditure (opex) from the transport agency. Capex includes any spending on infrastructure, such as painted bus lanes, new bus shelters, reconstructed intersections, new vehicles, etc. Opex refers to the annual, ongoing spending required to keep the buses running – drivers’ wages and benefits, fuel, maintenance costs, etc. Any changes in the costs of owning and operating private vehicles are generally counted on the “benefits” side of the equation, rather than the cost side.

Streslau’s transport agency will be able to recover some of these costs from public transport fares. If it’s like New Zealand, fares might cover 50% of the operating costs. If it’s more like most European cities, fares might be lower and only cover 15-20% of operating costs. In any case, it’s not important here, as CBA doesn’t distinguish between public and private costs. It’s worth knowing, though, that the “C” reported in a BCR doesn’t always reflect the net cost to government.

CBA principle: Compare projects against a “do-minimum”

In transport, it is seldom possible to do nothing. Even if nothing else is changing with your transport system, it will still be necessary to spend a significant amount of money on maintaining and operating the system. Even if car congestion is an acceptable level and traffic volumes aren’t growing, you’ll have to spend money fixing potholes and resurfacing streets. Likewise, a PT system that’s standing still will still be paying drivers, buying replacement vehicles, and repainting bus shelters and stations.

Consequently, an important part of transport CBA is the identification of a realistic do-minimum, or the minimum amount that could be done to ensure that the existing system continues to operate. The do-minimum forms the baseline against which you evaluate all other options. In some cases, the do-minimum will include some costs that you are able to avoid by implementing another project.

To return to Streslau: let’s say that the city’s transport agency has found that if nothing else were done, it would need to add new turning lanes to three intersections on the road from A to B to avoid gridlock. The do-minimum should therefore include the cost of three intersection widenings. If adding a new bus route would avoid the need for this spending, it should be subtracted from the net cost of the bus route.

Likewise, transport CBA should consider the net benefits rather than the total benefits. For example, let’s say that the owners of a large pickle factory in neighbourhood B are planning on relocating the plant to a provincial town in two years’ time. This will reduce the number of people commuting by car along the road from A to B, whether or not a new bus route is introduced. This should be considered when evaluating the new bus route, to ensure that the project doesn’t take credit for some reductions in congestion that would have occurred anyway.

CBA principle: Consider costs and benefits over a longer time period

Transport infrastructure projects often have impacts over a long time period. For example, every day tens of thousands of people travel across the Auckland Harbour Bridge, which was opened in 1959 and expanded a decade later. Similarly, Wellingtonians make over ten million trips a year on the regional rail network, which was electrified in the 1930s. Transport projects often have significant up-front costs that must be compared against benefits that accrue over a longer period.

In order to do so, CBA typically forecasts future costs and benefits over a long time horizon and discounts them to present value (PV) for comparison. In order to discount future effects, it is common to use a discount rate that reflects trade-offs between costs/benefits experienced today and those experienced at a future date. For example, using a discount rate of 10% means that if you expect to earn $100 in a year’s time, it will have a PV of $90 (=$100*(1-0.1)^1).

Using a higher discount rate implies that you place a lower value on future outcomes relative to future outcomes, and vice versa. The choice of discount rates can have some interesting implications for decision-making, as previously discussed on this blog.

In New Zealand, BCRs must be calculated using standardised evaluation periods and discount rates to ensure that they can be compared. However, it’s important to know that different agencies have different rules:

  • NZTA, which funds road projects out of a dedicated fund, announced last year that it would use an evaluation period of 40 years and a discount rate of 6% to assess its projects.
  • The Treasury, which funds most other government investments, including rail infrastructure projects, continues to require an evaluation period of 30 years and a discount rate of 8%. In effect, the Treasury’s approach places less weight on long-term outcomes than NZTA’s approach.

Let’s conclude by considering how this may apply to our Ruritanian example. Let’s assume, for the moment, that Streslau’s transport planners have determined that the project will have the following net costs and benefits:

  • $15 million in net capex at the outset, for purchasing new buses, building bus shelters, painting bus lanes, and adding bus priority to some intersections
  • An additional $2 million in ongoing annual opex to run the buses
  • Net annual transport benefits relative to the do-minimum scenario in which road users cannot avoid congestion on the road that start at $6 million in the first year of operation and increase by 5% per annum as bus ridership increases.

These costs and benefits are summarised in the following graph:

Peter Ruritania CBA example

It is difficult to look at this graph and determine whether the long-term benefits – shown in the green line – outweigh the significant up-front costs and ongoing operating costs – shown in the blue and red bars. However, Streslau’s transport planners can use the principles described above to calculate the present value of future costs and benefits. The table below calculates the present value of costs and benefits over a 30-year time horizon, using three alternative discount rates.

As you can see, it looks like Streslau’s transport planners will be able to make the case that the new bus line will benefit the city.

Peter Ruritania CBA summary

Next time: Wait, what do you mean when you talk about transport benefits?

9 comments to Guide to economic evaluation of transport projects: Part 1 – Principles of cost-benefit analysis

  • Darren

    We’re just closing out a research project for the NZTA that looks at international data on the variation of the value if travel time savings with total and proportional time, as well as income (equity approach) and how this fits into other currently quantified benefits. I’ll seek permission from NZTA to publish on here.

  • Simon

    Thanks for the article Peter. I have a concern about the agency’s methods. You say they use BCRs to decide between options and projects but in the same breath use the example of prioritisation within a fixed cycling budget. Why have fixed budgets for different modes? For example if 20 cycling projects with an average BCR of 4 can’t fit in 53 million but we’re spending billions on extending already mature roading infrastructure we have a methodological problem.

    • Luke C

      that is where the Government Policy statement published by MoT comes in. This sets the overall funding bands that NZTA must work within, hence leading to crazy outcomes like the RONS taking the money.

    • There’s a certain irony that, a decade ago, cycling projects were funded from the same bucket as roading projects. It was hard to get funding for cycling projects because there were no specific benefits determined for cycling (e.g. like health benefits) and no simplified procedures for low-cost bike projects, making it hard work to get a decent BCR. Then more research refined things like health benefits, crash saving predictions, etc and simplified procedures for walk/cycle projects made it easy to obtain rather good BCRs for cycling projects. But now cycling doesn’t compete with roading; it is funded from a separate bucket of money that is very limited in value… (not to mention that it also has to overcome the “strategic fit” hurdle as well as a BCR now)

  • MFD

    “CBA principle: Identify all costs and benefits to society”

    The fact that NZTA has introduced additional criteria raises a number of concerns. Why should “strategic fit” be included if the CBA if all costs and benefits are captured? “Strategic” implies some sort of strategy and that, in turn, involves costs and benefits that can be evaluated in the same manner as those for the CBA. It seems to me that, in respect of the RONS, the “strategic fit” element is shorthand for “the government wants to do it in spite of a poor CB ratio”.

    • Exactly. And what then is ‘effectiveness’? Something also not captured in the benefits? Effectively, it appears to be ministerial whim.

      • Fred

        As Bevan says below, the way of calculating a BCR is so hugely flawed and overly weighted to mythical time savings benefits that we need strategic fit & effectiveness to add ability to do anything but stupid road projects. Just because RoNS abuse strategic fit doesn’t mean it’s a bad idea.

  • The predominant benefit in NZTA’s BCR calculations for its roading projects are the “travel time savings”. These occur primarily at peak hour and are the small daily savings estimated for individual drivers, which when taken for all traffic over a projected 40 year period and valued at an hourly rate, becomes a massive benefit. Enough to justify a very expensive new roading project.

    However there are fundamental issues with this approach, including:
    1) NZTA places a value on the travel time savings for a commuting driver who has to start work at peak hour (say 8:30 am). But if we think about what that commuter does with the extra few minutes we’ve saved them (eg: sleeping a little longer, watch a bit more breakfast TV, etc.) and is that really a benefit to society that we want to spend billions to achieve?

    2) The travel times savings that are projected often short-lived because of a thing called “induced traffic”; where an increase in road capacity causes ever more people to drive as they change the time, route or mode to take advantage of the new road capacity.

    3) The travel time savings are only estimated for the local effects of the new roading, not the wider network effects. Eg: the Puhoi to Warkworth motorway’s BCR takes into account estimated local travel time savings, but does not consider the impact of the extra traffic it will create on the Northern motorway into Auckland. I would image in many such cases that the local travel time savings will be negated by the extra congestion caused downstream.

    Another key issue with NZTA BCR calculations is that they ignore things that they find hard to put a financial value on. Hence many of the significant costs of new roading projects are ignored, including: air pollution, carbon emissions, community severance, traffic noise, unsafe streets for children to walk or cycle, health issues exacerbated by car dependency, economic reliance on cheap oil, etc, etc. Perversely, NZTA describes these very real costs as “intangibles” and give them little/no value in the BCR formula, but then take things that are not real (eg: the artificial financial constructs of “travel time savings” and “vehicle operating cost savings”) and gives them significant value to load the Benefit/Cost equation.

    I have been told that the use of travel times savings to sway BCR’s in favour of new roads was developed by the likes of General Motors to support the building of the US Interstate motorways. I’d welcome anyone shedding some more light on that.

  • One of the fascinating parts of transport advocacy is that there is usually a reasonable logical solution to a transport problem but the actual decision making process is intensely political (as it involves large amounts of tax/ratepayer money). The BCR is supposed to highlight the best logical option but becomes subject to debate through the political process.

    There are also some inherit shortcomings to relying on BCR which I am sure you will identify later on. I think one good real-world example of this was the 2005 decision by the Greater Wellington Regional Council to retain the trolley bus service based on a business case with a BCR of 1.1 (see http://www.gw.govt.nz/assets/council-reports/Report_PDFs/2005_156_2_Attach.pdf).

    [Also Bevan: The BCR of this business case DID include benefit estimates for Greenhouse Gases, Local Air Quality, Noise, increased patronage, Reliability and even "Willingness to pay".]

    Subsequent experience showed the 2005 trolley bus retention business case was wrong in several elements included it’s cost estimates, travel time assumptions and reliability leading to the recent decision by GWRC to scrap them in 2017.

    I also hope you cover the political misuse of the BCR to “fix” the best option over others.

    I do commend you, Peter, for providing a guide to the Economic Evaluation of Transport process that may help interested readers identify when it is, and is not, being used correctly.

Leave a Reply