This is the second and final post discussing some broad ideas for building a better city. The first post discussed the dynamic nature of cities and argued that a focus on appropriate pricing and incentive mechanisms was important to managing urban ills without stifling beneficial change. This part discusses how we might identify policy areas in need of improvement, and why we should care about efficient policy.
2. Cost benefit analysis is important for identifying opportunities to improve policies
Over the last 170 years, New Zealand cities have been shaped by a wide range of policies, ranging from planning regulations to public investments to government land-holdings to tax and subsidy policies. Many of those policies serve (or served) a useful purpose, and most are good intentioned. But it’s almost inconceivable that all of them are efficient. Cities are dynamic places, and policies put in place in past decades can easily become outmoded and begin to distort prices and limit people’s choices.
Fortunately, as I have tried to suggest in the previous post, we’ve got more policy alternatives than we think. We don’t have to use yesteryear’s solutions to solve next year’s problems. But how do we know what we might have to change?
In my view, cost benefit analysis, or CBA, has an important role to play in identifying which policies are good and which need to change. If you want to learn more about CBA, you can delve into the technical guidelines published by organisations like the Treasury and the NZ Transport Agency. But CBA is not really as complicated as the tedious official guidance makes it sounds. It boils down to a rather simple question:
“This policy has some benefits and some costs. Do we expect the benefits to exceed the costs?”
If the answer is no, perhaps we should do something different instead.
Cost benefit analysis can be applied to a wide range of policy questions. It’s commonly applied to evaluate public investment options – that’s what NZTA uses it for – and less widely used elsewhere. But the principles can readily be extended to a wide range of urban policies. In a paper I wrote for this year’s NZAE conference, I looked at a couple of approaches to evaluating the costs and benefits of planning regulations. I found that analysis of property sales could be used to identify cases where planning rules distorted prices and prevented people from making useful investments – as well as cases where planning rules could provide benefits by managing localised externalities.
Here are a few good examples of how CBA can help in making better decisions.
Back in 2009, the new National-led Government worked with the Green Party to design and implement an insulation and clean heating subsidy programme. In the first four years of the programme, roughly 180,000 homes were insulated and heat pumps were installed in around 60,000 homes. A cost benefit analysis undertaken in 2011 (Grimes et al, 2011) found that the project’s benefits exceeded the costs by a factor of five:
The overall results suggest that the programme as a whole has had sizeable net benefits, with our central estimate of programme benefits being almost five times resource costs attributable to the programme. The central estimate of gross benefits for the programme is $1.28 billion compared with resource costs of $0.33 billion, a net benefit of $0.95 billion.
This finding provided a strong rationale to extend the programme to 2016 and trial a rental warrant of fitness programme to improve home weathertightness and safety performance in selected NZ cities.
A second good example is the Ministry of Transport’s recent analysis of the economic performance of state highway investment. I reviewed their analysis in a series of posts last year (parts 1, 2, 3, 4). Among other things, they found that the economic efficiency of road spending had fallen since 2008, with projects with low benefit-cost ratios being selected over projects with higher BCRs.
This is a valuable finding that should be taken seriously by policymakers and the public. It suggests that there may be opportunities to significantly improve the value that we are getting out of transport investment. That being said, it also suggests that policymakers have chosen to take a more optimistic view about project benefits than indicated by conventional CBA procedures. Sometimes this is warranted – it’s difficult to accurately account for some benefits – and sometimes it’s not.
This reminds me of a third example. I was struck by recent comments by Wellington’s Deputy Mayor about the city’s new publicly funded convention centre and film museum:
Deputy Mayor Justin Lester said the museum would become New Zealand’s most significant man-made attraction and an international draw card.
“It’s a little bit when Disneyland first opened in California, but in a Wellington context… In the 150th year since Wellington became New Zealand’s capital, there are only a handful of moments that rival the significance of this announcement.”
I haven’t read the business case, so I don’t know what assumptions were made to sell the project. But if the financial and economic forecasts require Wellington to become the “Disneyland of the south”, I would be very nervous.
This leads on to a very important consideration when using CBA results: To get the real story, it’s important to dive into the data and calculations that sit behind the headline figures. In some cases, people make claims about projects that are not backed up by their analysis. For example, they may require unlikely things to happen in order for the hypothesised benefits to materialise. In these cases, a properly-done CBA should also provide you with a means of understanding the risks inherent with the policy.
3. In an efficient city, there is time and space left over to lead a good life
But why is efficiency important? At the start of this post, I argued that good urban policies facilitate agglomeration economies in both production and consumption. This, in turn, enables cities to succeed in attracting new businesses and new residents. (The alternative of urban stagnation or economic decline is not really very appealing.)
Or, as Stu argued in a post last year, efficient urban policies provide us with an abundance of good things. Efficient urban planning policies allow us to have abundant, affordably priced housing, without sacrificing public goods. Efficient transport policies enable us to have abundant access. Good management of public assets allows us to combine a productive economy with a good supply of public goods.
Furthermore, generally prioritising efficiency in our urban policies means that we will have more resources left over for all the non-economic things that make life beautiful and enjoyable. For example, allowing people to use land efficiently by building more housing in areas that are accessible to jobs and amenities will also allow them to avoid commuting long distances to sprawlsville. This in turn frees up time to spend with the ultimate in non-economic investments – children and families.
Last December, I saw how things could be a little different. It was the first day that LightPath / Te Ara I Whiti cycleway was open. (Incidentally, I think it should be called PinkPath.) I skipped the mass ride organized by Bike Auckland in favour of a drink elsewhere, but checked out the new cycleway on the bike ride home.
Now, I wasn’t extraordinarily enthusiastic about the project. It seemed to be too far over to the west edge of town and so I wasn’t sure how many people would want to use it. But it has surprised me. I’ve been up and down it eight or ten times since then, and there are always people out on it, even at 10pm. They are having fun cycling – a relatively new concept for Auckland.
PinkPath was designed to be fun to cycle on and fun to see from a distance. It sends a message: “Auckland will give you new choices about how to travel. Rock up on your bike.” It didn’t cost much – less than 1% of Auckland’s annual transport budget and a disused motorway off-ramp. But that money and space can easily be consumed by inefficiency elsewhere in the system.
Which leads me to my conclusion: the good things in life are not necessarily expensive, but they can easily be crowded out by bad urban policies. So get the prices right, do some CBA, and live a better life in a better city.
How transport projects are evaluated has always been of interest to me. I believe that although the standard cost benefit analysis approach that lies behind the NZTA economic evaluation manual has its flaws, the resulting BCR is still an important factor in determining whether a project, or a particular project option, should proceed. I don’t really buy the argument that a project with an unfavorable BCR should be trumped by “strategic” reasons to enable it to proceed. If the strategic reason is any good it will probably be reflected in the BCR, particularly if wider economic benefits (WEBs) are taken into account. I put the Puhoi to Warkworth business case in this category (BCR 0.92) , along with the eye-wateringly expensive Additional Waitemata Harbour Crossing for cars and trucks (BCR 0.4).
Recently I took a look at a number of documents on the East-West Connections project – formerly called the East-West Link, released by the NZTA . At this stage they’ve completed an “Indicative Business Case” (IBC) – essentially, an initial investigation of the options for improving connectivity in the area. They’ve published the IBC alongside a number of technical appendices.
This is a welcome step as this is the first time that the wider public is getting a decent look at the project, including all of the options on the table. NZTA’s decision to release the full documentation, without redacting large sections of the analysis, is really good for transparency.
So let’s take a high-level look at some of the specific trade-offs between costs and benefits of the different options. To jump right to it, NZTA’s conclusion is that Option F should be progressed to a Detailed Business Case. Here’s a picture of Option F, which involves a new highway along the Onehunga foreshore:
For context, Options C, D and E also involved building new roads (part of the way) along the foreshore, while Options A and B entailed upgrades to existing roads, including freight lanes. A summary assessment of this “short list” is available in Appendix O of the business case.
NZTA conducted a cost-benefit analysis (CBA) of these six options. CBA for transport projects typically compares:
- The financial costs to build and operate the project, and
- The monetised economic benefits of the project, including user benefits such as travel time savings, vehicle operating cost savings, and reliability improvements and other benefits such as vehicle emissions reductions (or increases) and effects on economic productivity (“agglomeration”).
While CBA does have some weaknesses, largely due to shortcomings in the modelling tools available to us, it’s a conceptually robust way to assess project options. Especially in the case of road projects, NZTA’s approach should capture the majority of the economic benefits arising from projects, including productivity improvements for freight users.
This is the summary table:
You can see the net present value of the total benefits exceed the total costs for each option – i.e. the total benefit-cost ratio (BCR) for each option is above 1.
However, there are two problems.
The first is that the NZTA has made somewhat arbitrary assumptions about agglomeration benefits, which in theory reflect the productivity gains arising from improving connectivity between businesses. Rather than formally modelling it using the procedure specified in Appendix A10 of NZTA’s Economic Evaluation Manual, they’ve simply assumed that agglomeration impacts will add 25% on top of transport user benefits for each option.
As Stu previously highlighted in the case of the Mill Road highway, which included a similar “fudge factor” for agglomeration benefits, there is no real reason to do this (other than the rather circular argument that the same thing is being done for other projects).
In the case of East-West Connections, there is a stronger argument to be made for agglomeration benefits, as this project will serve a busy commercial/industrial area. However, it’s still necessary to do the analysis to establish their existence and magnitude! I (perhaps cynically) suspect that the 25% figure has simply been used to make the BCRs all appear higher than they otherwise would be. For public relations purposes it is preferable to have a higher BCR than a low one, even though the purpose of the exercise is an option evaluation rather than an assessment of the absolute economic worth of the project.
The second problem with this table is that it is not consistent with NZTA’s own requirements. Section 2.8 of the EEM sets out requirements for calculating and reporting BCRs. That section requires an incremental analysis of costs and benefits:
In other words, if you are choosing between two options, one of which is considerably more expensive than the other, it’s not enough to say that the more costly option has a BCR above 1. It’s actually necessary to show that the added (incremental) benefits of the costly option exceed the added (incremental) costs.
This is an important step in cost-benefit analysis as it shows you whether spending that extra bit of money for a more expensive solution is justified. Failing to do an incremental CBA is basically an invitation for gold-plating and overspending – i.e. find a worthwhile project, and then jack up the costs as high as possible.
So let’s take a look at an incremental BCR analysis of the East-West options. For simplicity I’ve focused only on Options A, B, and F – the two cheapest options, and NZTA’s preferred option. (Options C, D, and E are fairly similar to F in terms of total costs and total benefits – including them wouldn’t get a different result.)
Here’s a picture of Option A, which is an upgrade of SH20 and the existing Nielson St route to SH1:
And here’s Option B, which is pretty similar but also adds a south-facing ramp to SH1:
I’ve ranked the options from least to most expensive:
- Option A has total costs of $200 million and total benefits of $850 million. Consequently, it has an incremental BCR (relative to spending nothing) of 4.3. In other words, Option A seems like a good project.
- Option B has total costs of $500m and total benefits of $1650m. This means that it has incremental costs of $300m (i.e. $500m-$200m) and incremental benefits of $800m (i.e. $1650m-$850m). Its incremental BCR, compared to Option A, is therefore 2.7. This suggests that it’s well worth spending the extra money for Option B.
- Option F has total costs of $800m and total benefits of $1550m. Relative to Option B, its incremental costs are $300m and its incremental benefits are -$100m. Its incremental BCR is therefore -0.3.
In other words, if the NZTA were to follow their own economic evaluation manual it shows that Option F is not great value for money. It costs a lot more while actually delivering fewer economic benefits than Option B. Negative BCRs are generally not a positive sign that a project is a good idea.
This isn’t to say that we shouldn’t build Option F, or that we should build Option B. There may be some significant positive or negative effects that aren’t captured in this analysis and that may tip things in a different direction. For example, existing traffic modelling tools may not capture travel time reliability benefits very well. Similarly, we haven’t taken a look at environmental costs – on the one hand, Option F paves over the remainder of the Onehunga foreshore, which is negative; on the other, it potentially moves trucks away from the town centre and residential areas, which might be a good thing.
Admittedly I’m not a professional economist, but to me the incremental BCR analysis does highlight several questions that need to be answered:
- Given the fact that Option F costs more than Option B while delivering fewer quantified economic benefits, is there evidence that other unquantified benefits, such as travel time reliability, are sufficiently large to justify the added costs?
- Given that the project is primarily intended to improve convenience for freight users, has the government asked freight companies and shippers in the area if they would be willing to invest their own money to pay for Option F?
- Given the results of the Basin Reserve Flyover hearings, in which a Board of Inquiry found that the incremental economic benefits of the Flyover weren’t sufficient to outweigh the added environmental/amenity costs, is there a risk that approval for Option F won’t be forthcoming?
And finally, given the results of an incremental BCR analysis, isn’t there a case to also progress Option B for a more detailed assessment in the next stage of the work, given that it maximises economic benefits at a lower cost?
Back in July, I went down to Wellington for this year’s New Zealand Association of Economists conference. I really enjoy NZAE – people attend because they’re genuinely excited about sharing their ideas and learning from other people. (Stu Donovan and John Polkinghorne were also there.)
I was presenting a paper on using hedonic analysis of property sales to assess and compare the costs and benefits of planning regulations. The empirical side of the paper was an analysis of the impact of dwelling size, lot size, location, and amenities such as the presence of old buildings on property sale prices.
I used these results to consider the rationale for heritage preservation policies. In doing so, I asked three key questions:
- Is there evidence of positive spillovers (“externalities”, in economese) associated with old buildings?
- How large are those spillovers relative to other things that people value, such as living close to the city centre or having more living space?
- Is a blanket heritage control that limits the demolition of building likely to be optimal? In other words, are the positive spillovers from old buildings large enough to justify making it more difficult to develop in some areas?
The first question is very important. As I discussed the other week, people argue that old buildings should be preserved because they are valuable to their inhabitants. To my mind, that is not a good case for government to get involved. If heritage buildings are mainly valuable to their inhabitants, then those people can probably sort things out without the need for any rules.
But if there are positive spillovers from heritage, there may be a case to regulate. That’s because decisions made by a property owner about whether to demolish a heritage property may not take into account the impacts that their decisions may have on other people.
Many – although certainly not all! – old buildings have aesthetically pleasing exteriors. Simply put, they’re nice to look at. (This may simply reflect a selection process – i.e. people built ugly buildings 100 years ago, but they’ve been demolished.) The presence of these buildings can make an area more attractive for passers-by and other residents.
Central Post Office – now known as Britomart (Source)
There are a number of ways that we can measure the public value of aesthetically pleasing old buildings. For example, people may visit areas with more old buildings more often and spend more time walking the streets. (Although I caution that there’s a risk of omitted variable bias here, as areas with older buildings also tend to have older, more walkable street networks.) They may spend more money in shops in these area. Or, importantly, they may be willing to pay higher prices to live around old buildings and enjoy their aesthetic characteristics more frequently.
In my paper, I used residential property sale data to identify the existence of positive spillovers from old buildings. I’ll spare you the details of the number-crunching, but basically, I used four years of recent property sales data to determine whether people are willing to pay higher prices to live near old (pre-1940) buildings.
The results suggest that there are modest positive spillovers from old buildings. On average, every additional pre-1940 building in a neighbourhood was associated with a 0.3% increase in the price paid for neighbouring dwellings. Some individual buildings are likely to have stronger spillovers, of course – not all old buildings are created equal! And there are likely to be some spillovers that aren’t captured in residential property prices.
But as heritage policy is often a very local event – people tend to advocate for the preservation of buildings in their suburb or neighbourhood – it’s likely that this measure captures many of the spillovers that matter. Which leads us on to the third question: When is a blanket heritage control likely to be optimal?
The downside of a blanket control is that it will make it more difficult (or even impossible) for people to redevelop sites or make additions to existing homes. My analysis of recent property sales showed that the quantity of floorspace has a strong effect on property values. I estimated that a 10% increase in the size of a dwelling was associated with a 4.8% increase in its sale price, holding all other factors constant.
Based on this result, I asked: How much additional floorspace would be required to fully offset the loss of aesthetic spillovers from neighbouring pre-1940 buildings? In other words, what’s the point at which people might be indifferent between preserving heritage and getting opportunities to intensify their properties?
The results are mapped below. Darker greens and blues indicate areas with larger positive spillovers from old buildings. Yellow colours indicate areas where there are few if any spillovers. Of course, there are likely to be a number of subtleties that I wasn’t able to pick up in the data, such as the quality of heritage properties in different areas.
Change in floorspace required to offset loss of heritage spillovers (Source: Nunns, 2015)
One interesting thing about this map is that it suggests that the value of heritage preservation may be relatively low compared to the value of opportunities for intensification almost everywhere in the city. Even in the most heritage-y parts of Devonport and Ponsonby, it would only take a 30-40% increase in floorspace to fully compensate for the loss of localised spillovers from all the pre-1940 buildings in the neighbourhood. That isn’t an unreasonable possibility given that these areas have standalone houses sitting on crazily expensive land. (And the fact that many of these buildings would be preserved by their owners anyway.)
So what should we make of this?
First, an important caveat: these results are not definitive. They’re based on a piece of quantitative analysis that captures overall trends but omits qualitative aspects of the aesthetics of old buildings. In some areas, it may under-estimate the contribution of individual buildings that are especially attractive. In others, it will over-estimate the magnitude of spillovers, because the old buildings in the area are simply not that flash.
But even taking that caveat into mind, there may be room to optimise heritage preservation by focusing blanket heritage controls in areas where evidence of positive spillovers is strongest. So it’s encouraging to see that Auckland Council is refining its position on heritage controls in the Unitary Plan. (And dispiriting to see the NZ Herald’s alarmist one-sided take on the issue. Pro tip to the editors: articles like this are why I do not buy your newspaper. I spend money on other print media, so you’re missing out.)
It’s also worth remembering that blanket controls aren’t the only way to preserve heritage. Heritage schedules can be used to target protections to individual buildings with notable aesthetic or historic value. And councils can directly fund the preservation of notable buildings by buying up and renovating them. In some cases, these may be a more efficient way of ensuring that we maintain the good bits of the city at a reasonable cost.
What do you think an optimal heritage preservation policy would look like?
Last week I started taking a look at publicly-owned golf courses. I argued that they are different from public parks in several important respects. While public parks are freely available to all Aucklanders, golf courses are only open to paying golfers. As a result, we need to treat golf courses differently – not as a tax-funded public good, but as a business that must pay its way.
This week, I will take a look at some of the “opportunity costs” associated with using land for golf courses rather than alternative uses, such as public parks or housing. My central question is this: Do the benefits of using land for golf outweigh the benefits of developing the land for housing? Or is it the other way around?
Let’s start with a look at some broad trends. First, here’s what’s happened to the price of housing over the last two decades: it’s gone up significantly. This is a strong indication that demand for housing (and more intensive urban land uses) is increasing. While predicting the future is difficult, most people expect housing demand (and prices) to continue rising in the future.
Second, here’s a short-term forecast of revenues for Auckland’s 39 golf clubs from a 2013 report on future prospects for golf facilities. According to the report, golf club membership has been declining by around 1.6% a year. Unless something major changes, this trend will continue and put many golf courses under financial pressure:
Effectively, rising demand for housing and falling demand for golf mean that using large amounts of publicly-owned land for golf courses will become increasingly inefficient. Here’s one way of thinking about the benefits of the status quo (to golfers) versus the benefits of redevelopment (to people who otherwise wouldn’t be able to buy or rent homes in the area).
I’m going use Chamberlain Park as a case study, but the same approach could be generalised to other publicly-owned golf courses. Here’s a picture of the course, which occupies 32 hectares in Mount Albert:
According to the local board, Chamberlain Park currently hosts over 50,000 rounds of golf a year. Let’s be generous and call it 55,000 rounds. According to the club’s website, green fees are $30 on weekends. This means that the total annual value of golf rounds played at Chamberlain Park is $1.65 million. In present value terms (i.e. extending this forward 40 years into the future and applying a 6% discount rate), this equates to $26.2 million.
Now, let’s consider alternative uses for the land. Let’s assume that we would develop it for housing and commercial uses, with a substantial amount of land reserved for public parks. We don’t have to look too far to find a good example of this kind of development. It’s exactly what’s happening at Wynyard Quarter, which will have a mix of medium-density residential and commercial buildings, a substantial waterfront park, and a linear park running the length of Daldy St:
The important thing is that if development is master-planned appropriately, it can lead to more housing and better public spaces. That’s certainly happening at Wynyard, but it could also happen in Chamberlain Park if redevelopment enabled better connections between new public parks, the Northwestern cycleway, Western Springs, and Mount Albert in general.
So let’s start by assuming that we would reserve one third of Chamberlain Park – 10 hectares – for new parks and playing fields. That’s the same size as Grey Lynn Park, which attracts 100,000 people to the Grey Lynn Festival on a single Sunday – i.e. twice as many people as Chamberlain Park sees in a year.
The remainder – around 22 hectares – could be developed as new neighbourhoods, possibly along the mid-rise, mixed-use lines of Wynyard Quarter. I’m going to assume, further, that around 25% of that space would be devoted to streets, which is pretty typical of new developments. That means that after providing some sizeable public parks and laying out all the streets, we’d have around 16 hectares that could be built on.
Now, current land values in the Mount Albert area are in the range of $1500 per square metre, or possibly higher. That’s a reasonable estimate of the value that people place on the opportunity to live in the area. That means that the total benefit of redeveloping Chamberlain Park for housing is around $240 million (i.e. $1500/m2*16 ha*10,000m2/ha). These benefits would accrue primarily to the people who end up living in the area, but it could also keep housing prices from rising as rapidly and thus have wider benefits.
In short, the benefits of redeveloping Chamberlain Park – even after leaving aside a substantial area for public parks – are nine times larger than the benefits of the status quo for golfers (i.e. $240m/$26.2m = 9.2). Because demand for housing is rising at the same time that demand for golfing is falling, this figure is likely to increase, not fall.
This doesn’t necessarily mean that we have to redevelop Auckland’s publicly-owned golf courses, but it does raise some questions. First, given the fact that redevelopment is likely to be vastly more beneficial than the status quo, why isn’t it being put forward as an option in the Chamberlain Park consultation?
Second, why isn’t the opportunity cost of using lots of land for golf being recognised the prices charged by golf courses? As we’ve seen, people would place a quite high value on being able to live or work on the land occupied by some golf courses. In principle, that should be factored in to green fees, but in practice it isn’t. In the next instalment, I’ll explore this question further – it turns out that publicly-owned golf courses enjoy a large subsidy from ratepayers.
What do you think about the benefits of alternative options for using golf course land?
One of the best things done recently by governments on both sides of the Tasman has been the establishment of Productivity Commissions tasked with investigating the economic efficiency (or lack thereof) of various policy areas. The NZ Productivity Commission, which only started up in 2011, has been diligently building an evidence base on issues as diverse as international freight, service sector productivity, and land use policies.
Transportblog reviewed their recent inquiry on using land for housing, which had some excellent insights and recommendations. Not all of the recommendations made by the Commission have been picked up (yet), but they form a useful basis for future policymaking.
We’ve been paying attention to the NZ Productivity Commission, but it’s also worth keeping an eye on the Australian version, which is grappling with many of the same issues. They recently published a nice summary of their 2014 inquiry into public infrastructure, which highlighted some key steps for improving the efficiency of public investment.
The Commission highlighted the scale of capital investment. Australia’s expecting to spend megabucks on new capital investment, most of which will go into the built environment – i.e. houses, roads, railways, water infrastructure, etc:
In 2013-14, there was an estimated $5.1 trillion or more of installed capital that was available for use in the Australian economy — over three times the value of production in that year (figure 3.1, top panel). Over the next 50 year period, the Commission has estimated that new capital investment will be more than five times the cumulative investment made over the last half century to around $38 trillion in today’s prices (PC 2013).
The largest component of today’s installed capital is in the form of non-dwelling constructions — which consists of non-residential buildings (i.e. buildings other than dwellings, including fixtures, facilities and equipment integral to the structure) and other structures (including streets, sewers, railways and runways) — which amount to over $2 trillion (or 43 per cent of the total). The next single most important component relates to dwellings, amounting to around 35 per cent of the total installed capital (figure 3.1, bottom panel).
Given this, it’s important to get new public investments right:
Additions to the stock of capital will usually increase output and add to labour productivity. However, for productivity to improve, the growth in output must exceed the growth in inputs. Poorly selected projects can detract from productivity as the resources they use would have delivered a higher output elsewhere in the economy.
The Commission had some stern words about the quality of decision-making about public investments. Their analysis suggests that “there is considerable scope to improve the quality and efficiency of government investment in public infrastructure investment in Australia”.
This could probably be translated as: “You guys are doing a bunch of stupid stuff. For the love of god, please stop it!”
So how could infrastructure investment improve? The Commission discusses public-private partnerships (PPPs) as a potentially useful option, but observes that they are not a magic panacea for bad project selection. In some cases, the PPP process can drive governments towards costly, oversized solutions:
Most relevant to enhancing the efficiency of the provision of public infrastructure is improving project selection processes. Australia’s cities and towns generally function adequately and assets undergo usual maintenance, although problems have emerged in some major cities. Nevertheless, the Commission found numerous examples of poor value for money arising from inadequate project selection and prioritisation. In particular, there was a bias toward large investments despite the returns to public investment often being higher for smaller, more incremental investments. In part, this was because the private sector is more interested in financing large investments (due to the costs involved), and governments have increasingly seen public private partnerships (PPPs) as a way of harnessing not just finance, but expertise in project delivery and operation.
Furthermore, the fact that PPPs typically involve complex negotiations and ongoing relationships between governments and private investors can make it difficult to properly assess costs, benefits, and risks. In essence, this creates a situation in which private sector involvement does not result in a better outcome, due to muddled incentives.
The Commission concluded by encouraging Australian public infrastructure providers to conduct rigorous, consistent cost-benefit analysis for all major projects and – equally important – to publish the results prior to committing to a project:
A key recommendation of the report was that governments should undertake a comprehensive and rigorous social cost—benefit analysis to all public infrastructure investment projects above $50 million. Such analyses should be publicly released during the commitment phase and be made available for due diligence. In general, cost-benefit analyses should be done prior to any in-principle commitment to a project or as soon as practicable thereafter.
Doing this would have avoided the farcical situation facing Melbourne’s East-West Link, an A$8.6 billion project to build a massive tunnel to link up several motorways:
As far as I can tell, the project was announced and confirmed before a business case was completed or published. As a result, nobody really knew whether it was a good idea. The project was subsequently cancelled, albeit at a significant cost for cancelling the contract.
Do the figures stack up?
The truth is that we don’t know. Those against all new major road projects may not care about the figures one way or the other, but those who follow these things closely say the project is unprecedented for its lack of transparency. It’s been a nagging political problem for the government, and a key reason the East West Link is so contentious.
“Normally we would see more detail, and historically it’s been much clearer on what basis we are proceeding with projects like this,” [infrastructure consultant William] McDougall says.
“This is new for Australia,” says [transport policy lecturer John] Stone. “The fact that through all these court cases and all this political focus the government has never released its business plan – it released a back of envelope estimate – means probably there’s nothing to back it up. If they had a better number they would have put it out there.“
Fortunately, New Zealand seems to do a better job when it comes to consistent cost-benefit analysis of transport projects (although we don’t always take the findings seriously). However, there is always room for improvement. The Commission highlights three key factors that can undermine the effectiveness of cost-benefit analysis:
- Optimism bias. There is a systematic tendency for project appraisers conducting cost-benefit analysis to be overly optimistic — the bias is toward overstating benefits, and understating timings and costs, both with respect to initial capital commitment and operation costs. Over estimates of traffic forecasts on toll roads and tunnels are a particular problem…
- Treatment of risk and uncertainty. Costs and benefits are expected values based on the probability of different outcomes. Cost-benefit ratios may be sensitive to certain assumptions which have to be made without sufficient evidential support. For example, inappropriate assumptions about allowance for project risk in the discount rate (that is, the risk premium) may alter the ranking of projects and lead to suboptimal project selection…
- Treatment of ‘wider economic benefits’. Infrastructure projects create direct benefits for users of the resulting service provided by public infrastructure. Where cost-benefit analysis is done, such benefits are routinely estimated and included. However, projects can also create wider economic benefits and costs. For example, investment in transportation infrastructure brings consumers closer to more businesses, potentially facilitating greater competition and leading to a more innovative and a dynamic economy. However, such wider economic benefits are hard to quantify and their inclusion in a cost-benefit analysis has the potential to show one project to be superior to another purely because of differences in the way such benefits are defined and estimated. Cautious and consistent treatment across options of wider economic benefits is warranted.
Transportblog’s highlighted a few of these issues in the past. I’ve discussed optimism bias on the benefits/usage side here and optimism bias on costs here and here. Stu’s covered off treatment of wider economic benefits here. And it’s probably high time we took another look at discount rates / risk premia…
Disclaimer: in professional life I have done some work on ports, including co-authoring the 2012 PwC report on future scenarios for Upper North Island ports. This post doesn’t reflect the views of my present or past employers or clients. It’s just a quick thought experiment, based on some data and a few assumptions.
The Ports of Auckland (POA) are back in the news due to their new reclamation plans. As usual, this has attracted both critics and proponents. POA’s plans have been criticised for their negative environmental impacts on the Waitemata Harbour, the loss of views of the Hauraki Gulf, and the fact that they will limit our ability to re-use port land for other purposes. On the other hand, they’ve been defended due to the economic role that POA plays in Auckland – it’s New Zealand’s largest port of import and also a significant port of export.
As this suggests, there are both pros and cons to having a port located right on Auckland’s front door. How should we weigh them up?
Here’s one way of thinking about the question of whether we should prefer having POA in Auckland, or whether we would rather close it down and move our freight elsewhere:
- The costs of moving the port would primarily relate to the added freight cost for Auckland’s imports and exports
- The main benefit would be that we could repurpose POA’s land for alternative uses, such as housing, offices and retail, or public spaces.
How large are these costs and benefits?
The costs of relocating the port
One realistic way to look at the cost of port relocation is to ask: How much more would we have to spend to get the same outcomes?
If we closed down POA and shipped Auckland’s imports and exports through the Port of Tauranga (POT) instead, we would have to pay more to move those goods by land between the two cities. This would represent a net cost to New Zealand’s economy.
We can get a rough sense of these added costs by looking at current land transport costs and port volumes. According to an NZIER report published last month, in the year ended June 2014 POA handled:
- around 968,000 twenty-foot-equivalent containers per year, 203,000 of which were trans-shipped to other ports in NZ;
- around 207,000 cars; and
- some other random stuff, like bulk cement.
Now, based on figures published in the 2012 PwC report (see Table 4 on page 76), the cheapest way to move goods between Tauranga and Auckland is by rail. It costs approximately $600 to move a single container by rail between the two cities. (Or around $750 to move a container by road.) While KiwiRail doesn’t currently ship cars by rail, rail operators in other countries do. Let’s assume, therefore, that it costs around the same amount ($600) to ship a single car.
Based on these land transport costs, we’re looking at an added annual cost of around $580 million. Yikes. A quite large sum. In reality, this is probably a bit on the high side, given that some of these goods will not originate from or be destined for Auckland.
In addition, we would forego the $66 million in annual dividends that POA pays to Auckland Council. So the total annual cost of relocating the port would be around $650 million.
The benefits of port relocation
Although the costs of moving POA entirely out of Auckland are high, we might be willing to bear them if the profits from land development were sufficiently high. So: How much would the land have to be worth to justify relocating POA?
Well, we know that, in order for it to be worth doing, repurposing the port land for residential and commercial uses, or public space, would have to yield at least $650 million per annum. That figure represents the minimum annual return that we would require from POA’s land.
Let’s assume, for a moment, that Auckland Council could get an average rate of return of 8% on its port land if it were put to other uses. This suggests that in order to obtain an annual return of $650 million, POA’s land would have to be worth a total of around $8.1 billion. (Calculated as follows: $650 million in annual profits / 8% rate of return = $8.1 billion.)
According to Wikipedia, POA has a total of 55 hectares of wharves and storage areas. If that were worth $8.1 billion in total, it would mean that the land would be worth around $15,000 per square metre. That’s roughly what it would take for moving the port to be a net benefit for the economy – city centre land values above $15,000 per square metre.
Now, this is in the range of current land values in the city centre – albeit on the high side. So redeveloping the port could, in principle, provide net benefits for Auckland. The case might get stronger if land values continue increasing and if the downtown revival continues at pace.
However, I don’t think this quick, back-of-the-envelope analysis proves much. For one thing, the benefits of port relocation are probably overstated due to the fact that it would be quite difficult to redevelop 55 hectares of downtown land quickly. It might take decades to realise the value of port land for alternative uses.
For another, it would be quite difficult to compensate the “losers” from the process – the firms and workers who would be worse off as a result of higher transport costs to their location in Auckland.
So, what should we do with the port?
As this analysis has (hopefully) shown, there are both costs and benefits to moving POA. And, for that matter, to leaving it in place or expanding it.
Moreover, the costs of moving POA are not infinite, which means that the benefits of doing so may at some point be large enough to justify the move. But they are very large, which means that we would have to be confident that we could actually redevelop port land in a reasonable timeframe.
It’s also important to recognise that there are other risks in moving the port, as well as uncertainty about some of the costs that I’ve cited. In my view, there are three main limitations to this analysis:
- First, I’ve assumed that there are no technical constraints to doubling freight volumes at POT. This is probably not realistic – expanding that port would be costly both financially and environmentally.
- Second, I’ve assumed that shipping lots more goods by rail between Tauranga and Auckland won’t drive up the price of rail freight. In reality, KiwiRail (or the government) would have to pay for quite a few track upgrades and purchases of rolling stock, which may drive up the costs of rail freight.
- Third, I’ve assumed that it would actually be feasible to redevelop POA’s land, and that redevelopment of port land would create added value rather than simply diverting growth from elsewhere in Auckland. This is not unreasonable, but it won’t be a rapid process. As the Wynyard Quarter shows, it can take over a decade to active and develop a substantial chunk of new land.
Lastly, there are likely to be problems with the timing of funds. In principle, land development profits could be used to pay for infrastructure upgrades. In practice, it won’t work so neatly, as infrastructure requirements will be front-loaded while development profits trickle in over a period of years or decades.
In other words, actually moving the port is likely to be a costly and risky enterprise. It will be difficult to overcome the risks and up-front costs associated with doing so – meaning that we should expect the port to stay in downtown Auckland.
Port location: What do you think?
This is the second post in a series on the Ministry of Transport’s working paper on New Zealand’s capital spending on roads, which was prepared as an input to the 2015/16 Government Policy Statement (GPS) on Land Transport Funding. It was released to Matt under the Official Information Act just before Christmas. Previous posts:
In the previous post, I took a look at the MoT paper’s findings on the economic efficiency of state highway spending. MoT showed that since 2008 spending on the Roads of National Significance (RoNS) has gone up, while benefit-cost ratios have gone down. As a result, we have almost doubled our spending on state highways without achieving any more economic or social benefits from that spending.
This week, I’ll take a look at a different question: Is it possible to spend our road budget more efficiently? If we chose to build other roads instead, would we get more benefits from them?
The MoT paper examines this issue quite comprehensively, and comes up with an unambiguous “yes”. But before I get into it, it’s worth reviewing the system that the Government is currently using to assess transport investments. Projects are ranked on three criteria:
- Strategic fit [i.e. is this project trying to do something that the Government cares about?]
- Effectiveness [i.e. will this project actually do what it’s intended to do?]
- Benefit and cost appraisal [i.e. will this project deliver more benefits than costs?]
In short, the BCR is only part of the picture. In practice, it’s less important than strategic fit. However, it’s still an important criteria for determining whether we are getting good value out of our transport investments, especially as many of the strategic outcomes that the Government wants are accounted for in a transport cost-benefit analysis.
With that in mind, Section 5.4 of the MoT paper compares BCRs for local road and state highway projects which have committed funding versus those that will probably receive funding or which will remain unfunded.
This analysis, summarised in the chart below, shows that BCRs for state highway projects tend to be lower than BCRs for local road projects whether or not they have committed funding or not. This might be an indication that too much money has been allocated to new state highways – effectively, there are worthy local roads that are going unfunded.
Another worrisome finding is that BCRs for “committed and approved” state highway projects are considerably lower than projects that are merely “probable” or which have not been given funding. This suggests that even within the state highway budget, funding isn’t going to the projects that offer the best returns.
However, the MoT paper notes that these figures include “significant spending on large strategic projects” – the Auckland Manukau Eastern Transport Initiative (AMETI) in local roads and the RoNS in state highways. Is it simply the case that a few big funding calls are skewing the results?
Here’s what the chart looks like with those projects removed. As you can see, “committed and approved” state highway projects other than the RoNS also offer a lower return than the “probable and reserve” projects that may or may not get funding. What the hell is going on here?
Elsewhere in the paper, MoT sums up the situation as follows, with a nod to the idea that traffic forecasts are over-predicting growth:
It also compares these figures with BCRs for other transport spending, including NZTA-funded PT infrastructure and services and walking and cycling projects, and concludes that:
In other words, the focus on big state highway projects means that the Government is passing up higher-value spending that serves other modes. Unfortunately, the paper doesn’t offer a lot of additional analysis. But it would be interesting to know how much analysis NZTA or MoT has done on the bus infrastructure projects that are needed to get good transport outcomes in Auckland, such as the Northern Busway extension, the Northwest Busway, extensions of the AMETI busway, and bus interchanges to support Auckland’s New Network.
With all that in mind, how would we be spending money if cost-benefit analysis was the key criteria?
Section 6.2 of the MoT report contains a number of colourful charts to illustrate how we could be doing things differently. Here’s the bit that stuck out for me. It classifies new state highway projects, excluding RoNS, according to their BCR (vertical axis), funding priority (horizontal axis), and total cost (size of bubble).
If BCRs were the key criteria for project funding, the black-coloured bubbles would be de-funded and the red-coloured bubbles funded in their place:
As you can see, if the Government were focused on getting the highest benefits out of its transport budget, it would have to de-fund most large state highway projects that are currently underway. Yikes.
It’s not clear what conclusions MoT’s drawing from this analysis, as the final paragraphs are entirely blacked out. However, I’d be surprised if they weren’t a bit skeptical of the way that public money is being spent…
Next week: MoT’s analysis of roads spending by region. Preview: Canterbury’s getting a raw deal.
Next Tuesday, the Government Economics Network and Auckland Council are hosting a seminar entitled “Economic evaluation in Auckland – new ideas and challenges“. It’s on a topic that I personally find very interesting – some readers may also be keen:
Estimating the economic impact of transport interventions using the Gross Value Added approach.
Current transport appraisal methods, with their focus on the economic welfare benefits and costs of transport investment, are well grounded in theory and widely used. However, these methods do not provide estimates of extra Gross Domestic Product and extra jobs, nor the spatial distribution of any economic gains and losses. Gross Value Added (GVA) models, have recently applied in the United Kingdom and the United States to account for some of these effects.
In this presentation, Anthony Byett, outlines the results of NZTA-commissioned research on the development of a GVA model for New Zealand. The research uses 2001 and 2006 census data from the 72 sub-national territories, and applies the model to a proposed additional Waitematā Harbour crossing. Promisingly, the model reveals productivity gains from local agglomeration and points to some productivity gains from wider connectivity as well. However, the building and use of the model also reveals shortcomings with the measurement of effective densities and the ability to reach inferences about regional distribution. Nonetheless, the model did prove insightful in highlighting where the benefits of another harbour crossing will likely lie.
Economic evaluation and Cost Benefit Analysis: Implications for practitioners, government agencies and Auckland Council.
Chris Parker, Auckland Council’s recently appointed Chief Economist, will reflect on recent developments in economic evaluation, including the NZTA research using the Gross Value added approach, and discuss some of the implications for practitioners, government agencies and Auckland Council.
The two speakers promise to be pretty interesting. Anthony Byett has led some pretty interesting work into the productivity of road networks. Chris Parker has just been appointed as the Council’s new Chief Economist following on quite a bit of work in transport appraisal at consultancy NZIER.
The seminar is being held at the Council Chambers in the Auckland Town Hall from 1pm to 2:30pm on Tuesday 17 February. You can RSVP at the GEN website.
This is the second post in a series on the Ministry of Transport’s working paper on New Zealand’s capital spending on roads, which was prepared as an input to the 2015/16 Government Policy Statement (GPS) on Land Transport Funding. It was released to Matt under the Official Information Act just before Christmas. Previous posts:
As I said last week, MoT’s paper suggests that there are big issues with the land transport budget. Current road spending does not seem to represent good value for money. To their credit, MoT appear to be acknowledging this. However, it doesn’t seem to have percolated up into the investment decisions being made by the Government.
This week, I want to look at what NZTA’s money (the National Land Transport Fund, or NLTF) is being spent on, and how economically efficient that expenditure has been.
Section 4 of the MoT report contains a lot of useful data on past and future spending on roads. Here’s what’s happened to the roads budget over the last 15 years, and what’s expected to happen over the next decade:
Basically, about a decade ago we started spending a lot more on new or improved roads. A lion’s share of new spending went to state highways, in spite of the fact that local roads carry more traffic. As we have previously discussed at length, this spend-up coincided with a flattening of growth in vehicle kilometres travelled. (It also coincided with an acceleration in price inflation for civil construction.)
In other words, we’ve spent a decade spending increasing amounts of money on roads for which demand is not increasing. And the last three Government Policy Statements plan for state highway spending to increase further.
In order to pay for state highway spending, it’s been necessary to divert money from other activities – local roads, maintenance, PT, and walking and cycling have all taken a hit. The Government has also raised petrol taxes several times. The MoT report offers some analysis of how spending priorities changed between the 2008 GPS and the 2012 GPS.
The following chart compares projected spending ranges for new and improved state highways (the darker uppermost bands) and new and improved local roads (the thinner, lower bands). It shows that funding for state highways – the Roads of National Significance – was raised by around half a billion dollars a year, while local road funding was cut back.
One would hope that the Government’s decision to allocate vast amounts of funds to state highway projects was based on a sound economic rationale. Unfortunately, there is no hard evidence of this in the MoT paper. Section 5 of the MoT paper analyses benefit-cost ratios (BCRs) for road spending. It notes some caveats with the data – BCRs for some projects had to be inferred from “efficiency scores” – but there is enough data to paint a picture.
Here is MoT’s picture. It is not a pretty one:
Essentially, MoT finds that average benefit cost ratios for state highway projects declined significantly in 2008/09 and have stayed low ever since. An eyeballing of the graph suggests that BCRs prior to 2008 averaged a bit over 3.5 – meaning that state highway projects were expected to return $3.5 in social benefits for every dollar invested. Since 2008, they have averaged a bit over 2 – meaning that state highway projects now only return $2 in social benefits for every dollar invested.
MoT’s analysis of this graph is entirely blacked out in the released document. Nonetheless, the implications are simple: we have almost doubled our spending on state highways without achieving any more benefits from that spending. BCRs aren’t everything, but it’s really, really hard to understand why the Government would want to spend money so ineffectively.
The answer is that they feel that the Roads of National Significance offer a better “strategic fit” with their overall objectives for the land transport budget. I’m not necessarily opposed to this evaluation approach. In my experience, cost-benefit analysis invariably has some blind spots. Using qualitative “strategic fit” criteria can allow policymakers to take account of broader goals that aren’t well covered in NZTA’s Economic Evaluation Manual.
However, I don’t think that strategic fit should override all other analysis. If you think that a project is important for supporting a productive economy, that’s fair enough. But if an evaluation of the project’s impact on freight costs and agglomeration effects in urban areas results in a low BCR, you should question your prior assumptions about its economic benefits. It’s foolish to think that four-lane divided highways are magical devices for creating economic growth. Economics simply doesn’t work that way.
Next week: Do we have better options for spending the transport budget?
In several recent posts I’ve taken a look at people’s revealed preferences for roads (nobody’s willing to pay directly for them) and public transport, walking, and cycling (people are queuing up to get on the train). In those posts, I’ve argued that observing how people vote with their feet (or their wallets) can teach us a lot about demand for different travel modes.
Rail is now growing too fast to be un-fit for survival.
But as any economist knows, markets have two sides to them: demand and supply. As transport infrastructure has a lot of “public good” characteristics, it tends to be provided by government agencies such as Auckland Transport and the New Zealand Transport Agency. (These agencies wouldn’t say no if a private company turned up and offered to build a new motorway at no cost to them… but that’s not going to happen any time soon due to the fact that most recent private toll roads have failed financially.)
As a result, we have to consider how transport agencies make decisions about what to supply to the market. I’ve written a few posts on the basics of cost-benefit analysis, which is one of the tools that they use to decide which projects to build.
But is cost-benefit analysis robust, or are the results systematically biased in a certain direction? Thinking about this question led me to re-read one of my favourite papers on infrastructure costings (don’t laugh!): Bent Flyvbjerg’s “Survival of the Un-fittest: Why the Worst Infrastructure Gets Built – and What We Can do About It” (fulltext pdf). Flyvbjerg takes an empirical look at hundreds of major infrastructure projects around the world, finding that cost overruns are all-pervasive:
- 9 out of 10 projects have cost overrun.
- Overrun is found across the 20 nations and 5 continents covered by the study.
- Overrun is constant for the 70-year period covered by the study, cost estimates have not improved over time.
In addition, benefits are systematically overestimated in ex-ante evaluations. The result is that a number of bad projects get built on the back of over-optimistic business cases. Flyvbjerg attributes this to “cognitive and political biases such as optimism bias and strategic misrepresentation”. (This is a polite way of saying “lying about the project to ensure that it gets built.”)
So how do New Zealand’s transport agencies stack up against Flyvbjerg’s analysis? Fortunately, we’ve got some empirical data to investigate this question with. Between 2009 and 2012, NZTA conducted and published a number of post-implementation reviews of (mainly) road projects that it funded in part or fully. Matt did an excellent job summarising the data in a post last year.
While the projects aren’t necessarily representative of all road projects, they do run the gamut from small pavement upgrades to multimillion state highway expansions. NZTA provided data comparing ex-ante and ex-post evaluations of costs and benefits for 69 projects in total. I subjected the data to some basic statistical analysis, finding that:
- The average project had a cost overrun of 34% – a difference that was found to be highly statistically significant, meaning that there is a less than 1% probability that the observed difference happened by chance.
- The average project had actual benefits that were 28% lower than expected – although as this difference was not statistically significant we can’t determine whether it simply reflects random chance.
In other words, NZTA and regional transport agencies seem to have had some issues accurately costing road projects. And the errors they are making are not random – they have systematically underestimated costs. This can be seen really clearly if we graph the data in histogram format.
Here’s the data on construction cost overruns, in percentage terms. The size of the bars represents the number of projects. Bars to the right of the black line indicate projects where costs were higher than expected. As you can see, costs were higher than expected for the vast majority of projects – sometimes to a quite significant degree (i.e. over 100% more expensive than planned).
And here’s a similar chart for benefit overruns/underruns. This shows that although estimates of benefits have in some cases been wrong by a quite large amount, most of the errors are clustered closer to the zero line. This shows that while NZTA or transport agencies often miss the mark on their estimates of benefits, the errors are sometimes positive and sometimes negative. In other words, optimism bias seems to be less pervasive when estimating benefits than when estimating costs.
This data has (or should have) important implications for the way we plan and fund transport projects. It suggests that it’s necessary to be much more conservative when estimating the costs and benefits of road projects. This is especially important in light of the fact that NZTA’s funding is being devoted in large part to major motorway projects – the kind of “megaprojects” that Flybjerg identifies as posing the greatest risks for good project evaluation.
Unfortunately, NZTA stopped publishing post-implementation reviews in 2012, so it’s impossible to say whether agencies have used this data to refine their cost estimates. I hope they have, but there are indications that optimism bias is still running rampant. Take, for example, NZTA’s long-term forecasts of road traffic and public transport patronage, which blithely disregard the market realities. Or, more concretely, there’s the strange case of the Additional Waitemata Harbour Crossing traffic forecasts, which Matt picked up on a few years ago.
A 2010 business case for the AWHC, which would be New Zealand’s most expensive infrastructure project of all time, found that the project’s benefit-cost ratio was a mere 0.4 to 0.6. (Indicating that it costs about twice as much as it returns in benefits.) But, as it turns out, this figure was based on traffic modelling that overestimated actual traffic across the bridge in 2008 by almost 10% – in spite of the fact that the actual data was available at that point. That’s some serious optimism bias right there…
Auckland Harbour Bridge Traffic volumes (actual and forecast)
Finally, it’s also worth noting that Flyvbjerg finds that cost overruns (and benefit underruns) tend to be a more serious issue for rail projects than for road projects, especially in the United States. Unfortunately, we simply haven’t completed enough rail projects to robustly evaluate whether the same holds true in New Zealand. However, there are some signs that recent public transport infrastructure projects have outperformed their business cases – as seen in NZTA’s post-implementation review of the Northern Busway and booming ridership at Britomart.