One of the criticisms that we have had of the NZTA in recent years has been that there they don’t appear to go back and check if the what they predicted to happen turned out to be accurate. This is pretty important as they are spending huge amounts of our money on various projects and as we showed last week, it appears that sometimes they aren’t even using real data in their business cases. I reported on the review of the northern Busway back in June so I decided to ask them (via an Official Information Act request) for any other post implementation reviews that they have done in the last 5 years for the Auckland region. Here is what they provided me, first up the letter that came with the response:
Five post implementation reviews of state highway projects in the Auckland region have been carried out over the last five years. These are as follows:
- State Highway 1 Stafford to Esmonde bus priority lane
- Newton Road to Western Springs auxiliary lane
- Puhinui intersection grade separation
- Northern Busway
- Greenlane East amalgamated improvements
Please refer to Attachment G for summaries of these reviews.
While these reviews have identified ways that we can improve our project planning and management practices, they are not a statistically significant sample. For this reason, we consider their results and findings to be indicative rather than conclusive.
The NZTA has two initiatives underway to better measure the outcomes from our investments. These are:
- Approved Organisations (including the NZTA’s Highways and Network Operations group) will be required to formally agree performance measures at the time of funding approval and to report back at agreed dates. In time, this will provide a much more statistically robust picture of the performance of the improvements programme overall. This initiative is in the final stages of development and is to be applied to funding approvals from 1 July 2012. We recognise that it will take time to gather the post-implementation information and we will need to continue with our current programme of post implementation reviews in the meantime.
- We are reviewing our post implementation review methodology. The intention is that the new methodology will focus on the underlying benefits of a project rather than being based on a recalculation of the benefit-cost ratio. The new methodology will investigate and explain variancesin both cost and benefit achievements in greater detail so that lessons can be learned and applied to future decision making.
You will note that some of the reviews are for projects that were started some years ago. Where applicable, one of the pieces of information we collect before starting projects is the crash history of the section of road over the previous five years. To allow us to rate how effective the project is in reducing the number of crashes, we need to compare the earlier crash history data with that collected at least three years after the project is completed.
What I found immediately interesting was how few projects were on the list considering the amount of NZTA projects that have gone on in recent years, e.g. why has no PIR been done on the section of SH18 from Greenhithe to Albany Highway, or the extension of SH20 to Maioro St. I’m sure there have been quite a few other projects finished within or close to that timeframe. I was also surprised by just how long the explaination was but that perhaps made a bit more sense after reading the attached document. It was really just a summary of the outcomes but there are certainly very concerning comments from the auditors. Here are the road based ones.
Newton Road to Western Springs auxiliary lane
Greenlane East amalgamated improvements
Puhinui intersection grade separation
So while I agree this isn’t a statistically significant sample, it certainly isn’t a good look. It hints that there are certain elements within the organisation that are so determined to build roads they will ignore economic advice and funding conditions to get things done. It really makes me wonder what else has been going on within the organisation and will the big motorway projects like Manukau and Hobsonville etc. get similar results. We certainly shouldn’t be thinking of embarking on building another harbour crossing for a cost of $5 billion if we can’t get projects like these correct.
Here are the comments for the other two projects:
State Highway 1 Stafford to Esmonde bus priority lane