Auckland Transport accused of ‘marking its own homework’ in speed reduction evaluations

Four successive independent reviews of Auckland Transport’s contentious speed-reduction programme were produced by an engineer who previously worked at AT developing the programme — a situation one critic called “marking your own homework”.

Transport engineer Lewis Martin worked on Auckland Transport’s (AT) Road Safety Team for three years, starting in late 2017. His first 12 months were spent as a graduate road safety engineer.

Martin worked extensively on the Safe Speeds Programme, a sweeping review of and reduction of speed limits across the city, aimed at reducing the number of serious crashes and consequent death and injury on the roads.

In October 2020, Martin left AT and joined the transport consultancy Abley.

Seven months later, Abley was contracted to carry out work for AT. Martin was assigned to prepare “an independent peer review” that considered the agency’s proposed plan for the second phase of its speed limit reductions under the programme.

Among its key components was an assessment of the speed programme’s process and methodology used in identifying and selecting roads for lower speed limits. The report was very favourable.

Thereafter, in 2022, Abley was contracted and Martin prepared three independent evaluative reports on the first phase of the Safe Speeds Programme, which was implemented in 2020 and lowered speed limits on some 880km of city roads.

The reports provided monitoring and evaluation of crashes, deaths and injuries on the affected roads after 12 months, 18 months and 24 months. They found large reductions in serious crashes across the roads where speed reductions were introduced, though they bear significant caveats including small sample sizes and the inclusion of anomalous periods of pandemic-era lockdown.

A fourth evaluation of data at the 42-month mark is understood to be complete but has not yet been published (Martin also worked on this report).

The Herald understands AT was aware Martin would be working on the reports when the work was contracted from Abley.

While the reports were prepared by Martin, in each instance the work was reviewed by the head of Abley’s transport team Paul Durdin. Additionally, the peer review was reviewed by Dale Harris, a principal transport and spatial adviser at Abley and the 18-month evaluation was reviewed by Chris Blackmore, a senior transport planner at Abley.

An AT spokesman said the agency did not consider that Martin’s work presented a conflict of interest or a risk of bias; no mitigation process was followed.

”We are satisfied with the quality of work from Abley, and with the results. We have international awards recognising our work and it is backed up with evidence and data. We have followed international best practice at every step of the process.”

He said the reports were done “following correct protocol, with all checks and balances in place”, and Abley was a highly regarded, evidence-based consultancy. In addition, he noted the data used in evaluation was sourced externally, from the Transport Agency’s crash database.

He said the Abley evaluation reports were peer-reviewed by “other highly regarded engineering consultancies”.

Last month, in an open letter to the Government, more than 80 New Zealand and international professionals from fields ranging from medicine to speed management and engineering emphasised the link between higher road speeds and deaths and serious injury. They also warned speed limits should be set based on “solid evidence”, and cited AT’s speed limit reductions as an example.

The spokesman said Martin’s work at AT on the Safe Speed Programme was technical and within the context of a large team.

Abley chief executive Steve Abley, told the Herald Martin is a well-credentialled and able young engineer, and the relevant reports were reviewed and approved by more senior staff.

”Our work is undertaken independently of Auckland Transport and represents our firm’s independent expert analysis, including our expert opinions and recommendations. I understand those recommendations have led to some minor changes at AT’s safe speeds programme,” he said.

The Auditor General’s guidelines in considering possible conflicts of interest include matters of potential predetermination or bias.

The underlying risk with predetermination, the advice says, is the same as for conflicts of interest: there is a risk of “tainting” decision-making because of bias or the appearance of bias.

Auckland Ratepayers’ Alliance spokesman Sam Warren said the perception of bias was “a big problem for Auckland Transport” since it looked to him like it had been “marking its own homework”.

”For Aucklanders to have confidence that the [speed] programme stacks up, [I believe] it has to pass the sniff test without issue. Any whiff of bias would cast doubt on the programme’s credibility, and any potential conflict should have been properly managed.”

The Ratepayers Alliance is linked to the NZ Taxpayers’ Union.

The reports have been relied upon by both the Auckland Transport Board in its decision-making to continue its programme of speed limit reduction, and by the Auckland Council, including in a recent vote to oppose the Government’s plan to raise speed limits on some roads.

The Herald raised the matter of Martin’s involvement in evaluation of AT’s speed reduction programme with the office of Auckland Mayor Wayne Brown. Brown declined to comment.

Speed limits (and related evaluations) sit atop an active political fault line

Auckland Transport’s speed reduction programme, rolled out in three phases to date, remains contentious, as does its use of some evaluation findings.

For example, when unprompted by evaluation findings, 46% of Aucklanders’ recently polled by Verian, for AT, favoured the lower speed limits, while 38% opposed them.

However, the views of residents became very considerably more amenable to the lowered limits when respondents were first informed that an evaluation of the lower speeds found a 30% reduction in road deaths, and a 20% reduction in serious injuries (the figures were drawn from the 24-month Abley evaluation).

That first percentage reduction is controversial. The figures underlying the 30% drop in road deaths is the difference between an average 6.4 fatalities a year over five years before the speed limit change, and an average of 4.5 fatalities a year in the two years afterwards.

The sample in each case was so small that the difference between them was not statistically significant.

It bears noting there were more instances of serious injury (an average 68 a year before the changes and an average 53.5 a year afterwards). This reduction, or the two data sets rolled together, is more statistically meaningful.

While the Abley evaluation report itself warns of the very small sample sizes, its use by AT sometimes has not, including in past paid advertising to solicit Aucklanders’ feedback on the speed limit changes and in polling.

Auckland’s speed programme, and measures of its success, mark a political fault line. The programme flows from AT’s so-called “Vision Zero”, a goal of eliminating road deaths and serious injuries by 2050; it’s an aim that married the previous Labour Government’s “Road to Zero” plan, which aimed to reduce serious injury and death on New Zealand roads by 40% by 2030 (from the 2018 level). Speed limit reductions were an essential part of both plans.

Since last year’s election, however, the National-led coalition Government has begun efforts to reverse at least some of the speed limit reductions made since 2020.

Both Transport Minister Simeon Brown (also the Minister for Auckland) and Prime Minister Chris Luxon argue there are economic drawbacks to the lower limits, including reduced travel times and lost productivity (the Government also emphasises that alcohol and drugs are the highest contributing factor to fatal crashes on roads). This position, in turn, has been extensively contested.