Instead, Mr Crosby said he was a strong believer in government departments setting up well-resourced autonomous offices in regions, as the New Zealand Transport Agency (NZTA) had done in Tauranga.
"NZTA set up regional offices, but more importantly they gave the regional manager significant delegated authority," he said.
"And then things happened a lot quicker. I think that's a very good model to follow. Put good people in the regions who are well-resourced, and give them delegated authority to do stuff and make it happen quickly."
Tauranga's former deputy mayor and city councillor Kelvin Clout said Mr Peters' overall concept sounded good in theory, but he suggested it was unlikely to work in practice.
And while relocating an entire government office to a smaller regional city could benefit it, "in reality you could struggle to get staff to relocate to smaller provincial towns," he said.
However, Mr Clout also supported the idea of government departments having strong regional offices, and argued that it made good sense for Tauranga, as the biggest city in the Bay of Plenty, to be the base for regional central government offices.
Priority One communications manager Annie Hill said it would be great for the Bay of Plenty - and particularly Tauranga - to have more government departments represented in the regions.
"They tend to pay good salaries and it shows the government's support of the regions as well as the main centres," she said.