"It would be done with the greatest of speed to restore some balance to our economy.
"And for many civil servants they will be able to buy a house and buy it in much quicker time ... many workers will welcome the opportunity to ditch long commutes for a drive of a few minutes and the outdoor recreational opportunities on their doorstep."
Asked if government workers would be happy to move away from the city, Peters said it wouldn't be up to them.
"Their proximity to ordinary people might actually improve them."
Today's announcement by New Zealand First is part of the party's regional policy. Peters has stepped-up campaigning in the provinces ahead of next year's election.
The idea of moving government work out of the cities has previously been examined by Labour, and in 2014 Finance Minister Bill English said the idea had merit - but moving was a call for ministry chief executives to make.
In July, Local Government New Zealand members voted for a remit to have some state services shifted from Wellington to the regions. The proposal was put forward by Nelson city councillor Matt Lawrey.
At the time, Lawrey told Radio New Zealand that the proposal would save taxpayers through lower rental costs, and mean provide some protection for government services if a natural disaster hit Wellington.