While it was not yet clear where the central city might be built, Mr Robertson said it was clearly not going to be by the airport.
He said a short-term lease might have been understandable, but the long-term move raised questions about the Government's commitment.
"That's the question for (Earthquake Recovery Minister Gerry) Brownlee, is the plan actually happening? Because if you're looking at the future of the Christchurch central city, nine years without those guys is a long time.''
However, Mr Brownlee said the Government remained committed to the plan and it was wrong to fixate on the length of the lease.
"The Government leases a lot of properties, there are always uses for properties that are under Government lease,'' he said.
"What eventually happens to that space, where the call centre eventually locates, it's only a very small part of the total number of government servants who work in Christchurch.''
Mr Brownlee said he was pleased the agencies had opted to remain in Christchurch at all, and that moving into the central business district was not possible at the moment.
"And, frankly, if you asked them when can they go back, they can't give you an answer to that,'' Mr Brownlee said.
"There'd be greater outrage if the call centre had actually left Christchurch, and I think people have to calm down a little bit.''
Asked whether other agencies were likely to set up outside the central city too, Mr Brownlee said he could not say because those were operational matters for the departments.
"The commitment to go back in remains the same, there's a comprehensive plan being drawn up by all agencies of government to return to a suitable space in the CBD,'' he said.