Treasury boss John Whitehead has warned public sector heads if they do not face up to a "fundamental rethink" to trim fat out of their departments, they will have change forced upon them.

In a speech to government department chief executives in Wellington yesterday, Treasury Secretary Mr Whitehead said the public service needed to move out of its traditional comfort zone and take some risks to ensure it delivered services as cost-effectively as possible.

"There is a stark alternative to mobilising ourselves as public servants. If we don't rise to the challenge and make real progress, change will occur - but it will be done to us rather than by us."

The Government's edict was for better services without spending increases - and Mr Whitehead said nothing should be off the table to try to lift the productivity of the state sector.

Options included contracting out more services to the private sector, merging administrative services with other departments to lower costs and cutting projects despite the possibility of staff cuts.

The public sector employs nearly 250,000 workers - and Mr Whitehead's speech has angered the union representing public servants, the Public Service Association. Its national secretary Brenda Pilott said she rejected the suggestion within the speech that privatising government services could increase productivity.

"We're amazed Mr Whitehead says we should be privatising public services when bad management in the private sector has created the worst global recession since the Great Depression.

"The effective way to lift productivity in the public sector is for the staff and employers to work together on this issue, not by privatising public services in the false hope that this will automatically lift productivity."

In the speech, Mr Whitehead says "tough decisions" are needed. Staff numbers working in the core bureaucracy had grown by 44 per cent since 1999 - a far greater number than those affected by recent redundancies.

Mr Whitehead told the government departments more savings would be sought through the "line-by-line" reviews of spending that have become a regular part of the Budget process.

Mr Whitehead said no formal target was set for savings in the reviews "but I don't know if any departmental or agency chief executive would argue they couldn't think hard about better utilising 5 or even 10 per cent of their baseline. I know I have."

While the initial round of reviews early this year had resulted in $2 billion in savings over the next four years, Mr Whitehead said that only isolated the "low-hanging fruit".

He said the fiscal outlook and economic downturn meant it was now of particular importance to "up our game".

"Without a very firm focus on growing the economy and controlling government spending, New Zealanders will be looking at more public debt, more money on interest payments, less room for government to move - and higher taxes."

Since National came into power, chief executives and ministers have had to negotiate "performance improvement actions" to ensure they monitored the success of their departments and costs.