The Government's caution was heavily criticised last week when it announced plans to freeze the the pay of public servants.
Under new government rules, three-quarters of people working in the public sector are unlikely to get a salary bump until at least 2024.
No government employee earning over $100,000 a year will get a pay rise until 2024. And those with salaries between $60,000 and $100,000 will need to prove exceptional circumstances.
This led to accusations of austerity measures, with many also questioning whether it was a good idea to freeze the wages of doctors and nurses after all they had done for the nation.
The Unions immediately hit back, expressing concern that doctors and nurses may move to Australia if the measures are put in place.
Robertson responded to those concerns, saying that the use of the term "pay freeze" was not accurate.
"The guidance breaks down three categories for public sector pay, 'lift' (those at $60,000 or below), 'adjust' (those between $60 and $100,000) and 'hold' (those above $100,000)," Robertson said.
"It is important to note that this is not as has been reported, a pay freeze.
"Pay for public servants such as teachers, nurses and police officers will continue to increase as they move through the pay bands we have previously agreed, pay equity discussions continue and there is still a collective bargaining negotiation to go through."
Robertson said New Zealand could not take the recovery for granted.
"Further waves of Covid-19 around the world underline that we are still in a highly volatile and uncertain global environment.
"There are still sectors and regions in New Zealand struggling to deal with the impacts of the pandemic. We have also taken on considerable debt to support New Zealanders lives and livelihoods through Covid-19"
Robertson said a balanced approach was required by the government.
"We need to continue to be careful with our management of the economy, while also providing stimulus to accelerate the recovery and to make investments where they are needed the most to tackle our long term challenges in priority areas such as climate change, child wellbeing and housing."
Robertson said there was a bit more space in its operating and capital allowances to support the economy in line with its wellbeing approach while also keeping a lid on higher debt levels with an eye to lowering debt once the recovery was secure.
"An investment-focused recovery that supports all New Zealanders is the way to ensure that our finances remain sustainable. It is also the way in which the Government will continue to tackle the long-standing issues that we were elected to address."
Robertson said it would not all be done in one Budget.
"I view this budget as one of a package of three this term to advance our priorities and return our books to a sustainable fiscal position."
Robertson said Budget 2021 would still be very much a Covid Budget in that it was still dealing with the ongoing effects of the pandemic and that it still loomed large over the global environment.
He said the Government's first job was to protect the health of people.
"Our investments in keeping our border secure and in rolling out the vaccine programme are both substantial and necessary. Our goal is to have every New Zealander over the age of 16 vaccinated before the end of the year."
This would allow New Zealand to open up more opportunities to connect with the world in a planned and careful manner.
But he said the Government also recognised the economic impacts of Covid were still being felt and supply chain issues were affecting businesses here and across the world and was likely to continue for some time.
Last week the Government announced an extra $200 million for the tourism sector.
