Robertson said he had asked ministers to do a stocktake of the Covid spending in their portfolios, and return any un-spent money. About $926 million was released back into the fund.
It is a small portion of the total $50 billion Covid-19 Fund Robertson set up in the 2020 Budget, funded primarily through debt. Less than $10 billion of that fund remains.
Robertson said it was important to target the funding where it was required. "We have committed to only spending what we need to in terms of our response and recovery."
The Budget would also begin the process of paying down the debt that was racked up to deal with Covid-19.
"New Zealand's economy continues to perform better than forecast, but we know the impacts of Covid-19 are still being felt around the world. We will strike a careful balance between continuing to invest in our recovery and beginning the task of reducing down the debt we took on to support New Zealanders through Covid-19."
He said there were "no costless decisions in our response to Covid-19" but he stood by the actions the Government took.
"It was expensive, it was not perfect, but it has given us a headstart we are determined to follow through on."
He emphasised the resilience the economy had shown through Covid-19, despite dire forecasts of a year ago.
"Our stronger than expected economic performance means we have more options than we have expected to invest in critical services. But we are also mindful that we have an obligation to future generations to prioritise spending, and over time reduce the debt we incurred fighting a one in 100 year economic shock."
He said while sectors such as tourism and international education had suffered, New Zealand's unemployment levels and GDP was strong compared to other countries it traditionally compared itself to.
He put that down to trade, noting strong exports in logs, fruit, and wine as well as dairy and meat.
"Even in the tough times of Covid, consumers all around the world need what New Zealand sells."
However, he said the ongoing impact remained uncertain, and that meant the Budget still had to focus on keeping New Zealand Covid-free.