Mr Joyce said staff were now keeping up with current payments, and the focus had shifted to resolving the six-month backlog of issues dating back to the system's launch.
He said most complaints from schools were about past issues that needed to be resolved.
The backlog was very significant, and the worst part was that it took time to resolve the issues.
"So this week we're setting up a backlog clearance unit, which is effectively going to double and treble the size of the people working on that."
Mr Joyce said he had seen a draft copy of the technical review report on Novopay, which is due within the next fortnight.
After seeing the full report, Mr Joyce is expected to make a decision on whether to ditch Novopay for the old Datacom system.
But he would not speculate today on what decision he would make.
"It's a decision that has to be made really carefully because no matter which way you go on that, there's pain, and the last thing I want to do is create more pain than necessary.
"We have to have a cold, hard look at whether it will be quicker to remediate on the plan that we're operating on now, which we're continually ramping up, or cut across back to the old system."
Mr Joyce said Datacom had presented the Government with a proposal and he had met with them in recent days.
Asked whether Datacom was up to the task, he said: "I think they could do it if they needed to."
But he cautioned against seeing such a move as a quick-fix.
"The reality is they would still have to deal with all the backlog and they'd also have to take teachers back to the old system, which would then have to be upgraded down the line to another system."
Hundreds of teachers last week protested outside National MPs' offices throughout the country to vent their frustration at progress with the Novopay system.
The teachers' union PPTA is considering legal action against the Education Secretary over Novopay. Prime Minister John Key has said compensation to prevent that was possible if the PPTA agreed to a package.
