But he said there were plenty more people to picket than just National MPs - including former Labour education minister Trevor Mallard and his then adviser Chris Hipkins.
"I understand they want the thing solved, but actually there's a wider range of villains in this ... This problem goes back many years and in fact this is the second pay system which has been difficult."
Mr Joyce said there had been similar issues with the pay system for teachers introduced in the 1990s.
"So there's something about the way we pay our teachers in this country as well which causes a problem whenever the software is changed ... We do have to get to the bottom of that."
Mr Joyce said a lot of work had been put into fixing the Novopay system and a number of errors had been stabilised.
He said 80 bugs had been fixed two weeks ago and another 150 would be fixed in two weeks' time.
"It's still unacceptable levels - roughly just under 2 per cent of the last pay were errors and it should be down around half to 1 per cent in a mature pay system.
"And that's an important thing to point out, too. There's always errors in a pay system and in fact this time last year, the old system was delivering about five or six hundred teachers that weren't being paid."
Asked if he still had faith in Talent2, the company which built the Novopay system, Mr Joyce said "maybe" - but there was a long way to go.
Mr Joyce said there should be no teachers who were not getting paid.
"The system is obviously not delivering pay for some people, but there are systems in place to make sure that people get paid immediately afterwards if there's a problem."
Some schools were also making payments to teachers.
NZEI president Judith Nowotarski said the union had been to weekly meeting with Mr Joyce but had not seen a reduction in the level of frustration from members.
Ms Nowotarski said the action would not interrupt the school day.
The union wants the Government to agree to a package of support measures for schools, and to ensure adequate staffing and service levels at Novopay and the Ministry of Education.
NZEI spokesperson Tanya Gant was picketing outside the office of Waimakariri MP Kate Wilkinson.
Those there were fed up and demanding action from the Government.
"We want them to fix the system. We want them to actually put in a proper timeline and what they should have done was actually done a proper trial in the first place."
Ms Gant said they had public support.
"Waves and horn toots. Everyone's well aware of Novopay and the problems it's caused our school and staff."
One school principal said this morning's national Novopay protest was also about thanking the community for its support.
Around 30 staff, students and parents from Target Road School in Glenfield wer among hundreds up and down the country who were rallying to mark six months since the troubled Novopay system began.
Principal Helen Varney said the point was to tell the community teachers' frustrations continue, and also to thank them.
"Teachers who haven't been paid for five weeks. We've got parents who bring them in food. We've had baking brought to schools. People give them grocery vouchers."
Around 50 people have been protesting outside Hekia Parata's office in Porirua.
Mana MP Chris Faafoi said the protesters received good support from the public.
"There were quite a lot of beeps. There wasn't more than five seconds where we weren't getting beeps, so we know there's lots of public support. Parents of kids know that their teachers are getting frustrated and that their principals are spending way too much time on Novopay and not enough time on educating their kids."
Mr Faafoi said if something wasn't done soon about the system, National MPs' could expect more protests.
Did you witness or take part in a protest? Please send your story, photos or video here.
- APNZ with Newstalk ZB
