Under the changes, young people who earned cash-in-hand for babysitting or lawnmowing would not have to file a tax return at the end of the year.
But children earning under $2340 who paid tax to their employers could no longer claim it back.
Treasury officials said the changes would make the Government $14 million a year over the next four years.
Mr Key - a paperboy in his youth - said he found out about the move at Cabinet on Monday and did not regret it despite the publicity.
In a speech at the Langham Hotel in Auckland today, Mr Key said the change was done primarily to try to tidy up the tax code and most people didn't claim the PAYE exemption.
"A lot of people didn't know they were entitled to them so they didn't bother claiming. The amounts were fairly small and overall we have been trying to clean up the tax code."
He rejected criticism the Government was picking on paperboys and said he had never claimed the exemption.