The 65-year old voluntarily left Sri Lanka last week and arrived at Christchurch International Airport on Friday.
He was allowed through immigration before being arrested in the city a few hours later.
After spending a weekend in police cells, McGrath appeared from custody at Christchurch District Court today.
Crown prosecutor Mark Zarifeh opposed bail, citing the seriousness of the alleged offending and that he was a possible flight risk.
But Judge Jane Farish did not agree he would take flight, especially since he had returned to New Zealand voluntarily, knowing he would face arrest.
She granted bail on strict conditions that he reside at his sister's residence in
Christchurch, report to a police station three times a week, has no contact with children, and does not go within 100 metres of any primary or secondary school.
His passport had already been surrendered and he can not apply for a new one.
Judge Farish remanded him until December 17 which will allow him time to study the new charges.
At his next hearing at Christchurch courthouse, McGrath will either accept extradition or oppose the application.