They also found Facebook chats with a woman where Kelly posted a number of images of young children and babies and described in details the sexual acts he was going to perform on them.
In August 2014 he contacted 24-year-old Auckland woman Krystal Harvey.
Kelly asked Harvey to send him photos of children wearing nappies and also ones where they were naked.
He then encouraged her to make a video of her performing a sex act on a 1-year-old child.
Harvey initially stated that she wouldn't do that, but claimed that Kelly threatened her if she didn't do what he wanted.
But she later admitted sending the video to Kelly for monetary gain. He paid her $300.
Kelly's offending was planned and premeditated, Judge Paul Kellar said today.
The judge said that Kelly had arranged and encouraged Harvey to commit the offence, record it, and send him the footage.
"She should, and could, have said no," Judge Kellar said.
"But the offending would never have occurred if you didn't encourage it."
Judge Kellar noted that infants are the most helpless members of society.
Kelly was also found with images of naked young girls in sexual poses, and other photos and videos, where children were involved in sexual acts with adults.
There was also a video of a woman performing an oral sex act on a dalmatian dog.
In February 2014, he contacted two other women online and offered money to have time alone with their young daughters, both aged under 5. Kelly described in detail the sex acts he wanted to do on her.
Kelly was found with about 70 images he had taken of himself wearing only a nappy, as well as eight images of animals involved in sexual acts with children and adults.
When questioned by police, Kelly tried to blame another man.
Defence counsel Andrew Bailey today said Kelly is now aware of the seriousness of his crimes and the harm he has caused.
He described Kelly as being "younger than his years" and was immature.
Kelly earlier pleaded guilty to 16 charges, including one of sexual violation by unlawful sexual connection, five of making an objectionable publication, and 10 of possession of an objectionable publication.
Harvey was sentenced at Manukau District Court earlier this year to eight months' home detention and 100 hours of community work.
But the Crown appealed the sentence and a jail term of three years and nine months was later imposed at the High Court.
Judge Kellar noted that Kelly himself had been abused as a child, and that he had been aged 19-20 around the time of the offending.
As he jailed him for five years, members of the public gallery sobbed and hugged each other.
