In 2004, he was jailed for sexual offending against a girl under 16 and doing an indecent act with a girl under 16.
At the end of his prison sentence, in December 2011, he was subject to an extended supervision order for 10 years and reporting obligations under the child sex offender register.
One of the conditions was to not posses or use any computers or electronic equipment capable of storing images - unless granted by his probation officer.
But, on August 8 last year, police visited Jolly's home.
They found a device plugged into the back of his television and took him away for questioning.
Jolly denied he owned the internet-capable device and then astonishing fled the police interview.
The next day police returned to his home and arrested Jolly, they also found a hidden cellphone.
On the phone were 3942 images and 20 videos.
Twelve of the images and videos were sent to and assessed by the Films, Videos, Publications and Classification Office. Eight were deemed objectionable.
Some of the images showed young women or young teens engaging in sexual acts, including one video depicting a woman being drugged and raped.
Prior to his latest offending, Jolly had 18 previous convictions for breaching his extended supervision order, including in 2014 when he used a smartphone to contact a teenage girl on Facebook.
Then in December 2016 his offending continued when he cut off his electronic monitoring tag and went on the run.
He was later found on State Highway 1 near Pegasus township, 25km north of Christchurch.
His extended supervision order conditions also ban him from going near schools, playgrounds and parks as well as associating with anyone under the age of 16.
However, mid-last year, Jolly was allowed to move into a Whangārei home within a kilometre of a children's health camp, a daycare centre, two primary schools and a high school.
Corrections moved him out of the address only days after he moved in - after neighbours identified him through the Sensible Sentencing Trust's online database.
Jolly's lawyer Nigel Cooke argued at the May 29 appeal hearing in the High Court at Auckland that his client was not integrated into the community, has no social network, support, or friends.
"He's a lonely man," Cooke said.
"I think he's been crunched. Even if a couple of months were knocked off it would make a world of difference."
After Cooke argued there had been a significant period of time where Jolly had not offended, Justice Patricia Courtney quipped: "I don't think you get any credit for not offending against any children while you're in jail."
Despite his history, in a June 8 judgment released to the Herald last night, Justice Courtney allowed Jolly's appeal and allowed a discount for his guilty plea, quashing his sentence and substituting it with an end sentence of one year and 11 months' imprisonment.
In 2015, Jolly's history was highlighted by the Herald, which led to calls for an urgent review of the legal system.
