New Zealand First leader Winston Peters said his party welcomed the work Little was doing in the criminal justice sector.
"As part of his work the Justice Minister is developing a range of options for criminal justice sector reform. One recommendation was to repeal the three strikes legislation. This recommendation was included in a paper due to go before Cabinet today.
"New Zealand First has welcomed the Justice Minister's decision to not proceed with recommendations to Cabinet today," Peters said.
"The caucus looks forward to working with him on achieving a balanced reform package."
The New Zealand First caucus would consider its legislative position on the three strikes repeal at its caucus meeting tomorrow.
The Sensible Sentencing Trust this morning ran a full-page ad in the New Zealand Herald calling on Little to drop his plans.
Founder Garth McVicar has credited New Zealand First leadership for the backdown.
"They know the New Zealand public will not support a return to the days of failed soft-on-crime policies that do not deliver justice and marginalise victims at the expense of the 'rights' of dangerous recidivist criminals.
"We say 'thank-you New Zealand First for doing the right thing'."
McVicar said Little should also drop "his outrageous plans to weaken the bail law, increase the already over-used home detention" and that he should push forward to build more prison space.
Under the law, which was passed by the National-led Government in 2010, a person with three warnings after serious violent, sexual or drugs convictions must be sentenced to the maximum jail time without parole unless the sentencing judge believes it would be manifestly unjust.
Simon Bridges said Little had been "hauled into line" and had clearly been making promises he had not done the work to keep.
"And what is genuinely concerning is the embarrassing backdown comes a week after it was confirmed that the Government hadn't even bothered to seek advice or follow due process before axing [new offshore] oil and gas exploration.
"It is policy on the hoof and shows why Labour is so reluctant to make decisions and is instead sending everything to working groups, 122 and counting.
"When they do try and make a decisions they make a mess instead."
The open letter to Little published in today's Herald ad was signed mostly by relatives of people who have been killed or severely abused by offenders.
For every relative's name is a victim's name - many of which are chillingly familiar: Sophie Elliott, Michael Choy, Philip Nisbet and 17-year-old Augustine Borrell, who was stabbed to death outside a party in Herne Bay.
"We, the undersigned, are all personally the victims of serious violent crime or close relatives of someone killed, maimed or sexually abused,'' the letter reads.
"We are all deeply concerned about the proposals to repeal the three-strikes law, make bail easier and generally to let people out of prison who need to be there.
"We believe the proposed changes are misconceived, misguided and certain to create more members of the club to which we, the undersigned, all belong - the club which no one wants to join.''
Charlie Borrell, whose teenage son Augustine was murdered by Haiden Davis in 2007, said Little's announcement was a big relief and felt "common sense had prevailed".
"I thought 'wow that's fantastic news. I suppose there's some common sense there'."
Borrell said two-thirds of the population supported three strikes in the polls so was pleased New Zealand First has listened.
"The results are showing there has been a reduction in second strikers under the three strikes law."
For him personally, he said it meant her could breathe a little bit easier. "I'm a lot happier. I wasn't looking forward to seeing another family go through what we did during out time. So it's just a bit of a sigh of relief."
Davis would be released next year and Borrell said his biggest fear was bumping into him in the community.
