More jailed mothers should be able to look after their children in prison, a report says, and parent-child units should be extended to fathers for the first time.
The Unlocking Prisons report, launched by lobby group JustSpeak, says keeping convicted parents and their children together is likely to reduce their chance of reoffending.
Mothers have been able to bring under-2s into jail since 2008, when the Labour government adopted a Green Party private member's bill. Some jailed parents can get independent units within prison, and others can use specialised facilities to look after their child up to 12 hours a day.
There are 14 parent-child units in the three women's prisons, for a population of 533 prisoners.
Justspeak said more parent-child units should be established and Corrections should consider independent units for fathers in prison.
Corrections Minister Anne Tolley agreed that the units were highly valuable, but said demand was low.
"When I visited Christchurch Women's Prison recently there were a number of empty rooms. These units are extremely important for families - but there is no point in building more just for them to lie empty."
Mrs Tolley said state prisons were increasingly following the lead of private prison operator Serco, which introduced family-friendly visiting areas including facilities to bathe children.
JustSpeak said family-friendly measures were more important than ever because the rate of female imprisonment was increasing faster than for males.
The comprehensive review of prisons made a number of recommendations, including locking up fewer criminals and using more community-based measures for rehabilitation.