Katie Marshall, mum to 8-month old Audrey, says she felt the pressure first hand.
Having difficulty with breastfeeding from the start, Marshall started supplementing Audrey's feed with formula.
And after all the information she got about the benefits of breastfeeding, she felt guilty.
Marshall avoided mums' groups and was reluctant to see the Plunket nurse. As a result, she had little information on bottle-feeding.
"I felt terrible for even asking for help. It really stopped me from enjoying being a new mum."
Eventually, she said, she got over it.
"There's a misconception that mums who are formula feeding are lazy, or just want to go and drink all the time and have their own life."
Marshall says that is not true, and although she is an advocate of breastfeeding, mums should be a lot less judgmental of each other's choices.
"At the end of the day Audrey knows I'm Mum, whether I give her a bottle or not."
A lactation consultant at Wairarapa DHB, Yvonne Stuart, says that though their focus is breastfeeding they support and respect mums with their independent decisions about feeding.
"Sometimes they experience some pressure, but they're not pressured by society, more by themselves."
Wairarapa DHB is signed up to the World Health Organisation and Unicef-endorsed Baby Friendly Hospital and Baby Friendly Community 10-step programmes to get more mums breastfeeding.
As a result, the DHB has increased the number of mums leaving the hospital exclusively breastfeeding from 28 per cent in 2004 to 83 per cent last year.
Stuart says the aim is to make breastfeeding the norm, and the "more it happens in public the more normal it becomes".
She is one of just two lactation consultants in the region, which presents some challenges, particularly travelling to remote communities.
But overall, she says, support services for breastfeeding are good in provincial areas.
"They are picking up on issues and referring mums on for help. It's one of the benefits of small towns.
"People like Plunket and Whaiora are busy, but not so busy that they see a problem."
Andie Collett, co-leader of the Masterton La Leche League, says the baby-friendly programmes are working so well that the advocacy body does not have an active group in the Wairarapa.
says mums get a lot of advice and support in the first six weeks, although it drops off a bit after that and it can be hard to find like-minded people.
"In bigger towns it's easier to find people who think the same as you and to get information."
She says she has never experienced any negativity when breastfeeding in public.
New mum Anya Ogden is also a neonatal nurse, so needed little support for breastfeeding, but says there could always be more.
"In Wellington they probably have that little bit more access to things.
"Everywhere could do with more help but it's more pronounced in smaller areas."
She loves it when cafes have signs up which say it's a breastfeeding-friendly place.
"It just gives you that little bit of confidence that you're invited to breastfeed there rather than it just being put up with."