Ms Kay encouraged her to go to Work and Income to get emergency assistance because there would be a bond as well.
But Work and Income staff at the Gisborne office said they could only offer her an appointment eight days away. The Work and Income regional director has since issued an apology for this.
A phone call to Women's Refuge was also met with a closed door because the young woman was not fleeing from domestic abuse, and a call to a Salvation Army number went to an answerphone.
Ms Kay said she and her staff could not turn the woman and her baby away.
"I'm looking at this girl with her four-month-old baby and thinking, 'what is happening in this country?' Is it really possible that there is more support for a penguin than for a mother and her child here in New Zealand? What happens to people like this?''
Ms Kay and property manager Carol Martin contacted an owner in Australia to see if he would mind if she stayed in his vacant house that was on the market.
He was happy to help and forego the bond temporarily considering the circumstances.
Ms Martin drove them to the house and stopped to pick up some groceries and essentials. All they had with them was a backpack full of clothes and a foam squab.
"She was a nice girl. I drove her around and I talked to her about going home to Auckland but that wasn't an option.''
"Here is this little girl with a four-month-old baby. What about that baby? That was our issue, where was she going to go?''
Salvation Army community ministries co-ordinator Bev Hauiti said the mother received food parcel assistance from them the next day.
She also confirmed a lack of emergency housing in Gisborne.
"A lady came in with three children and they had been sleeping on the beach.''
Women's Refuge manager Vi Pirini confirmed they do not help women who are not being abused.
"That's our only criteria - that they are women and that they are experiencing or witnessing domestic violence. If they don't come under that category, it becomes a housing issue and we don't provide emergency housing.''
The young mother said she was very grateful and happy to have her own space after arriving from Auckland in April, pregnant with her first child.
"I love Gisborne. When I moved here, I was staying with my cousin. We fell out and I had to find somewhere else to go. I stayed at a friend's for a night but I didn't want to put my problems on them.''
Work and Income East Coast regional director Annie Aranui said given her circumstances, the mum should have been given an emergency appointment.
"We met with her today, after visiting her at her new place, and have helped with the rent and bond. Other assistance has also been provided and we have discussed what support she needs going forward.''
