A resident near the Flaxmere home of one of the baby girls said it was "horrible" to see a family tragedy in her street.
"I saw the baby come out in a little baby coffin, it was really horrible," the woman said.
Police at the scene had appeared to be "investigating" for a "few hours" after they arrived about 7am.
"I saw the father come out, he was a mess, he was absolutely distraught and in tears.
"You could see the mother trying to keep it together because she had other children to care for, but the father was really, really upset.
"It's an absolute tragedy and horrible to see a newborn baby die.
"Even though I didn't know the family I really was affected by it for a number of days."
She said she had seen the baby a day earlier and described her as "looking really healthy".
"They seem like really nice parents as well, it's just really sad for them."
People at the family home declined to talk to Hawke's Bay Today yesterday.
One neighbour said he had noticed police on the street a few weeks ago, followed by a funeral.
He said police had been at the address the day and night beforehand.
"It's always really sad when it's a young baby."
Another neighbour said she did not know anything about a baby but about a month ago a friend of hers had seen a distressed man run out of a house around the corner and flag down police.
A couple who live on the street said they were aware of the death of a baby girl, thought to have been about 3 months old, but they did not know the family.
"Everyone pretty much keeps to themselves around here but we did acknowledge the passing of their child."
They said the death was kept very quiet and they believed a post-mortem examination was carried out before the baby was cremated.
"I'm very saddened by it."
Police are also investigating how a 5-week-old boy from Waipukurau received multiple fractures and head injuries. He was admitted to Hawke's Bay Hospital on November 1 and transferred to Starship Hospital in Auckland later that day.
An Auckland District Health Board spokeswoman said last night the 5-week-old boy was now in a stable condition.