He said he would not accept Muslims or any non-Christians praying at council because they believed in different gods.
Mr Bielski told RNZ he only said those things because the press were not present and it was in the confidence of the meeting.
But he stood by his comments.
"Obviously if I said it there, and if it's not twisted and everything like that I would stand by it because that's exactly what it is.
"I tell you what, there is a lot of things said when the press are not there because we have the confidence to talk about some things," he said
Steve Bielski declined to be interviewed further and hung up the phone.
Fellow councillor Shane Casey witnessed the outburst at last month's meeting.
"A councillor was not happy that we were considering bringing in other faiths.
"The councillor's mannerism was agitated and upset and was quite adamant the council should only be opened by, you know, a Christian group.
"Everyone is entitled to their opinions and how they feel," Mr Casey said.
However Shane Casey said he does not agree with Mr Bielski.
Another councillor said they were gobsmacked by the outburst.
And the chief executive, Richard Templar, said he told the room Muslims, Jews and Christians all pray to the same god before carrying on with the meeting.
Manawatū District Council Mayor Helen Worboys said she was not at the meeting but no decision was made, it was just a discussion.
"It's certainly not my opinion, I'm very happy and very open about all faiths and all cultures, we are a diverse community so my own personal opinion is that we would welcome anyone on board.
"But like I said, councillors are entitled to their own opinion and they are free to express them," she said.
Helen Worboys said Mr Bielski would not be reprimanded for his comments.
She said the councillors have had similar animated discussions about the Māori wards. (-RNZ)
