Leaving the House, Mr Mark at first claimed he had only mouthed the words. When told it had been picked up by microphones, he admitted it was out of order.
The MP said he had been referring to a "group of MPs that were behaving the way they were".
Mr Brownlee said the swearing was inappropriate but Mr Mark had now apologised.
New Zealand First leader Winston Peters would not comment because he did not hear the original remark.
Asked whether he had "dropped the f-bomb" in the House, he said: "Well ... look, not so that you heard it."
It is not the first time Mr Mark has been caught out. He was filmed giving the fingers to National MP Tau Henare across the debating chamber in 2006.
Former Labour MP Steve Maharey was also forced to apologise in 2007 when said "f*** you" to National MP Jonathan Coleman in the House during an argument.