Labour MP Louisa Wall's marriage equality bill, which legalised same-sex marriages in 2013, exempted celebrants nominated by a church or other approved organisation whose "religious beliefs or philosophical or humanitarian convictions" do not allow same-sex marriage.
But Baker, who conducted weddings as a pastor for Baptist and other churches for more than 25 years, is now retired and applied to become an independent marriage celebrant after a couple asked him to marry them.
He was on a mission trip in Europe at the time and had to turn them down because his application led to a long exchange with the Internal Affairs Department culminating, after that couple's marriage, in a rejection.
"I've had three other people ask me to marry them. I said I can't now," he said.
"I was surprised. There seemed to be no reason except there was blatantly a prejudice against those of a particular faith.
"Marriage originally was a religious institution. Now it looks as if the institution has been hijacked by a minority, a radical minority."
Another rejected applicant, Christchurch Salvation Army volunteer shop assistant Lesley Erikson, 53, said she was "shocked" when her application was rejected because she would not perform same-sex marriages.
Montgomery said the application form for marriage celebrants did not ask about same-sex marriages, but he had instructed staff to ask about it when they interviewed applicants.
"When I am personally appointing independent marriage celebrants [it is to] provide a public service, so if the celebrant says when we ask in the interview that they are unwilling to provide service to anyone who is legally able to get married, including same-sex couples, then they will not be approved," he said.
He advised would-be celebrants rejected for this reason to join an approved church or organisation.
"The easy option for them ... would be to align with an approved organisation or to create their own organisation the same way as the Church of the Flying Spaghetti Monster did," he said.
"I approved them [in 2015] on the basis that they had clear philosophical beliefs, albeit unusual ones, thus they were able to nominate marriage celebrants."
Wall said decisions on who could become marriage celebrants were entirely up to Montgomery.
"It's got nothing to do with my bill, it's about how he is determining who is a suitable person to be an independent celebrant," she said.
Justice Minister Amy Adams, who is in charge of the Marriage Act, said the issue had not been raised with her until now. "I've asked my Justice officials for advice," she said.
