Mr O'Connor also said he was worried about "single issue" groups trying to capture "naive" Grey Power members to promote their cause.
Mr O'Connor told the Northern Advocate his discussion with Mr Gray got a bit heated.
"I called them to let them know I wasn't interested in them using our body to further their legalise cannabis programme. Medicinal cannabis is totally different from legalising cannabis for recreational use," Mr O'Connor said.
"We will not support legalising marijuana for recreational use."
He said the body had not considered the issue of medicinal marijuana, but if one of its independent federations brought a remit to consider the issue to a national conference then it would be, but he stressed the science had to back it, and it would then be up to the members to vote on.
"We are aware of a growing trend for activists and single issue groups to attempt to use the Grey Power brand to promote their causes. We get all sorts of weird and wonderful ideas popping up from time to time from the fringes of society but we are very careful about who and what we support nationally," Mr O'Connor said.
Mr Gray said he believed Mr O'Connor was attempting to censor it's membership and override their wishes. Mr Gray said his party and other cannabis organisations had not even lobbied Grey Power Otamatea.
"It is obvious that Beverley Aldridge has done her own research and come to her own conclusions about the science around medical cannabis. Mr O'Connor should be standing by his original statement that local chapters were entitled to take on their own causes," Mr Gray said.
"Mr O'Connor seems to think his members are feeble minded when he said 'a lot of our members are very old and naive and they can be easily taken in by causes that aren't a good look for Grey Power'."