Richards said the move came "absolutely out of the blue" and he alleged that Holder had breached franchise agreements that had "years to run on them".
Holder this afternoon issued an email to members, through public relations agency Acumen Republic, in which he apologised "for any concern or confusion caused to you during the change-over".
"As background, on Friday 8, February we regrettably had to take steps to end our franchise agreements with Club Physical in Westgate, Botany and Three Kings because we believed the franchisor had not been meeting its obligations to us and that this was putting the future of the clubs - your clubs - in jeopardy.
"This wasn't a decision we took lightly, but we had to do this in order to offer you the best possible fitness experience.
"Subsequent to us cancelling the agreements, the franchisor has threatened legal action against us."
Richards has confirmed that Club Physical has entered an application to the High Court this afternoon seeking an injunction to block Jolt Fitness from operating.
The letter said all members have been transferred to Jolt Fitness and those who wished to cancel their Jolt Fitness memberships would be dealt with "on a case by case basis".
Richards said members who wanted to keep working out at Club Physical could transfer to another branch at no cost.
"We're saying 'you can use all clubs while this is going on' but I don't know how long we can afford to make that offer. These sorts of things costs heaps of money."
Rachel Sturges, who signed up to the Three Kings Club Physical ten days before it changed to Jolt Fitness, said she was given no warning the rebrand was coming.
"When I signed up, they must have known it was going to change but they didn't say anything. The whole thing's been very badly done."
Sturges said she signed for the Club Physical gym specifically for certain classes which have since been scrapped.
She said she had spent the week trying to get answers from staff about whether she could cancel her membership.
"Nobody down there is giving any answers. There were other members there saying 'we don't know what's happening either'. It feels as though the staff don't even know what's going on.
"I might stay, but only because the location's really convenient to me."
Other affected members took to Facebook out of frustration.
"When I got the text message saying 'business as usual' last week I assumed the same classes would still be on, but unfortunately they aren't," said Hayley Sharp on Jolt Fitness's Facebook page.
"Will you be bringing back pilates, hot hula, flex, or any of the other Group X classes? They're the primary reason I joined CP Westgate in the first place, and the current jolt classes don't appeal to me as much."
Richard Beddie, head of Fitness NZ, issued a general letter yesterday stating the importance of gym franchises honouring their contracts.
"We believe allowing franchise operators to break franchise agreements without due regard for contract law would be both without precedent, and more significantly, damaging for our industry."
Beddie said he has not seen the contract between Club Physical and Jolt Fitness, and therefore could not say whether it had been breached, but his main concern right now was for affected gym members.
"These people have joined Club Physical for a number of reasons. They may have joined for the location, which hasn't changed, but they may have joined for the classes and I understand the classes have all changed now."
He said it he would be very surprised if Jolt Fitness refused to let members end their contracts.
Holder told members in his letter today he could not comment further on the matter while a legal process was underway.
"We are disappointed that this process is causing distress and confusion to some of you and we are anxious to resolve this situation as quickly as possible," he said.