The soldier's family are being supported by members of the New Zealand Special Operations Force and the New Zealand Army.
An Auckland Rescue Helicopter was sent to a vessel near Channel Island about 6.30am, a spokesman said.
Maritime New Zealand said it was reportedly a "medical incident" but the Defence Force announced this afternoon Taylor had died during an exercise.
A witness at Port Jackson in the Coromandel told Radio New Zealand this morning that he had seen a soldier brought ashore there in an inflatable boat, accompanied by about 15 people in black, some of them armed.
The witness told the broadcaster he had been told the soldiers had been training on a boat off Channel Rock and that the soldier had fallen.
He described how ambulances arrived and he saw paramedics work for about 90 minutes in an attempt to revive the soldier.
The Defence Force said earlier that the soldier's next of kin have been informed and a family liaison officer assigned. Police have also been informed and all inquiries are being directed to them.
A police spokeswoman said police were made aware of a death off the Coromandel Peninsula this morning, and will be making inquiries on behalf of the coroner.
'
Defence Minister Mark Mitchell has extended condolences to Taylor's family.
"I offer the Government's most sincere sympathies and my own to the soldier's family. Our thoughts are very much with you at this terrible time," Mitchell said.
"No words can adequately describe this sudden and tragic loss. Your family member was a man who commanded high respect amongst his fellow soldiers and embodied the Army's core values of courage, commitment, comradeship and integrity."
Mitchell said the tragedy would be felt by the wider Defence Force, particularly others in the Special Operations Force.
"As Minister I've been supported by the SOF and have witnessed their professionalism on operations and in training. They take on difficult and demanding tasks and regularly accept significant risk. They do so to serve and protect their fellow New Zealanders.
"I understand there will be a high level of public interest and outpouring of sympathy for the family, but I ask that everyone give family, friends and colleagues the time and space they need to grieve."
The RSA has also paid tribute to Taylor.
National RSA spokesman Hamish Stewart said any death of a service person is devastating to the organisation.
He couldn't comment on whether the RSA would be in touch with Taylor's family.
"In this instance we'll follow the lead of the Defence Force", he said.
The special operations force (NZSOF) is described by the defence force as "highly capable", and says it works to pre-empt threats to New Zealand domestically and aboard. The force includes the SAS.
In 2012, Private Michael Victor Ross from Kaitaia died during a training exercise at Lake Moawhango, in Waiouru.
The Court of Inquiry found his death was accidental, but a lack of adherence to safety procedures contributed to Ross falling overboard and drowning.
On August 13, 1990, six young men died on an army training course on Mt Ruapehu.
A Military Court of Inquiry found the main cause of tragedy was the inadequate skill and experience of the two instructors.
Tina Grant, widow of SAS corporal Douglas Grant who died in a firefight six years ago in Kabul, Afghanistan, said she couldn't comment but she did offer her deepest condolences to soldier's wife and children.
"My heart, my thoughts, everything go out to his wife and his family and especially the boys that he was working with. I have been sitting here feeling deeply saddened for his wife and family, because I know."
She confirmed the squad focused purely on counter-terrorism activities within New Zealand waters, whereas the SAS is internationally based.
"So these gentlemen do not all the same training as our special forces guys because obviously they're just New Zealand-based and they do the counter-terrorism stuff."
Grant, who works for the Defence Force as a liaison officer for families of the fallen, said a solider didn't have to be killed in action for it to be a shock; any sudden death leaves the family in pieces.
"Because it's a sudden death and you grieve, in my opinion, it's the worst because you don't get to do the finish-off, you don't get to do the final goodbye, that sort of stuff. Whereas if it's a terminally ill situation you can put things in place, you can discuss the finality of things and say your goodbyes, but when it's a sudden loss there's no finish. So for me, that was the hard part."
However, she said the Defence Force "did an amazing job" of supporting grieving whanau when one of its staff or soldiers died.
"They're a bloody good employer."
