Under New Zealand regulations, any ship heading from New Zealand to Antarctica needs permission from New Zealand authorities.
Prime Minister John Key said he was concerned about the potential danger.
"People take their own risks, but it's a highly risky environment and we would want to make sure that they do everything they can to preserve their safety.
"That's an area where you need authorisation to go, and we believe the Norwegian sailor doesn't have the right authority. We're concerned about what safety provisions might be on the boat and the risks to the New Zealander that's on that boat."
But asked if the Government had considered rescuing the New Zealander from the ship, Mr Key said: "I don't think we'd go that far."
The Foreign Affairs Ministry is providing consular advice to the New Zealand man's family, though it is unclear whether the New Zealander has asked for assistance.
Mr Andhoey told Norwegian media that he planned to sail to McMurdo Sound to find out what happened to his three crew who died when the yacht Berserk sank last year.
He wanted to leave a wreath and bottles with messages in them.
Berserk at the time was waiting for Mr Andhoey and his companion Samuel Massie, who were attempting to cross the South Pole on quadbikes.
They were rescued from the ice and faced severe criticism in the aftermath.
New Zealand officials served Mr Andhoey with a deportation order on January 23, after he failed to declare that he had been deported from Canada when he arrived in New Zealand.
He left Auckland the following day.
Norwegian media reported it may be another week before Nilaya reaches Antarctic territory.