The spokesperson said to date information given to the Government department indicated it was a "food quality" issue.
"At this stage, there is no 'food safety' issue. There is no risk of people becoming ill," he said.
"MPI is closely monitoring this situation and will be working with the official verifier and manufacturer to ensure company systems are adequate to manage any food safety implications."
The milk supplier has not been identified.
Today Countdown spokesman James Walker said as a result of four customer complaints the affected milk was removed from select North Island stores.
The supermarket advised the supplier, who investigated immediately.
"Our supplier identified the issue and resolved it over the weekend.
"They informed us the concern with the product was an issue impacting taste, and it is not a food safety issue," he said.
Customers who bought a bottle are being urged to return it for a full refund or replacement.
Some strong reactions to the milk have been shared on the Herald's Facebook page:
Kelly Ann Harre: Not surprising! We were buying it but I was constantly pouring it down the sink because after being open for more than a day it stunk and tasted like it was off. It never got close to reaching expiry. It would look, smell and taste rotten 3-5 days before expiry. Good call Countdown Supermarkets
Jasmine Mann: No!!! I had this. I spat it out as it tasted sooo yuck! I can't believe it