Now it's just under 3 million. South Africa was the mohair "power house", Milne says, but political and other changes there have seen production dwindle.
However, markets remain hungry for the angora goat's fibre, which is used in high-end garments and home textiles.
Ninety per cent of the mohair produced in New Zealand is exported through two privately-owned warehouses.
Producers' output is 'pooled' and sorted into lines at buying events twice a year.
"I can't remember the last pool where the mohair wasn't all sold. It's been a long, long time," Milne says.
MPNZ's conference last year included a well-received presentation by GT Ferreira, a world-renowned mohair producer from South Africa.
He lifted the sights of local producers and in particular suggested they should put much greater emphasis on premium mohair.
Achieving 'good' vs 'premium' mohair is not a huge leap for farmers, Milne says.
"Essentially it's about matching good quality mohair with staple lengths in excess of 115mm. Sometimes it's easy for people to think, 'let's shear it now and be done with it'. But if they leave it, say, two or three more weeks they can get into the premium line. It's a matter of clarifying in producers' minds exactly what they're targeting and how to do it."
Milne says the two buying warehouses came on board with GT Ferreira's advice and put everything they could into an inaugural premium mohair pool last year.
"They achieved a 10 per cent increase in price on their first attempt. And the feedback from the buyers was that if they can get the [supply] volume up, 10 per cent is only the beginning."
The Ashburton conference includes site visits to one of the big warehouses near Rangiora, to Five-Star Beef's feed lot (farming angoras and cattle offers a number of complementary advantages), to Ashford's spinning and craft factory and Ray and Donece McEwan's angora farm.
More information for attendees to chew over will come from guest speakers David Williams, a mohair buyer from Australia, and scientist Dr Mark Ferguson.
Ferguson now works for Merino NZ on the Perfect Sheep programme but his background is in angora goats, particularly at the high profile Mallee Park stud in Australia.
Merino is the closest to mohair as a quality natural fibre. He'll talk about genetics, and how feed and nutrition affect the quality of mohair.
Milne's interest in goats goes right back to when she had a pet goat as a child.
"I'd always thought I'd like more to do with them and when Richard and I moved to a new dairy farm, we specifically looked for a place that also had land for goats."
Federated Farmers, which has been involved in advocating for the goat farming industry on resource management matters for more than a decade, has also promoted goat's competitive gross margins, the animals' advantages for controlling weeds on pasture land, and as a viable means of diversification.
Milne looks after and shears their 220 angoras herself and when the couple switched out their dairy herd for beef and dairy bull cattle, Milne felt she had time to give something back and was persuaded by the then chairman to join the board of Mohair Producers NZ.
Barely six months later, the chairman resigned and when no one else would step forward, Milne took on the role.
"It's been a rather steep learning curve," she says, in classic understatement.
Last year's visit by GT Ferreira stirred up new interest in the local mohair scene and Milne and her board want the Ashburton conference to continue that momentum.
There are fewer than 200 mohair producers in New Zealand and to grow the market, more need to be attracted in.
"We're not getting large numbers but we are getting a steady trickle of newcomers. Quite a few of them are younger and they're frequently females, who seem to get on with goats and enjoy them.
"I guess it's balanced by older ones leaving the industry. Regenerating is the word to use."
The average price for mohair at 30-32 microns is about $20 a kilo, and a good goat is doing two and a half to three kilograms every six months, Milne says.
"Our message is 'give it a go' - perhaps not with large numbers to start with; I wouldn't recommend that to anybody. But just take on a few and see what happens."
Milne says a good option is to initially run mohair wethers, "then you don't have to worry about breeding, kidding and so on but you're still doing the same dollar return. Wethers can return $100 a year just as easily as a doe.
"A wether can be run for three or four years, returning the same as a good big fat lamb, but you haven't got the work involved."
Farmers can go high end. Two producers have just imported Australian genetic animals that have been bred out of bucks from GT Ferreira.
The 90-115kg animals shear "big weights of mohair" but deserve to be treated in a way akin to a good dairy cow, Milne says.
"You need to feed them well and keep them on good country.
"But if you want to stay with the more NZ-type of angora, you can use more secondary country to run them. Just remember when it comes to shearing, you don't want to downgrade the mohair by having them on blocks where the fleece will be contaminated and stained."
- For more information on the Mohair Producers New Zealand conference, and registration papers, email dburt@fedfarm.org.nz.