John Key has told Lockwood Smith he must forfeit the immigration portfolio in any National-led government as a result of his ill-judged comments about seasonal workers, the Herald understands.
Senior frontbencher Maurice Williamson has also forfeited any return to the transport portfolio after more embarrassing comments this week on road tolls.
The National leader is thought to have conveyed this message to the two accident-prone former ministers during personal phone conversations yesterday after having lost patience with them over blunders.
It is not certain they would be kept out of a Key-led Cabinet altogether but their place as ministers is no longer guaranteed - as it was before the campaign began.
It is possible the pair could be among the group of ministers earmarked for rotation by Mr Key - ministers given a job on condition they move on after a couple of years to make way for younger blood.
Dr Smith is National's longest-serving MP, having been elected in 1984, and is a former Trade Minister.
Mr Williamson was elected in 1987 and promoted back to the front bench by Mr Key after a serious falling-out with former leader Bill English.
Dr Smith's comments represent the biggest blunder of the campaign so far.
They were made in an interview with the Marlborough Express on Tuesday and published yesterday.
After speaking to winegrowers in the region, he said he wanted the Recognised Seasonal Employers scheme - which allows temporary work permits for Pacific Island workers - expanded to include Asians.
"There are some skills in the vineyard that some people are perhaps better at. For example, some of the pruning ... some of the Asian workers have been more productive ... because their hands are smaller."
Dr Smith also thought employers should not have to be solely responsible for training workers.
"Some of them are having to teach them things like how to use a toilet or shower," he said.
Dr Smith issued a statement headed "RSE comments out of context" in which he said: "Presented in the way they were, I can see that my reported comments may have caused offence. I sincerely regret that because it certainly was not my intention."
Mr Key said in New Plymouth yesterday that the apology was "wholly appropriate".
He said he accepted the apology "and I think we should move on".
Prime Minister Helen Clark called them "absolutely daft comments".
"I don't think he could have engaged brain before mouth.
"If you were going to judge who was fit to pick fruit by the size of their hands, presumably you would never pick men, only women, because generally we have smaller hands."
Maori Party co-leader Tariana Turia said Dr Smith's remarks had been "utterly racist".
"He had no right to be making comments of that nature."
She said Dr Smith was completely out of touch with real people.
"He can smell the power coming his way and he thinks that's going to give him licence to speak out as he likes about any other people. It won't be tolerated."