The issue was the Maori broadcasting funding agency, Te Mangai Paho, for which he was responsible.
The Opposition dragged up disturbing examples of mismanagement, but as minister he seemed incapable of giving plain answers in Parliament.
Horomia labelled the attacks racist. Winston Peters said it was inverse racism for the Prime Minister, Helen Clark, to be tolerating such a poor performance from a minister.
Horomia required close political management by Helen Clark.
Horomia stalled further questions about Te Mangai Paho until his fix-it man, Wira Gardiner, had sorted the place out.
Those were very difficult days for Horomia and his party may have wondered why Clark had shown such confidence in him that she insisted on a smooth selection process from the party in 1999.
But then along came the Court of Appeal's foreshore and seabed decision, opening the way for iwi to claim customary title, the very same month that Te Mangai Paho was blowing up.
And in many ways the foreshore and seabed was the making of Horomia and thrust him into a new leadership role.
Instead of being the incoherent minister who couldn't answer questions in the House on broadcasting funding, he became a pivotal player in one of the most complex and potentially divisive issues to have confronted a modern New Zealand Government.
Michael Cullen and Helen Clark drove Labour's response and Maori responded to them with a large hikoi.
But Horomia's extensive personal networks in Maoridom were crucial to keeping the channels open, and while he could not prevent Tariana Turia from leaving Labour, he clearly remained a true friend as evidenced by her heart-broken tribute to him.
Retaining his seat in 2005 against Turia's new party increased his standing in the Labour Party.
Horomia's third and final term in Government was nowhere near as controversial as his second term.
Then he spent the last four years of his life in Opposition, which can be as controversial as you make it. He didn't make much of a fuss in Opposition.
Not quite like his going.