Mr Key said this afternoon he was powerless to do anything if a defiant Mr Gilmore said he would not resign
But Mr Key said he had not lost control of the MP after Mr Gilmore defied his suggestion to resign.
He told media this afternoon that he still could not reconcile Mr Gilmore's word with the events at a Hanmer hotel last week.
"Mr Gilmore has to be able to look the New Zealand public in the eye and say his version of the events that actually occurred.
When told the MP would not resign, Mr Key said: "It's in his hands now".
"He can make his own decision."
This morning Mr Key signalled strongly Mr Gilmore should quit.
He told Radio Live he had not received an honest answer from Mr Gilmore about the dinner at the Hanmer Hotel where he allegedly told a barman who refused him alcohol he would have the Prime Minister fire him.
When Mr Key was asked if the backbench MP should quit, he said: "If he can't reconcile what happened on that night in the way that he's described it to me, then the answer would have to be 'yes'.''
While interviewed on Radio Live he described Mr Gilmore's story as being like "quicksand''.
The matter was yesterday referred to National Party president Peter Goodfellow, suggesting Mr Gilmore won't be a runner for the party list prior to the 2014 election.
Mr Key said this afternoon the MP was a "distraction" from what had been a good week for National, with the float of Mighty River Power and improved job numbers.
"The vast bulk of Kiwis would say: 'Who cares about Aaron Gilmore?'."