David praised the 'Finding Dory' star for doing other things alongside her long-running show and admitted he should have done more himself.
He said: "I did not. All I cared about was myself. I was looking through the wrong end of the telescope."
Elsewhere on the programme, the presenter - who now hosts 'My Next Guest Needs No Introduction' on Netflix - recalled a funny incident in a hotel that left him convinced he was going to be sent to prison.
Ellen's producer Mary used to work on The Late Show and she and David were once playing catch on the 14th floor of 30 Rockefeller Plaza in New York City - only to accidentally break a window with a baseball, which flew down onto the crowds outside.
David quipped: "Well how many are dead?
"So I'm stunned and I go over to where the window is and I'm just kind of hiding and looking down there...and people were looking up because shards of glass had rain down on the Avenue of America. And I see a guy down there point up and go, 'Oh hey there's David Letterman!' and I just thought I was spending the rest of my life in prison."