The latest revelations emerged from the singer's daughter Nancy, who said he hit "rock bottom" during the divorce and secretly wished he could have his time again. Sinatra once said if he could turn back the clock he would have stayed with her mother, she told CBS.
Nancy Sinatra, who had a hit in 1966 with These Boots Are Made for Walking, said: "My mum had one great love of her life and that was my dad. To this day she still loves him and adores him. I asked him once, if he had it to do all over again, would he leave Mom for Ava? And he said no.
"I think he hit rock bottom pretty much at that period [the divorce], in terms of work and emotion, and it was tough. I never knew anyone, and I still don't know anyone else, who felt things as deeply and took things to heart the way he did."
Sinatra had three children with his first wife before they divorced in 1951. He then wed The Killers star Gardner but the union lasted just two years.
Despite their split, Gardner still used her influence to help the singer - securing him the role of Private Angelo Maggio in the 1953 movie From Here to Eternity.
The intervention proved decisive, salvaging his career and winning him an Oscar.
His daughter also revealed how Sinatra, who died in 1998 and would have been 100 this year, was a large baby and suffered scarring to his left eardrum when forceps were used to deliver him. He spent his life preferring to be photographed from his right side.