"We were in trouble. I was really panicking," he said.
"I was having sleepless nights. We were probably six weeks away from the beginning of the shoot, and we hadn't settled on anyone else, and I was torturing myself by watching Sherlock on an iPad at 4 o' clock in the morning."
Jackson eventually resolved the conflict by taking the "radical" step of agreeing to shut down production on The Hobbit for two months while Freeman went to film the second season of Sherlock.
He told reporters in Wellington for the premiere of The Hobbit: An Unexpected Journey. the delay was a good thing as it allowed him to review and edit the film and make adjustments.
"It's not the sort of thing you normally do on a film, but I'm incredibly pleased we did it, it's the best thing that ever happened," he said.
Jackson also explained the reasons why he felt Freeman was perfect as Bilbo, The Hollywood Reporter said.
"Martin was the only person that we wanted for that role, and that was really before we met Martin."
"We knew him from The Office and Hitchhiker's Guide [to the Galaxy], and we just felt he had qualities that would be perfect for Bilbo. The stuffy repressed English quality. He's a dramatic actor, he's not a comedian, but he has a talent for comedy."
The Hobbit: An Unexpected Journey is set to be released in New Zealand on December 14.