Western tourists were among hundreds of people inside the building who realised it was on fire only when they saw burning debris falling past windows and clouds of smoke outside.
Irishman Kenneth Flynn was having a drink in the second floor lobby with his girlfriend at 9pm local time when she noticed something falling outside.
He said: "I went outside to have a look and by the time I walked out the whole building was just ablaze. The heat was so intense outside.
"It was unbelievable. By the time I came back in I had lost her because the whole place was in panic and people were crushing each other trying to get out and get down the stairs and jumping over railings. I couldn't believe how fast it actually happened."
"There was no fire alarm. I could hear no fire alarm going off."
"I'm not going to say I didn't panic myself, I was absolutely terrified to be honest," Flynn said.
"I've never experienced anything like it. It was surreal."
He managed to find his partner and they got out safely.
A number of guests told the Washington Post, however, that the evacuation was well organised.
"I was on the 45th floor, the fire was on the opposite side.
"However, our room was full of smoke," said one 47-year-old tourist who did not want to be named, adding that the "evacuation was extremely professional" and carried out by hotel security staff.
Irish singer Anita Williams, who was performing at the hotel when the fire began, told the BBC that people left in a "stampede".
"We left everything. There was debris falling down. It [the fire] just shot up through the entire hotel. Everybody was screaming, everybody was running ... I thought: 'This is a film'."
BBC correspondent Nadia Huraimi watched the blaze. She said: " I knew right away it was the Address Hotel in Dubai's downtown district that overlooks Burj Khalifa.
"I also knew that it was fully booked five years in advance.
"There were also thousands of people in the streets below and in the adjacent buildings, all waiting to view the highly anticipated New Year's Eve fireworks display."
About an hour before the planned midnight fireworks, Dubai's media office said that about 90 per cent of the blaze was under control and that the New Year's Eve celebrations would go ahead. The fire came as cities including Brussels and Paris cancelled New Year celebrations because of terrorist alerts.
It was reported to have started on the 20th floor, and quickly spread up the outside of the building, which one witness said went up "like paper".
Internal fire extinguishing systems appeared to have worked, giving everyone inside the building time to get out alive and saving the internal structure.
The 302m building, completed in 2008 at a cost of US$230 million ($336.8 million), has 196 hotel rooms and 626 serviced apartments and is part of the same Downtown Dubai development as the world's tallest building, the Burj Khalifa.
The Burj Khalifa was at the centre of Dubai's New Year firework display, which went ahead despite the fire.
The Address Downtown had been advertised as "the perfect place to watch the New Year's Eve fireworks".
Last year the Emirate broke the world record for the largest ever pyrotechnic display with more than 400,000 fireworks being set off during a six-minute show which ushered in 2015.
BBC World Service Middle East editor Sebastian Usher said the fireworks display was "a huge prestige event" for Dubai and that authorities would want "the images that people look back on next year to be of the fireworks - and not of the blaze".
