Mr Key padded his way in bare feet past people in various states of song, prayer or contemplation, with an army of security, photographers and officials in tow.
Some buildings had a look of Franklin Rd - the Auckland street that attracts thousands of visitors to see its festive decorations each Christmas - about them with their strings of coloured lights hung over them.
The 99 metre pagoda stands on a platform well above sea level and can be seen for kilometres.
It has also been associated with resistance over the years with now Opposition Leader Aung San Suu Kyi reportedly having addressed protests there of several hundred thousand in 1988.
It was also the rallying point for marches in the so-called Saffron Revolution when 20,000 monks and nuns led protest marches for a week.
Almost 90 per cent of Burmese are Buddhist.
Mr Key described the pagoda which is 2600-years-old as "incredibly remarkable" and a moving place.
The father of President Thein Sein became a monk after his wife died, which is not an uncommon practice.