Senior Sergeant Gael Mockford said the body was reported to police by a member of the public on Sunday afternoon.
Police were in the process of identifying the body.
Security camera images taken at the hospital showed Mr Smith walking away from the entrance on Pembroke St on March 16.
His whanau members from Kawhia, Waikato, Bay of Plenty and Auckland scoured the city in a vain search for him.
His brother, Steve Smith, said the body that was found was that of his brother and that he had fallen about four metres down a steep bank.
"It is sad, but it is closure for the whanau," said Mr Smith.
"It has been three weeks to the day that he went missing - it was pretty near to the hour when they found him.
"It's not a good outcome obviously, but it is closure."
Mr Smith said that after police had finished their formal identification process, his brother would be taken back to Rakaunui Marae in Kawhia for his tangi.
He said he was grateful to whanau members and strangers who had helped search for his brother. additional reporting APNZ