He began playing them, then making them, and now he makes all his own instruments.
"It's better if you make them yourself, because it's in your control. And there aren't that many people who make them, anyway."
Mr Kavanagh said he could never choose a favourite among his taonga puoro.
"They're so personal. That's like asking me to choose which one of my children is my favourite - I like all of them for their own reasons."
"When I was overseas and I was homesick, my taonga puoro were like having my whanau with me."
His daughter was born to the sound of the taonga puoro - Mr Kavanagh played them during her birth - and his son has a few instruments of his own and is showing an interest in playing.
Mr Kavanagh, a former te reo Maori teacher, now spends his time playing gigs - he has a band called Hui-A - and teaching workshops throughout the country.
He said he particularly enjoyed teaching teenagers how to carve the musical instruments.
"Teenagers are always using their hands to text on their phone or play on their computer. This is using their hands - but when they've finished they get something out of it that they've made themselves.
"It's really hard work, carving the instruments, and I think they enjoy that," he said.
Mr Kavanagh is unassuming about his music, but his achievements are impressive.
He collaborated with American composer Chris Tin and producer Ben Mullon on the album Calling All Dawns, which won a Grammy Award for Best Musical Crossover in 2011.
Later this month, Mr Kavanagh will travel to London to take part in a special indigenous festival that will coincide with the Olympic Games. He will give a lecture at the British Museum about his taonga puoro, and a performance will be filmed by the BBC.
"I'll be playing the taonga Captain Cook brought back to England with him, so I'm looking forward to that."
Tonight Mr Kavanagh will perform with his band, Hui-A, 7pm, at the Whanganui Regional Museum. He will also be holding workshops with Wanganui schoolchildren at 9.30-11.30am and 12.30pm-2.30pm.