"Colin has changed the face of sport in Hawke's Bay. He has been instrumental in creating a highly successful organisation focused on getting more people active and supporting the development of other sport organisations and clubs throughout the region."
Mr Stone said it was a difficult decision to move to Wellington.
"It has been a humbling experience to lead Sport Hawke's Bay through this growth and I now feel that the time is right to take on similar challenges and projects at a national level."
He said a highlight of his tenure as CEO was pulling the region's major stakeholders together for the development of Napier's Pettigrew Green Arena, which opened in 2003.
In his role at Sport New Zealand he would work closely with national and regional sport organisations.
"Sport New Zealand have just finalised a draft of a strategic new plan with a focus on community sport. It's a new system that aims to be world-leading in terms of participation numbers. I'm very excited because [community sport] is my area of expertise."
He would also step down as a board member for Horse Of The Year and Hawke's Bay Regional Sports Park.
Mr Harvey said a recruitment process to replace Mr Stone was under way, with the position advertised locally and nationally. He anticipated strong interest in the role.
"[Mr Stone] has been incredibly successful and had a massive impact on the organisation and Hawke's Bay. It will be a highly sought after role because of that."