The 19-year-old, who has amassed career earnings of more than $10 million, said it was difficult hearing people doubt her loyalty to New Zealand.
"I never thought about [switching allegiances]. I've represented New Zealand at the world amateur championships and a lot of amateur events overseas, so it was a little tough for me to hear the doubts," Ko told She's Got Game.
"That seems to have settled down now and hopefully everyone can trust now that me and my team are [committed to New Zealand].
"I'm proud to have the New Zealand flag on my bag and to represent New Zealand at an Olympics."
Ko heads to Rio - the Games officially open on Saturday but New Zealand's women's football team play today - fresh off a disappointing performance at the British Open this week, where she finished in a tie for 40th, 15 strokes behind winner Ariya Jutanugarn of Thailand.
Lydia Ko on the British Open:
The result blemished Ko's remarkable record in majors, after finishing in the top three in the past five tournaments, winning two of them and losing in a playoff at the Women's PGA Championship in June in Washington state.
While disappointed with her outing at Woburn, Ko says it is important she doesn't dwell on the performance and allow self-doubt to set in as she prepares for the biggest sporting spectacle in the world.
"When I'm on the course I try to play aggressive and confident, but at the same time play smart - and if you're not playing well, try and be okay with it. If you beat yourself up next time you have more doubts about yourself."
Women's golf will be among the last batch of medals awarded at the Rio Olympics, with the final round to be held on the penultimate day of the Games.
For the full interview with Lydia Ko, catch She's Got Game from 10am.