New Zealand, like many other countries, grapples with how it balances its relationships with China and Taiwan.
China is New Zealand's largest trading partner, but Taiwan is a like-minded democracy and leading chip maker.
China claims Taiwan as part of its territory after it separated from the mainland after a 1949 civil war.
But US President Joe Biden has said multiple times his nation would defend Taiwan if China attacked it.
Tensions ratcheted up last month when US House of Representatives Speaker Nancy Pelosi visited the island nation and China responded by launching a massive naval exercise around Taiwan.
New Zealand recognises China's "one China policy" and so does not have formal diplomatic ties with Taiwan.
Peters told guests that Taiwan - with few natural resources, a small land area, and population of 24 million people - had in a short time become a world leader in advanced economic activities, according to a statement published by President Tsai.
He also called for co-operation and courage among the nations present - which included former officials from nations, such as Thailand, India and Luxembourg - to be a force for good in the world.
President Tsai thanked those present for pushing for Taiwan to play a greater role in the international community.
"I want to thank you for your longstanding attention to peace and stability across the Taiwan Strait, and also for supporting and speaking up for Taiwan's participation in the international arena," she said.
Peters also told guests the world would be a better place if Taiwan was left in peace.
"We are saying, if left peacefully to continue, we – the rest of the world – will be the beneficiaries, and go on being the beneficiaries, of Taiwan's success," he said, according to media outlet Stuff.