The competition was set up in a knockout format, with winners from each fight progressing to the next round.
Warren went up against the Hungarian champion in his 70kg class, making history with the result.
"I was a bit nervous at the start but once we shook hands I got used to it," he said.
"He was a very strong guy but I stopped him in the first round with a body rip, and his coach threw in the towel."
The knockout made him only the second Kiwi to win a fight at the world champs.
However, his second fight did not go as well, with Warren losing on points to the Australian champion following their three-round bout.
"It was frustrating for me, just his style of fighting. He was short so he kept wanting to come in," he said.
"If I had won the fight I would have got into medal contention."
He said although it was a frustrating fight, he was still happy to have won his first bout.
"It was overwhelming, even for the team. I think they were quite shocked because I was the only one who won a fight out of the Kiwis," he said.
Out of the 54 countries competing, Warren finished with a ranking of 10th overall in his weight class.
Warren said the home favourite from Ukraine won the 70kg weight class overall.
The tournament had given him more drive and ideas to continue to grow as a boxer.
"It has motivated me to learn different techniques and to teach other people at the gym as well," he said referring to the Rotovegas Boxing Gym.
"Over there they all swap stances and fight unorthodox. It sounds strange but it seems to work, so my dad has started training me in southpaw."
The natural right-hander trains at the Rotovegas Boxing Gym, which is attached to his family's house in Owhata.
His long-term goal is to compete at the 2016 Rio Olympics before turning professional.
Warren comes from a family of boxers with his father, Aaron Warren, currently his coach and his two older brothers both experienced fighters in the sport.
His father Aaron said all nine fighters from New Zealand who attended the world championships did the country proud and all lasted the distance in their respective fights.