Head coach Colin Batch said it was a disappointing finish to what had been a great game from New Zealand.
"We came into this game with a lot of confidence and played really well up until the final six minutes, which is hugely disappointing," he said.
"We knew the Germans were going to fight back, they have won the last two Olympic gold medals and have the ability to score right at the death.
"I think we are good enough to be in the semis so to be knocked out like that is hard to take. We surprised a few people here in Rio and finally got it together as a team so it's hard to be cut short.
"The guys are hugely disappointed, mostly because we played such a good game and deserved our lead.
"To finish like that is a bitter pill and something that will hurt for a long time."
New Zealand opened the scoring in the 17th minute when Simon Child went on a run and linked up with Hugo Inglis, who jinked his way into the circle and slipped the ball over the diving goalkeeper.
Veteran defender Shea McAleese doubled the lead in the 48th minute when he got a great deflection on a penalty corner set piece.
Germany lifted the intensity after the McAleese goal and put immense pressure on the New Zealand defensive line, in particular goalkeeper Devon Manchester.
Moritz Furste put the Germans on the board four minutes from fulltime with a well-worked penalty corner flick, before levelling the score at 2-2 three minutes later with another drag flick.
Germany dealt a cruel blow when Florian Fuchs scored the winning field goal with just one second remaining on the clock.
