After arguing for their lives, they were allowed back to their rooms to collect clothing with the patch attached, where they tried to put them back on with safety pins.
"They all laughed at us," Vadja told ABC radio before the ceremony. "We were marched about 600 yards down the street to a military barracks, and lined up in front of a machine gun in front of a wall.
"Suddenly civilians arrived and we were told it was Wallenberg, then the atmosphere changed and we were taken back to a protected house.
"Everyone looked at us as though we had come back from the grave. It was really unbelievable."
No one knows how many others were saved by Wallenberg, but estimates run as high as 100,000.
The 34-year-old special envoy used bluff and false documents to confer Swedish citizenship on condemned Jews in a desperate race against the Nazis' accelerating efforts to slaughter as many as possible as Soviet troops neared Budapest.
Shortly after the Hungarian capital fell in 1945 Wallenberg was arrested for reasons that remain unclear and sent to Moscow's notorious Lubyanka prison, where Soviet officials declared in 1947 that he had died from heart failure.
However, doubts persist, with reports he survived as late as the 1980s.