"We asked organisers if the Speaker or New Zealand's Ambassador to Israel could attend in her place, however this was unfortunately not possible," Peters said this week.
However, his office has now confirmed it was only made aware of the invitation by the Ministry of Foreign Affairs on January 16, a week ago.
"At [this] point all efforts were made to arrange the attendance of the Speaker. However, this was unfortunately not possible," a spokeswoman said.
In a statement, an Mfat spokesperson confirmed the delay and said the department had known since September.
"Visibility of the invitation, which was considered by the Ministry in September, and how the Ministry was handling it should have been provided to his office much earlier."
The Herald understands Dame Patsy indicated to Israeli officials she would be unable to attend several months ago, instead sending a letter that has been described as touching and inspiring by Israel's ambassador to New Zealand, Itzhak Gerberg.
It was only after Peters learned of the situation last week that officials scrambled to see whether Parliament's Speaker, Trevor Mallard, could attend instead - leaving the organisers just days to try to make room.
Peters earlier told Radio NZ Israel had cited security issues as the reason for declining, but could not say what they entailed.
The Herald understands they potentially related to the timeframe.
Mallard earlier told the Herald his bags were packed on Tuesday night to fly out to Israel.
"I got the message to unpack. Apparently there was something to do with security," he said.
"You can't go to commemorations unless you've got the total support and backing of the people who are hosting you."
National Party foreign affairs spokesman Gerry Brownlee has called New Zealand's absence embarrassing and disgraceful.
He said he didn't accept that officials had failed to tell Peters.
"It beggars belief. I cannot believe foreign affairs officials would not have put that to the foreign minister," Brownlee said.
"It is simply not a credible thing to have happened."
In lieu of attending, Dame Patsy has sent a written letter to the forum, discussing her visit to Yad Vashem, Israel's official memorial, in 2017 and her sadness over New Zealand's rejection of some Jewish refugees in the 1930s.
"It is only through a deep understanding of the lessons of history that we can ensure atrocities such as the Holocaust never happen again," she wrote.
"May your conference reinforce the message of diversity and inclusion to which we in New Zealand are firmly committed."
New Zealand's Ambassador to Poland is, meanwhile, attending a commemoration event at Auschwitz.
Prime Minister Jacinda Ardern is speaking at a Holocaust Remembrance Day event in Auckland next week.
