Haylett-Petty said the team is burning at having to wait another seven months for their next Test, against Fiji in Australia, after losing 37-21 to England at Twickenham.
"It's definitely frustrating, it would be nice to cap off the year and get some reward for all the hard work we've put in," Haylett-Petty said.
"I do think we're one of the hardest-working teams out there.
"So it is frustrating but gives us an opportunity to now look at what we need to work on and it gives us a lot of time to work on it for June next year."
Haylett-Petty was controversially yellow-carded late in the match by referee Jaco Peyper for a late hit on English fullback Mike Brown, though it should only have been a penalty.
"I don't know what I can say, obviously a bit disappointing, we were still chasing the game," Haylett-Petty said.
"I felt like probably a penalty, bit rough. Yeah, I was definitely disappointed."
Wallabies in 2016
Lost to England 28-39
Lost to England 7-23
Lost to England 40-44
Lost to All Blacks 8-42
Lost to All Blacks 9-29
Beat South Africa 23-17
Beat Argentina 36-20
Lost to South Africa 10-18
Beat Argentina 33-21
Lost to All Blacks 10-37
Beat Wales 32-8
Beat Scotland 23-22
Beat France 25-23
Lost to Ireland 24-27
Lost to England 21-31
Played 15
Won 6
Lost 9
For 329
Against 407