"While Danny was very much looking forward to once again representing New Zealand and contesting for a medal in Tokyo, he has made the hard decision to focus his attention on retaining his playing rights on the PGA Tour for the 2021/2022 season."
As it stands Ryan Fox and Lydia Ko will be the only New Zealand competitors in Tokyo.
Lee competed at Rio in golf's return to the Olympics, finishing 27th and plans to attend the next Olympics in three years time.
"Danny has always taken great pride in representing his country and looks forward to doing so again in the future. While he has one eye on the 2024 Games to be played at La National in Paris, France, his current focus will be to retain his playing rights and secure a 2021/2022 PGA Tour Card and representing New Zealand proudly on the worlds leading Tour," Golf New Zealand said.