"We ran out the front door, the car was parked in front of the house in the driveway. I opened the door and discovered a rather big bloke was in the driver's seat," he told Yahoo News Australia.
"We just started screaming at him as loudly as we could non-stop to get out. He told us to 'shhh' and we stopped and thought 'huh? What are we doing?' And resumed screaming."
Rice added that the man even had the nerve to ask them for a lift, which the couple refused before demanding the keys to the car back.
He returned their keys and told them he hadn't stolen anything else.
The newlyweds had gone from a "complete state of relaxation to unbelievable chaos in seconds", with Rice adding that he was happy the incident didn't unfold 10 minutes earlier.
The couple reported the man to police, who arrived at the scene promptly.
After the incident, the couple said they were "pretty rattled" and would ensure to keep the backdoor locked during the day.
However, they do see the funny side of it, revealing that they got to meet their neighbours for the first time in six months since they moved in.