Geoscience reported the earthquake had a magnitude of 5.3, down from their earlier estimate of 5.5.
It was almost 10km deep with an epicentre between Trafalgar and Moe in the southeast.
Anthony Atkin, the duty manager of the Criterion Hotel at Trafalgar, just west of Moe, said he feared for his life when the quake hit.
"I thought the roof was going to fall down," Mr Atkin said.
"Everyone in the hotel ran outside, it was like a train was coming through the hotel.
"I've been here for 44 years and never felt anything like it."
He said the quake lasted for around 20 to 30 seconds "but it felt like a lifetime".
He said alarms had been set off in many shops in the town.
Moe police constable Chris Hand was enjoying a cup of coffee when he felt the ground moving.
"I had the cup sitting on the table and it spilled over," Const Hand told AAP.
"We've had a lot of calls but no one is reporting any damage or injuries at this stage.
"It was a decent-size rumble, the biggest I've ever felt.
"The ground moved for 15 to 20 seconds."
State Emergency Service spokesman Lachlan Quick said there had been about 30 calls for help, mainly in Gippsland, but no reports of injury.
"It's been mostly low level building damage - a chimney collapse in Moe, a garage has collapsed, some cracked walls but nothing major," he said.
In the state's northeast Bright resident Mandy Ditcham said she felt her house shake just before 9pm and initially thought it was high winds.
"I was sitting by the computer and I felt the ground shaking," Ms Ditcham said.
"I thought it must have been really windy outside. It was making the house shake... it lasted for about 10 seconds."
Phone lines and the internet ran hot as the tremor was felt all over the alpine area from Wangaratta to Harrietville.
In Melbourne, millions felt the impact.
Ron Smith from Kew said his whole house shook.
"We heard a roar and thought it was the wind," he told AAP.
"The floor and the walls shook, the whole house shook."
Jan Turnbull of East Bentleigh was in bed when the room started moving.
"My lamps were shaking and the windows were rattling, it was quite strong," she said.
Victorian MP Bruce Billson told a late-night sitting of federal parliament that members of his community were shaken and puzzled following the quake.
"There are plenty of people with shaken nerves, wondering what was going on," Mr Billson told the chamber.
"To our community, our thoughts are with all those responding to anxious calls and hope there is no injuries or substantial damage."
- AAP
