The Brumbies were given every chance to set themselves up with plenty of ball in the first half against an ill-disciplined Waratahs, who gave away four penalties in the opening four minutes and lost Will Skelton to the sinbin for a dangerous high tackle.
Christian Lealiifano's hot-stepping try put the home side up before Folau got the Tahs on the board with a typically powerful charge. A second Waratahs' yellow card, this time to Dean Mumm in his 100th match for NSW, didn't hurt them as they went to the break level at 8-8.
Things heated up with an all-in melee that ended with a bloody - and possibly broken - nose to Matt Toomua.
It only served to fire up both sides, with a five-pointer to Joe Tomane for the Brumbies quickly balanced out by Phipps' slick try.
A penalty try from a scrum with 12 minutes left gave the hosts a 10-point buffer, and ultimately sealed the game after the Waratahs were denied late by the TMO because of an obstruction.
Nigel Ah Wong capped it off for the Brumbies with a final-minute try.
The Waratahs enjoy a bye next week, while the Brumbies travel to Perth to face the Western Force.
Larkham was impressed that his side found a way to claw back.
"There was a lot of pressure applied at the breakdown for both teams but we backed ourselves to find the space in attack - in previous years we probably wouldn't have done that," he said.
"It was a good little victory." He praised his "unbelievable" front-row scrummagers, which has been a focus of Larkham's in recent years. Waratahs coach Daryl Gibson admitted the Brumbies won the battle of the scrum - aided by a few "rough calls".
"I felt overall we really showed some good passion and real desire and commitment," he said.
"What we lacked was a little bit of poise at crucial times and coming away from the area with points, in terms of accuracy - particularly at the breakdown.
Gibson confirmed Hegarty had a suspected ACL injury, while Polota-Nau refractured his arm and Horne hurt his knee.