The Brumbies had taken a 16-15 lead at halftime, but the Sharks set themselves up for victory early in the second half by piling on two tries through Willem Alberts and Lwazi Mvovo.
In the end, it was enough to secure them victory in a game where the lead changed seven times.
The game was tight from the outset, but the Sharks were the first to draw blood when fullback Riaan Viljoen scored controversially on the wing in the 22nd minute after appearing to knock the ball on.
But the Brumbies were quick to wrestle back the lead - and they did it in style.
Jesse Mogg pulled off the try of the night after leaping onto a teammate's pass a good 50m out and then delivering a kick-and-chase that bounced perfectly over Odwa Ngundane's head and back into Mogg's hands just before the line.
Viljoen crossed again in the 35m minute to give the Sharks another lead, before the Brumbies stole it back before the break courtesy of a Christian Lealiifano penalty.
The Brumbies finished the night with more injury concerns, with Matt Toomua re-injuring his knee and Henry Speight also limping from the field.
-AAP