Dan Fridd scored twice and Scott Saurin once for the hosts while former Central League striker Andy Pickering claimed both the goals for the visitors.
Said player/coach Andy Jeffs: "Dan scored with 30 seconds to go so we're very happy.
"It's a very big upset and we fought really hard.
"There's no way we should have won the game because we had just the bare 11 players," Jeffs said.
The Rovers are second on the premiership table.
Rangers (D1) 2 Rangers Prems 0
Is it a derby when two teams from the same club meet each other rather than different clubs in the same city?
It hardly matters because there's no running away from the fact that the Western Rangers' first divisioners sent their premier team packing at St Leonard's Park.
Scoreless after regulation time, the two goals came in the second spell of extra time in the 2-0 victory to the minnows.
Luke Woon broke the deadlock in the 109th minute after latching on to a defensive blunder and Dylan Morgan added another in the 118th in similar circumstances, according to premiership coach Ritchie Howard.
"We had a goal disallowed for offside in the last 10 minutes of regulation time, which was the only clear chance in the game," Howard said.
Vale 1 Eskview 0
Maycenvale United FC are the only other premiership side still in the hunt after pipping division-one campaigners Tremains Eskview United 1-0 at Akina Park, in Hastings.
Eskview, who are the div one league champions this winter, succumbed to a goal from Jacob Rosandich following a freekick in the second spell.
"It was a cup classic. Standouts for us were Campbell Whitworth and Hamish Carruthers who led a back four who have only conceded once in the last four games," Vale spokesman Ethan Dent said.
Losing coach Tony Bartlett said it was a trying game in the wind.
"We missed two really good chances but [Vale goalkeeper] Shaun Peta was brilliant," Bartlett said, adding they were "all over" the hosts in the second spell with the set-piece goal the difference.
Bartlett said Vale thoroughly deserved the win, considering they had the bare 11, although his men let themselves down.
It remains to be seen if Eskview will try to regain their premiership berth after they were relegated several seasons ago.
"It's been 21 years since the club won a trophy," Bartlett said, revelaing he intended to lodge a report to his club committee in a bid to gain promotion to the prems next season.
Eskview have only lost 3-2 to Taradale Vikings this season and drawn two matches.
Bartlett, who has nine league titles, including Central League and two div one bragging rights, predicted Marist would win the HB Knockout Cup this year.
-In the only premiership league game, defending champions Your Solutions Taradale AFC predictably overwhelmed Hastings Old Boys 6-0 at Taradale Park.
-In a Lotto Federation League match at Guthrie Park, Cole Murray Group Havelock North Wanderers beat Palmerston North Marist Reserves 4-1.
The Bruce Barclay-coached villagers are third on the table on 26 points, a dozen points adrift of leaders Wanganui Athletic while former Central League campaigners Team Taranaki are perched on the second rung on 33 points.
