It marked their first public appearance together since Burnham was blocked from standing as an MP in the forthcoming byelection by Starmer loyalists.
They are expected to sit on a panel together next week – another sign of their “closeness”.
Starmer’s grip on the leadership looked weaker by the hour, as Labour MPs welcomed the show of force from those thought to be moving against him.
One described it as “good” to see Rayner and Burnham out and about together, with another saying “they have always been really close”.
Meanwhile, Mandelson looked like he was preparing to fight back against the looming police investigation into whether his sharing of sensitive Government intelligence with Jeffery Epstein constitutes misconduct in a public office.
A top white-collar crime defence lawyer was seen entering the former US ambassador’s Regents Park home.
With rivals circling and allies wavering, the Prime Minister now has a choice to make.
Does he fight on, even in the face of the bleakest of outcomes, or fall on his own sword?
Those close to him are saying the Prime Minister is agonising over his own future.
In private, it is reported he is see-sawing between anger and self-reproach after acknowledging he took Mandelson’s word that he “barely knew” Jeffrey Epstein at face value. And all without properly interrogating the former ambassador, despite publicly available evidence concerning his links to the disgraced paedophile billionaire.
One Labour MP and long-time friend of Starmer said he is “absolutely devastated” about the unfolding crisis.
“He will be incredibly angry, he will accept himself that it’s his fault – because it is – and he will be regretting allowing McSweeney to be the interviewer of Mandelson.
“Keir might be really sh*t at politics but honestly, he is a very, very decent man.”
That decency may not be enough.
Polling conducted by Survation for LabourList shows a third of Labour members now want Starmer to resign as Prime Minister over the Mandelson affair.
More than three-quarters believe his chief of staff, Morgan McSweeney, should go.
Among MPs, the mood is increasingly unforgiving. One Labour backbencher described the Labour Party as a “mess”.
“The anger on Wednesday, Thursday and Friday was palpable,” they said. “There was a lot of talk last night that Morgan McSweeney was going to go. But the Prime Minister is still fighting for McSweeney’s survival, which is completely tone deaf.”
Karl Turner, the Labour MP and vocal critic of the Government’s judicial reforms, warned Starmer that sacking McSweeney was the only way to save himself.
“Keir Starmer is in a terribly difficult position. But I don’t think taking bad advice from a bad adviser is a capital offence.
“However, if he hangs on to his chief of staff beyond the Gorton and Denton byelection results on February 26, then his position becomes untenable.
“To recover, Sir Keir needs to sack McSweeney and then bring in Gordon Brown, begging on your knees if you have to, to come and advise on cleaning up politics.”
It is clear that MPs want blood.
Many are tired of Starmer deflecting responsibility and not owning up to his mistake, with one backbencher saying “he’ll always find someone, other than himself, to blame”.
Another Labour MP was blunter still, telling The Telegraph: “I think we are in the s*** quite deeply. Most people in the parliamentary Labour Party now are of the mind that Starmer is finished.
“The PM has relied on terrible advice. Ministers decide and advisers advise. The responsibility in consequence is in the Prime Minister’s hands.”
On Thursday (local time), Rachael Maskell, the former Labour MP for York Central who now sits as an independent, said the party needed to “move forward” to gain back support.
“I don’t believe we can have the PM in place – it is inevitable that the PM is going to have to step down,” she said.
Maskell later said that she believed the future of the Labour Party should be put first and, if this was not recoverable, then Starmer’s position was untenable.
Graham Stringer, the Labour MP for Blackley and Middleton South, said the Prime Minister was “finished” when he appeared on GB News. He added: “It’s embarrassing some of the policy documents that have gone out and some of the policy decisions.”
Even loyalists like Jack Abbott, the Labour MP for Ipswich, has acknowledged that last week was “arguably the most challenging since coming into power”. But he later insisted that the Labour Party should prioritise policy over infighting.
On Saturday morning (local time), Starmer was spotted with his wife at a synagogue, looking subdued. Perhaps the battered leader was searching for spiritual guidance; even salvation.
Either way, patience within the parliamentary Labour Party has evaporated.
With rivals circling, allies distancing themselves and the party demanding accountability, the Prime Minister is facing the greatest political battle of his life.
Short of divine intervention, many MPs believe his fate is already sealed.
Sign up to Herald Premium Editor’s Picks, delivered straight to your inbox every Friday. Editor-in-Chief Murray Kirkness picks the week’s best features, interviews and investigations. Sign up for Herald Premium here.