The chapel is on a private road just a stone's throw from the Royal Lodge, the residence that the Duke of York shares with the Duchess of York, his ex-wife.
It is understood that the private ceremony for August Brooksbank and Lucas Tindall is the first time that two Royal babies have been baptised together.
Princess Eugenie and Jack Brooksbank, her husband, were due to christen their nine-month old son back in July. However, the event at the All Saints Chapel was cancelled due to a Covid scare.
Mike and Zara Tindall's son was born in March.
The second cousins share the same middle name, Philip, in tribute to the Duke of Edinburgh, their late great-grandfather