The Jenner boys are not nearly as close with Kylie and Kendall as the girls are with their other half-siblings, the Kardashians.
Brody revealed in February of this year that he didn't even know Kylie was pregnant until after she delivered baby Stormi.
The Page Six source also revealed that Brody and his blogger fiancee are set to have an LA reception too, which may have more Jenners in attendance.
The same site claimed the reason Caitlyn did not attend the island ceremony was because she was too busy with "work commitments".
Caitlyn - who was pictured visiting fast food joint Chipotle with her companion, Sophia Hutchins, in Malibu on Sunday - recently admitted she found it "tough" to maintain a relationship with her children.
The I Am Cait star said her children - Burt, 39, Casey, 37, Brandon, 36, Brody, 34, Kendall, 22 and Kylie, 20 - have all "moved on".
She told Broadly last week: "I spend a lot of time by myself here in the house. I have a lot of children, but sometimes just because of circumstances, maintaining a close relationship with your kids is very tough. They all have lives. They've all moved on. We're just human beings; we're going to be here for a very short time. We come and we go and at the end, when it's all said and done, hopefully your family is going to be there."