The businessman's affair with his former mistress, and now partner, began around 1980 and she began working for his company the following year, according to the High Court.
She has worked there ever since, the High Court said last year.
Evidence before the Family Court indicated that throughout his marriage, the businessman had other sexual relationships with other women. His wife was unaware of all these affairs and when confronted he always denied them.
According to his ex-wife, the businessman's affair with his current partner was clandestine. It took place in early mornings or when he left family holidays to return to "work".
The man, according to the High Court, in 1998 told his wife he was participating in a yacht race in Australia and even purchased wet weather boating gear to support this ruse.
He was, in fact, spending a holiday with his current partner in Perth.
The wife alleged in the High Court that the former mistress and her ex-husband's claim they were in a de facto relationship only surfaced in 2011 or 2012, specifically to assist him opposing her in the Family Court.
The businessman's former mistress argued she was pursuing her entitlement and that her relationship with the man was a devoted one, in which she had a strong focus and passion for his business.
The former mistress rejected the assertion she manufactured the de facto relationship argument late in the piece, that she colluded with her lover on this and that she had done so to keep his business interests in his hands.
The Family Court's Judge Anthony Walsh in 2014 found there was no qualifying de facto relationship between the man and his former mistress.
This, and other parts of that decision, was then appealed to the High Court, but Justice David Gendall agreed last June that the relationship between the man and his former mistress was not a de facto one.
They then sought leave to take the case to the Court of Appeal, but Justice Gendall denied this bid last December.
Now the Court of Appeal itself has also declined to take the case.
Lawyers for the businessman and his current partner told the Court of Appeal last month that the lower courts got it wrong.
Those courts, they claimed, set the bar for what counts as a de facto relationship "too high" by failing to take into account the features of a "contemporaneous de facto relationship", which they said were different from an "orthodox de facto relationship".
"A contemporaneous relationship is by its very nature usually clandestine...," they argued.
The judges also placed an "excessive weight" on comparisons with the businessman's marriage and failed to take into account the ongoing defacto relationship between him and his current partner.
The proposed appeal would provide an opportunity to correct these errors, which they argued, had caused "serious injustice".
But Justices Wild, French and Brown were not convinced:
"There have been concurrent findings there was no de facto relationship in existence. The challenges to that finding are essentially case-specific and do not, in our view, involve a question of general or public importance such as to outweigh the cost and delay of a second appeal," Justice French said yesterday.
"The fact this litigation has been going on for some nine years reinforces our view. We agree with Mr Ormsby [the wife's lawyer] that justice now requires finality," she said.
