Ludlow appealed against both sentences in the Court of Appeal, arguing that Judge Bouchier's sentence was too high and Justice Toogood's sentence should not have added an extra nine months.
He submitted his culpability was similar to defendants in other cases who had received lower sentences, and he should not be jailed for more than three years.
But the court last week dismissed his appeal, noting that his premeditated offending had led to total losses of $14 million.
He took the money from vulnerable victims, knowing he had no right to, and spent it for the benefit of himself and his family - including the purchase of four villas in Fiji.
The court earlier this year dismissed Ludlow's appeal against conviction.
National Finance, whose core business was car finance, was placed in receivership in May 2006, with investors owed $21m.
Some investors have recovered 49 cents in the dollar, although others have been left completely out-of-pocket.