"The Environment Court considered a 'high degree of artificiality' was involved in treating the structures as three separate structures," the judge said.
Walmsley was made to remove the fort in 2016, but the council appealed the Environment Court's decision due to its reasoning, which it said had an effect on the interpretation of the District Plan.
The Environment Court ruled the council had erred in its interpretation of the definition of "ground level" - the argument was around whether the structure should be measured from the top or the bottom of the retaining wall between the properties.
The retaining wall was only near the boundary, not on it.
The council's lawyer said the Environment Court's decision meant "a retaining wall now has the power to significantly alter the development potential of an uphill property".
"I think it is highly unlikely that property owners would have expected the results we have seen in this case from the council's administration of the District Plan," the judge said.
"On the one hand, the Environment Court ordered the removal of a structure it described as a 'contrivance undertaken to get around rules' preventing the construction of a fence, whose adverse effects were 'extreme' and 'severe' and which was 'offensive' and 'objectionable'.
"On the other hand, the council maintains the structure was compliant and a 'permissible activity' under the rules. For a structure such as this to be classed as 'permissible activity' is nothing short of anomalous."
He said that on a property where a retaining wall was by the boundary, a two step process should be engaged.
"Ground level" will be at the point where the front surface of the wall meets ground, and the point "at the boundary" is where the ground level intersects with the boundary.
The judge dismissed the appeal.