He said it was also clear Mr Walmsley thought the couple would do "nothing about the situation".
Judge Dwyer went on to say the Walmsleys received expert advice prior to the construction of the "so-called children's play structure", and thought they could get "around any difficulties with the district plan".
Earlier this year, the Environment Court ordered the fence be removed and it was being dismantled a month later.
Mr Aitchison previously told the Herald his Maida Vale Rd home was devalued by a minimum of $900,000 when the fence was built.
He said the fence battle had gone on for 20 years, its roots in a strange old struggle between a neighbour and a long-gone property developer.
"The poles went up in March. He actually put [in] these poles, like a hundred poles, and they sat for about a month," Mr Aitchison said.
"And then all of a sudden, three guys arrived. It went up over about 10 days I guess.
"The structure was a kids' fort, and Wellington City Council gave it the thumbs-up."
But Mr Aitchison said that no children seemed to use the neighbouring property, where a group of young adults lived.
The Aitchisons could not be reached for comment this evening.