His decision came after Prime Minister Bill English revealed that Barclay had told him last year he had audio recordings of Dickson - just hours after Barclay denied any recordings were made.
Police have reopened the investigation into Dickson's allegation that Barclay had secretly recorded her.
They had previously found insufficient evidence to carry out search warrants of take matters further, following a 10-month investigation during which time Barclay declined to be interviewed.
Barclay has not been at Parliament since confirming he would leave politics. A Fairfax reporter approached him for comment outside the Pig and Whistle pub in Queenstown and was told he was not making comment.
Patterson said he knew Dickson from his own time as a National supporter, and said her treatment by the party had disappointed people in the area.
"At a local level a lot of people have known the truth but they have been prepared to turn a blind eye to it. So that's not really the southern way - people trade on their integrity down south.
"Yes, I think they have [lost votes]. But I think it has got to be seen in a wider groundswell of discontent. In many ways it would have been better for us if Todd had stayed, because he was quite unpopular within the local electorate, particularly the more rural parts of the electorate who felt he ignored them. We were probably disappointed in the end that he's not standing."
Patterson said key issues in the electorate included a sense the regions had been neglected by the Government while money went to infrastructure projects in Auckland.
Foreign ownership of farmland and supply chains was a point of difference to National, he said, and the NZ First policy to put GST back to the provinces had strong appeal given the tourism spend in Queenstown and Milford.
