Tully suffered from minor dog bites and police were continuing to search for the weapon.
He would be medically examined before being formally interviewed and charged, Mr Knowles said.
The capture would be a "huge relief" for local community, he said.
"It gives them some closure and safety tonight to think they can go home and not worry," he said.
"We had the whole area contained but as I said, it's the small town New Zealand ... dealing with things ... it's been horrific today.
"Nothing will give the families who lost loved ones closure but I think a community can go to sleep tonight with the thought that the person we believe may be responsible is in custody."
Do you have more information? Email us here.
Locals applauded police and the armed offenders squad as they arrived at the local police station this evening, with Tully in their custody.
According to witnesses, a man wearing a black balaclava and carrying a shotgun entered the Work and Income office on Cass St about 10am today. Shots were then fired.
Watch: Police storm house looking for killer
One witness, who did not want to be named, said he saw a staff member run from the office after the shooter yelling: "You bastard, you've blown her to bits."
The witness said he stopped his van in the middle of the road when he saw a man in a black balaclava with a shotgun on his back walk in to the Winz office.
"It was just surreal. I stopped and got out of the van because I thought 'that's not right' then I heard two shots and I legged it straight to the office."
Suspect Russell Tully was arrested late this afternoon
Tully was interviewed by The Ashburton Guardian for an article just 12 days ago.
The paper reported Mr Tully was living in a tent on the Ashburton riverbed, and was looking for a house from Housing New Zealand.
"I want somewhere to live. I can't see why I can't move in here when no one is living here," he was quoted as saying in the article.
The article said Mr Tully had been back in New Zealand for three years after working in mines in Australia.
He has a skin condition which affects his joints, and after three years of no work, he had no money, the article said.
As police searched throughout the day, the Ashburton community was advised to stay indoors. The New Zealand Defence Force assisted police with the manhunt.
Ashburton Mayor Angus McKay said Ashburton was a very community-minded town and the tragedy would affect many lives. He became emotional when talking about it: "It just hits home real hard."
Prime Minister John Key said he was "shocked and disturbed" at the deaths of two people at the hands of a gunman in Ashburton.
Mr Key said his "sincere condolences" went to the family and friends of the dead and the injured.
"There will be many questions to be asked and answered in the coming days about how this happened."
Watch: Fatal shooting in Ashburton
All Ministry of Social Development offices in Ashburton, Timaru, Oamaru, Christchurch and Balclutha were closed today.
Today's Ashburton tragedy comes after a long series of attacks on Work and Income offices by angry beneficiaries, and was the first to result in deaths.
