Mr Brown rejected reports he was weaving on the road and hit road pegs and other vehicles.
He said that after he kicked some children off the bus for playing up "they wanted to get one back on me, and this is the opportunity. I have been given an unfair go. With the school kids, especially senior ones, you can't stop any playing up. You can't kick them up the backside now".
Jenny Kirby, whose 16- and 14-year-old children were on the bus, said she was adamant they were not lying.
"When you've got 40 kids all telling you exactly the same thing, that's what you believe. He definitely needed their help."
She said Mr Brown had been contradicting himself and that the students on board had not overreacted.
Mrs Kirby said she had been wary about the 82-year-old driving the children for an hour each morning and afternoon on the 50km journey.
The driver said a doctor had given him a medical certificate, and had been told there was nothing wrong with him other than a flu. He maintains he is still up to driving a bus at the age of 82.
Asked to confirm if he had tried to call in sick before the incident, Mr Brown said: "It's true in a way."
He was waiting to be interviewed by police, and said he had been told not to speak to media.
Mr Brown said he had been driving for 65 years.