But Mr English said the argument about what interest rates banks charged "is really a judgment about how competitive your banking system is".
"Do we think New Zealand's banking system is sufficiently competitive? Our view is, yeah it is."
"There's plenty of other banks in New Zealand beside the big four Australian banks. People can take their business there."
But Mr English warned the one thing "worse than banks making too much money is if they don't make money at all".
Mr Hayes' question was welcomed by Green Party co-leader and committee member Russel Norman who last week highlighted the four big Australian-owned banks' high profits.
If the Government wanted to ensure productivity improved in the economy, "surely you should be concerned by a giant sector which is earning three times the average return on assets internationally and three times the average return on equity against other New Zealand businesses," Dr Norman told the Herald.