He said it could also result in fewer rental properties - and higher rents as landlords tried to claw back the losses.
"Your typical property investors are average mums and dads - not wealthy cigar-smoking fat cats.
"This move would certainly stop them investing, but in the process it would quickly lead to a shortage in rental housing which would fall back on the Government - so it would end up costing the taxpayer a lot more in the long run."
Church also disputed Little's claim it would even things up for first-home buyers, saying families were being closed out of the housing market by high loan-to-value ratios, not investors.
Little denied it would hurt small investors, saying many only used it for a short period when interest payments were high.
"The people we are going after are the multiple property owners who deliberately set up a property portfolio to run at a loss, offset it off other income and demand a taxpayer subsidy - that's coming to an end."
He said those who had entered the property market on the basis of using negative gearing to offset losses would have five years to adjust as the policy was phased in.
Labour released figures showing 35 per cent of non-residents who owned rental properties in the last year had declared rental losses totalling $47 million. That was about 8000 of 23,000 overseas-based owners. The figures also showed about 91,000 taxpayers declared rental losses in 2015 and of the $149m claimed in tax breaks on that, $60m was for the top 10 per cent of income earners.
National's campaign chairman, Steven Joyce, said removing the tax breaks for property investors would not have the effect Labour claimed - and would hit mum and dad investors more than Little believed.
He said Little's claim few small-time investors used it was "pulled out of the proverbial", saying negative gearing was used for all loss-making investments - not just residential property.
He said tax working groups under Labour and National had concluded getting rid of negative gearing was unlikely to result in more housing supply and the most likely impact was higher rents. "You'll end up with fewer houses being built and higher rents."
He said countries which had ring-fenced housing losses still suffered fluctuations in house prices.
Joyce said the bright-line test to tax capital gains on houses which were on-sold within two years and removing depreciation had helped.
"Those two steps are the least likely to reduce supply and least likely to push up rents. If you combine those with the Reserve Bank's [loan-to-value ratios] you will see Auckland house prices at the moment are flat to falling. The challenge is to not do too much on the demand side so you end up reducing the level of supply."
Little insisted it was an important step to help first-home buyers who did not benefit from the tax breaks.
"This will create a level playing field for home buyers and help families get a fair shot at buying a place of their own."
Andrew King, executive officer of the NZ Property Investors' Federation, said the advantage the tax breaks gave to investors was over-exaggerated. He estimated that removing the ability to claim losses for rental property providers would increase the cost of providing the average home from $6184 a year to $10,293 - an increase of $79 per week.
Little said the International Monetary Fund and the Reserve Bank had advised removing the tax breaks.
Last year, the IMF said ring-fencing tax losses on housing investments would weaken a significant price driver in real estate.
Negative gearing means that if the cost of a rental property in mortgage interest, maintenance, water, and rates are more than the income from rent, the owner can get a tax deduction on the loss. It is seen as an incentive to invest in housing.
