Deputy Prime Minister Winston Peters has been kicked out of the House for the first time since the Government was formed last year.