Labour withdraws from man-ban