A fortnight ago England were the third ranked team in the world but Eddie Jones' side now sit sixth following Sunday's second test defeat to South Africa.
The loss was the side's fifth in a row and saw Wales and South Africa both move past England in the World Rugby rankings.
The All Blacks remain in top spot, a position they have held since 2009.
Ireland hold second place following a victory over Australia on Saturday. The defeat saw the Wallabies drop to fourth with Wales now the third ranked side in the world.
South Africa jumped up from seventh to fifth, while Scotland dropped one spot to seventh after a shock loss to USA.
Jones took over as England coach following the 2015 World Cup and the side won their first 17 tests under the Australian - and 28 of their first 29 tests.
However England ended their Six Nations campaign with defeats to Scotland, France and Ireland before dropping the first two tests in South Africa.
The third test is at Cape Town on Sunday.
World Rugby rankings
1 - All Blacks - 93.99
2 - Ireland - 89.20
3 - Wales - 85.94
4 - Australia - 85.41
5 - South Africa - 84.65
6 - England - 84.35
7 - Scotland - 81.83
8 - France - 79.10
9 - Fiji - 78.51
10 - Argentina - 76.73