Interpol is also assisting with the investigation, helping Japanese police to co-ordinate with the South African authorities, sources told Kyodo News.
The raids are believed to have started early in the morning when banks were closed, buying suspects more time to leave the country before police investigated the crime according to the Yomiuri newspaper.
Standard Bank, which is one of South Africa's largest financial services groups, released a statement confirming that it had been the "victim of a sophisticated, co-ordinated fraud incident".
It added: "This involved the withdrawal of cash using a small number of fictitious cards at various ATMs in Japan."
The incident is the latest in a string of increasingly sophisticated ATM robberies involving fake credit cards across the globe.
One such incident reportedly involved a pair of heists totalling around £31 million, after cyber thieves disabled withdrawal limits on ATMs around the world.
There is little that customers can do to protect themselves in the event of such attacks.
Dr Steven Murdoch, a security researcher at University College London, said: "It looks like something has gone wrong at one or both banks and there's not really anything the customer can do to protect themselves."