Police identified him only as a 28-year-old foreigner, without releasing a name.
Police also showed photographs of detonators, ball bearings and a metal pipe that they believe was intended to hold a bomb.
"The bomb materials are the same, similar or the same type" as those used in both bombings, police chief Somyot Poompanmoung told reporters.
He added that the suspect had traveled in and out of Thailand since January 2014.
Police also found "a number of passports from one country," Prawuth said.
He did not name the country, but photographs showed stacks of passports that were similar to those from Turkey.
Earlier, Prawuth said that authorities had not yet determined his nationality and dismissed Thai news reports saying he is Turkish.
Images of a Turkish passport with the apparent suspect's picture were posted on social media.
"The passport you see is fake," said Prawuth, referring to the online photos.
"We don't know if he is Turkish or not."
A Turkish government spokesman said he had no information on the suspect held or any possible Turkish link to the attack.
He spoke on condition of anonymity in line with Turkish government rules that bar officials from speaking to journalists without prior authorization.
Asked what could be the motive for the bombing, the police chief told reporters, "it's a personal grudge...not international terrorism."
He did not elaborate or give a clear explanation.
The man faced charges of possessing unauthorized explosives, Prawuth said, and was taken to a military base for further interrogation.
The blast at the Erawan Shrine was unprecedented in the Thai capital, where smaller bombs have been employed in domestic political violence over the past decade, but not in an effort to cause large-scale casualties.
The shrine is a popular tourist destination, particularly with Chinese visitors, who are an important segment of the lucrative tourist market.
At least six of the dead were from China and Hong Kong.
It sits on the corner of a busy traffic intersection with a nearby overhead walkway in a neighborhood full of upscale shopping malls and five-star hotels.
No one has claimed responsibility for the blast, sparking a variety of theories into who might be behind it.
Possible suspects include parties seeking to avenge Thailand's forced repatriation of ethnic Uighurs to China.
Uighurs are related to Turks, and Turkey is home to a large Uighur community.
Other theories included Muslim separatists from southern Thailand, opponents of Thailand's military government and feuding factions within the security services.
Soon after the bombing, police released an artist's sketch of a man seen in a security camera video leaving a backpack at a bench then walking away from the open-air shrine.
A separate camera showed the man, wearing a yellow T-shirt, on the back of a motorcycle taxi leaving the site.
The man seen in the video was believed to have carried out the bombing, which police said was likely planned by a group of people.
They indicated in Saturday's news conference that the man arrested was not the bomber seen in the video.
"We believe he is a culprit in the same network. More details will be given later," Prawuth said.
Police have been criticized for releasing conflicting statements and rapidly hosing down the crime scene at the shrine before all forensic evidence was recovered.
Many accused authorities of rushing to clean up the bomb scene to reassure the public - especially foreign tourists - that security in the city was back to normal.
Police say they have been handicapped by low-quality and broken surveillance cameras and a lack of sophisticated image-processing equipment to clarify the fuzzy images in security videos, which were the only firm evidence they had.
- AP
