In 2004, police went to the house in Cleveland, Ohio, where three women were apparently imprisoned for a decade and interviewed the prime suspect, but found no signs of their captivity.
Officers visited the home of Ariel Castro, a bus driver, in January 2004 after he left a child on a school bus and went for lunch while working for the local school district.
Investigators knocked on the door but left when no one answered, having been within metres of where Amanda Berry and Michelle Knight are believed to have then been held.
The third victim, Georgina DeJesus, disappeared months later.
Castro, 52, was later "interviewed extensively" about the bus incident but was never charged and remained off the police's radar for anything other than traffic violations until he was arrested on Tuesday.
He and his two brothers - Pedro, 54, and Onil, 50 - are now believed to be responsible for one of the most extraordinary kidnapping mysteries in American history.
At the heart of the case is the question of how the three women could be held captive for a decade in a major city without any suspicions being raised.
Cleveland police defended their policies as "solid" yesterday and said they had never received any indication that the house in Seymour Ave was being used as a prison. A police online statement said: "Upon researching our call intake system extensively, only two calls for service from police are shown at that address.
One call was from the resident, Ariel Castro, reporting a fight in the street. The second call was in relation to an incident regarding Ariel Castro and his duties as a bus driver."
Martin Flask, the city's director of public safety, said: "Our initial review indicates that nothing was provided to the city of Cleveland by any of the neighbours that live on that street or anywhere else regarding activities at that home."
But Castro's neighbours yesterday challenged that, saying they had heard screams and warned police on at least two occasions over the years. Police showed up at the house both times, the neighbours say, but never went inside.
"A few years ago we heard a scream," said Juan Perez. "We called the police. But it went away. We just had no idea."
Elsie Cintron, who lives three houses away, said her daughter once saw a naked woman crawling on her hands and knees in the backyard several years ago and called police. "But they didn't take it seriously," she said.
Another neighbour, Israel Lugo, said he heard pounding on some of the doors of Castro's house, which had plastic bags on the windows, in November 2011. Lugo said officers knocked on the front door, but no one answered. "They walked to the side of the house and then left," he said.
Neighbours also said they would see Castro sometimes walking a little girl to a neighbourhood playground. And Cintron said she once saw a little girl looking out of the attic window of the house.
"Everyone in the neighbourhood did what they had to do," said Lupe Collins, who is close to relatives of the women. "The police didn't do their job."
Cleveland police came under heavy criticism in a separate case a few years ago following the discovery of 11 bodies in a man's home and backyard in another poor section of the city. Neighbours had long complained about foul odours, and the victims' families charged that police did not take the reports of missing women seriously.
The community was trying to reconcile their image of the "happy-go-lucky" Castro with the crimes he is suspected of committing.
His Facebook page reveals a passion for motorcycles and bass guitars and an upbeat attitude, with his last post on May 2 reading: "Miracles really do happen, God is good."
A month earlier he celebrated the birth of a new grandchild, noting "that makes me Gramps for the fifth time".
He did not mention that one of those grandchildren had her throat slashed by her mother, Emily, Castro's daughter, six years ago.
Emily's mother, Grimilda Figueroa, was seen walking in the street in Fort Wayne, Indiana, carrying the bleeding 11-month-old child on April 4, 2007.
When police arrived at Emily's home they found she had cut her own wrists and tried to drown herself in a nearby creek. Both she and her baby survived. She is now 24 and serving a 25-year prison sentence in Indiana.
Castro's son Anthony wrote an article about the kidnappings in 2004 while studying journalism, even interviewing the mother of DeJesus about the "tragedy" of her daughter's disappearance.
Anthony Castro told the Daily Mail newspaper that he now speaks with his father just a few times a year and seldom visits his house. He said that on his last visit, two weeks ago, his father would not let him inside.
"The house was always locked. There were places we could never go. There were locks on the basement. Locks on the attic. Locks on the garage."
- additional reporting AP