As Rosengart began to argue for a private hearing, Spears interrupted him to say "I can talk with it open."
"My dad needs to be removed today," she said, adding that she would like to see him charged with abusing his position.
She spoke so rapidly she was at times difficult for the court reporter and the dozens of media members in the courtroom to understand.
She said she wanted the conservatorship to end immediately but not if it required going through any more "stupid" evaluations. She said she did not want another opportunity for "people to question my intelligence for the millionth time".
She described being denied things as basic as coffee, her driver's licence and her "hair vitamins" by the conservatorship.
"If this is not abuse, I don't know what is," Spears said.
She recalled that "I thought they were trying to kill me" in the early years of the conservatorship as she was being overworked and constantly examined.
She began to sob as she talked about taking comfort in a therapy dog when she felt abandoned by her family.
James Spears would not be stepping down as Rosengart challenged, his attorney Vivan Thoreen said in court, adding he only ever had his daughter's best interests in mind.
Thoreen said Britney Spears had many inaccurate beliefs, among them that "her father is responsible for all the bad things that have happened to her".
"Whether it's misinformation, lack of correction, or being wrongly advised, I don't know," Thoreen said.
Thoreen emphasised that for nearly two years James Spears has had control only over his daughter's money, ceding power over her life choices to a court-appointed professional, Jodi Montgomery.
Thoreen argued an evidentiary hearing was necessary to look into the allegations Britney Spears made at the June 23 hearing, including she was being forced to take medications and use an intra-uterine device for birth control.
Thoreen said she found it odd Britney Spears would describe those circumstances, which the pop star called "abusive", then approve of Montgomery continuing in her role as overseer.
Montgomery's lawyer, Lauriann Wright, responded that the last thing Britney Spears needed was to be put on the stand and cross-examined over the truth of what she said during her emotional testimony.
"I don't think it's a secret that we're dealing with someone with mental illness," Wright said.
Wright added that it's been "strongly recommended by her medical team that her father Mr Spears needs to be off the conservatorship".
Britney Spears has been under court supervision since February 2008. She was in the midst of a public meltdown at the time and her family sought the conservatorship for her protection.
She has had throughout the proceedings a court-appointed attorney to represent her interests, Samuel Ingham III, but he resigned after the dramatic hearing three weeks ago in which the pop star told Penny: "I just want my life back."
Fans from the #FreeBritney movement outside the courthouse cheered the decision to appoint Rosengart, then cheered Rosengart himself when he walked out.
In court, he called into question whether the conservatorship should ever have been put into place, and said he and a team of attorneys from his firm would be taking a close look at the details of the arrangement.
"This is not working," Rosengart said. "We know that."
Republican congressman Matt Gaetz made a brief appearance addressing Spears' supporters outside the downtown Los Angeles courthouse, at one point shouting "Free Britney!"
In Washington DC, a group of Spears supporters demonstrated at the National Mall in support of the singer.
The case had for several years operated with little drama, though questions about its length and the singer's feelings about the proceedings arose in recent years. That has culminated into intense public scrutiny of the court proceedings, which have been conducted largely in secret due to medical and private information about the singer and her condition.
There has also been focus on the complex network of lawyers and others involved in the conservatorship.
Penny again gave little indication of what moves she'll make next, scheduling another hearing for late September and expressing hope that the warring factions will find common ground for Britney Spears' sake.
"Everybody should be working collaboratively," the judge said. "It's not about anybody else, it's about her."
