A married Sikh couple were told they could not adopt a white child because of their "cultural heritage", the family claim.
Sandeep and Reena Mander, a British couple from Berkshire, said they were instead advised to "adopt a child from India".
The couple, business professionals in their 30s, told an adoption agency they were very happy to adopt a child from any ethnic background, according to Daily Mail.
Despite being found to be suitable adoptive parents after a home visit, the couple claim the agency said they shouldn't apply because only white children were available.
Adopt Berkshire, the county-wide agency run by the Royal Borough of Windsor and Maidenhead, said white British or European parents would be given preference.
The Manders have no close links to India but are of Sikh heritage, the Times reported.
They have now begun legal action to overturn the decision, claiming discrimination in the provision of services.
The Manders have been trying to conceive for seven years, and have had 16 sessions of IVF treatment.
Mander, 35, said: "Having attended introductory workshops organised by RBWM and Adopt Berkshire, giving an adopted child - no matter what race - the security of a loving home was all we wanted to do.
"What we didn't expect was a refusal for us to even apply for adoption, not because of our incapability to adopt, but because our cultural heritage was defined as 'Indian/Pakistani'."
It is legal for agencies to give preference to parents from the same ethnic group, but government guidelines stipulate different racial backgrounds should not prevent a couple from adopting.
Falling adoption numbers have been blamed on the "misguided" belief that children should only be adopted by families of the same ethnic background.
The Manders' case is being backed by the Equality and Human Rights Commission.
Commission chairman David Isaac said: "There are many children who are waiting for a loving family like Sandeep and Reena to help give them a better life. To be denied this because of so-called cultural heritage is wrong."
In 2015, senior judges ruled on a test case that allowed two black African boys to be adopted by a white British couple.
In spite of the cultural differences, the Court of Appeal said the couple were perfectly suited to adopt the brothers - one of whom has Down's syndrome.
Social workers believed that white parents couldn't met the children's needs, including preparing them for the impact of racism.
But Lady Justice Black ruled: "The boys had a number of needs, not limited to important considerations related to nationality, culture and race, and there was no perfect way in which to satisfy all those needs."
The eldest boy, 15, had suffered emotional harm at the hands of his birth parents.
His younger brother, 8, was removed from the parents shortly after birth.