It is believed Little was last alive in the mid-1970s.
"His identity remained a mystery for almost 40 years," a police statement said.
"In October 2021, the DNA Doe Project provided the names of potential living relatives, who were able to provide Frank's name.
"A close relative provided a DNA sample, which was analysed by the Ohio Bureau of Criminal Investigation Crime Lab.
"Not much is known about his disappearance and death. Our sympathies to the family during this difficult time."
The DNA Doe Project is a non-profit that uses genetic genealogy to identify bodies which can't be identified by other means.
The band released a statement to CNN saying that Little was part of the band in the early days and that they hadn't heard from him since, but wished his family and friends closure.
"He came with us when we first ventured out of Cleveland and travelled to Los Angeles, but he also was in love with a woman in Cleveland that he missed so much that he soon returned back to Cleveland after a short amount of time," the statement said.
Little served in the US Army for two years, which included a deployment to Vietnam during the Vietnam War, according to the police statement.
Little had a daughter who passed away in 2012, and he has a son who has not yet been located or identified, it said.
When Frank played with The O'Jays in the group's early days he wrote songs including Pretty Words and Oh How You Hurt Me, TMZ reported.
After his tenure, the group scored its biggest hits including Back Stabbers, For the Love of Money and Love Train.