Innovative AI tools and avatars that interpret sign languages are emerging quickly. Photo / 123rf
Innovative artificial intelligence (AI) tools and avatars that interpret sign languages are emerging quickly, raising potential alarm bells around their development.
Deaf Aotearoa general manager for adults and seniors, Lara Draper, said the development of AI tools has raised questions about language rights, accuracy and ethics.
Draper, who recently attended the SLxAI Summit held in Boston, United States, told the Herald it was important to look at who was creating these tools and who would be responsible for the implications if they were incorrect.
“The use of AI in this space should be about more access and about talking to the Deaf communities about what they need on their terms, rather than outsiders who get very excited by their new hot technology,” she said.
Draper said the development and implementation of AI avatars was happening rapidly for different sign languages used globally.
She said this was being done in two ways: one with significant human oversight or with heavy reliance on an AI system.
Draper said avatars can be made preferably by a native speaker wearing a motion capture suit, with sensors placed across the body.
For sign language, sensors would have a particular focus on the face, arms and hands. Then, a 360-degree camera set up would capture the movements in real time.
Draper said this footage would then be fed into an AI system and a Deaf language expert would check what has been captured for accuracy.
“Compared to a straight system without a human in the loop, where you’re just working straight with the data that’s coming in,” Draper said.
“The technology has moved so fast that AI is now learning and capturing much more of the finer detail that you see in sign languages, particularly on the face.
“Many people think sign languages are just what happens on the hands, but what happens on the face is also meaningful and grammatical.”
Draper said things such as eyebrow and shoulder movements can add meaning.
“It’s a very rich, multi-dimensional language. It is a 3D language,” Draper said.
She said AI translation from a spoken or written language is easier when compared to sign languages, which are often communicated with a higher level of complexity.
“What we are finding is that it’s easier to translate from the written word to an AI avatar in sign language, but the other direction is too complex at the moment,” Draper said.
Draper said the new technology had raised concerns around job security, cultural appropriation and the potential consequences.
She said the method without significant human oversight was more at risk of “flattening” the language.
However, a positive of the non-human system was its accessibility for straightforward information in places where you can’t have a live interpreter, such as for airport announcements.
“If we’re talking about higher risk situations, in the legal sphere, in the health system, mental health assessments, in an educational setting... there would be far too much risk to try to use AI interpreting and translation in those contexts without any oversight or supervision,” she said.
Draper said the tools should be treated as something to increase access for Deaf communities, but they shouldn’t replace human interpreters.
She said that although there were fears, AI could not replace a human interpreter entirely because it can’t yet fully capture the emotion behind how sign languages are expressed.
Draper said another concern was around who was taking responsibility for this technology and the oversight of its development and policy.
“Certain hearing people get very excited about creating some kind of entity. They’ll come to the Deaf community and think of them as an afterthought... But once you have gone too far, you’ve really failed the community already.
“Whatever those technological advances are, they always inevitably fail because they haven’t considered Deaf people from the outset.”
Draper said there was also concern about cultural appropriation, and that it was important for Deaf people to preserve their languages.
“They see the risk of that being taken away from them if people start changing their languages, manipulating it, getting it incorrect.
“It’s crucial that we’re involved because we do have concerns that outside-led companies and entities will just march along without even talking to us, let alone considering our thoughts.”