Air-guitar playing may never be the same again, thanks to a Kiwi innovation that could transform interactive games like Guitar Hero.

The state-of-the-art sensing gloves created by StretchSense, a University of Auckland spin-out company, draw on soft sensors set to replace wearable technology with a new generation of devices called "disappearables".

Along with gaming, the technology could help change the face of robotics, sports, motion capture and more.

StretchSense has already developed a range of other wearable applications with its soft-sensor technology - which might be described as rubber bands with Bluetooth - enabling the user to record real-time information about themselves while running, jumping or playing video games.


In Las Vegas this week, the latest innovation was revealed before hundreds of people at the global consumer electronics convention CES 2016.

In a performance at The Venetian casino, they were used to play air guitar on the Indian song Jai Ho, made famous by the 2008 film Slumdog Millionaire and by girl group The Pussycat Dolls.

The glove's inbuilt sensors allowed the performer to play virtually, without the use of a physical instrument, as the technology was able to measure the motion of the performer's fingers.

Creating this level of interaction in an unobtrusive and natural way had previously been a challenge for tech developers, because it required the measurement of precise hand movements.

"The glove conforms to and moves with the hand," said company co-founder Todd Gisby.

"It accurately captures our natural behaviour without getting in the way."

The gloves were an example of emerging "disappearables" - invisible, soft and lightweight devices that could integrate seamlessly with the wearer's clothing.

"Disappearables like our glove are more than just technology you don't see, it's about taking advantage of actions that already come as second nature to interact with technology in a way that's indistinguishable from normal behaviour," said Mr Gisby.

Motion-capture gloves had been around for some time but the underlying sensor technology had always held their development back, he said.

The gloves' sensors enabled the full precision that was needed for virtual reality, motion capture and the gaming industries, and had further potential uses for virtual musical instruments, robot controllers, and sportswear applications.

The gloves are among many colourful examples of Kiwi wearable tech innovations.

Others have included a mini-camera with 1080-pixel definition, a device that could help people unable to move or talk translate their thought patterns into computer commands, "smart fabric" for bandaging, a voice-restoring microphone and sensors to help people with hearing impairments communicate in dark or low-light situations.

How smart gloves work

• The gloves draw on soft-sensor technology developed by Kiwi company StretchSense, which has been described as like rubber bands with Bluetooth.

• A sensor hub, about the size of a button, on the back of the hand captures the motion data and integrates it with on-board accelerometers to recreate the movements and shape of the hand.

• In a demonstration by an air guitarist at the CES 2016 convention in Las Vegas this week, the gloves showed how the data could then used to synthesise the performance in real time.

• By fusing art with technology, the performance showed how wearables are moving beyond simple body tracking tools, and will soon provide seamless ways for users to interact with the virtual world.