Beyond the tech babble: How Ikea, KFC, Adidas and others are using VR/AR

The most bored I've ever been – I'm talking a transcendental level of boredom – was when I was at primary school and my mother and older sisters would drag me to a shop called Millers in Dunedin. It was a retail warehouse of clothing fabrics and buttons and reels of cotton and absolutely nothing whatsoever of any interest to me. I would scream on the inside and plead to leave on the outside.

If only I could have transported my youth to the current age - or perhaps even a few years into the future - then that feeling of everything being irrelevant wouldn't have been nearly as pronounced as it was then.

The World Economic Forum (WEF) believes the next decade will be a golden age for consumers, with lots of personalised choice, transparent pricing, and frictionless fulfilment.

The key drivers of success over the next decade will be centred on building a deep understanding of and connection to the empowered consumer, promptly incorporating disruptive technologies, embracing transformative business models in both the offline and online space.

Customer experience – or "CX" if you're in the know – is front and centre of business strategy nowadays. Because retail is such a viscerally competitive sector, it's also one of the most relentlessly innovative in looking for new ways to engage customers.

So, what can we learn from this seething test bed of experimentation?

Retail is always evolving. Mail order catalogues dominated in the 1800s for stuff you couldn't buy at the local grocery store, department stores took off in the 1900s, and online shopping has been displacing the TV shopping channels that took off in the 2000s.

Even for shopping at physical stores, a 2017 study reported 84 per cent of customers use a digital source before or during a purchase. What's next?

Well, one thing to watch is immersive technologies, including augmented reality (AR) and virtual reality (VR), which the WEF picks as one of eight technologies disrupting retail and which some of the biggest retailers in the world are heavily investing in.

In fact, there is a fast–growing ecosystem of retailers big and small, marketers, and tech experts all prototyping like crazy, and that includes in Aotearoa.

Digital retail

There are loads of digital platforms which are designed to engage customers in retail. In China alone, more than 350 million people have live-steaming apps. One platform, ShopShops, livestreams shopping events from physical shops in the US, a host demonstrates products, her assistants answer questions about size etc, and customers place orders on Taobao for direct shipment. There are loads of platforms designed to make the fulfilment process seamless, too.

We all heard about how Amazon Go uses machine learning and computer vision to enable customers in its physical stores to avoid queues and physical checkouts by using an app.
An offshoot of KFC in China lets customers pay at a "Smile to Pay" terminal which incorporates a 3D camera and "liveness" algorithm to ensure a video isn't being used to trick the terminal.

AI and bots are helping retailers deepen their relationship with customers. Aside from apps allowing customers to track their order, some brands use AI to "socially listen" to every mention of their brand on social media, and choose when and how to respond (55 per cent of retailers are adopting AI to optimise customer experience, and one estimate is for 85 per cent of all customer service interactions to be by bot by 2020).

Audi encourages owners to post images of their cars which then get assembled into an Instagram collection.

This is ingenious stuff, but perhaps just the building blocks for a truly immersive experience.

Playing with augmented reality

AR overlays the virtual onto the real world.

Alibaba partnered with fashion brand Guess in its FashionAI concept store in Hong Kong. This includes every item having a digital tag which allows customers to interact with smart mirrors (Memomi is one supplier) and smart fitting rooms for advice about fitting and complementary suggestions. Customers check in and out of the store via an app.
Adidas has gone further, partnering with Microsoft Kinect on a body scanner at its physical stores which then allows people to virtually try on a full set of clothes. Once you're scanned you can also try on clothes virtually at home. Ralph Lauren, Converse, Gap and Uniqlo offer the same.

One celebrated use of AR is Ikea's app which allows people to see what a piece of furniture might look like in their home. Lowe does the same for kitchen visualisation.
Some research revealed that 60 per cent of customers prefer stores with AR, and 40 per cent would pay more if they could experience the product through AR. Another study claimed 71 per cent of shoppers would favour stores with AR.

Lacoste responded by enabling its LCST shoe brand app to not just let you try on shoes virtually, but also to generate window displays, in-store signage, and promotional postcards. Some 30,000 users engaged with 3D products using it. American Apparel launched a similar app, including customer reviews and pricing.

Cosmetics brand Charlotte Tilbury partnered with AR provider Holition to install AR "magic mirrors" in-store. After scanning, you can see yourself with ten different looks in less than a minute. L'Oreal Paris's MakeUp Genius app – with over a million downloads – lets you paint your face virtually mixing and matching as you see fit.

Sephora, another cosmetics brand, is trying to remove friction from purchasing: you upload a selfie before applying makeup, meaning it's a one-click process to buy.

Timberland installed a virtual fitting room using Kinect technology, with the main goal of driving more foot traffic. Topshop did the same in its Moscow store.

Airwalk linked AR to the pop-up shop phenomenon – you could only find where to buy a pair of limited edition Airwalk Jims by downloading their app and making your way there. It generated US$5 million (NZ$7.7m) in earned media, and its busiest ever ecommerce weekend.

Toys "R" Us targeted its Play Chaser app at kids in-store, which includes the mascot, Geoffrey, coming to life. It was downloaded more than 100,000 times.

My favourite AR retail idea is by Adidas, which sold sneakers that unlocked a virtual world when you held the tongue up to a webcam. The best bit is you then used the sneaker as the control to navigate.

So what's happening with virtual reality?

AR interacts with reality, but the aim of VR is to be truly immersive. With two out of three consumers being interested in virtual shopping, there's clearly demand.

Some retailers are using VR to create magnet experiences to attract customers to physical stores. Galeries Lafayette has a virtual rollercoaster experience in its Paris store. You can ride a virtual water slide down Oxford St at Topshop's flagship store in London. BT Sport created a live VR experience of a match between Chelsea and Arsenal.

Others do the same, but focused on product. Audi used ZeroLight to create a virtual experience allowing you to walk around and explore a car. North Face used VR in-store to transport customers to a 360 degree, stereoscopic immersive experience of Yosemite National Park, enhancing their storytelling about endless exploration. Toms shoes put Samsung headsets into 100 of their stores featuring a trip to Peru with panoramic views of schools and kids getting boxes of free shoes, thereby showcasing their social responsibility pitch of donating a pair of shoes for every pair they sell.

Marks & Spencer placed a pop-up VR show room in homeware stores in various UK cities. It allowed customers to virtually place upmarket homewares to "create their ideal living space".

Westfield mall extended its mall experience by showcasing its fashion collections to consumers at home who had an Oculus Rift headset. You can experience a new Lexus using Oculus Rift too. Tesco and Cisco (hardware) also offered VR customer tours.

Designer clothes retailer Ted Baker used VR in an advertising campaign that immerses you in something like a Tim Burton movie.

eBay moved away from physical stores altogether by launching the first virtual department store in 2016, promoting it by giving away thousands of headsets they called "shopticals". By all accounts, impressive though it was, it wasn't really immersive.
Fashion house Karen Millen recreated its flagship store in a 360-degree panorama accessible through a few platforms, but it didn't enable purchasing as part of the experience.

Alibaba launched Buy+. Customers use a cardboard headset and smartphone to shop virtually at Macys and other US department stores giving customers more options but also allowing US retailers to sell into China without any physical presence. In fact, you can get virtually chauffeured to the store in a pink Cadillac with an engaging driver. More than 8 million people used it the first week alone. You can also use voice-activation to select and pay for products.

Myers and eBay in Australia went in the opposite direction by creating a VR department store that did not mimic a physical store. This is more imaginative. Customers can "fly through an infinite space of products" with no walls, ceilings or escalators. The experience includes "sight search" which allows you to go to a product if you stare at it for a few seconds. You can only purchase by removing the headset and using the eBay app, though.

Mastercard and Swarovski launched an app that solves this pain point. Once customers have logged in and authenticated their bank account details, they can browse home accessories in a decorated home by moving their head, and pay by using a Masterpass button.

Aotearoa

As always, New Zealanders are in the thick of it. In early 2016, Comvita gave customers the opportunity to experience the source of their mānuka honey – including a thrilling helicopter ride up the Mohaka river - by using a VR headset in-store.

Kiwi Property and advertising agency 99 created a 360-degree VR fashion show viewable only at Kiwi Property malls. 99 and Rush Digital also worked with New World on its AR Epic Easter Hunt, a Pokemon Go-like in-store egg hunting app.

Recently, the Herald reported on top NZ hairstylist Richard Kavanagh's invention of Piiq, which connects a tablet and smart mirror for 360-degree Imagineering for new looks. It's being trialled at Rodney Wayne salons, and L'Oreal is poised to roll it out across 1000 affiliates in Australia, and then globally. It's offered as a subscription to salons, reflecting another fast-growing trend.

The AR/VR sector in Aotearoa is booming, leading to the NZVRAR Association being incorporated in 2016 to build a strong and vocal community; build capability and growth through education; promote the sector to grow the economy; and advocate for the sector. There's lots of government and local body support.

Its first report, in September 2017, talked about a local innovation supporting retail:
Consumer behaviour specialists, Lumaten partnered with Rush Digital to develop SHOPPER360 offering a custom-built virtual retail environment to test scenarios for packaging, point of sale, pricing strategies, product mix, marketing campaigns, store layout, category layout and other shopper-related challenges. Viewed through a VR headset, this virtual supermarket allows users to interact as they would in a real supermarket allows users to interact as they would in a real supermarket, whilst measuring and analysing for insights.

With Amazon and other global online retailers just some of many threats facing NZ retailers, it's obvious they will need to look at every possibility to deepen their engagement with their customers, and that includes immersive technologies.

Where are immersive technologies headed?

The World Economic Forum predicts AR/VR retail will generate US$4 billion by 2025, as part of a fast-growing AR/VR market that will generate nearly US$100b by then. Market analyst IDC is more aggressive in predicting the AR/VR market will hit US$215b by 2021 by doubling year-on-year i.e. exponential growth.

GlobalData digital retail analyst Andreas Olah is bullish, too. Retailers may have largely just experimented with AR/VR so far, but she says this is expected to change as major supermarkets, department stores, fashion retailers and DIY stores look to roll them out.

The emergence of "social VR in gaming", such as Facebook's Spaces or Microsoft's AltspaceVR, allows multiple players to be immersed with each other across platforms.

Industry commentator Emily Campbell is excited about this approach in that retail merchants can give social recommendations and social brand experiences to real shopping purchases. These group entertainment experiences can involve mall-style immersion in VR experiences with multiple audience participation.

She also writes about how a VR firm called Body Lab – recently sold to Amazon - is working on true-to-life avatars.

In parallel, Vive and Rift are VR systems which have external sensors that track you around a room safely. They also allow you to approach 3D products, look under counters, open drawers and pick up products with the same kinetic sensation as the real world.

Magix lets you change the colour of a couch by a simple hand movement. These advanced VR attributes will soon let you buy goods with a simple gesture, too.

Last August it was reported that Walmart had applied for two patents relating to showrooms and fulfilment centre management. That followed their purchase of VR start-up Spatialand. If you need any more evidence of their determination to stay at the front of the retail race, they employ more than 3000 people in their innovation unit @WalmartLabs.

Dan O'Shea, another retail expert talks about research into haptics, like feeling wet if your avatar enters a swimming pool, and then really goes out there.

"Let's see what happens with Neuralink et al in the coming months and years, because that may be a big game changer," he says. "Interfacing with the whole brain with the machine makes VR possible without clunky hardware, and without the need for more or less effective haptic suits … and THAT redistributes the game completely."

Put all this together and you really are talking about immersion.

Will immersive retailing dominate?

There are lots of questions. Technical issues include coping with demands for ever-increasing computational power – lag needs to be eliminated - and battery life, step changes in eye tracking and expanding the field of view of headsets, and reducing the weight and increasing the comfort of headsets.

The user community is still small. New platforms will stimulate demand, but prices will need to keep dropping, too. They'll need to move beyond the first mover geek factor, too – one teenager in my house described VR as "lame".

There are ethical and wider community impact issues like who owns the data, what impact it will have on social interaction, and what happens to traditional retail precincts?

Nevertheless, immersion in retail looks like it's growing exponentially.

AR helps stores stand out from their competitors, to personalise offering to customers, retail assistants with product knowledge, with stock management, and with deep analytics about customer behaviour. It helps customers with product knowledge and experimentation, store navigation, customer reviews, to engage in interactive fun, and to enjoy frictionless purchase and fulfilment.

VR does most of that, and also allows retailers to test ideal layouts and product placement, to reduce the size and number of their physical stores, to create fantastical stores that don't obey the mundane laws of physics, to train staff, and to project a rich customer experience around the globe. Customers can enjoy outlandish virtual experiences in-store, or join faraway friends in a virtual store where they can manipulate products in more ways than they could in a physical store.

Fears about negative impacts on people and places can just as readily be argued as strengths: for example social media has enabled us to engage with lots more people than we traditionally could in the physical world.

Within a few years, we'll all be fluent in immersive technology – not just in retail - and we'll come to see the current technology as the equivalent of the first wave of "cellular phones". Now, who's working on smell-o-vision (as long as it's not the smell of Millers or any other fabric warehouse)?

• Kevin Jenkins is managing director of professional services firm MartinJenkins, and a director of digital automation firm Quanton among other roles.