Inside Amazon’s AI chip lab - could it overtake Nvidia?

In a laboratory on the outskirts of Austin, Texas, a team of engineers are designing Amazon’s custom computer chips, including those used to train and fuel the latest artificial intelligence models.

“We started from a blank sheet of paper and we looked at these algorithms and we developed architecture that is optimised for these types of workloads,” Annapurna Labs director of engineering Rami Sinno told Markets with Madison.

Amazon acquired Annapurna, an Israel-based business, in 2015.

“They were a little startup at the time investing in custom silicon and we thought they were really interesting,” Amazon Web Services vice-president of compute and networking services David Brown told Markets with Madison.

The purchase defined Amazon’s transition from an online retailer to a technology software and hardware behemoth that sells cloud storage space, builds data centres and creates its own chips to go inside server racks.

“In 2018, we launched our first Graviton chip, we launched our second one in 2019, and today we’re on Graviton four,” Brown said.

“And with Graviton silicon we’ve been able to give customers about 40% better price performance, which is just an incredible number.”

And then came machine learning.

“That was the next adventure,” Brown said.

Annapurna’s most recent chip is called Tranium - named such because it’s used to train frontier AI models.

Amazon Web Services was building a cluster of 400,000 Tranium chips for OpenAI’s competitor Anthropic, the owner of Claude.

Could that scale see Amazon overtake the most dominant AI chipmaker Nvidia?

Go inside its lab in Texas to find out in today’s episode of Markets with Madison above.

Madison travelled to Austin, Texas, courtesy of Amazon Web Services. The company had no editorial control over what Madison produced.

Get investment insights from executives and experts on Markets with Madison every Friday here on the NZ Herald, on YouTube and wherever you get your podcasts.

Sponsored by CMC Markets.

Disclaimer: The information provided in this programme is of a general nature, and is not intended to be personalised financial advice. We encourage you to seek appropriate advice from a qualified professional to suit your individual circumstances.

Madison Malone (nee Reidy) is host and executive producer of the investment show Markets with Madison. She joined the Herald in 2022 after working in investment, and has covered business and economics for television and radio broadcasters.