The 31-year-old was at the time playing rugby for Bath during his two year break from the NRL ahead of the 2015 Rugby World Cup in the UK.
Burgess told the Nova breakfast duo West sized him up as a "poor person" as they shared a meal and some drinks before heading upstairs to a penthouse to party some more.
"Honestly, it's pretty much what Russell said yesterday," Burgess said.
"It was a weird night. I'd driven from Bath over to the Dorchester (in London) for a session with Rusty.
"We were just sitting down having dinner. Obviously the drinks were flowing, as usual when me and Rusty are together. Kanye just walked in and sat down. I didn't think it was Kanye at the start because he was so small. I couldn't believe it and I didn't really care too much, but then I thought, that's pretty cool.
"He brought his laptop in. He was showing everyone his new (Yeezy) range and he also called me poor that night.
"He was showing me the new Yeezy stuff that he'd designed. He'd just been to Paris Fashion Week. Anyway, I'm going, 'That's cool. That's really cool'.
"And then he says, 'I'm making clothes for poor people just like you'.
"I thought, aww. I was kind of chuffed that he'd called me poor. I was like, 'aww, that's cool'.
"So I was like, 'Yeah, I'm a peasant'. But you can come and eat food off my table, Kanye. That's fine.
"That's what he said. It was hilarious."
Burgess later said he told West: "Gee, Kanye, if you think I'm poor, let me take you up to Dewbury (in northern England) where I'm from, if you really want to see poor people".
Burgess admits he fell under West's spell that night and repeatedly blew smoke up his backside when the pop star opened up about his plans for the future.
The retired Rabbitohs star also revealed West's bizarre choice of cuisine that night as they chatted away for several hours.
"He ordered two chocolate cakes and he was like slurping these chocolate fondant cakes. And I'm watching him eat these cakes going, 'What a peasant'.
"He called me a peasant and I am watching you eat chocolate cake like you've never eaten it before.
"I never get starstruck and I'm not really worried about what you do as long as you're a good person — and he was cool. He was a cool guy. He was really open to be fair."
He also says several emails from Kim Kardashian popped up on West's laptop screen as he showed off images of his clothing range to Burgess and Crowe.
Burgess admitted to the moment Crowe claims the star athlete upset West as they sung along to one of West's tracks from 2015.
Crowe said it was more than a little awkward when Burgess got the lyrics wrong as the trio were all singing along to West's Four Five Seconds hit which he produced alongside pop star Rihanna and Beatles legend Paul McCartney.
"Obviously it's a good tune and I was pretending to like it and I was singing along," Burgess said.
"I was totally wrong. I was off-tune. I was off-beat. Different words. Different days. I was way off, but I was having a good old time."
Crowe said he had met Kanye on several occasions over the years, starting with a get-together in Sydney.
"He wanted to meet me because he was doing a record with Jay-Z and they wanted to use some stuff from Gladiator … on one of their records," Crowe explained.
The actor then detailed "one of the funniest nights I've ever had with Kanye".
Crowe was in the restaurant with Burgess when Kanye approached the pair.
"Kanye comes down because he'd heard I was there and he's like, 'Can I come and hang?'" Crowe recalled. "I was like, 'yeah man'.
"So he came down and we had a couple of dumplings or whatever. He brought his computer with him. He was just scrolling through a bunch of stuff he had been working on."
Crowe said Burgess repeatedly got the days of the week wrong as they are mentioned in the hit track.
When Crowe told Burgess he was singing the wrong lyrics, Burgess replied: "It doesn't matter what day it is. Kanye, does it matter what day it is?"
As Crowe recalled: "Kanye just looked at him across the table and said, 'yeah, it matters what day it is'."
Hard to believe there hasn't been a second meeting between Burgess and West.
