He pulls out a photo of all five band members on a hotel bed in The Hague, in 1964.
"This is us signing records just before our first gig outside the UK, which ended up in a riot, a really nasty one. It wasn't just little girls running onstage. They tore this beautiful concert hall to pieces, ripped all these beautiful tapestries off the walls.
"I remember the police horses going up the steps outside. Very peculiar. I suppose it was a bit like those early rock and roll movies where people tore up all the cinema seats."
At the start of 1965, the band played to 100,000 fans in Australia and New Zealand, playing venues as small as the Civic Theatre in Invercargill, the town that Keith Richards famously dubbed "the arsehole of the world".
No one in the 60s imagined this pop music lark would last five years, Jagger says, let alone 50. "Everybody thought, 'let's make a bit of money and invest in a hairdresser's or something'. No one thought it would carry on, including The Beatles - though actually Bing Crosby and Frank Sinatra had had screaming girls since the 1940s. But then the whole longevity thing happened and the industry changed."
The high point was the Stones' first worldwide Number One with (I Can't Get No) Satisfaction in July 1965 when Jagger was still only 21.
"We'd been popular until then but that was a big moment for us. There's been some awful points of course, mostly to do with drugs and imprisonment - getting busted and put in prison, that was not good," he says with enormous understatement.
"All of those things are very distracting from what you actually want to do in life and that's make music. You have to spend a lot of time and energy defending your behaviour against the law and the press, which was such a big deal in those days.
And all those things we were accused of then are more or less everyday behaviour now. Now it wouldn't mean much. It was a different time - though now if you say something you regret on Twitter or make anti-Semitic remarks or something it can be a low point and a lot of actors have suffered from that."
The band never felt much of a sense of competition with anyone, apart from The Beatles. "When you're 19 you feel competitive," he says. "When you're older, you don't feel like that any more. U2 and Bruce Springsteen have been around for so long I can't remember a time when they didn't exist. I first saw Led Zeppelin at Madison Square Garden and thought they were really good, a different kind of music with a different approach. You need some healthy competition, really. It's like football - 'well they're useless, we'll show 'em'!"
Why weren't the Stones on the bill at the Queen's Diamond Jubilee concert last month?
"We got the call from Gary Barlow but we just weren't ready," he shrugs, "but I thought Grace Jones and that hula hoop was pretty good. I don't see many shows these days but I did see Olly Murs in New York a few weeks back, playing to a really tough audience and he worked the room so hard. I thought, 'he'll give up soon', but he didn't let go and, in the end, they did get up and dance."
The old rascal is now 68. What advice would he give his teenage self in the earliest band photos?
"None!" he cackles. "You can't give a 19-year-old advice, he won't listen to you. Completely pointless. They don't take any notice."
How have the Stones lasted so long? "There's lots of lessons you learn. You have to keep rolling with the punches, you have to absorb the good things and the bad things. It's not all going to be good, same as anybody else's life. All of your life isn't going to be wonderful and up and you can't expect it to be.
"The thing about showbiz is it has a kind of forced projection where everything is wonderful - although some do the opposite, the Amy Winehouse view that everything is awful all the time.
"But the old-fashioned view of showbusiness, which is the model followed by most people, is that you are always up. If you appear somewhere you're always happy, you're always smiling, you're always full of energy. Of course things aren't really like that: you have to fight through that and you either come through it or you don't. In the end you have to be optimistic.
"It's nice to get to this milestone of 50 years and I hope we can play and do a lot of things to celebrate it - though I don't think there's many ambitions left that haven't been achieved. The Rolling Stones as a band have been pretty lucky, really. We don't have a lot to complain about."
- Daily Mirror
