Rory McIlroy's recent withering criticism finds an echo down the ages in the contemptuous verdict of the Irish Golfer magazine: 'The Olympic Association may call the golf competition it promotes the World Championship or 'The Supremacy of the Utmost Surreal Horizon' or any other high-sounding title which rings sweetly in American ears; nevertheless, a grandiose title cannot, per se, give prestige to a meeting.'
Such sentiments clearly won the day, for the Canadian gold medallist George Lyon turned up to defend his title at the 1908 London Olympics . . . and found he was alone. The golf was duly cancelled - and the sport began its long years of exile.
Fraser's book details how close it came to being ended in Atlanta in 1996, with the organiser Billy Payne - now chairman at Augusta National - lobbying hard for it to take place at the home of the Masters.
The IOC were won over, until it was pointed out that golf taking place at a men-only course with not a great history for race relations might not be the smartest move.
Then there is the Hitler Trophy, the book's eye-catching title. Here, thanks to the author's painstaking research, we have the definitive account of the extraordinary story of the Fuhrer's quest to have golf included in the Berlin Games in 1936.
When he didn't get his way, he organised his own tournament at Baden-Baden - complete with his own, personally gifted trophy - that was an Olympic event in all but name as far as the Germans were concerned.
As Fraser points out: 'Flags were raised at the opening ceremony and at the trophy presentation; anthems were played and medals presented. The winners were even given fir trees in a mirror of the Berlin Games which saw each gold medallist receive an oak.'
It's a matter of great debate as to whether Hitler was going to present his own trophy to the winners. When Germany led at the halfway stage, did he make plans for the long drive, only to change his mind when the hated British turned the tide and ended up the winners?
The Hitler Trophy, after a long and arduous journey in itself, now resides at Hesketh Golf Club near Southport, the home course of one of the two winners, Arnold Bentley.
So does Bentley's prize of a fir tree, planted within sight of the clubhouse and which has thrived, despite members from long ago forsaking the toilets on occasion to underline their loathing of the German dictator with their own watery ritual.