Prince William will host the October 7 event and also present medals to 150 volunteers. A groundsman will work with royal household gardeners to create a pitch within the 16ha garden.
Polytechnic's opponents will be fellow Chiswick club Civil Service FC, the only founding FA club still in existence.
"We've been around 138 years, so we're slightly younger than they are," Mr Brown told the Herald.
"When you pull into the train station, they're on the righthand side, we're on the lefthand side. I think the first time we played each other was about 120 years ago."
Mr Brown, who played for Wellington's Miramar Rangers in New Zealand, said the Kiwi players - Tom Lancaster, Nick Brandford and Justin Fredrickson - were all national representatives at youth level, or had national league experience. All were fully amateur now, and next month's game would probably be the biggest of their lives.
"I don't think anything could really compare to this ... On the prestige stage, this is second to none," Mr Brown said.
While Prince William would be in attendance, he said, the Queen was unlikely to watch the game.
"I think she'll be based at Balmoral for the day, which I think may be one of the reasons why it's been allowed to go ahead, I don't know if she'd be too keen to see a bunch of blokes playing football on her backyard."
Regardless, he would be on his best behaviour.
"I'm going to have to watch myself on the sideline and keep the expletives down to an absolute minimum, ie, zero."