Prince Philip also reportedly made a joke about a man with a beard, prompting laughter from those who heard.
As he walked with Princess Anne, he pointed at the gentleman, who had long, ginger facial hair, he asked a bodyguard: "Is that a terrorist?"
An onlooker in the crowd said: "The man appeared to be his with wife and child. Philip was wishing lots of people a Happy New Year and then he spotted this guy with his distinctive beard.
"He pointed at him in a funny way and turned to one of his Royal bodyguards, saying: 'Is that a terrorist?.'
"He was obviously having a little joke, but he said it within earshot of the man who burst out laughing and appeared to find the whole thing hilarious. I think Philip knew he was going to be overheard.
"I'm sure Philip didn't mean any harm. His grandson Prince Harry has a ginger beard, so perhaps Philip had been cracking jokes about that over Christmas."
Ian Smith, 39, added: "Everyone knows Prince Philip has got a sense of humour, although some people night have though it was a risqué comment."
The Queen was among those walking into the church, but Prince Harry and Meghan Markle weren't seen at the service, and nor were the Duke or Duchess of Cambridge.
After the 50-minute service, at which the congregation sang the National Anthem and Christmas carols including It Came Upon the Midnight Clear, As With Gladness Men of Old and Hark the Herald Angels Sing, the Queen walked to her car as an audience applauded.
Prince Philip decided to be politically correct after he walked out of the church, and didn't make any more edgy remarks.
However, his attentions were again captured by a dog, this time a large rare breed Black Russian Terrier called Boris.
Neil Richards, 55, who owns the dog, explained: "Boris is an unusual dog and quite big. Philip asked what breed he was and when my wife said he was a Black Russian Terrier, he looked surprised and joked, 'A terrier?'
"It was nice that he spoke to us. It is a good example of his sense of humour. We brought Boris to see the Royals at the church two-years-ago, but nobody noticed him then."
Buckingham Palace declined to comment on Prince Philip's jokes.
Prince Philip's gaffes
"British women can't cook" to the Scottish Women's Institute, 1961
"It is very difficult at all to see how it is possible to become immensely valuable by singing what I think are the most hideous songs" to Tom Jones, 1969
"I declare this thing open, whatever it is." During a visit to Canada in 1969.
"If you stay here much longer you'll all be slitty eyed" to British exchange students in Beijing, 1986
"You can't have been here that long — you haven't got a pot belly" to British tourist in Hungary, 1993
"How do you keep the natives off the booze long enough to pass the test?" to Scottish driving instructor, 1995
At a tree-planting ceremony in Hyde Park in 2011, the Queen met 16-year-old Army cadet Stephen Menary, who lost an arm and most of his sight in an IRA bomb attack. When the Queen asked him how much he could see, Philip interjected: "Not a lot, judging by the tie he's wearing."
"If you're near that music, it's no wonder you're deaf" to children from the British Deaf Association, standing by a steel drum band, 1999
"Oh it's you that owns that ghastly car is it? We often see it when driving to Windsor Castle" to Elton John, 2001
"You look like you're ready for bed" to President of Nigeria Olusegun Obasanjo, who was wearing traditional dress, 2003
"The Philippines must be half empty, you're all here running the NHS!" to a nurse at Luton and Dunstable University hospital, 2013
"I wish he'd turn the microphone off" muttered at the Royal Variety Performance as he watched Sir Elton John perform, 2001.
"It is my invariable custom to say something flattering to begin with so that I shall be excused if by any chance I put my foot in it later on" at a speech in 1956
