Cowell and Silverman have often been seen on vacation together and the 36-year-old brunette is a regular during his annual trip to St Barts.
Meanwhile, an assistant to Simon's representative called the matter "sensitive", telling People: "This is a very sensitive issue and all media enquiries are being dealt with by his U.S. attorney."
Comedian David Walliams appeared to confirm Cowell's baby news on Twitter.
He wrote: "Oh yeah, and @SimonCowell is doing a baby with a lady."
Walliams - who recently welcomed his first child with wife Lara Stone - then jokingly added: "For the record @SimonCowell and I tried to have a baby together for many years.
But I just couldn't get pregnant."
While Simon was previously against the idea of fatherhood, he revealed a change of heart last year.
He said: "I do really, really like kids, because I can talk to them and listen to them. The problem has always been how much time you need to devote to bringing up kids. I've always dedicated all my life to work, and at 52, I've probably missed the opportunity."
- AAP and Bang! Showbiz