Mr Trump once said: "Building this unit, if you look at the columns and the carvings, was harder than building the building itself."
But when he met President Barack Obama at the White House he also took time to look at that building and was said by aides to have liked the neoclassical architecture, the New York Times reported.
Having run as an outsider candidate determined to "drain the Washington swamp" Mr Trump is said to be reluctant to spend all his time in the capital.
It is possible he may spend weekdays in the White House, returning to Trump Tower or his Mar-a-Lago club in Florida at weekends.
The moving of his family to the White House is complicated by the fact that Barron is part way through a school year in New York.
Mr Obama moved in to the White House the day after his inauguration despite his daughters being in school in Chicago.
Security has already been beefed up at Trump Tower since the billionaire won the Republican nomination with people entering the building having to show identification.