A White House spokesman declined to offer an immediate comment on Trump's remarks.
Outlining a potential bipartisan deal, the lawmakers discussed restoring protections for countries that have been removed from the temporary protected status program while adding $1.5 billion for a border wall and making changes to the visa lottery system.
The administration announced earlier this week that it was removing the protection for El Salvador.
Trump had seemed amenable to a deal earlier in the day during phone calls, aides said, but shifted his position in the meeting and did not seem interested.
Graham and Durbin thought they would be meeting with Trump alone and were surprised to find immigration hard-liners such as Republican Bob Goodlatte and Senator Tom Cotton at the meeting.
The meeting was impromptu and came after phone calls this morning, Capitol Hill aides said.
After the meeting, Marc Short, Trump's legislative aide, said the White House was nowhere near a bipartisan deal on immigration.
"We still think we can get there," White House press secretary Sarah Huckabee Sanders said at the White House press briefing.