Sri Lanka has resisted calls for an international investigation of abuses such as civilian deaths during its long civil war.
The Commonwealth is a grouping of 54 nations that were once part of the British empire. The ministers' meeting comes two months ahead of the Commonwealth leaders' summit in Colombo, after which Sri Lanka is set to serve as the organization's chair for two years.
Amnesty said Sri Lanka should be barred from hosting the summit because of its "disturbing human rights record."
Friday's meeting in New York "is an opportunity for the Commonwealth to show some real leadership on human rights," said Polly Truscott, Amnesty's Deputy Asia-Pacific Director in a statement. "The organization has been shamefully silent so far about Sri Lanka's human rights crisis - including the persistent lack of justice for past crimes and ongoing attacks on human rights defenders and other activists."
Human Rights Watch said the holding the summit in Sri Lanka "casts serious doubts on the Commonwealth's commitment to supporting human rights, democratic reform, and fundamental human right."
There was no immediate government response to the statements of the two rights groups, but in the past top Sri Lankan officials have rejected such criticism saying those two groups are biased.
Meanwhile, Australia and Britain also members of the Commonwealth are encouraging engagement rather than isolation of Sri Lanka in the face of criticism over its human rights. Both nations are encouraging countries to participate in the November summit.