Up to 40 Sydney Harbour Bridge workers have walked off the job over concerns two apprentices were exposed to asbestos on the iconic structure, the construction union says.

The CFMEU said between 35 and 40 Roads and Maritime Services (RMS) workers downed tools on the bridge on Thursday in response to the incident.

The two apprentice workers were exposed to asbestos on Wednesday when contractors were brought on to the bridge to do plumbing work, the union said.

"These are two young apprentices on the start of their working life and they now have the spectre of asbestosis and mesothelioma above them," CFMEU organiser Bard Parker said in a statement.


He said a "lack of care and proper planning by Roads Maritime Services management" was to blame for the incident.

Mr Parker said uncovered asbestos had been "piled up" near the workers' lunchroom and toilets.

He said workers on the site were "rightly upset" that they might have unwittingly carried asbestos fibres home in their clothes.

There would be no further work on the bridge until a meeting with RMS at 7.30am (AEST) on Friday, the union said.

RMS released a statement on Thursday confirming a "small amount" of asbestos was found during plumbing repair work on the bridge on Wednesday night.

The agency said samples of the potentially lethal substance were taken for testing.

"Given the location, there was no risk to the safety of workers of the public," RMS said.

It said a licensed contractor was removing the asbestos on Thursday.