While the majority of those polled by ISM continue to be positive about business conditions, "there is an underlying concern regarding the uncertainty of the future economy," the institute said in a statement.
"That's been the process this whole recovery-you get one strong quarter and then things start to slow down again," Scott Brown, chief economist at Raymond James in St Petersburg, Florida, told Reuters.
All eyes now are on Friday's monthly payrolls report, which is expected to show employers created 200,000 jobs in March. The unemployment rate is forecast to stay at 7.7 per cent.
In afternoon trading in New York, the Dow Jones Industrial Average declined 0.74 per cent, the Standard & Poor's 500 Index fell 1.04 per cent, while the Nasdaq Composite Index dropped 1.29 per cent.
In Europe, the Stoxx 600 Index finished the day with a 0.9 per cent slide from the previous close. Benchmark stock indexes in Frankfurt, London and Paris weakened as well, giving up 0.9 per cent, 1.1 per cent and 1.3 per cent respectively.
Commodities including gold, silver and oil also took a hit from the sobering data on the world's largest economy, reflecting concern demand might ease.
Conversely, US Treasuries became more attractive. The yield on the benchmark 10-year note shed five basis points to 1.81 per cent, the lowest since January 24, according to Bloomberg. The 30-year bond yield fell five basis points to 3.05 per cent.
In Europe, the ECB is widely expected to maintain its key interest rate at a record low 0.75 per cent when policymakers meet on Thursday. Two of 56 economists surveyed by Bloomberg predict the ECB will further reduce its rate to 0.5 per cent.