A spokesman for the whalers, Glenn Inwood of the Institute of Cetacean Research, confirmed the men were on board the vessel.
"They are unhurt, they are being questioned and there has been no decision on anything beyond that at this stage."
New Zealand-based Mr Inwood said it was wrong to say the incident happened in Australian waters. "Australia has legal jurisdiction out to 12 miles. The equivalent is 19km. This ... happened at 40km out."
Sea Shepherd's captain, Paul Watson, urged the Government to take action before the vessels left Australian waters.
"I think the Australian Government would be very embarrassed if an armed Japanese vessel can just pick up Australian citizens in Australia and then take them away to Japan," he said.
The three activists are Geoffrey Owen Tuxworth, 47, of Perth, Simon Peterffy, 44, of Bunbury, and Glen Pendlebury, 27, of Fremantle. AAP