Police said the maritime launch was directed to the location and the Westpac Rescue Helicopter was put on standby.
Before they reached the fishermen, reports from the public came in that the boat had sunk and all three were in the water.
"The helicopter was immediately put in the air but the police launch was fortunately on scene shortly after and was able to pull the males and a dog from the water," police said.
The two fishermen were checked by ambulance staff but did not need treatment.
They had been in the water for about 15 minutes, and were being pushed further into the harbour by the wind.
Police said they were concerned the pair had gone out in rough conditions. While both were wearing lifejackets, one had failed to do his up.
"It is a timely reminder to all people venturing out on to the water to be mindful of your own and your boat's limitations and take all reasonable and sensible precautions at all times."