Around 22 Houthi fighters were killed in the opening of the battle, while three coalition fighters died, according to medical sources.
The UN's main concern is that the fighting will damage the port or stop the arrival of food, medicine and fuel.
Around 22 million people in Yemen are dependent on aid, with at least eight million on the verge of famine.
About 10,000 people have been killed since fighting began in 2015.
The International Committee of the Red Cross said the attack was "likely to exacerbate an already catastrophic humanitarian situation in Yemen".
Anas Shahari, a spokesman for Save the Children Yemen, said that the port appeared to operating at nearly full capacity yesterday and that most of the fighting was happening on the southern outskirts of Hodeidah. But if the battle drew closer to the port or to the densely populated city centre, the effects could be devastating.
"There are 300,000 children in the city and many of them are malnourished and exclusively reliant on aid. If we leave them without assistance, a lot of children are going to die," Shahari said.
Yemen's internationally recognised Government, backed by the Saudis and the West, said it had "exhausted all peaceful and political means" to retake Hodeidah.
"The effort to liberate Hodeidah is the beginning of a complete victory," the Government said. The Saudi-led coalition alleges that Hodeidah is being used by the Houthis to smuggle weapons from Iran, including ballistic missiles which have been fired into Saudi Arabia.