Turkish ballots are usually stamped before they are handed to voters but the board announced at the last minute it would accept unstamped ballots unless they could be proven to be fraudulent.
"The High Electoral Board has failed by allowing fraud in the referendum," CHP deputy chairman Bulent Tezcan said the party's headquarters in Ankara. Erdoğan's supporters took to the streets nearby in celebration.
The Yes vote gives Turkey's Government the authority to scrap the country's century-old parliamentary system and replace it with a presidential model.
Opponents have warned the new system will send Turkey lurching towards dictatorship as it would concentrate unchecked power in the hands of Erdoğan, who has jailed opponents and cracked down on dissent since a failed coup against him last year.
The new constitutional system will get rid of the role of prime minister and transform the presidency from a largely ceremonial position into a vastly powerful post as both head of state and head of the government.
The president will be able to appoint senior judges, declare a state of emergency, dissolve parliament and in some cases issue new laws be decree.
It will also theoretically allow Erdoğan, who has dominated Turkish politics as president and prime minister since 2003, to stay in office until 2029.
Most of the changes will not come into effect until after Turkey's next presidential election in 2019 but Erdoğan's towering position in Turkish politics means its unlikely that anyone will be able to challenge him.
Turnout was high in the election with more 86 per cent of the country's 55 million eligible voters casting ballots that were simply marked Yes or No. All three of Turkey's largest cities appeared to have voted No but Erdoğan gained enough support in rural areas to claim victory.
Three people were killed in an apparent political dispute at a voting station in the Kurdish city of Diyarbakır early on Sunday and two officials from the main opposition party were beaten at another polling station by unknown assailants.
At a secondary school in the Kosuyolu neighbourhood of Istanbul, voters mostly stuck to familiar scripts after they cast their ballots: Yes voters said they had faith in Erdoğan while No voters said they feared too much power would be given to one man.
But others offered surprising reasons for their vote.
Insaf Akay, a 37-year-old mother in a headscarf, said she was tired of being discriminated against by secular Turks, describing how she had been spat on in a market. "I think there will be more freedom for people like me under the new system," she said.
Mustafa Sacat, 62, said he normally voted for Mr Erdoğan and his AKP party but did not want to give up on the parliamentary system that has governed Turkey since 1920. "I like Erdogan but I want to keep the Parliament system."
The Turkish President staked his personal credibility on a Yes vote and every Turkish city and town was plastered with thousands of posters bearing his image. During the campaign he rallied nationalist voters by equating the No camp with terrorism and stoked an explosive diplomatic row with the EU.
