The R&A, which oversees golf, responded to the vote by decreeing that Muirfield would be prevented from hosting the Open Championship in the future.
But Alliss, 85, defended the ban today, insisting women can play at the club for free as guests but would baulk at paying a joining fee.
Alliss told The Telegraph: "The fact is if you talked to the wives of members of Muirfield they would be horrified at the prospect of being allowed to join.
"The wives can use all the facilities at the club but it doesn't cost them anything. If they had to pay to join they would be horrified.
"I spoke to women at Muirfield when the Open was last held there in 2013. I said: 'Are you all crusading to be members?' and they said: 'No fear'.
"This is nonsense. Women have always been allowed to play at Muirfield.
"I think it will be very sad if Muirfield didn't host the Open Championship because it is one of the best courses the Open is played on. It is bull****."
He went on to suggest the letter sent to members was "probably written by a woman". He added: "It wouldn't surprise me if a lady sent that around. The clubhouse is full of bloody women. They love going there for nothing."
The letter, headlined 'the risks', was signed by 33 members and circulated to fellow golfers at the east Lothian club and claimed: "The introduction of lady members is bound to create difficulties... They are likely over time to question our foursomes play, our match system, the uncompromising challenge our fine links present, our lunch arrangements. It will take a very special lady golfer to be able to do all the things that are expected of them."
Stuart McEwan, the Club Secretary, said: "I'm sure the letter might have swung the vote. That's what you call campaigning."
He added: "It's a blow for the club, a blow for the community and for Scotland. I can tell you how disappointed (the committee) are and how involved they have been with trying to push this through. It is a shame that it didn't happen."
Henry Fairweather, the club captain, said: "It is frustrating that we have got so close to achieving a two-thirds majority, without doing so." He said that "older members" might have been " more resistant to change."
Of the 616 members who cast a ballot, 397, or 64%, voted in favour of admitting women, while 219, or 36%, voted against. A further 14 votes in favour of change were needed to reach the two-thirds majority.
Muirfield, which last staged the Open in 2013, will now be removed from the rota of courses eligible to host the tournament.
Martin Slumbers, chief executive of The R&A, said: "The Open is one of the world's great sporting events and, going forward, we will not stage the Championship at a venue that does not admit women as members."
Royal Troon, which stages this year's Open in July, is the only remaining club on the Open rota with a male-only membership policy. The club is expected to push through a vote on women membership before this year's event, heading off a potential boycott of the tournament. It will be too late to strip Troon of holding the Open in the summer.
David Cameron condemned the vote. He said in a radio interview: "My general rule is that sports clubs should be totally open to both sexes, and it's outdated not to do that, particularly if you think that you're up to hosting important championships."
Nicola Sturgeon, Scotland's First Minister, said: "This decision is wrong and indefensible.
"I understand and accept that, as a private club, it is for Muirfield to decide on its membership - but at a time when Scotland is a country where women can get to the top in politics, law, business and other fields, this sends the wrong signal."
Heather MacRae, a professional golfer, tweeted her disapproval of the club, reflecting a view that Muirfield was horribly entrenched. "The 1st time I played Muirfield I had to sit outside after the game as I wasn't allowed in. I sat and watched a member go in with his dog!" she wrote.
Muirfield has hosted the Open 16 times. The course is home to the Honourable Company of Edinburgh Golfers, a society which was founded in 1744 and which claims to have both set down the original rules of golf and to be the oldest recorded golf club in the world.
