The following morning, the twins were back on site, when Richardson approached them to raise his concerns.
"How you guys are at the moment, is concerning me," he told them.
"I think emotionally you guys are struggling. You've been getting an absolute pounding social-media wise. You guys are struggling, you have to admit over the past couple of weeks, it's really ground you down."
He then continued: "I think if you continue in this state, you won't get the house finished to the standard that I want, you want and I think the market will want. What I feel is best for you guys is if you just flag this week.
"It's not an option. It's non-negotiable," he told them.
The pair are the first team in Block NZ history to be ordered off-site.
A teaser for Sunday night's episode showed the remaining three teams gather for judging at the Challenge Warehouse - and no sign of the twins.
In a statement, a MediaWorks spokesperson confirmed The Block Facebook page is moderated.
"We have an important duty of care responsibility to our contestants across all Three reality shows. It was acknowledged during last night's episode that there has been a significant amount of negativity on social media directed towards Julia and Ali in recent weeks and combined with the stress of participating on the show, we decided that it was necessary for them to have some time off. The Block NZ and associated Facebook pages are moderated and comments that are abusive, offensive or aggressive are removed."