"Please think one more time about our review," he pleaded with the judge.
The couple's son, who was born in 2012, has suffered from autism and seizure disorder since he was one year of age.
The boy's mother later added: "The treatment [for autism] is so expensive in our country ... when we go back to Bangladesh we don't have anything, we are so poor."
In July last year the tribunal ruled, that while "unfortunate", the family do not have exceptional humanitarian circumstances which would make it unjust or unduly harsh to deport them back to Bangladesh.
The family arrived in New Zealand in October 2015 on visitor visas before the parents held work visas until November last year.
In November 2015 the family also lodged a claim for refugee and protected person status, claiming fear of serious harm due to the father's political associations with the Bangladesh Nationalist Party.
However, the claim was declined in June 2016 and an appeal against the decision was also declined on the grounds that the political claim was not credible.
Since the family's arrival in New Zealand, the boy has enjoyed medical care and treatment for his conditions, while also attending pre-school with the assistance of a teacher-aide.
At the tribunal hearing, the family argued that if they were to return to Bangladesh they would be unable to pay for the boy's medical care, which they said would amount to about $1315 per month.
The parents also argued their son will face a strong societal stigma due to his disabilities.
A Unicef report, provided to the tribunal, said the rights of children with disabilities to quality health care is not yet realised in Bangladesh.
"Such services tend to reach a small number of children, primarily in wealthier groups and urban areas," the report said.
Several letters from paediatricians in Bangladesh, relating to their observations about the boy's conditions and treatment, were also provided.
One paediatric neurologist and developmental paediatrician in Dhaka said that she had treated the boy since he was a year old.
She said the treatment did not "mitigate his substantial consequences".
In its decision, the tribunal accepted that the boy will "experience some stigma" and that it is unlikely he will be able to access the same level of treatment found in New Zealand.
"Given the serious deficiencies in the provision of education for the disabled in Bangladesh, the son's developmental progress and life skills will be far better assured if he is able to remain in the New Zealand health care and education system. It is therefore in his best interests that he is able to remain in this country," the tribunal said.
But it said, that throughout the family's stay in New Zealand the son has placed significant demands on medical, rehabilitation and special education services and he can be expected to do so into the future.
No estimated costs were provided to the tribunal for the son's ongoing care in New Zealand, but specialist education services conservatively estimated a cost of more than $35,000 per year, while annual costs for occupational therapy may reach $9000.
The tribunal further added that New Zealand was not obligated to provide "unrestricted access" to its health or special education services simply because a child's home country provides a lower standard of care.
"It is unfortunate", the tribunal said, that the boy will not have access to the same level of medical treatment, however, "the same could be said about the opportunities for most children from countries less developed than New Zealand".
"While the son has particular needs, his family are alive to those needs and will have the benefit of professional input into their child's autism, delayed development and epilepsy in Bangladesh," the tribunal said.
The tribunal said the son's circumstances may be similar to other disabled children in Bangladesh, and may even be considerably better than disabled children from poor families living in rural areas.
The tribunal said that any harshness involved in the son and his parents' deportation does not go beyond what is acceptable to maintain the integrity of New Zealand's immigration system.
Justice Duffy said "no one would argue" against the principal that children with disabilities are better off in New Zealand than in developing countries.
"That is a fact of life," she said.
"The issue here is, what is so exceptional on humanitarian grounds?"
Justice Duffy, who expressed her sympathy for the family, reserved her decision on the application for leave to appeal.
