Dr Sijen from the Netherlands Forensic Institute told the court last week she arrived at her conclusion that brain matter was present on the shirt based on tests performed on RNA in the sample.
RNA indicates which part of the body cells came from -- different to DNA which would indicate who the cells belonged to.
Prof Buston told the jury Dr Sijen did not follow written instructions when conducting tests on the sample.
"When you bake a cake you should follow instructions. But if you don't follow instructions, you should explain why you are not following the instructions."
There were problems with timings and the test, he said. The instructions said the testing time should be for 90 seconds, rather than 30 seconds used in Dr Sijen's test.
She also used less "primer" to detect RNA then the instructions specified, Prof Buston said.
"The method they are using for the detection of RNA is not really fit for purpose.
"I would be highly reluctant to accept the results of the [tests] because of the technique that has been applied."
Prof Buston also pointed to the process the institute used to test the RNA, Reverse Transcription, which gave variables, but it was the only test available to use.
He was also surprised to see that tests were performed on the RNA years after the tissue was deposited on the shirt because he did not realise the RNA would last for so long.
He had since discovered it was possible, but great care was needed to be taken with the samples, including how they were stored.
If the RNA was degraded, results might not be reliable, he said.
The trial in front of Justice Simon France continues.
