He was made professor of physics in 1984 and became the founding director of the MacDiarmid Institute for Advanced Materials and Nanotechnology.
Over the years he received huge international recognition for his research, including being elected a Fellow of the Royal Society of London.
It was the field of molecular physics, and specifically the properties of fluids using the technique of magnetic resonance, which cemented Sir Paul's name as a leading scientist.
Sir Peter Gluckman, the Prime Minister's scientific advisor, described Sir Paul as new Zealand's most distinguished public scientists and in the world of molecular physics has been a giant a "giant''.
"Over the last decade, Paul had became our most public and most recognised scientist. And he did so in a way that led him to be warmly received by all.''
He believed New Zealand was not using its innovative capacity well enough and was a strong advocate for the physical sciences and the potential of advanced manufacturing which would arise, Sir Peter said.
Last year he was named as the New Zealander of the year for his outstanding leadership for more than 30 years as a scientist, teacher, science administrator and communicator.
"Sir Paul is passionate advocate for addressing and finding ways to reverse the overseas 'brain drain'. In his view, widening New Zealand's economic growth engines can be achieved by using science and technology to create industry that is `world class and home grown'.'' his New Zealanders of the year award said.
His fight with cancer also took a public and scientific approach when he tried experimental intravenous vitamin-C treatments in battling it.
In June last year he was advised to take a break from chemotherapy to establish the full extent of the cancer's spread so he used the break to trial high-dose vitamin-C infusions.
"Let me be clear. I do not deviate one step from my trust in evidence-based medicine,'' Sir Paul said in his blog at the time.
"Why would I not try it?''
"Am I mad? Probably,'' he said.
In January, Sir Paul ended the vitamin-C treatment, saying there was "absolutely no evidence'' it had worked.
Victoria University vice-chancellor Pat Walsh today said the university was deeply saddened by his death.
"He was a leading light in the field of nuclear magnetic resonance, and in addition made a significant contribution to communicating science beyond the scientific community and to debate about New Zealand's future prosperity.''
"During his treatment for cancer, though an immensely difficult time for him and his family, he went about his work with characteristic good humour and stoicism.''
In 2010 the university awarded him an Honorary Doctorate of Science because of his numerous contributions and academic achievements, publishing more that 240 articles in scientific journals in the past 35 years.
Acting Prime Minister Bill English said Sir Paul Callaghan was an "outstanding New Zealander who made his mark as a world-leading scientist".
"He fought a valiant battle with cancer, and has been taken from us far too early,'' Mr English said.
Sir Paul earned the respect of everyone, including those who disagreed with him, and was a true public intellectual.
Sir Paul received many honours, almost too numerous to mention, throughout his career, Mr English said.
"Including several prestigious European awards, academic recognition from both Oxford and Cambridge Universities, the Rutherford Medal, a fellowship to the Royal Society of London, a Prime Minister's Science Prize, and the New Zealander of the Year award in 2011.''
He was also knighted in 2009.
Science and Innovation Minister Steven Joyce said Sir Paul also believed in the commercialisation of science, and used him founding Magritek - a Wellington-based company at the cutting edge of MRI and NMR technology - as an example.
"He believed science was not only about great ideas, but getting value from those ideas. Magritek leads the world in portable MRI technology, and wouldn't exist without Sir Paul's drive and innovation.
"His legacy to New Zealand will be a strengthened commitment to the power of scientific endeavour in leading innovation.''
Labour Leader David Shearer said Sir Paul had a brilliant mind.
"He was not only one of New Zealand's leading scientists, he was a pioneer. His cutting-edge research in the development of nuclear magnetic resonance methods has had an enormous impact in the areas of medicine, physics and biology.''
Sir Paul leaves behind a great legacy of ideas which the country would take forward into a new future, Mr Shearer said.
Green Party co-leader Russel Norman said Sir Paul was a great scientist and public intellectual.
"It is rare to have one person be a leader in so many pursuits. His public service was exemplary.''
"Sir Paul's passing leaves a major gap in our public and intellectual life. He will be missed.''
He leaves behind his wife Miang and two children.
