It is Brazil Rugby's dream to put on a good fight in front of the home crowd at that Olympics, although they acknowledge they are very much an Eric the Eel in the sport.
Mr Key said although the womens' team was successful internationally, the men were still struggling.
"That is why the Brazilians came to New Zealand and asked for help and support. It's a financial money-making deal for the Crusaders. They don't do this for love, it is essentially an export of sports technology and they get paid for it."
The Crusaders' Darryn Collins is in Brazil full time as head strengthening and conditioning coach for the teams and other coaches travelled over for shorter stints.
He had been in Brazil for seven months and coached about 80 players - under-19s, the mens XV and the mens' and womens' Sevens.
He said the Brazilian sides might not have any great success in the immediate future, but would be a lot stronger in five to 10 years - and he was hoping that the Rio Olympics in 2015 would help in the hunt for talent.
Mr Key told those gathered that he would be happy if Brazil moved up in the rugby world - as long as they didn't move ahead of the All Blacks.
Earlier in the day, Mr Key and his wife Bronagh visited the top of the Banespa building, which hosted the oldest bank in Brazil and is the highest building in Sao Paulo.
Tonight he is expected to take a bit of time away from official duties for a spot of sight seeing before flying to Brasilia tomorrow, where he will meet with Brazil's President Dilma Rousseff.