Carlos Osorio, general secretary of the Brazilian bid, told Reuters: "Overwhelming, spectacular, unbelievable."
The Brazilian delegation broke into singing their 'Marvellous City' song, all waving flags and hugging each other.
The announcement, which was delayed by several nervous seconds as IOC president Jacques Rogge struggled to open the envelope, left Brazil [ Images ] President Luiz Inacio Lula da Silva and much of the rest of the Brazil bid team in tears of joy.
"Like in every competition there can only be one winner," Rogge said.
"Tonight I have the honour to announce that the Games of the 31st Olympiad are awarded to the city of Rio de Janeiro," Rogge said.
In the final round of voting, Rio polled 66 votes to 32 for Madrid.
Carlos Nuzman, Rio bid leader, hugged President Lula, both in tears and said: "We did it, we did it."
Former tennis champion Gustavo Kuerten, a bid ambassador, said: "This is an amazing result. We are going to be working from tomorrow to make it happen. Brazil will do the best it can from this great opportunity."
Spanish Prime Minister Jose Luis Rodriguez Zapatero told Reuters: "Rio was a great candidate. We put up a great fight."
In an astonishing start to the voting, Chicago, the odds-on favourite, went out in the first round, despite receiving unprecedented support from United States President Barack Obama [ Images ] and the first lady.
It was a rebuff for Obama, who had become the first sitting US president to address an IOC session.
Tokyo followed them out in the second round, leaving Madrid and Rio to slug it out in the final round.
President Obama had put his personal political credibility on the line by flying in to the Danish capital earlier on Friday to urge the IOC to choose his home town of Chicago. His wife, first lady Michelle Obama [ Images ], had spent two days in Copenhagen charming IOC members.
Almost no one had expected such an astonishing rebuff. Chicago finished last of the four bids in the first round of voting by 95 eligible members.
As none of the other three reached an overall majority, a second round of voting was held with Tokyo coming last, leaving Madrid and Rio in the third and final round.
The result of the final round of voting will be announced by IOC president Jacque Rogge at a ceremony starting at 5:30 p.m. British time.
Most observers had predicted a close contest between Chicago and Rio de Janeiro.
Though the US President and his wife produced strong appeals in the day's first 45-minute presentation by Chicago, they were almost certainly undone by the emotional tugs provided by Brazilian President Luiz Inacio Lula da Silva for Rio and former IOC president Juan Antonio Samaranch for Spain.
Lula raised the emotional stakes in his direct appeal to the IOC to stop favouring Europe, North America and Asia and take the Games to South America for the first time.
"This is a continent that has never held the Games," he said. "It is time to address this imbalance. The opportunity is now to extend the Games to a new continent. It's an opportunity for an Olympics in a tropical country for the first time, to feel the warmth of our people, the exuberance of our culture and the sensation of our joy."
Even more emotionally, Samaranch, now 89, pulled powerfully at the heart-strings of members when he spoke for Madrid. "I know I am very near the end of my days," he said. "May I ask you to consider granting my country the honour and also the duty to organise the Games in 2016?"
Obama's appearance, the first by a sitting US President at an IOC session, provoked huge interest from IOC members, even though they are used to being courted by major political figures.
Obama told the IOC: "I've come here today to urge you to choose Chicago for the same reason I chose Chicago nearly twenty-five years ago, the reason I fell in love with the city I still call home."