"I put it on and, for a few moments, became a medallist myself," Yannis Zavos told Apoyevmatini newspaper on Thursday.
"But the wife started teasing me."
The rotund, bespectacled Zavos, a cabbie for 42 years, said he never thought of keeping the silver medal, left by Dutch rower Diederik Simon in the back of his taxi.
"I didn't win it. Somebody worked very hard for that."
Crooked cabbies have been a byword among visitors to Greece for years and Zavos was keen to stand up for his profession.
"You hear a lot about taxi drivers but we are not all the same," he said. "There are a few in every profession who don't do their jobs properly and give everyone a bad name."
