The Briton, overall leader before the race and winner in Monaco last year with Brawn GP, parked his smoking car at the Sainte Devote corner at the start of the third lap after dropping to 11th place from eighth at the start.
The safety car had already been deployed after Germany's [ Images ] Nico Hulkenberg crashed his Williams in the tunnel on the opening lap, spreading debris across the track.
"I think it got a little bit hot on the way to the grid," Button told BBC television.
"We left a bung in on the left-hand side of the car that obviously you're meant to take out on the way to the grid. That's cooked the engine.
"It got very hot under the bonnet and I had to turn it off. I didn't want to leave oil on the race line. It would have been fine if we didn't have the safety car. That was what ended it," explained the 30-year-old.
"At such a slow speed you can't get any air into it to cool it down."
The Spaniard, twice a winner in the Mediterranean principality, started from the pitlane after crashing in final practice on Saturday but pitted at the end of the first lap for new tyres and then thrilled the crowd by overtaking a string of backmarkers.
After 22 of the 78 laps, the double world champion had worked his way into the points and up to eighth place as others ahead of him pitted.
Australian Mark Webber [ Images ] led from pole position.