Defending champion Nadia Petrova also went out, beaten 7-6 6-4 by fellow Russian and former U.S. Open winner Svetlana Kuznetsova in a closely-fought contest.
Mauresmo, the number two seed and twice a winner in Berlin, was leading Vakulenko 6-2 1-1 when they restarted their third-round match which had been suspended because of darkness on Thursday evening.
Vakulenko, ranked 53 in the world, stormed back at the restart, reeling off five straight games to win the second set and breaking second-seeded Mauresmo's serve four times in the match for a 2-6 6-1 6-2 triumph.
Mauresmo, back in action after a two-month layoff following an appendectomy, said she had completely recovered from the operation and was feeling fit.
"It's extremely frustrating," the Frenchwoman said of Friday's loss, adding that she was heading directly to Rome for next week's tournament, one of the last on clay before the French Open begins at the end of the month.
"I hope that I'll be able to get as much time as possible on the courts," Mauresmo said. "I need as much practise and as many matches as possible."
Henin had a far easier time against Maria Elena Camerin earlier on Friday, beating the unseeded Italian 6-1 6-3 in their rain-delayed, third-round match.
Henin, the top seed and a three-time winner here, must play Serbian fifth seed Jelena Jankovic later on Friday after rain earlier in the week created a backlog of matches.
The French Open champion took just over an hour to see off Camerin, pinning the Italian to the back of the court with her booming groundstrokes and breaking her five times.
If the Belgian beats Jankovic she will play third seed Kuznetsova in the semi-finals.
