The Dutch international, who joined on Thursday from PSV Eindhoven, pounced after Aiden McGeady's 20-yard drive came off the post midway through the second half.
Scott Brown had fired Hibs into an eighth minute lead following a fine run and pass from Moroccan midfielder Merouane Zemmama.
Celtic surged back though with Polish striker Maciej Zurawski levelling on 62 minutes and Vennegoor of Hesselink's winner came four minutes later in front of 58,078 fans.
Celtic top the standings on goal difference with 10 points from five games. Hearts also have 10 points after thrashing Inverness Caledonian Thistle 4-1.
"Physically he's (Vennegoor) no mug and on the ground also. I thought the link up play was fantastic," Celtic manager Gordon Strachan told BBC Scotland.
"The atmosphere today was special and that's why he (Jan) has come here.
"Although I thought we were alright in the first half, we did start sloppily."
Zemmama had given Celtic a fright after just three minutes when he raced in on goal to shoot past keeper David Marshall only for defender Stephen McManus to clear the ball off the line.
Celtic's Japanese midfielder Shunsuke Nakamura flighted a free kick against the face of the crossbar in the first half, but there were no takers for the rebound.
Hearts produced a convincing display after their midweek Champions League defeat by AEK Athens.
"It was a very important result for us. All the players tried to give their best... and the most important thing was to win and get the three points and we have got them," said manager Valdas Ivanauskas.
Hearts's goals came from Chilean striker Mauricio Pinilla, Jamie Mole, Andy Driver and Portuguese Bruno Aguiar while Graham Bayne hit back for Caley Thistle.
Rangers are away at Kilmarnock on Sunday.
