The fast bowler broke down with a strain in his right groin during the fifth day of the second Test in Adelaide on Tuesday.
"I knew I had a chance of not playing but once you find out you won't play it hits you harder than you realise," said a disappointed Gillespie in a CA statement.
Australia, who trail India 1-0 in the four-match series, are expected to recall fiery paceman Brett Lee from injury on Monday when they name their team for the third test in Melbourne starting on December 26.
The home team, though, will again be without fast bowler Glenn McGrath, who has been ruled out of the series following ankle surgery.
"It's now a case of getting things right for the rest of the summer as we have plenty of cricket left to play," said South Australian Gillespie, who has led the attack in the absence of New South Wales pair McGrath and Lee.
Australia team physiotherapist Errol Alcott said: "Jason has strained his right groin and although the medical staff is confident it will resolve over the short term, we felt it was prudent for him to miss this next Test match."
The fourth Test in Sydney starts on January 2.
