The England defender thumped a 25-metre shot past Pompey keeper Jamie Ashdown after 75 minutes to revive the champions who had been largely outplayed by the home side.
Campbell's goal, his first for Arsenal since August 2003, leapfrogged the Londoners over Everton into second place with 38 points from 18 matches. Chelsea have 43 points after a 4-0 home win over Norwich City on Saturday.
Arsenal's creative players were subdued against a Portsmouth side who were unlucky not to maintain their unbeaten league run since manager Harry Redknapp resigned last month.
While Arsenal enjoyed plenty of possession it was Portsmouth who carved out the game's best chances with Ricardo Fuller twice wasting great opportunities when through on goal.
Arsenal's Spanish goalkeeper Manuel Almunia, playing his sixth game since replacing first choice Jens Lehmann, was the busier of the two keepers, producing a late double save to keep out efforts from Steve Stone and Lomana Lua Lua.
"The goal came at the right moment for us, it was a mental blow for Portsmouth," a relieved Arsenal manager Arsene Wenger told Sky Sports.
"Many teams would have lost here today, but we showed great character and we are back in the (title) race now."
It was only Arsenal's sixth clean sheet in the league this season, and Wenger praised Campbell after he came in for criticism for Chelsea's goals in the 2-2 draw at Highbury last week.
"It was a great shot and I'm very pleased for him because he was not happy with the goals Chelsea scored last week," said the Frenchman. "We looked much more solid today.
"We are very hungry, determined and we have the talent to make the race very interesting."
Earlier Liverpool produced an impressive comeback to beat Newcastle United 3-1 at Anfield.
Dutch forward Patrick Kluivert shot a weakened Newcastle in front after 32 minutes but within six minutes a Titus Bramble own goal and Neil Mellor's well-taken goal swung the game in Liverpool's favour.
Milan Baros's 11th goal of the season sealed the points for Liverpool on the hour before Newcastle midfielder Lee Bowyer was sent off for a second caution.
Liverpool stay sixth with 28 points from 18 matches while Newcastle languish in the bottom half of the table with just one win in their last eight league games.
