However, the League Two team's supporters were soon crying into their beer as two goals inside three minutes before the break put Championship side Plymouth in front.
The Carling Cup's sponsors gave Posh fans two crates of beer to celebrate their 570-mile round trip to Devon and Plummer soon became their toast with his first senior goal.
He lashed home the ball from 12 yards after his fellow defender Mark Arber had crashed a header against the crossbar following Paul Carden's 21st minute corner.
Plymouth, seeking to reach the second round of the competition for the first time since 1992, fought their way back on level terms when David Farrell's clumsy challenge on David Norris gave Paul Wotton the chance to equalise with a powerfully-struck 35th-minute penalty.
Within three minutes, the hangover set in as recalled striker Scott Taylor turned the tie on its head, seizing on midfielder Akos Buzsaky's deflected shot to shoot home unmarked.
Peterborough manager Mark Wright said he did not think Argyle should have been awarded the penalty from which they equalised.
"Their player kicked David Farrell's leg and went down," he said. "I think it was a poor decision,
"Do I feel hard done by? Not really, because Plymouth are a very good side, but I would like the referee to look at his decision."
Argyle manager Bobby Williamson said: "I think we can play better than that. Lots of times we gave the ball away needlessly.
"But I always look for perfection. You want players to raise their standards as high as possible, and that's all you ask. We're pleased to be through, though."