Argyle won the third round tie at the second attempt to earn a fourth round tie at Barnet on January 27 with first-half goals from leading scorer Barry Hayles and David Norris.
However Posh, who parted company with manager Keith Alexander on Monday, shocked their Championship rivals by taking the lead in the 13th minute when Aaron McLean, who had earned the minnows a second crack at Argyle with the equaliser in the first game's 1-1 draw, took advantage of some defensive hesitation.
Stand-in manager Tommy Taylor saw McLean latch on to right-back Adam Newton's through ball ahead of Argyle defender Hasney Aljofree and goalkeeper Luke McCormick to stroke the ball home.
Argyle levelled within four minutes as Hayles stormed into the penalty area, exchanged passes with Reuben Reid and hit a well-placed shot that sneaked past United goalkeeper Mark Tyler for his tenth strike of the campaign.
A splendid solo effort from Norris midway through the first half effectively settled the game. After collecting the ball from Hayles, the Argyle player of the year rode a host of Peterborough tackles before steering the ball expertly past Tyler.
Peterborough refused to be daunted by the task that now faced them and took the game to their hosts, especially when midfielder Peter Gain replaced Gavin Strachan at half-time.
Posh should arguably have levelled matters when substitute Simon Yeo controversially beat the offside trap to send a shot in from ten yards that McCormick saved exceptionally well.
McCormick again came to Argyle's rescue by kicking the ball off his line after substitute Trevor Benjamin had worked an opening.