Defender Sean St Ledger headed into his own net eight minutes from time after a mix-up with goalkeeper Mark Tyler which was welcome relief for Chris Turner's County who had soaked up a barrage of pressure and gone behind to a second minute opener from James Quinn.
Northern Ireland international Quinn fired home after Mark Arber had seen a powerful header cleared off the line, before St Ledger made a bad error inside his own penalty area.
County substitute Michael Malcolm punted in a hopeful ball into the box but St Ledger saw his backward header clear the advancing Tyler and into the empty net.
Stockport showed early on just why they have yet to keep a clean sheet this season, gifting Peterborough an early goal after some poor defending.
With two minutes gone Peterborough skipper Paul Carden whipped in a corner and Arber saw his powerful header cleared off the line by Stockport's Harpal Singh.
But the ball fell to David Farrell on the edge of the box. He teed up former Stockport loan midfielder Quinn who fired home from inside the penalty area.
The visitors nearly doubled their advantage five minutes later when Adam Newton supplied Farrell, but the midfielder dragged his shot wide of the far post.
There was no let up in the pressure and Richard Logan came close to making it 2-0, his 20-yard drive flying inches over the crossbar.
Carden saw a deflected shot well-saved by Duke a minute later, before Sagi Burton had a shot blocked at the back post after a clever pass from goalscorer Quinn.
Stockport had their first chance in the 34th minute. Mark Robinson hit a sweet volley which Peterborough keeper Tyler did well to save, before ex-Manchester City defender Tony Vaughan saw a header saved.
Newton saw a shot on the turn saved in the 35th minute as Peterborough looked to double their advantage, and on the stroke of half-time Carden had a fierce drive tipped over the bar by Duke.
Stockport started brightly in the second half and Matthew Hamshaw found Vaughan free at the back post with a teasing corner but the experienced defender volleyed wide.
At the other end substitute Dean Holden had a shot saved by Duke from a Carden corner, while Peter Gain went close, only to be denied by the keeper inside the six-yard box.
Duke produced another fine save, this time from Holden in the 63rd minute. Holden manufactured an acrobatic volley eight yards out but Duke was equal to the shot.
A minute later Burton saw a header hit the left-hand post as Peterborough continued to dominate and then in the 75th minute Logan had a shot saved from close range after a Quinn flick-on.
But despite their domination, St Ledger's late own goal gave the home side something to smile about, although they were lucky to get it.