The Robins took a first half lead but were pegged back by a determined Peterborough side with 15 minutes left.
As injury time loomed, the home side won a corner when Peterborough goalkeeper Mark Tyler flew to save from a Gavin Caines free-kick.
Grant McCann's flag-kick was only half cleared and substitute Craig Armstrong drove a low shot back through a crowd of players and inside the far post to secure Cheltenham's second consecutive win.
Having survived an early scare when Adam Newton shot just over the crossbar for Posh, it took Cheltenham 36 minutes to create their first real chance of note.
John Finnigan, with a long-range shot, and Aaron Brown, with a header, both missed the target as the Robins struggled to break down their visitors, and Tyler was finally called upon to save an on-target shot when McCann tested him with a 25-yard free-kick.
A goal had seemed unlikely at that stage and it took a piece of uncharacteristically hesitant goalkeeping from Tyler to open the scoring.
Cheltenham forward Kayode Odejayi fed Brian Wilson on the right and his hopeful cross caught Tyler in two minds. He started to come then changed his mind and the ball fell for McCann to score with a simple header at the far post.
Cheltenham continued to carry the greater attacking threat at the start of the second half and went close to increasing their lead when McCann volleyed just wide then Odejayi, turning sharply onto a pass from Wilson, smacked a shot against the base of the far post.
Having been given a let-off Peterborough sensed there was still something in the game for them and their cause was helped by some nervous defending by the home side in the latter stages.
First, Dean Holden drilled a shot against the post from 18 yards out then, with 75 minutes gone, Holden broke forward after a spell of Peterborough pressure and crossed from the left. St Ledger went for his cross at the near post but missed it, although Quinn was following up behind him and made no mistake from close range.
At that point a draw looked by far the most likely outcome, although Cheltenham sealed the points with their late rally to make it four wins from 11 matches this season.