They looked down and out when they went two behind but inspired changes saw them claw a way back with goals from Danny Crow, his tenth this season, and a second from the outstanding Aaron McLean.
Then with a minute to go, Rovers skipper Stuart Campbell sensationally gambled with a 25-yard free-kick that flew in just under the bar.
Even then Posh were not finished, for in added-on time, Josh Low thundered a shot against the bar and Dick Butcher headed just over the top.
Though the late win boosts Rovers hopes of seventh-place in the table, while the Posh after losing eight matches in a row see a revival after their midweek draw with Swindon and this so-narrow reverse.
Rovers took the lead with a long throw by Andy Sandell, which swirled in front of Peterborough's big centre-back Ben Futcher.
As he stretched out his arms, a hopeful shout of 'hands' went up as the ball bounded off a knee. It was a lucky deflection for Rickie Lambert, who hit it on the turn, wide of keeper Mark Tyler.
Posh were stunned and in the next ten minutes Rovers' top scorer, Richard Walker, had two chances to bury them but shunned them both.
First, he had time to control a cross at the far post expertly delivered by right-back Ryan Green but Tyler made a comfortable mid-air save.
Then minutes later, Walker was slow to react to another good ball, this time from Sammy Igoe.
Rovers though forgave him early in the second half when he lashed in a 30-yard volley, after the ball had been knocked down by Lambert.
Earlier George Boyd, who has yet to score since joining the Posh from Stevenage, clipped the outside of Steve Phillips' left post.
Rovers were also rescued by the referee, who ruled that Aaron McLean had fouled Phillips before trickling the ball over the line.
McLean, by far the most enterprising Posh on show, sparked a revival which gave Ferguson everything to work on at half-time.
Ferguson's analysis looked in doubt when Rovers doubled their lead, but his substitutions rapidly changed the entire thrust of the match.