It is the fifth successive season that Rotherham have exited the competition at this stage against lower league opponents and they very much contributed to their own downfall after being caught by swift home counter attacks.
Despite having nine-goal top-scorer Trevor Benjamin ruled out with flu just before kick-off, Posh had the better of a lively opening spell and Rotherham goalkeeper Neil Cutler did well to block a third minute strike from Aaron McLean before Richard Butcher hooked over the bar a minute later.
Home skipper Guy Branston, who spent five years at Rotherham earlier in his career, came out on top in a close quarters tussle with Will Hoskins, although the Millers hit back immediately when Delroy Facey's shot was blocked and Eugen Bopp fired the rebound well wide.
Posh boss Keith Alexander signed McLean on loan to boost his fire power but the Grays frontman proved equally effective in build-up play.
After Michael Keane's poor corner, McLean released Adam Newton whose 29th minute cross from the byline was dispatched into the roof of the net by Butcher to claim his third goal of the season.
McLean, whose transfer is likely to be made permanent in January, should have doubled the home lead only to blaze high over the bar when set up after Danny Crow got the better of David Worrell.
Skipper Colin Murdoch moved up to head too high from a corner just before the break but Posh survived and deserved to go in ahead at the break.
Rotherham started the second period strongly with Hoskins sending a header a foot too high from a Ritchie Partridge centre.
Branston then cleared a Murdoch header off the home line and Mark Tyler was again impressive palming away a fiercely struck 25-yard effort from Bopp.
Posh doubled their lead on 59 minutes when they stormed forward after successfully defending a corner by Bopp with Crow clipping a precise pass for McLean to bear down on goal and fire home a low right-foot shot.
Rotherham were again caught napping at the back in the 72nd minute when Crow nipped in to pinch a cheeky goal as Jude Stirling's long throw sailed unchallenged into the penalty area.