Williams was red-carded less than a minute after coming on as an injury-time substitute for a foul on Jack Lester which gave Chesterfield a penalty that was saved.
But Williams took far too long to make his exit and had to be told to leave the field by the referee before the spot-kick could be taken.
Chesterfield were clearly annoyed by what was blatant gamesmanship and Williams is almost certain to be reported for refusing to leave the field of play.
It was a sour ending to what had been an excellent game which the Posh won through goals by Aaron McLean and George Boyd, with Lester replying late on for the home side.
But they needed a superb penalty save by Joe Lewis, who was mobbed by his team-mates at the final whistle, after he dived to his left to keep out Peter Leven's spot-kick to ensure the win.
Peterborough had been coasting towards victory in a game that turned on a costly error by Barry Roche, who dropped a low cross allowing McLean to bundle in his 22nd league goal of the season.
The visitors needed no assistance for their second two minutes after the interval when Craig Mackail-Smith sprinted away down the right and his deep cross was brilliantly volleyed in at the back post from 12 yards by Boyd.
Peterborough should have scored again when Adam Newton cut through Chesterfield's defence, but Mackail-Smith failed to turn in his low cross and Roche had to tip over a 25-yard shot from Charlie Lee.
McLean was denied by a goal line clearance in the 80th minute and the visitors were comfortable until a corner fell to Lester inside the six-yard box and he put away his 23rd league goal.
When Williams dragged him down Chesterfield had a great chance to salvage a point, but Lewis emerged as the hero with a full-length save that could be worth so much more come the end of the season.