Full-time score from The Den
With the the two teams locked at 0-0 and the game heading towards a penalty shoot-out, Williams produced a textbook header to power home a pinpoint cross from Carlos Edwards securing The Lions' first home win after a run of four successive defeats on home soil.
The terrier-like Ben Thompson showed great tenacity and persistence to keep hold of the ball under pressure and work a yard of space before letting fly with a dipping 18-yarder on 71 minutes that bounced narrowly wide as The Lions continued to press for the opening goal.
Less than 60 seconds later Upson ghosted into the penalty area brilliantly and saw his goalbound effort saved by Alnwick. Mark Beevers then clipped the woodwork with a deft flick from another Williams' cross, yet still that elusive goal would not come.
Onyedinma did well to weave his way into the penalty area, but with Edwards in space on the overlap, went for goal himself and failed to make a clean connection with the ball squirming wide of the target.
It looked as if the game was heading towards a penalty shoot-out, but when the fourth official held up the board indicating five minutes of added time, the home crowd found their voices once more and the players responded in kind. With just three minutes remaining Edwards received the ball on the right, looked up and picked out Williams with a superb deep cross that the midfielder rose majestically to head home for a fine winner.
Millwall: Archer, Edwards, Craig, Beevers, Nelson, Williams, Thompson, Onyedinma (Webster 90+4mins), O'Brien (Cowan-Hall, 81mins), Marquis (Gregory, 69mins), Upson. Subs not used: Girling, Powell.
Scorer: Williams (90+2mins)
Peterborough United: Alnwick, Taylor, Bostwick, Vassell (Couilbaly, 72mins), Maddison, Almeida Santos (Smith, 63mins), Washington, Anderson, Davey (Beautyman, 20mins), Elder, Forrester. Subs not used: Henry, Angol.
Bookings: Vassell (35mins), Bostwick (62mins)
Ref: K. Hill
Att: 2,050 (away 222)