Shaun Cummings' 90th-minute winner dumped reigning Premier League champions Leicester City out of The Emirates FA Cup on Saturday afternoon.
Neil Harris made just one change from the team swept aside Port Vale on Tuesday night, with Shane Ferguson coming in for Jed Wallace. Lee Gregory and Shaun Hutchinson both made a welcome return to the Millwall bench.
An end-to-end opening culminated in the first shot of the game, which went the way of the visitors, as Demarai Gray shot wide.
Byron Webster made a stunning block to prevent Ahmed Musa handing The Foxes the lead on 15 minutes, with Jordan Archer saving from Shinji Okazaki from the resulting corner-kick.
Ferguson had an attempt blocked as The Lions look to gain the ascendancy shortly before the half-hour mark, but they could not find a way through the Leicester defence.
On 41 minutes, Steve Morison's header from a Ferguson free-kick was deflected over the crossbar, whilst from the following corner-kick, Calum Butcher headed straight into the arms of Foxes goalkeeper Ron-Robert Zieler.
Williams drilled a shot low into the path of Zieler as the second-half got underway, whilst Musa also saw a deflected effort fortunately fall into the hands of Archer.
The Lions went down to ten-men on 52 minutes, as Jake Cooper - booked towards the end of the first-half - picked up a second yellow card for a foul. In the wake of the red card, Harris moved Tony Craig to centre-back, with Ferguson filling in the left-back slot.
Despite their disadvantage, Millwall kept coming, with Williams' rasping drive making Zieler work just after the hour-mark.
Substitute Jamie Vardy drove wide with 76 minutes on the clock, whilst at the other end, Aiden O'Brien headed over from a Cummings cross.
Archer spread himself to pull off a magnificent save with ten minutes to play, after Vardy's shot fell to Okazaki just five yards out.
The goalkeeper again performed heroics with 85 minutes played, as he palmed away Okazaki's deflected cross, which looked bound for the back of the net.
With 90 minutes on the clock, The Lions took the lead. Good work down the right-hand side found the onrushing Cummings, who had no qualms with shrugging off his marker and sliding the ball past Zieler.
The Den erupted - it was the defender's first strike since his goal in the 3-0 Third Round win over AFC Bournemouth.
Referee Craig Pawson blew his whistle after five minutes of injury-time - Millwall are in the Quarter-Finals of The Emirates FA Cup!
Millwall: Archer, Cummings, Craig, Williams, Onyedinma, Butcher (Abdou, 75'), Webster, Ferguson (Romeo, 86'), Morison, O'Brien (Gregory, 78'), Cooper. Subs not used: King, Hutchinson, Worrall, Smith.
Sent off: Cooper
Scorer: Cummings (90')
Leicester City: Zieler, Chilwell, Musa, King, Amartey, Kapustka (Albrighton, 64'), Wague (Wasilewski, 70'), Okazaki, Gray (Vardy, 73'), Mendy, Benalouane. Subs not used: Hamer, Ndidi, Mahrez, Fuchs.
Attendance: 18,012 (3,114)
Referee: Craig Pawson