After a solid 2-2 draw against Watford Ladies in midweek, The Lionesses were looking to take all three points from a young and talented Durham.
With striker Emma Whitter out for the season through injury, Shauni Griffiths moved up from the development squad and started in a 4-3-3 formation for Millwall.
The Lionesses got off to a flying start with a goal after just five minutes when Griffiths slid the ball through to Lillie Bilson and she calmly slotted the ball past keeper Katie Luther and into the back of the net.
With the early goal settling the visitors, Millwall continued to play attacking, free flowing football on the 3G surface.
Bilson could have added a second goal on 13 minute when Cherelle Albert's corner found her in space, the striker's low shot was saved on the line and after a goalmouth scramble Durham finally cleared the ball out of their half.
The next 15 minutes belonged to Millwall who produced wave after wave of attacks forcing the Durham keeper into saves from Lily Agg, Griffiths and Naomi Cole.
Durham nearly caught Millwall on the break when captain Dixon twice teed up the dangerous Zoe Ness and Jess Round, but keeper Quantrill was quick off her line to avert the danger.
Millwall continued to push for the second goal and in the 38th minute were rewarded when Bilson smashed home a 30-yard pile-driver to send The Lionesses in 2-0 at the break.
Kim Stenning came on for Albert in the early minutes of the second half and the visitors continued to dominate proceedings.
A third goal followed on 56 minutes when Cole delivered the perfect ball for captain Dionne Lennon, who rose high to head the ball down and into the back of the net.
With the game coming to a close, Millwall made their final two subs in quick succession, Shirvea Edwards coming on for Griffiths and Frankie Strugnell replacing Lilli Maple, who had worked non-stop in the middle of the park throughout.
Bilson completed a stunning hat-trick moments later, driving home another 30-yard strike that flew into the top corner to make it 4-0.
Durham tried to rally and pushed for a consolation goal which duly arrived when Kylie Davies was judged to have brought down striker Yasmine Hardy in the box. The referee pointed to the spot, Dixon stepped up and sent Quantrill the wrong way for a final 4-1 scoreline.
Millwall face London Bees next for their first home game of the season at The Den, on Sunday 27th April KO 2pm. Click here for ticket information