The talent contest had some shock results at the weekend, with Janice Robinson facing instant elimination after coming bottom of the public vote.
The judges' vote then went to Deadlock, and LMA were sent packing over Brendan Murray.
Now Dermot has revealed he was particularly surprised by this turn of events, having backed the choir to go far in the competition.
Speaking to the Metro at this week's Pride of Britain Awards, he said: "I thought the choir were going to be around forever.
"I thought at least semi-final, Liverpool would get behind them. It’s odd. Normally the critique they get on a Saturday sort of translates with the vote. And it didn’t.
"Molly got a pretty harsh critique, the boys…they were safe. It’s interesting, I kind of like that the public aren’t listening to the judges."
Sixteen contestants made it through to the live shows this year - but they are only running for six weeks.
The high number of contestants means Dermot is bidding farewell to them at rapid pace.
"Simon [Cowell] was adamant we wanted to put 16 though, which is fantastic but creates its own problems," he commented. "So we have to get through them.
"It’s going to mean were going to have double eliminations all the time. So no one is particularly happy happy, joy joy."
LMA's mentor Robbie Williams has also expressed his shock at the latest results.
The group initially appeared safe with two votes to one - but it was Robbie's wife Ayda Williams who initiated Deadlock, their first major disagreement of the series.
The X Factor continues this Saturday at 8.35pm, when contestants will sing spooky songs for Fright Night.