Although nothing is yet officially confirmed, it seems Nicole Scherzinger definitely won't be back.
Louis Walsh and Sharon Osbourne appear to be safe, despite previous reports suggesting they too were going to be axed.
Louis told the Daily Mirror newspaper previously: “I have a signed, water-tight contract, as has Sharon Osbourne. Of course you’ll see us back on the show.
“Sharon and I have a three year contract, and we’ve only done two, so this will be our third year. Simon doesn’t break contracts.”
Now Louis has suggested who could replace Nicole alongside himself, Sharon and Simon.
He said on The Ray D’Arcy Show on Irish telly: “I don’t know who will fill Nicole’s shoes. Mel B maybe or Rita Ora? I like Beyonce’s sister Solange."
“It could be anybody from Little Mix," he teased.
Louis continued: "It could be anybody from the Spice Girls – except Posh. She does not talk. I want somebody who is fun.”
For now, we'll have to wait and see.
Meanwhile, it's not just the judges set or a change on The X Factor this year.
Yet another shake up to the show may see the live episodes go back to being Saturdays only.
It would be the first time since series 5 in 2008 - that Alexandra Burke won - that X Factor airs on Saturdays alone.
A source said earlier this year: “One idea being looked at is whether getting rid of the Sunday show would allow ITV to put all its focus and resources into making Saturday night’s X Factor even more spectacular.
"Nothing has been decided but there are lots of ideas being thrown around.”
A spokesperson for The X Factor said: “No decisions have been made about the new series.”