Two spin-offs of Simon's long-running music talent contest are airing this year, beginning with X Factor Celebrity in a fortnight.
It will be the show's first series in over a decade to feature just three judges, with Louis and Nicole Scherzinger sitting alongside the Syco boss.
However, it seems Nicole is not staying on for the second special edition, in which past contestants will compete to be crowned champion of champions.
Contrary to rumours that Simon would also be bringing back past judges, it seems he'll be replacing the Pussycat Doll with someone not as familiar to UK viewers.
According to The Sun, Simon's old mate Paula Abdul has signed up for the all-stars version.
The pair became close when they appeared on American Idol together between 2002 and 2009, the height of its success.
She followed him to the US version of The X Factor in 2011, but left after one season.
"Paula is a close friend of Simon’s," a source said. "They’ve always kept in touch since working together in America.
"She is part of the Syco family and they all work closely together.
"Paula and Simon have great on-screen chemistry and their banter is funny and entertaining.
"Simon has handed her a lucrative contract to ensure she’s happy and it’s worth her while financially."
While Simon and Paula's partnership is famous in the States, she may be a more puzzling choice for an all-stars special on this side of the pond.
The 'Opposites Attract' singer has never been a full judge on The X Factor UK. Therefore, she hasn't mentored any of the acts returning for a second shot.
She did, however, appear briefly as a guest panellist during the third series.
Nicole is apparently unable to continue due to her other work commitments.
She has put her judging skills to good use recently, also joining Australia's Got Talent and The Masked Singer US.
The X Factor's run of spin-off specials begins on ITV in October.