But fans of Nicole needn't worry as Simon Cowell hasn't had a sudden change of mind about this year's panel.
Instead, Mel B was drafted in as a guest judge to stand in for Nicole at some of the try outs in London while filming earlier this year.
Simon confirmed back in the summer that he would be joined by Nicole, Sharon Osbourne and Louis Walsh on this year's panel of 'fan favourites'.
While Nicole declined the chance to appear in a Broadway production of Cats to join the show she couldn't clear her schedule completely.
Therefore America's Got Talent judge Mel was flown over to take a seat for a third appearance on The X Factor.
Mel was previously a guest judge in 2012 and also appeared as a full time judge for the 2014 series.
“This is a diary clash that Nicole had booked in before she agreed to do X Factor and couldn’t change," a source revealed earlier this year.
They told The Sun newspaper: “Mel has been working with Simon as a judge on America’s Got Talent so was available to cover.
“He also knows she’ll come in and take no prisoners so is the perfect temporary replacement for the day.”
The X Factor 2016 auditions continue next Saturday and Sunday night on ITV.