The former Spice Girl joined the panel last year but it's not 'unlikely' she will be invited back to the show for this summer's twelfth series.
It' as Simon Cowell does his best to revamp the show, having already replaced host Dermot O'Leary with Caroline Flack and Olly Murs.
Simon will be back on the panel while it was previously reported that Cheryl has signed a £2 million deal to return.
But Mel is set to leave with a source declaring today: “It is now looking very, very doubtful she will come back, Simon has made up his mind."
Mel's still working on Simon's other shows including America's Got Talent but a bid to "freshen up" X Factor on this side of the Atlantic means she looks likely to be cut.
The source added to the Daily Mirror newspaper: “Some fans will be disappointed though as some viewers felt she was the best thing about the show last year.”
However a spokesperson for The X Factor insisted that no decisions had been made yet.
According to The Sun, Mel could get a repreive if bosses fail to sign up a replacement.
An insider told the tabloid: “If they fail to find a suitable candidate there is a very small chance she [Mel] could be back.”
As well as Mel B, Louis Walsh's spot on the panel is also up in the air.
He said last week he was "done" with the show but then this week claimed he may be back after all, insisting: "There are loads of rumours flying around at the moment but none of them are true."
The X Factor 2015 will start on ITV later in the year.