The switch would've made Alesha the star of no fewer than three prime time judging panels having previously been a part of Strictly Come Dancing.
But the singer says she'll be staying firmly put on BGT which she thinks is by far the "better" show.
"I love Britain’s Got Talent and the fact that it's a variety show," Alesha told the Daily Express newspaper.
Speaking about why she prefers Britain's Got Talent over The X Factor, Alesha continued: "I love that it’s light-hearted family entertainment… they are tonally very different.
"I think the tone of BGT suits my personality more. I can be serious and strict when I need to be, and I think you would have to be more like that on The X Factor. I am really happy - our show is better!"
Britain's Got Talent will be back next month on ITV for its latest series and it's the sixth year Alesha has been on the panel alongside David Walliams, Simon Cowell and Amanda Holden.
"We’ve become like a bonded, dysfunctional family – and it works!" she said. "The thought of doing it without them is quite a scary thought."
Ahead of the new series, Alesha added: "Each year the friendship grows and we’ve become more comfortable with each other.
"I feel blessed and honoured to work on one of the best shows on television, but obviously I won’t be doing it when I’m in my 70s."
BGT will be back on TV in April.