Earlier this month, Simon revealed that he wants to postpone the sixteenth regular series of his signature talent contest.
Instead, he intends to air two spin-offs later this year: one with celebrity contestants, and one with ‘all-star’ favourites getting a second shot at stardom.
But it seems the returning acts wouldn’t include Olly, who finished in second place on the sixth series.
The ‘Moves’ singer, who this year celebrates the tenth anniversary of his successful career, confirmed on This Morning that he’s not interested in a comeback on the show that made his name.
“I’ve heard about this,” he said. “I’ve been asking everyone at ITV if they can give me a clue what’s going on.
“I wouldn’t go back on it, no.”
Olly’s decision isn’t surprising, given that he now has judging experience as a coach on rival show The Voice UK.
However, he does support his ex-mentor Simon’s plan to rest the main version of The X Factor for a year.
He continued: “If they are going to take a break from the show, I think that’s a good thing.
“It does need a little break from what it was doing, I think Simon has seen that and I’m glad.
“I love the show more than anyone, it changed my life. I watch it like everyone else, and I’ve always felt something’s not clicking right, so it does need a break.”
Bosses are yet to confirm details of either of the proposed spin-offs.
Viewers can catch Olly on The Voice UK every Saturday night at 8.00pm on ITV.