ITV's biggest talent contest is getting a huge shake-up this year after its last series reached new ratings lows.
The next public edition has been postponed, with Simon Cowell planning to air two new spin-offs instead.
However, Matt - winner of the highest-rating series ever in 2010 - isn't sure the idea will reverse the show's fortunes.
Speaking to Smooth Radio this week, he claimed The X Factor has "waned" and that a break would give viewers "a chance to miss it".
"It seems like they’re wrapping it up, if you ask me," he commented. "It’s one of those things. Maybe give it a rest for a while.
"I say, if you eat spaghetti bolognese every night, you’re going to get sick of it. If you do The X Factor every year, it’s going to run its course.
"Maybe give it a rest for a couple of years, and then bring it back. Give people a chance to miss it."
He continued: "It is a great show, and it’s done great things for people, myself included.
"But if you constantly do things over and over, there will come a time when it starts to wane - and clearly it’s waned."
While Matt didn't explicitly state whether or not he'd join an All-Stars series, his closing comment didn't exactly suggest he's interested.
"Good luck with the new version," he added.
Earlier this week, fellow X Factor champ Shayne Ward ruled himself out due to his busy schedule.
Matt has maintained a steady career since his X Factor victory, with four albums under his belt.
His latest record, 'Time to Be Alive', was released last April after he re-signed with Sony Music.
Despite Matt's opinion, it seems The X Factor may be on our screens for a long time to come.
Reports suggest that ITV is in talks with Cowell's firm Syco to renew the format for another three years, until 2022.