The chat show legend is fronting Gary Barlow's new BBC One talent series, which is currently airing on Saturday nights.
Let It Shine is on the hunt for five singers to play the role of a band in a new musical based on the songs of Take That.
And Graham says that Let It Shine is better than its rivals because it only features acts who are "properly talented."
Speaking to the Radio Times, he said: "There are no Honey Gs in Let It Shine, the contestants are properly talented.
“I think the cruellest thing about some of these shows is when the contestants are promised something that’s never going to happen.”
Graham continued: “We’re not saying to these people that they’re going to have gold records and live in a Beverly Hills mansion. We’re saying you’re going to be on a bus, what we’re telling them is real.”
It's not the first time that Graham has been less than positive about The X Factor, previously claiming that the ITV series had "lost its credibility."
He said in 2015: “The people aren’t very good, so to all the judges’ comments of ‘I’d buy your album tomorrow’ you think, ‘You’re never having an album’.
"You wouldn’t throw 50p at them if they were busking in the tube. I think it’s lost its credibility.”
Let It Shine continues this Saturday night on BBC One.