Gary was a judge on the ITV series between 2011 and 2013, following the departure of Simon Cowell, Cheryl and Dannii Minogue.
Before signing up, Gary says he asked his contacts in the Music and TV business for advice.
Gary reveals: “To the last, they all said the same thing: ‘The only thing anyone cares about in that building is The X Factor. They won’t mind throwing you under the bus, using you to get a headline, feeding you a line that will end your career, as long as The X Factor gets a mention.’
"‘You don’t go in there alone, you need back up.”
Writing in his autobiography A Better Me, Gary says how he hired his own producer to help guide him through the show.
The Take That then recalled one particular incident from the 2011 series involving Misha B.
Gary alleges: “About half an hour before the show goes live, the producers would come in and and they’d go, ‘Oh my God. That Misha. She’s a bully. Can’t believe it. She is such a bully. In fact, you know what? You should say it. You should say it on air. She’s bullied everyone all week.
“They leave the room. Ben [the producer]: Do not say that.
“Later Misha comes on...Tulisa: I think you’re a bully. Louis: ‘I think you’re a bully.”
And in the extract, first published by the Daily Mirror, Gary admits that he also became paranoid that his dressing room had been bugged.
He said: "We developed a bad case of showbiz paranoia. Was the dressing room bugged?
"They always seemed to know what was being talked about behind closed doors, and eaten, too. What ever happened in that building always found its way to the press.”