Gary was already earning £1.5 million for the show last year, and could now pocket close to £2 million if he signs a new contract.
Negotiations with the singer are currently underway, with those usual sneaky sources claiming he's eager to be back behind the judges' desk.
"Gary loved being on the show and genuinely wants to be back," one supposed insider told The Sun. "While viewing figures were down a few million, the feeling is it would have been worse without Gary."
Meanwhile Gary himself confirmed talks were happening, but was less keen about revealing his thoughts on how they were going.
He said only: "I started talks two weeks before the last one finished. We are talking."
Open auditions for The X Factor 2012 kick off later this month, while last year’s bunch of contestants start their live tour in Manchester on February 25.
The show will return to screens on ITV1 in the summer.