The former One Direction star, who was a contestant with the group back in 2010, is being tipped for a guest mentoring role.
It comes after Harry's solo career launched to a great start this month with a number 1 single and debut album.
"Everyone is in agreement that Harry would be a ratings winner," a source claimed. "His chart positions clearly show he is set to become the biggest solo star from One Direction."
The insider told the Daily Star: "Not only does he seem to be a clear fans' favourite he's also being hailed by music critics as well.
"Nothing has been signed yet but Cowell and Co want to make it happen if schedules allow."
A spokesperson for The X Factor said of the report: "It is far too early to be speculating over live shows or Judges' Houses when we haven't even begun the judges' auditions as yet."
Meanwhile, when it comes to this year's X Factor judges, Nicole Scherzinger is all set to finally sign up to The X Factor again.
Simon previously said he hoped to keep the same judges as last year for this autumn's series, but that plan hit a bit of a road bump.
While Sharon Osbourne is said to have signed a deal and Louis Walsh has been shouting from the rooftops he'll definitely be back, Nicole's place on the panel has been in doubt for the past few months.
Recently there were rumours that Mel B or even Mariah Carey were being lined up to step in for Nicole, but now it seems the Pussycat Doll has made her decision.
According to The Sun, Nicole has finally re-signed up for another year behind the judges' desk.
A source said: “Nicole is so highly thought off by X Factor that there was always the intention to tie her down for another year. Once she heard other names being mentioned, she was keen to get her deal locked down and it’s now there to sign."
Officially however ITV have not yet confirmed this year's X Factor judges.
The X Factor 2017 will air later this year on ITV with Dermot O'Leary hosting.