The hard working singer is currently taking part in a 30-date tour and will then go straight into filming for this year's series.
It was confirmed earlier this month that he'd be fronting the twelfth run of the ITV show alongside former Xtra Factor co-host Caroline Flack.
“I was flattered to be asked. I grew up on the show, so it’s like coming home," he said this week.
Speaking to The Telegraph newspaper, Olly continued: “Caroline and I have great chemistry: we bounce off each other, she can be the serious one and I’m the joker.”
Olly went on to rubbish claims that he had 'shafted' former host Dermot O'Leary to get the gig.
He said: “It annoyed me, it was really nothing like that.
"Dermot’s one of my best showbiz mates, but we’ve had a great chat about it.”
However Olly is preparing for some critics when the show hits screens in August, but he declared: "If they don’t like me that’s fine."
On signing up Olly and Caroline for the show, X Factor boss Simon Cowell said earlier this year: "When they were on Xtra together, they were a brilliant double act. Since then, Caroline’s gone from strength to strength – and Olly has become one of Britain’s biggest stars.
"So now I’m looking forward to them fronting the main show."