The former Xtra Factor presenters have been widely rumoured to replace Dermot O'Leary as host of the ITV series after he stepped down last month.
But Olly insisted that nothing has yet been confirmed.
He told Irish radio station Spin 103.8: "I haven't got it yet. We've spoken about it in the papers but nothing's been finalised or nothing's been signed.
"We've had had a few discussions, that's about it really."
It had been reported that Olly would only take over hosting the main show alongside Caroline and admitted: "Me and Caroline have obviously spoken about the idea of it, but like I say, it's just an idea at the moment, nothing's been finalised."
And Olly insisted that he didn't want the move to be seen as him 'taking' Dermot's job.
"Me and Caroline we were never interested in taking Dermot's job, it was only, you know, Simon's idea of potentially having me and Caroline do it," Olly revealed. "I would never want to take Dermot's job because I love him so much and I wouldn't want to be compared to someone as good as him."
Speaking earlier in the year, Olly said if he were to end up hosting the X Factor he wouldn't try and 'be Dermot'.
"Dermot is one of my best mates in the industry," the former X Factor finalist remarked. “He’s my showbiz brother. I would never do the job on my own and never try to be Dermot O’Leary.”
The X Factor returns to ITV later this year.
Browse pictures of the potential X Factor 2015 presenters below...