According to The Sun, show boss Simon Cowell has entered talks with the Grammy award-winner - who had a worldwide hit with 'American Boy' in 2008 - after Kelly told him she wants to focus on her career Stateside.
A source said: "Estelle's name is very high on his list now and serious enquiries have been made as to her availability for auditions in May."
However, Gary Barlow, Tulisa Contostavlos and long-time panellist Louis Walsh all look set to return after they were apparently offered pay rises.
An X Factor spokesperson said: "This is pure speculation. No decisions on the panel have been made yet."
The X Factor's ninth series begins on ITV1 later this year.