Matt released the track last month as his first solo effort since winning the thirteenth series of the ITV show.
However, it has so far failed to set the charts alight, debuting at a lowly 83 in its first week.
The song - which you can listen and download here - came four places behind ‘Súbeme La Radio’, Matt’s collaboration with Enrique Iglesias, which first charted 13 weeks ago.
Streaming appears to be the reason for the embarrassing position. 'Sucker For You' did reach number 39 on the Official Sales Chart, but it failed to make the top 100 of the Official Audio Streaming Chart.
The song has since dropped down to Number 97, despite the release of the music video.
The result means Matt now has the lowest first-week chart position of any X Factor champ's first ‘proper’ single (excluding Steve Brookstein and Sam Bailey, who only released their winners' singles through Sony).
That record was previously held by Ben Haenow’s 'Second Hand Heart', which entered the chart at number 21.
The 24-year-old has had a slightly different start to his music career, signing to Sony’s RCA label, rather than Syco, Simon Cowell’s joint venture with the company.
He was also the first X Factor act since Shayne Ward to release an original track as their winner’s single.
'When Christmas Comes Around', which was co-written by Ed Sheeran, charted at 3 in December.
Matt recently told HuffPost that he isn’t concerned about what lies ahead. "Everybody is individual and we all have our own path," he commented.
"I just try not to think about it much and just do stuff without thinking. My path is my path and whatever is meant to happen will."
Earlier this year it was revealed that former X Factor contestant Fleur East has parted ways with Syco.