The favourite to win has undoubtedly proved a fan favourite among many female viewers, who have even been turning up at The X Factor house.
"Sometimes it's really shocking," he said this week. "Like the other day we were at the house just watching TV and all of a sudden we just heard a car horn go.
"And inside the car was a group of girls who were swinging their underwear, saying, 'We want Matt!'
"Sometimes I have to ask myself, 'Is this really happening?' but I feel like you just have to go with it and have fun.'"
Speaking to Reveal magazine, Matt denied he had been getting close to any of the other contestants on this year's show following rumours about him and Gifty Louise.
"I think no matter who I talk to, I have a romance with them. Gifty and I were such good friends, really close, but we're all close," he said.
Meanwhile, asked if he would ever pluck up the courage to ask out mentor Nicole Scherzinger after all their flirting, Matt went red: "I would die! I don't think I could ever."
He continued: "I wouldn't be brave enough to even ask her out or look at her with that thought. My best pick-up line for Nicole would probably be something like, 'You're really out of my league and I'm really sorry I've just said that, bye.'"
The X Factor 2016 continues this Saturday and Sunday night live on ITV at 8PM with the semi-finals.
Each of the top four acts will be singing twice and there will be guest performances from Lady Gaga and Zara Larsson.