Simon is well-known for the attention he gets from ladies - including Lauren Silverman, his girlfriend and mother of his son Eric, and Sinitta, who is frequently by his side on the ITV show.
But they have a new rival in Matt's grandmother, who apparently flew all the way over from Spain to be in Simon's presence at the auditions!
"She is your biggest fan!" the Louis Tomlinson lookalike explained. "She’s 'Simon this, Simon that, Simon this, Simon that'!"
Watching backstage, Nan was quite embarrassed at being outed, but soon exclaimed: "Ooh, I’d do anything to get in there!"
She got her chance when the judges awarded Matt four yesses, running in to the auditions room to congratulate him.
Nan was pretty much speechless as Simon greeted her with a hug, saying: "Nice to meet you! How are you doing?"
The media mogul then bid farewell to his "future girlfriend" and Matt and his family headed for home.
On their way, Matt asked his Nan what she thought. "That's Mrs Cowell to you!" she joked. "I shall be on the panel soon!"
There's more X Factor auditions to look forward to next Saturday and Sunday at 8.00pm on ITV.