Earlier this month, bosses announced Robbie and Ayda as two of three new judges on the ITV singing contest.
Their megabucks deal - thought to be worth up to £10million - makes them the first ever married couple on the panel.
Now Robbie has truly made it a family affair, by giving his father a turn at the judges' desk.
According to The Sun, Pete Conway was in the audience during filming of the arena auditions last week.
Suddenly, when Robbie fancied a break, he decided to swap places with his old man.
"Onlookers couldn’t believe it when Robbie just got out of his judge’s chair and plonked himself among them," said the newspaper's source.
"Even more surprising was just how good his dad was as a judge.
"But then he is an entertainer too, so he knows talent when he sees it and he can work a crowd."
During his brief stint as a judge, Pete apparently gave a glowing review to one act, classical trio Tre Amici.
"I’m glad I’m not up next! That one is going to be hard to follow," he is quoted as saying. "I’ve been here all day and seen a lot of acts.
"That’s the best we have seen."
It's not the first time that a star of Simon Cowell's talent shows has called in backup from their family.
During the tenth series of Britain's Got Talent in 2016, David Walliams' mum Kathleen Williams filled in for Simon when he was late turning up to one filming session.
She proved a surprisingly picky judge, buzzing almost every act she saw.
The X Factor is due to return to our screens next month.
Robbie and Ayda appear on the new judging panel alongside Simon and fellow newbie Louis Tomlinson.
Louis reportedly abstained from voting when Anthony Russell, one of last year's contestants, returned for a second chance.