The Irish music mogul claims that the ladies need the money because 'they need more make up'.
Reacting to the news he was the lowest paid of the judges, Louis said of Nicole and Sharon getting more cash than him: "They probably are [paid more] because they need more make up, more hair, more styling and more security.
"I just show up, jump out of the car and I'm ready in five minutes. They need at least three hours in make up!"
Speaking to heat magazine, Louis added: "We just laugh about stuff like that. We can slag each other off all we want - we don't care."
Simon had revealed details about the X Factor judges' pay earlier this year.
In the wake of the BBC gender pay gap scandal, Simon has revealed that the the women X Factor stars usually get paid more than the men.
Without revealing exactly how much the girls are paid, Simon told The Sun that Nicole and Sharon were both getting more than Louis for the current series.
“For once this is good news. Victory. Yeah, they definitely are. I just don’t tell Louis," he said.
Simon quipped: "He’s got enough money for a bum implant so he shouldn’t be complaining.”
And the music mogul added: “If they get the money, it doesn’t matter to me whether you’re a boy or a girl. But I would say we’ve probably paid girls more money than guys over the years.”
The X Factor airs on ITV on Saturday and Sunday nights.