Xtra Factor host Olly Murs has suggested that viewers who don't vote can't complain about the result.
Sunday night saw the first live X Factor results show, after all the finalists had performed for the first time on Saturday night.
Rylan Clark and Carolynne Poole were in the bottom two, with the judges deciding to send the result to a deadlock, which saw Carolynne leave.
It was a controversial decision to take the vote to deadlock, but Olly defended Louis Walsh.
He told the official ITV1 X Factor website: "He's entitled to his own decision and to be honest she did have the fewest public votes.
"Those people commenting about the decision should have picked up the phone to vote so she wasn't in the bottom two. Perhaps she chose the wrong song to sing. She has a brilliant voice though and it was sad to see her go."
Olly continued: "The only thing Louis possibly did wrong was to take as long as he did to make his mind up. It must’ve been unbearable for poor Carolynne. I imagine that people up and down the country were screaming at their TV sets."
And Olly revealed: "Gary was fuming.
"He was very upset to lose someone from his category. He’s passionate about his Overs and I can see why he walked off. I just hope Gary and Louis make it up in time for next week’s show."
The X Factor 2012 continues this Saturday night on ITV1, with the results show following on Sunday night with performances from Taylor Swift and Rebecca Ferguson.