The Hollywood star was crowned the first ever American champ of CBB UK on Friday night.
But despite the public voting for Gary, most of his former fellow housemates have insisted he shouldn't have won.
Over the weekend, Dee Kelly hit out at Gary's triumph, accusing him of "indecent exposure" in the house.
She said: "No matter how much I complained, nothing was done. I told producers 'it's not natural or normal' and that I didn't think he was physically or mentally fit to be in the house. If that happened in the street, it's indecent exposure."
And James Jordan has echoed Dee's comments and claimed that Gary was given an unfairly positive edit.
"He’s a vile, vile human being. Fact," James blasted.
He claimed: “When I took him for a shower I saw parts of his body that no young man should have to see, especially when they’re not your family.
“Wiping his s*** off the toilet seat after he’s been to the toilet, and then he’s sitting at the table with s*** on his hands and you have to tell him to go and wash them."
Gary has now hit back, saying: “Dee is a liar. I didn’t expose myself to her.
"I sat on a stool and she came in the door and screamed. I pulled down my trousers to put on a knee pad.”
And he said of the rest of his housemates: “They were rude. James and George were the rudest. They wanted to have someone to degrade.
“The only person I will stay in touch with is Audley. The others were abstract friends. I pray they can find their own maturity to find out there is more in the world than their opinion.”