TV watchdog Ofcom has revealed it's so far received 121 official complaints about the treatment of the actor by the other celebs.
Gary has found himself in a number of rows already, first clashing with James Jordan and then David McIntosh.
In scenes yet to air in the nightly highlights shows, Gary and Leslie Jordan also had a huge row, with Leslie claiming that Gary had alienated himself from every one of the housemates.
Speaking to Big Brother in the Diary Room earlier on in the week, Stephanie Pratt admitted that
Gary had been 'the odd one out' of the group, saying that everyone would nominate him if they had the chance.
In last night's show, Big Brother showed concern as they called James to the Diary Room to discuss the growing isolation of Gary from the group.
The Hollywood actor has also become the target of pranks from the other housemates, who were seen climbing into bed with him as he tried to sleep on Tuesday night.
It's also been claimed today that 70-year-old Gary is being monitored for his own safety as well.
A BB source told The Sun newspaper: “If it’s felt Gary could be a danger to himself or others he will be pulled out.”
A spokesperson for the show added: “There is a duty of care and a very robust process.
"There is a welfare team for the housemates before, during and after the show.”
It all means it's set to be another busy month for Ofcom staff, who are still dealing with the fallout from the recent series of BBUK.
Over 2,000 complained about various incidents in the latest series, with 400 writing in alone over last Friday night's finale where Helen Wood walked away with £100,000.
CBB continues nightly on Channel 5.