Two ejections and a number of warnings issued to the housemates have seen viewers writing in to the TV watchdog.
Ofcom will assess the complaints to decide whether or not to launch an investigation into the show.
Complaints have centered around the exits of Ken Morley and Jeremy Jackson while Perez Hilton's behaviour and antics have also resulted in objections.
The celebrity blogger was immune from the public vote this week but last night fans made it clear they had had enough, chanting 'Get Perez Out!' during the live eviction show.
Perez's controversial comments have included comparing Ken's exit to being "cancer free" and suggesting that Cami Li pretends to be a lesbian to win votes.
In last night's highlights, Michelle Visage took aim at Perez and accused him of setting back the LGBT movement "fifty years".
An emotional Michelle spoke to Big Brother in the Diary Room and complained about Perez's suggestive topless dancing in the garden: “What we just witnessed is an outrage”.
She claimed that Perez “sets our community back fifty years," adding “He’s an embarrassment, I do not want people judging the gay community on this ass.”
The total of 491 gripes to date are still well off CBB's worst ever year back in 2007 when 45,000 Ofcom complaints were made over the now infamous race row.
Celebrity Big Brother 2015 airs nightly at Channel 5.
Browse pictures of last night's live show action below...