He confided in Chelsea Singh: "I'm getting sick and tired of this gossiping and whispering behind people's backs to undermine them, it's transparent and childish and I'm actually getting quite angry about it now.
"If I know someone is b***hing about someone, do I say and create s**t? If I start telling you about it then I'm a gossip and a b***h and I won't be one."
Later, Andy took his problems to some of the housemates' faces, namely Lateysha Grace, Sam Giffen and Ryan Ruckledge.
He said: "I do feel particularly in the bedroom there are conversations about other people in the house that make me feel uncomfortable and I don't think are always very fair."
Andy pointed out comments made by Georgina about Evelyn and remarks from Lateysha about Hughie.
Lateysha hit back at Andy's claims but he told her: "The way I see it, saying something to someone's face doesn't give you carte blanche to be attacking them behind their backs."
Lateysha reacted: "Don't f**kng sit there and act as if I was slagging him off when I was saying everything to his face."
Things then begun to get heated with Sam lashing out at Andy for his accusations, while Ryan snapped: "You're probably the biggest game player in here!"
Big Brother airs tonight at 9PM on Channel 5.