Earlier today the group took part in a challenge where, amongst other things, they had to guess which tweets and public opinions were about which housemates.
And it seems as though Andy was left rather annoyed at being repeatedly picked on when the housemates put his name forward for all the negative messages.
"Every time I know they're writing my name before they say it," Andy ranted to Jason. "'Who's the ugliest? Andy' 'Who's this? Andy' 'Who's the most arrogant?'
"It can be any question you like, if it's negative, it seems to come my way and I just feel it's unfair."
He continued: "I never came on this programme expecting to win, I have no game plan, I'm not playing a game. I was nice and calm in the first few days because I had no reason not to be.
"But if someone is trying to have a go behind my back I will say something about it, I don't think there's anything wrong with that."
The former BBC Newsreader told Jason in the garden away from the rest of the housemates: "After a few days I got bored of Georgina's whining and I didn't like her trying to stir against me so I fronted up to her about it.
"I won't be bullied in this house, I'm not a coward. If they want me to tell them exactly what I think about them..."
The task and fallout will air in Sunday night's Big Brother 2016 highlights on Channel 5.