The Overs singer has her own room and doesn't socialise with the rest of the contestants, it's been claimed.
According to The Sunday People newspaper, Honey G - real name Anna Georgette Gilford - is "finding it hard".
A source told the newspaper: “It’s nothing vindictive towards anyone – they’re all nice people – but she just feels most of the housemates don’t really get her.
“She jokes it’s like living with a bunch of horny teenagers and feels like she’s back at school.”
Gifty Louise confirmed that Honey G separated herself from the rest of the group despite their attempts to involve her.
"She is quite a bit older so I suppose it’s that it’s a bit awkward hanging around with all of us a little bit," the 20-year-old is quoted as saying by The Sun newspaper.
Gifty explained: "We do try our best to get her involved in everything we do.
“In the house we all have dinner together but she tends to just have her headphones in."
And Gifty revealed: “She’s a rapper so she’s always practising her rap. There’s nothing wrong with that.
“When Ottavio and Bradley moved out of the smallest room it sort of made sense to give it to Honey.”
On Sunday night, Honey sailed through the latest results show as Freddy Parker left following the sing-off with Saara Aalto. Ryan Lawrie was saved from the bottom three with the life line vote.
The X Factor 2016 is back next Saturday and Sunday night on ITV at 8PM.
The theme is divas and there will be guest performances from Shawn Mendes and John Legend.