The rapper hadn't featured in updates from the house today (Tuesday) and the official Big Brother website revealed earlier today tonight that he had "left".
A spokesperson for Channel 5 confirmed that Ray J departed the series but did not revealed why he had gone, saying that details about Ray's exit will be revealed in Bit On The Side at 11PM on Channel 5.
However a report by website TMZ has claimed that Ray J was taken from the house to hospital because of "extreme tooth pain".
The article reported: "[...] a dentist came to the set, gave Ray 8 painkillers, 4 of which he was to take immediately and the remaining 4 the next morning.
"Ray woke up Tuesday in unbearable pain, took the remaining 4 pills and then pleaded for help. We're told producers agreed to take him to the hospital but on the way out of the house he blacked out."
Now Ray J has spoken up himself, accusing show bosses of 'ignoring' his pain.
"I’ve been complaining about my tooth for 3 and half a days," he said, "It got to the point where me and some of the housemates felt Like was I being ignored...
"We took petitions to say we wouldn’t do any more tasks until Ray J gets his tooth fixed."
Ray J says he eventually blacked out and was taken to hospital, where a quick treatment saw him bounce right back.
"I end up in the hospital, 45 minutes and I’m cool, I'm feeling fine and ready to go back in the house and they won let me back in," he alleges in a chat with TMZ. "I'm really hurt by it because it sucks because I was having such a good time."
Ray J said it was "really sad" to not be allowed back and branded the decision "unfair".
Asked if he would sue to get his rumoured $1 million fee, Ray J added: "I never like to do that... but this is just not fair... put me back in the house."
Celebrity Big Brother 2017 continues nightly on Channel 5.