Earlier today it was confirmed that the duo - made up of brothers Josh and Kyle - would be leaving the show.
In a statement, Josh Brooks said: "Due to events in my personal life I've decided that it's best not to be on the show at this time. While not everything that has been claimed is true I would like to apologise to everyone involved."
An X Factor spokesman added: "Following the allegations relating to Josh Brooks, we have decided by mutual consent that Brooks Way should leave the competition."
The twins have since taken to Twitter to post a longer message to fans.
"By mutual consent, we feel like it was best to walk away from the show this year. While not everything that has been claimed is true... I would like to apologise to everyone involved. And would like to thank everyone for there ongoing support and love," Josh wrote.
The boys, who first made judges' houses in 2014, went on: "A huge Thankyou to the team at @TheXFactor and @ITV for all your hard work and all you have done the last 2 years for us.
"And the final 10,we have loved every second of this journey with you all. We love you all dearly and wish you the best of luck. Miss you all."
Fans of the pair were quick to rally round them.
"What happened to you was awful,rumours destroy peoples lives,your so talented,you'll be fine," one tweeted.
Another added: "love you so much... X factor not gonna be the same without you two there xx".
ITV have confirmed an act will replace the pair with details to be announced soon.