The judge sparked fury after the opera group performed on the ITV talent contest's live semi-finals last month by saying: "Four sexy chocolate men with amazing voices. Polished to perfection. That was sensational!"
A whopping 133 viewers complained to Ofcom over the remark, claiming that her remark was racist.
However, the media watchdog confirmed today that it will not launch an investigation in to the matter.
"We had a number of complaints that some of the language used by a judge in this programme was offensive, but we will not be investigating," a spokesperson commented.
"It was clear that these comments made during a live programme were not intended to be derogatory. The judge went on to praise highly the performance by the quartet."
Indeed, Vox Fortura themselves seemed to love the cheeky comment, writing on their Twitter page afterwards: "If you think we are 'four beautiful chocolate men' can't get any better than that!!"
"I do! And so is my husband!" Alesha wrote back, despite not actually being married.
Vox Fortura ended up taking fourth place in their semi-final, with magician Richard Jones going on to win the series.