The newbie to the series revealed the truth about a rumoured awkward run in with fellow coach Jennifer behind the scenes.
He insisted: "When we came back from the Battle stage, I walked into a room and everyone kept talking about this naked thing, but I wasn't naked!"
Olly explained: "I had no top on because it was getting steamed and I just quickly grabbed a girl's jacket that didn't fit and threw it on to cover myself."
He added: "Jennifer found it funny and so the next day she gave me a robe as a present and said 'Next time you can't find something to wear, stick this bathrobe on'."
Meanwhile, cheeky Olly said he can't help but flirt with the show's acts.
But he says it's always strictly professional - and he'd never date a contestant.
Olly said: "I am naturally cheeky, flirty guy. I can't stop that! I would never step over the mark."
Fellow coach Will.i.am joked: "If he had all boys on his team, I would be worried, but if there's a hot female singer, then she's going to fail because he'll be distracted!"
"I'm fine. Just because I'm cheeky and flirty, it doesn't mean I'm not doing my job," Olly insisted.
The former X Factor finalist explained: "There is obviously some attractive talent, but I have to be strictly professional as a coach.
"I've had a bit of a flirt or two on the show, but I can't change the way I am. If a woman comes on that I find attractive I might give her a compliment, but I would never be unprofessional and date a contestant.
"It's the Essex boy in me. You can't take it out!"
The Voice UK continues Saturday nights on ITV.
This weekend sees the live final.