With the Blind Auditions over, the current series of the ITV singing contest has moved on to the Battle Rounds.
That means the four star coaches have started sending home people they fought hard to add to their teams.
As if that task wasn’t hard enough, Olly says he was conscious about upsetting his contestants’ nearest and dearest - as they were in close proximity to his chair.
“Obviously at this stage in the competition, the Battles, there is less fun,” he commented on This Morning.
“You’re hearing them singing, like, ‘That’s amazing! Oh, now I’ve got to tell one of you to go home.’
“Where I’m sat, you can’t really see it on TV, but I can see the families, just waiting to hear.
“So as I’m sat there deliberating who to pick, you know, I can just hear their parents right there!”
Olly has an idea of what the acts are going through, having shot to fame as a talent show hopeful himself on The X Factor in 2009.
He told how Simon Cowell recently tried to lure him back to his homeground as a judge.
However, while he’s still pals with his former mentor, the ‘Moves’ singer has pledged his loyalty to The Voice.
“Last year he tried to tease me back, and I just said, ‘I love The Voice too much’,” he explained.
“I spoke to Simon last year - all the papers write about us not getting on, we get on great.
“He tried, but I just love being on The Voice and I didn’t want to leave after one year. I wanted to give them my loyalty.”
Olly has also ruled himself out of returning to The X Factor as a contestant on the upcoming ‘all-stars’ series.
The Voice UK airs on Saturdays at 8.00pm on ITV.