Dean, who lives with his brothers, tells The Sun today how he was outcast as a child because of the crimes his eldest two siblings had committed.
"[My brothers] Stephen and Daniel got into scrapes in town. Then Stephen was done for armed robbery," Dean admitted. "I had no idea where he got the gun from but he did five years.
"I visited him every week and he would tell me not to end up there. It made me think."
The 21-year-old, who impressed Simon Cowell on last night's show with his group, went on: "When I was a kid my friends' mums wouldn't even let me in their house because I was a Kelly.
"It used to really upset my mum. People say it's parents to blame — but I couldn't have wished for better parents. It wasn't their fault."
And now he hopes Britain's Got Talent will help his family rebuild their reputation.
He added: "I'd love this to work so I can help get my family away. Music is my way out of the misery."