The actress was linked to the show last year, before former Strictly star Alesha Dixon was eventually announced as the new female panellist.
In an interview with OK! magazine this month, Samantha reveals that while she did "think about doing it", in the end she decided it wasn't for her.
"Simon Cowell flew me out to his mansion in Los Angeles. I was already committed to [TV show] South Pacific, and I kind of knew that, but to be honest, I wanted to see what Simon and his house were like!" she told the publication.
The soap star revealed: "He was bloody charming! I'd never been to LA before, and I spent a good few hours in his amazing house, just chatting.
"He's very easy to talk to and he smoked like a trooper - though weirdly his house didn't smell of cigarette smoke! He must have a magic machine that makes it disappear!"
She added: "It was amazing to witness LA like that and have the big black car pick me up.
"I took a sneaky photo of his house on my mobile - and I think he caught me doing it! It was a lovely meeting, but at the end of the day, it's just not me."
Alesha was joined by comic David Walliams on the panel this year, which also saw the return of Simon alongside veteran BGT judge Amanda Holden.
[cat]Britain's Got Talent 2012[/cat] continues on Saturday night on ITV1 at 8:30PM.