The former American Idol star, who is now one of the coaches on ITV's version of The Voice UK, reckons the X Factor boss can't offer vocal advice.
In an interview the Daily Telegraph, Jennifer said she "wouldn't really pay any attention" to what the music mogul says.
She said of singing for Simon on American Idol: "His role on that show was mainly to look at image, I think, and I wouldn't really pay any attention to what he says about vocals.
"That's not his thing. At the time Simon told me I was 'Too big, in every way.' I think he meant my hair, my voice, my personality, my songs. I didn't really understand how that was a negative thing. I said, 'Is big not what a star is?'"
Jennifer reckons her experience on a talent show will put her in good stead for The Voice but she vowed to remain firm with the contestants.
"If a note goes wrong, my body reacts. I've got sympathy for them, but there's no point lying," the Dreamgirls performer declared.
Jennifer is joined on The Voice UK 2017 coaching line up by Will.i.am, Gavin Rossdale and Sir Tom Jones.
The Voice UK airs Saturday nights on ITV.