The Olympian and pro partner Katya Jones found themselves in the bottom two on Sunday night, dancing for the judges to stay in the competition.
The panel ultimately saved the pair ahead of comedian Judi Love and her professional dancer Graziano Di Prima who became the fifth couple to exit the contest.
Speaking ahead of this weekend's show, Adam admitted: "It's unfortunate that we ended up in the dance off. I mean, I'm still carrying a bit of baggage from the weekend because it was tough to go to the dance off.
"And when the red light comes on, you're like, 'oh no I've gotta go straight over to Tess and talk about it'"
Chatting on It Takes Two, he continued: "Because it was so unexpected because I thought, 'you know what? 29 [points] solid'
"But it just shows you like with Rhys last week, no one is safe."
This week will see Adam and Katya dancing the Jive to Little Bitty Pretty One by Frankie Lymon & The Teenagers.
He confessed: "I have to find my confidence again now. I do deserve to be here… I think. It's dancing and I need to breathe.
"We worked so hard on the fundamentals last week, all the fundamentals, the character, the musicality everything.
"And I thought you know what, I'm loving this so much and it's unfortunate we ended up in a dance off but you can't control that."
The remaining couples will take to the dancefloor when Strictly Come Dancing returns on Saturday, 6 November at 6:45PM on BBC One.
And one more celebrity will leave in the latest results show at 7:15PM on Sunday, 7 November on BBC One, featuring a special performance from musical guest The Script.
Picture: Katya Jones, Adam Peaty - (C) BBC - Photographer: Guy Levy