He added: "[I've been struggling with my weight] from since I can remember, like since I was a baby.
"I used to get my mum to write a note to the school when I was like 8 so I wouldn't have to go to swimming lessons. So it's something that's been in me forever."
And the 26-year-old singer says he still struggles with his body image, which he blames as "the basis" of all his "sadness".
Speaking to Jameela Jamil during the premiere episode of her new 'I Weigh Interviews' Instagram series, Sam said: "It's the basis of all my sadness. Literally everything I've ever been sad about is my weight. I struggle with it every day. I get very, very dark and very sad."
The singer also revealed that he has considered having a sex change but now identities as neither male or female, instead describing himself as "non-binary and genderqueer" as he feels like he floats "somewhere in between" the two genders in his mind.
Sam said: "You do not identify in a gender. You are just you ... you are your own special creation. That is how I take it. I am not male or female. I think I float somewhere in between - somewhat on the spectrum. I've always had a bit of a war in my body and my mind ... I do think like a woman in my head at times. I've sometimes sat there and questioned, 'Do I want a sex change?' It's something I still think about, but I don't think it is."
The 'Stay With Me' hitmaker opened up about dealing with gender issues throughout his life, and revealed he stopped wearing 'boys' clothes' to school when he was 16, instead turning to fur, leggings and make-up before a switching to his signature suits to conceal his weight issues.
He also revealed how he can be "feminine in many ways" - including in his love life - and explained how he has been trying to adopt his "free" attitude to sex into his "thoughts on gender".
He said: "I've always been very free in terms of thinking about sexuality. I've tried to change that into my thoughts on gender... when I move, when I have sex with men, it's very feminine. I'm feminine in many ways -- and I've resented that."
-Bang! Showbiz