C'mon...you are asking a question that you already know the answer to.
Your entire question only points out the things that are dooming your relationship and you're still not sure if you should try to work things out with Sam?
It's bad enough that he's materialistic and "won't shut up about himself" (ugh!) but this is a long distance relationship, he has unresolved feelings for his ex (whether he's bitter or hurt, he needs to confront his feelings about her), he's insensitive and he isn't satisfying your needs.
What else do you need to know that there is somebody better out there for you? Believe me, don't stick out a realtionship with somebody who is unaffectionate, because you lose in the end, and he walks away unphased.
It's Good that He Lives Far Away
From what you say, it sounds like the only good thing about Sam is that he lives far away and when you do break up with him, your chances of running into him are slim. You deserve more than what he's giving you and you know it, so what are you waiting for?
It'll be the most satisfying weight loss of your life.