NO! You don't expect too much - this guy's just being a jerk.
Apologies for hurtful comments and insensitive remarks are all well and good, but if he keeps it up, he's obviously not committed to changing his behavior.
Long distance is tough to begin with, but it certainly isn't worth the trouble and extra effort if both parties aren't devoted to making it work.
In my opinion, you should chalk this up as a learning experience, tell him it's over, and find a man who'll appreciate you and reciprocate your feelings and love.
You're in a Great Social Environment
I don't mean go hunting for one, but you're in a social environment to begin with (college), and the good guys are out there somewhere.
You write that your feelings are slipping anyway, that you know you're not in love with him - go with your gut, finish it and move on. Experience here has nothing to do with it. If you feel like you're being neglected or not treated as well as you'd like, take control and walk away (especially considering you've raised the issue and he doesn't seem to want to change).
Only you know what's right for you, and there's someone out there who'll satisfy that. You two will find each other!