First, I just wanted to say that it's great that you accept and understand your feelings toward sex. Most people jump right into things without thinking of the consequences. But you seem to grasp the situation very well - VERY MATURE!!!
Anyway, I think it's great that your boyfriend is totally supportive of you. This should lesson the pressure of having sex. However, you're still having those conflicting emotions - I know how that feels.
Things Will Change
First off, you said that you did not want things to change if you did go through with it. Unfortunately, that is inevitable. And you'll never know until it happens. But that does not mean it will change for the worse. You may find out that you were more in love than you thought. But the truth is that it is a very big step for any relationship to go through so to be worried of the outcome is okay.
When In Doubt, DON'T
The fact that you have many doubts about giving your virginity to this guy or anyone says that you are not ready to have sex. My philosophy when in doubt, is if you are having any - you're not ready - whatever the case may be. When you are ready, you'll just know.
So if the relationship is going great so far, don't worry about a thing. Sex these days is so casual and you want to make sure that when you do decide to have sex, it will be with someone that you love and will not regret it down the road.
It's Normal To Be Afraid
You mentioned that you were afraid. Totally normal - but have you ever considered this? If your boyfriend loves you as much as you say he does, he may be just as nervous as you - or more. When you love someone, you want everything to be right, regardless of how many times you've had sex. So everything is different for every partner.
Forget What "They" Might Say
So you do lose your virginity to this guy and you're worried how everyone will react - forget about those people! You should be proud that you still are a virgin and when you do loose it, you will know that you waited until you were ready - so you did not act on impulse but rather maturely. Don't let people judge you unless they're the pope or something.
I'll leave you with this: You are very wise to accept your feelings and think things through. So you're doing everything right.
Regardless of age, when you're ready you'll know. It will all come naturally. So for now, just sit back and relax! Don't worry about things that haven't happened because it may sour the situation when it actually arrives. As long as you know you're doing the right thing - you have nothing to worry about!
Hope I've helped,