Dear Dating Again,
You ask the question, "is he just an old fashioned guy who's taking things slow or should I take it as a sign he's not that interested?"
My guess is that it's the former. You're right that it was bold of him to come over at 4 a.m.
He Was Excited To See You
He was excited enough to want to see you and be near you at that moment. The fact that you didn't connect physically probably means that it is something he is insecure about. If he is insecure, asking him, "hey don't you find me attractive?" probably won't help.
I guess the ball's kind of in his court. If he calls you again for a date, he's still interested. The physical connection will be something that develops. If he seems to want to be closer to you physically but it isn't happening, then I would gently raise the issue. Try to come from a place of strength: "I really like you, but I get the feeling you might not be very comfortable being intimate, is there anything I can do?"
He might be the kind of person who is uncomfortable with physical displays of affection. My parents have been happily married for 40 years, but I have almost never seen them kiss or hold hands. Keeping the physical intimacy a secret is kind of exciting.
He Didn't Regret Coming Over
You said that you cuddled in bed ... he clearly likes the intimacy with you.
Another possibility is that something else is going on. He may have a serious other relationship, and his approach/avoidance of you may be happening because he is guilty about the other person.