Why is it so hard for me to come up with something to talk about?

If you've known him for 10 years and are comfortable around him then you should just be straight with him and tell him what you're worried about. If other guys have given up on you quickly then you haven't had much time to get used to being with them and be yourself. Let him know you might be a little quiet at first and after a while you should be fine.