I think the "Will I regret this later in life?" question is a really good one. Will you regret not asking your friend to hang out in the summer? Probably. So ask him if he wants to hang out. It's not a weird thing to do or anything. The worst thing that can happen is he says no, but it sounds like you're already friends, so I don't see why he would say no unless he's going to be gone all summer or something.