This worry is very real to you and that's all that matters. However, I would try to relax around this guy and try not to put a lot of pressure on yourself. A potential friendship is not worth your mental health. You have a lot to offer; try not to worry about what other people think.