I think you need to be gentle with yourself and understand and accept your feelings. Obviously you don't want anything to happen with him given that he is family, but it's OK to have feelings. A lot of people think and feel "taboo" things and it can be pretty normal to feel them. As long as you don't worry about it too much it shouldn't be a problem.