I think it was abuse, and if it felt sexual to you then it was sexual abuse. Have you heard of covert incest or emotional incest? These are to do with psychological or emotional sexual abuse and possibly apply here.

We weren't there. You were. And it's okay to listen to your feelings about this and believe yourself.