I suffer from border line personality disorder, depression and have extreme self esteem issues.

ive been seeing this guy for maybe 2 months once a week. we just started to have sex and he slept over the second time. thing is he was 3 hours late to our hang out session so he came over at 11pm. this is a normal occurrence. every single time we've hung out he's been late. i like him a lot. hes 12 years older then me and everything i want in a man, minus a few flaws. he talks at me not to me. he doesnt ask questions about me. he says he likes kissing me and ****ing me but thats it. it's just about sex is my understanding. so i called him out on it and said i need boundaries either were together or were not. hes not dating or ****ing anyone else apparently. he doesn't talk to me during the week except to call and make plans so he can get some sexual action. he laughed when i texted him i need boundaries and said hed call me and i basically said don't bother. why would he want to talk to me? he doesnt care he just likes the sex so why cant he admit it.

why does this bother me so much. my heart hurts. i like him a lot a lot and i dont know why and i wish i didnt. i dont know how to not let him break me.