Being voluntarily celibate causes no issues. Being involuntarily celibate can cause lots of psychological problems. Wanting to have sex and never having it is a problem and can harm someone's self esteem. Not wanting to have sex and not having it has no downside at all. So it's all in how you feel about it.