It could very well be his own personal issues unrelated to you.

Unless you ask, you can only guess. Perhaps write a letter about it to him if it makes it easier