I am not a pro at human behavior, but the environment you describe sounds rather toxic. Even if they believe you lack at something, proper teachers should help you become better instead of simply making negative comments. Also, if your advisor prefers talking about his hobbies over giving you advice, talking to him about your feelings probably won't change much. But you can still try, I don't see it doing any harm. Have you considered getting a different thesis advisor? Sometimes, it is better to choose someone who is willing to help, even if his/her subject that is less interesting. Especially if not having enough attention hurts your feelings so much. I chose my thesis advisor almost purely about his helpful character, and it was the best decision ever.
Finally, do keep in mind that the teachers aren't your parents. They may act this way at younger ages, but in university many of them aren't like this. Some are more concerned about their research instead of their students, some take up teaching only because they can't find another job as scientists, some aren't indifferent but simply have zero skills in communicating with their students or helping them. Of course there are also some who are great for the job. Don't get all broken-hearted over it, it's their problem and not yours. Everyone does have their own problems, the trick is to find someone willing to care about yours too, at least in an academic level.
If you are to work with that advisor establishing a program from the beginning sounds like a good idea. It is good to be clear from the very beginning how you will work and how is he going to supervise you.