toomanycats is speaking alot of wisdom here.

I know this sounds blunt but you shouldnt be working your way towards a friendship. That in itself is crossing boundaries.

Ask yourself if you are looking at this man to meet your needs in therapy and also after its ended? Just because needs are not sexual does not make it necessarily appropriate.

The goal of therapy is not to keep this person in your life.

If he has counter transference and is actually looking at you to meet a need the he needs to process that and find another way thats more appropriate to meet his needs