The reason it's called "transference" is that a person transfers feelings that are really about somebody else, onto (for instance) the therapist. Transference happens in other relationships as well - I might get angry at a friend for doing something that reminds me of something annoying that my wife does, for instance. (That's a fictitious example. I have no wife.) I've read somewhere that love at first sight is actually transference - we can't have any actual well-founded feelings about somebody we've only just set eyes on, so such strong feelings have to be rooted in feelings we have had at some point for someone else.

And all transference is not about (romantic) attraction. As I said, it can lead to anger. And with therapists, it is also very common to have parental transference, where we get feelings that are actually about our parents, and turn T into a mother or father figure.