I think our perceptions and experiences are constantly shifting what we think. It could be while you were working with them you were more focused on what you were doing than the people whereas now you're doing something else you can focus on the people. . . and realize you didn't really like them.

There's opposites at work in our lives too, remember:

“Bran thought about it. 'Can a man still be brave if he's afraid?'
'That is the only time a man can be brave,' his father told him.”
~George R.R. Martin, A Game of Thrones

I know what you mean though; I've had those, "What am I doing here?" moments or, worse, "How did I get here?". Usually mine have come because I didn't pay sufficient attention to myself in the moment, just went along for the ride or because it seemed right in some way when I was approached with it (I need a job, any job). Sometimes it has only been when trying to bend myself to something like the proselytizing that I suddenly come to my self/right mind and realize it's "not me". Hard to always know sometimes if we have not had the experience yet but just lived in our head? The map is not the territory.