It's not wrong to question it at all. Doctor's misdiagnose all the time. Don't be afraid to question things if you don't feel like something fits. You know yourself better than your doctor.. if it took you this long to be comfortable, could you not have been comfortable with the doctor just because you weren't comfortable with his diagnoses? Or did you maybe leave some things out that would have gave a different thing to diagnose with you? Just some thoughts.

I think if you have a doctor that can't help you to understand why they diagnosed you the way they did or a doctor that would get offended with you questioning it then you probably need a new doctor anyway. Maybe you could just say something like "I've been reading about it on the internet and don't feel it fits me"..(but he might say don't believe everything you read on the internet, not understanding it's from reliable websites).. If he asks why then explain it. Maybe you could print the stuff you did find on personality disorders and check off what you do have and show why you think you should be diagnosed with that instead.

Or maybe just beat around the bush and get him to tell you why he thinks you have that and kind of say that it doesn't sound like you.