I'd say if your cocktail isn't super complicated and/or needs to be adjusted all the time and your primary is comfortable with it, yeah, good for it. A good doc is a good doc. Bad apples in every specialty, and there's no rule saying a PCP with very basic knowledge of psych meds, but is probably more humble and can accept when a patient with angency disagrees with them is always worse than a psychiatrist with a God complex thinking his prescribing philosophy that is clearly advised against by practically everybody that's researched the subject you don't trust that's already proved they are an irresponsible prescriber and caused harm.

Primary might not have enough specialized knowledge of meds and their effects and the effects on different conditions to be competent or comfortable with messing with a regime though.