I'm female and I wear men's boxer briefs and I never thought twice about it. I think anyone should be able to wear what makes them comfortable as long as it's appropriate for the situation. Like not wearing a bathing suit to school etc.