But it is. At least functionally in many cases. In the case of academia, think about what expertise is. It's paying a significant opportunity cost to often specialize in a single field. People who become experts in certain fields forgo other experiences and understandings. That's just a function of good old-fashioned linear time and how humans experience it as a finite resource. So if a bunch of physicists are, say, experiencing funding issues they'll be the first and the most knowledgeable people to raise the alarm. The reality is that basically no one outside that community exists who cares enough to do the homework. Even administrators and "non-physicists" within the group of people who understand funding issues for physics, are the same people who once were physicists, and they will draw upon the knowledge of what physicists need to make informed opinions. The world is incredibly incestuous and interconnected, there's no such thing as a detached unbiased person.
You can extend this to a lot of other concerns. There are traffic safety experts who are capable of having a lot of facts at their disposal, but might still see cycling as "recreational" because they've never been in a position to commute by bicycle. I can easily see where a cyclist's perspective isn't just something he should ignore in the interests of some external, and ultimately nebulous, standard of "balance" or "impartiality." Oftentimes what people are saying when they say, "as an X" is that they've had firsthand experience with the subject, and to please take into account that they might not be talking out of their ass.
You, as a free-thinking reasonably intelligent person, can calibrate that according to common sense. "As a mother" means little to me in certain contexts, because lots of people are mothers, and I can easily see lots of other mothers disagreeing in the same context. "As a mother, I'm concerned about rainbows appearing in sprays of sunlit water" means precisely that. They have children and are worried. Does it mean that I need to be worried? Of course not. "As a mother, I'm telling you it's crazy how much diapers cost." That's a statement where I can place and understand the perspective. It carries weight. That's because of instead of applying a simplistic rule that says anytime someone says, "As an X" I ignore them, I think about what that means on a case by case basis.