For a person buying raw plant matter from a regular seller, you’re right that the strength will be somewhat unpredictable.
How big of a deal that is depends very much on the drug’s overdose potential - taking morphine in the form of opium straight from poppies, for example, seems much riskier than taking psilocybin in the form of raw mushrooms.
If I were worried about unpredictable strength, I’d likely take the approach of grinding and mixing the plant matter up so a single dose is a fairly even cross-section of all the specimens obtained, starting with small doses that would be reasonable if the plant matter were of the highest strength available, and then working up to a desired strength of effect.
