The antipathy against Elsevier is more to do with the fact that it’s strategy is to deliberately publish rubbish articles that have been identified as likely intellectual dead ends, and then charge for access. How many times is the typical article cited in an Elsevier journal? Twice or less, ie. it’s subject matter is irrelevant. If you’re buying an article from Elsevier, you know you’re wasting your money, and that the taxpayer money that funded the researcher was probably a waste of time too, because the article will be insignificant, but probably not fraudulent.
Elsevier focuses entirely substantially on the long tail of academic publishing, because in many fields (biology excepted) no-one will send a manuscript to Elsevier as a first choice. Elsevier are the sloppy seconds of the academic publishing world, and everyone knows it. Rejected manuscripts filter their way to Elsevier after a couple of bounces from respectable journals. A publication in Elsevier is just above the cut-off of unprintable.
Effectively you need a government to sponsor the copy-editing work as a one-off expense, each time an article is published, and to provide a small hosting subsidy. The government should also pay rewards for cited articles. e.g. if your journal publishes a paper that gains 10 citations, the government should reward that with say a $500 bonus the next year - if it publishes a paper with 100 citations, a $5000 bonus, etc. That would incentivise the editors to select worthwhile material for publication.