AI: learning or counting?

Continuing the discussion from Diving into the anti-sunscreen movement:

That’s:

A) off topic; and
B) not how language models work. They’re not learning anything about the content of the text. Just how words statistically fit together.