I would argue that when people say “I told you literally a million times,” they AREN’T using the word literally to mean figuratively. It’s NOT clarifying that this is not a factual statement. Nobody believes that is is. Rather it just adds to the hyperbole. So it doesn’t really MEAN “figuratively” in that sentence. And I think that for the most part that is exactly how people understand it.