There’s certainly racism involved; the root meaning of “white trash” is something along the lines of “a white person so degraded that they’re no better than a black person”.
But I wouldn’t call the term racist in itself, as it doesn’t fit in the “racism = prejudice plus power” understanding of the issue that I prefer.
The “white” in “white trash” is the modifier, not the core. It’s you-are-trash-that-is-white; the trash is the insulting part, not the white.
I’ll occasionally use white trash in a reclaiming working class pride sense, but not often. Used outside that context, the appearance of the phrase is usually a warning flag that some heavily classist bullshit is about to be dumped.
I’ve no urge to restrict the language of others, but I do often wish that people would put a bit more effort into analysing their use of it. “White trash” is a concept used by the wealthy to sow division between the middle class and the working class.