Medievalist with a special interest in Ireland and England here:
They didn’t think in terms of “races” so much as “peoples”, and they did that by culture and language more than by colour. (Although physical characteristics were also well known: the Romans associated red hair with Germanic tribes, and by the time Britannia was being invaded, the Picts, with their red hair, were assumed to have migrated from Germania.)
By the time of Bede, the words for “a people” and “a language” overlapped. Bede’s description of Brittaina said
Haec in praesenti, iuxta numerum librorum, quibus lex diuina scripta est, quinque gentium linguis, unam eandemque summae ueritatis et uerae sublimitatis scientiam scrutatur, et confitetur, Anglorum uidelicet, Brettonum, Scottorum, Pictorum et Latinorum, quae meditatione scripturarum ceteris omnibus est facta communis.
There are in the island at present, following the number of the books in which the Divine Law was written, five languages of different nations employed in the study and confession of the one self-same knowledge, which is of highest truth and true sublimity, to wit, English, British, Scottish, Pictish, and Latin, the last having become common to all by the study of the Scriptures.
By the time this was translated into Old English for the Anglo Saxon Chronicles, this became
⁊ her sind on þis iglande fif geþeode: englisc, ⁊ brittisc, ⁊ wilsc, ⁊ scyttisc, ⁊ pyhtisc, ⁊ bocleden.
And there are on this island five languages: English, and British, and Welsh, and Scottish, and Pictish, and Book-Latin.
(Yes, that’s six in total, but note that “British” and “Welsh” were the same language. Also: “Scottish” meant “Gaelic”, because there was only one people, based in Ireland, who were expanding into southern Scotland, and they were called “Scotti” as much as anything.)
The word geþeode is derived from þeod, which is cognate with words like the Irish tuath and the Proto-Germanic Teuta, thus “Teuton”, and via þeudo to Deutsch. This word originally meant “a people”, and in Old English later expanded to “a language”, indicating how closely the concepts were intertwined.
The Anglo-Saxons and the Irish were on as good terms as two neighboring cultures could be. They traded, they swapped clerics, they occasionally skirmished, when the Scots and the Picts had more-or-less merged into one culture, they and the Anglo-Saxons butted heads at the Northumbrian border.
The real watershed happened after the Norman Invasion. First the Normans conquered England, then, a hundred years later, Strongbow led the Norman invasion of Ireland in 1170. And Prince John of England was soon made Lord of the newly “conquered” Ireland.
John visited Ireland in 1185. He was 19, and exactly as mature as you’d expect a 19 year old to be. He managed to annoy the local Norman lords, he actively insulted the Irish lords and kings who met him (the story is that he went around pulling on their beards because he thought it was funny), and when he got tired of everyone getting angry with him all the time, he went home to England.
In John’s court when he travelled was a Norman/Welsh minor nobleman and clerk named Giraldus Cambrensis, Gerald of Wales. And he saw an opportunity. So he wrote a book called Topographia Hibernia “The Topography of Ireland”.
And he made a special point to say that John’s lands were fine and valuable, but his reception wasn’t his fault, because these lands were spoiled by the animals living in them:
This people then, is truly barbarous, being not only barbarous in their dress but suffering their hair and beards to grow enormously in an uncouth manner, just like the modern fashion recently introduced; indeed, all their habits are barbarisms. But habits are formed by mutual intercourse; and as these people inhabit a country so remote from the rest of the world and lying at its furthest extremity, forming is it were, another world, and are thus excluded from civilised nations, they learn nothing and practise nothing, but the barbarism in which they are born and bred and which sticks to them like a second nature. Whatever natural gifts they possess are excellent, in whatever requires industry they are worthless.
Topographia Hibernia became, quickly, the standard reference work when anyone in England wanted to read up on Ireland, and it almost immediately became common knowledge that the Irish people were barely better than animals.
It only got worse after that point. (See: the history of Ireland from 1200 on.)
So you can trace the dehumanisation of the Irish back to c.1200, and a minor cleric’s desire to suck up to his boss.
It wasn’t a matter of colour, because that wasn’t really a thing at the time. And who was hated by whom depended to some degree on who their king was at war with.
There was definitely racism, though. It was just mediated through nationality (which was, as it still is, a complicated concept) rather than skin colour.