I grew up in the UK and learned to drive there. Moved to Canada in 1985 and US in 1988. For the last 12 years I've been traveling to Australia and NZ as well as the UK fairly regularly so I tend to alternate my driving from right to left and back.
In my personal experience it just makes me more vigilant since I have to check internally and be aware which side of the road is correct. This is helped by driving a car set up properly for that orientation - getting your head around driving a British car in France is a whole other difficulty setting.
It's not an easy thing - it takes a day or so to get reacquainted with the left or right side driving and then it becomes automatic (to me). I suspect it's much more challenging for people who are out of their comfort zone infrequently. I know NZ in particular has a big problem with wrong-side drivers, particularly when they have just got off a 13 hour flight and picked up a car or camper van. I've noticed in recent years that they are putting arrows on the road indicating the correct side to drive on after intersections and parking lot exits