The wrinkle here is that both the Cayman, and British Virgin islands are not completely separate countries, they’re “British Overseas Territories”, and have a governor that is appointed by the UK. The fact that they make an enormous part of their income from helping people to dodge taxes is probably part of the reason they were set up in the first place, and why it’s particularly embarrassing that the UK government can’t get them to co-operate. Also, the inhabitants are British Citizens, so you can see why the government might feel that they should pay some attention when the UK says ‘jump’.
In contrast you’ll notice that they’re not going after Panama or Lichtenstein (or Switzerland or Delaware for that matter). Britain can’t really complain about other tax havens until it gets it’s own house in order.
(edit, I’ve just read about this which is just another little wrinkle)
