As I understand it from a skim of the Spectre paper, the mechanism for extracting the desired sensitive information from the target program is not by reading the contents of the cache, but through a timing attack: If you want to know the value of the byte at address x, then you cause the program to speculatively attempt to access address y + *x, causing that address to end up in the cache. Then, you cause it to read values from y + 0 to y + 255, and infer from timing whether the read was a cache hit or miss.
It depends on finding a program which does certain types of accesses based on values you control (but evidently plenty of programs do), and a fine-grained timer to have a shot at differentiating cache hits from misses. The proposed mitigations I’ve seen include reducing precision of the timer available to javascript, and modifying compilers to detect programs which have the necessary characteristics to be exploited and automatically inserting guards.
