Originally published at: Using generative AI and brain scans to read minds | Boing Boing
…
2 Likes
This is a great set-up for a C’thulhu horror film.
1 Like
Reading through the full paper I thought this bit was interesting where they explained why the resulting images from different test subjects varied from each other:
Which kinda confirms something we alreay knew: that different people may perceive the same reality differently from each other.
In the future when this technology is a bit more advanced I can imagine all kinds of interesting psychological experiments that someone could come up with. For example you could probably easily discern someone’s political positions by showing them various pictures of politicians and seeing if the reconstruction images were showing nice, attractive-looking people or inhuman monsters.
3 Likes
Can anyone explain this in little words? I read through the paper, and I am usually pretty good at figuring out academic-speak, but I’m lost on this one.
Exactly what was the process?
They showed someone a picture, read their brain waves, and then… ? I can’t figure out how they got from the brainwave to an image. They say they didn’t need to train a model, but I can’t see how it worked without that.
3 Likes
I wonder if there is any correlation between the fidelity of the raw images retrieved compared to the ability of the person to view an image in their mind. From a brief Googling it appears that the visual cortex is also engaged when viewing mental pictures. I’ve read that the ability to see images in your mind varies from not at all to life like complexity.
1 Like
It looks like it’s in section 3.3:
The only training required in our method is to construct linear models that map fMRI signals to each LDM component, and no training or fine-tuning of deep-learning models is needed.
…
To construct models from fMRI to the components of LDM, we used L2-regularized linear regression, and all models were built on a per subject basis.
My 2-minute reading is that they do some linear regression from the fMRI readings to predict the images, then clean it up the result using stable diffusion. The per-subject basis part is probably necessary because the fMRI data is probably voxelized and brains aren’t all exactly the same size and shape, etc. My guess is that stable diffusion is doing a lot of the heavy lifting here; the predictions direct from the fMRI are going to be very noisy.
2 Likes
Skip to 1:00 for an explanation.
1 Like
I don’t purport to understand the technology here, but the clock tower example is interesting. Subj05 seems to have fixated on the watch face and not given much thought to the architecture. Maybe they’re a watch enthusiast? And one could speculate that Subj07 doesn’t like tall buildings much so mostly reflected on the monumetal and hovering nature of the construction. Whereas in my mind the same building has a fairly gentle and incremental elevation where the height is offset by clear horizontal lines. I am not saying I am more right of course, but it’s intersting that our views can differ like that.
1 Like
This topic was automatically closed after 5 days. New replies are no longer allowed.
