I wonder to what extent the neural network is associating sounds with the material itself (its texture and movement), and how much it’s purely learning about the motion of the drumstick. The latter would be less impressive, though still cool; if you think about it, with a sufficiently high frame rate you could simply measure the sound by seeing the vibrations through the stick, with no AI needed. Like that kind of remote audio bug that works by shining a laser onto a window or similar rigid surface.
Either way, there’s something neat about retrieving information that was never directly recorded. It’d be cool if it could get good enough to, say, extract on-set sounds from silent films.
