Man hacks his prosthetic arm to control music synthesizer with his thoughts

Originally published at: https://boingboing.net/2020/02/15/man-hacks-his-prosthetic-arm-t.html

9 Likes

I wonder if the learning curve for controlling such prostheses is different for people who were born without a limb vs. people who lost a limb later in life.

8 Likes

Writing as an able-bodied person aware of the comfort I enjoy as a result, this is so, so interesting to me. I love these stories. There are increasing numbers of examples of people using prosthetic devices to extend human capabilities beyond what an able body can do, and there is something so purposeful, satisfying, and just about this that I can’t quite put words to it. And if the technology gets far enough along maybe I get to use it for something someday.

The flipside of that is virtually all of the examples I have seen of this are by affluent white people with highly localized congenital deformities, or limb loss due to an accident in youth. The majority of prosthetic users in the USA are low-income, elderly people who have lost limbs due to complications from lifestyle diseases. (https://kk.org/extrapolations/prosthetic-limbs-forecast-to-2050/. quick citation, not the sole source of info). So there is another set of justice issues in play here related to nutrition, income, and so on. Note that the guy in video emphasized that the prosthetic he cannibalized was outdated and nonfunctional - presumably not of use to anyone in need.

Mods like his are fucking sweet and advance humanity. They should be better known and more widely celebrated. Can we do it in a way that also shines a positive light on less sweet-and-futuristic prosthetic users? Like diabetic amputees, veterans with ptsd, and people whose physical differences came with neurological differences. Lets make it easier for them to access increasingly more useful, sophisticated, and interesting services and devices as well.

6 Likes

I got a little frisson when I watched the prosthetic hand soldering the circuit board.

6 Likes

How long until we get to Robocot style cyborgs where you cut off limbs to replace them with “superior” techonology? Which I suppose is still better than if AI evolves faster than robotics and they instead cut out brains to replace with computers controlling a human body.

McLuhan said that every innovation serves to amputate the function it extends. But it’s also said that a tool achieves perfection - or maybe transcendence - when it gets used for something other than the purpose for which it was designed. So, “well done that man!”, and “In your Face, McLuhan!”.

4 Likes

I think it is going to be a long time, especially for your AI fears.

“So I just strap on this device, and it will read my thoughts and let me make music?”

“That’s right.”

“Great! Let’s do it!”

♫ ♩ ♪ ♬
Ooh-ooh, ooh-yeah yeah, yeah yeah
Yeah, yeah, yeah, yeah yeah yeah
7 AM, waking up in the morning
Gotta be fresh, gotta go downstairs

“Wait, I don’t think it’s working—”

♫ ♩ ♪ ♬
Gotta have my bowl, gotta have cereal (Cereal)
Seein’ everything, the time is goin’
Tickin’ on and on, everybody’s rushin’ (Tickin’ on and on)
Gotta get down to the bus stop
Gotta catch my bus, I see my friends (My friends)

“No! Please! I swear I’m not—”

♫ ♩ ♪ ♬
It’s Friday, Friday
Gotta get down on Friday
Everybody’s lookin’ forward to the weekend, weekend
Friday, Friday
Gettin’ down on Friday
Everybody’s lookin’ forward to the weekend

“Oh god! Make it stop!”

4 Likes

You don’t have to remove the limb, just re-purpose the signals.

2 Likes

That does conjure Automatic Jack, a tad. :slight_smile:

2 Likes

I’m looking at this thinking the next step is a full USB-implementation, allowing the user to do highly advanced things like simulating an entire synthesizer controller rather than having to rely on multiple wires to individual input ports.

And then, obviously, a keyboard and mouse implementation as well.

1 Like

I think the limiting factor here is that the prosthesis only provides two channels of control. Although he’s able to rotate the hand attachment still, implying there’s at least one more channel, but it’s not exposed to the removable hand part but rather controls a motor internal to the forearm (and so would require more invasive modifications).

3 Likes

It would be interesting to know how much plasticity there is to work with on the human side; assuming a best-case neural interface situation.

One doesn’t achieve any but the crudest motions by consciously thinking about all the parts that make them up(much less incorporating feedback from sensory neurons on the skin, sight of the moving hand, etc.) in real time. Unless one can develop an adaptation analogous to the one we use to control a limb suitable to the new peripheral the effective data rate is going to be pretty tepid. Might still make sense to expose it as a USB host or the like, just for compatibility’s sake; but if the user has to bit-bang the output the effective data rate is going to be quite low.

I’d also be curious to know what the actual effective data rate; best case across the board, would be. I found this interesting examination of optic nerves; and while nerve bundles do massive parallelism pretty well; spike rates on a given fiber are not exciting.

I’d be the last to deny the sophistication of biological systems; but it wouldn’t surprise me if the effective ‘bus speed’ for lower level tasks is actually fairly tepid. Humans can perform some processing tasks (like, say, anything natural-language related; and some image processing stuff) with spooky efficiency; but it’s not clear that the links between our brains and our environments, inbound or outbound, are all that fast. Which is impressive because we aren’t just brute-forcing the results we get; but means less headroom if some brute forcing is exactly what you want to do.

2 Likes

The joke will be on us when the master computer sets our rations to “Eels in Darkness”.

1 Like

The video description mentions a myoelectric interface, but I’m not sure that’s definitive.

This article might be of interest.

I wonder if he could do a hack involving an A/D convertor and control other things.

The bartender’s smile widened. His ugliness was the stuff of legend. In an age of affordable beauty, there was something heraldic about his lack of it. The antique arm whined as he reached for another mug. It was a Russian military prosthesis, a seven-function force-feedback manipulator, cased in grubby pink plastic.

But can it play music?

“Too artistic.”

3 Likes

Brings to mind the intelligent, brain controlled jet aircraft in Count Zero.

2 Likes

It would be interesting to see what other sort of controls could be devised, but CV, despite being an old, dead-simple mechanism, has the distinct physical property of true continuous voltage changes.

For a similar functionality via MIDI 1.0 it would be clocked to the serial baud rate for USB or DIN MIDI and if, for example, you implemented CC messages they would be restricted to 128 discrete values unless NRPN was used which would get you up to 2^14 discrete values, but is not well supported. MIDI 2.0 spec was just finalized which I believe offers 32bit resolution control messages, though!

I’m not saying it wouldn’t be useful, but since he’s already using a eurorack format modular analog CV and gate is also very useful, since it is how all intermodule routing is accomplished on that platform (no normalized connections.)

I can’t fathom what the actual user input is like in this particular case as I don’t know how the muscle movements are read by the electrodes (if I even have that correct) but this seems like an example of purely analog cybernetics!

Extremely cool. Synth people are always so rad…

1 Like

This is really great. Myoelectric control has a lot of potential for all kinds of things. For example: