How Mind-Controlled Bionic Arms Fuse To The Body

[Narrator] You're looking at a game changer

in prosthetics.

[Dr. Catalan] The only one today

using electrodes implanted in the nerve

and to have sensation.

[Narrator] The developer of this bionic system

is speaking to us from Ukraine,

where war has led to a crisis.

There is more than 15,000 people with amputations

in the country.

[Narrator] Let's walk through every step needed

to implant his bionic arm into a patient.

Bionic basically means

that it is a combination between biology and electronics.

[Narrator] But traditionally,

prosthetic arms are pretty low-fi.

Some are purely aesthetic,

made in silicone, but not functional.

Then there are functional mechanical ones

powered by a wire and a patient's own movements.

You can think about a claw or a hook

that can open and close,

and it has a system of gears like breaks for your bicycle.

[Narrator] Then you have fancy electric prosthesis,

where a patient can control the fingers independently

via electrodes placed on the surface of the skin.

But, how do you keep the arm in place?

[Dr. Catalan] This is normally done with a socket,

something that is on your skin, putting a lot of pressure.

[Narrator] It's uncomfortable and heavy,

so that's why the first step

in installing Dr. Ortiz's Catalan's bionic arm

is osseointegration,

or implanting a titanium structure directly to the skeleton.

Osseointegration made a a big splash in the medical field.

The first application were dental implants.

[Narrator] Eventually scientists

applied this to prosthetics

after discovering in the '50s

that if you attach titanium inside bone,

the bone cells can grow directly on the titanium,

making a very strong attachment to the residual limb.

Say you have a transhumeral amputation above the elbow.

The surgeon will place

a titanium implant that looks like a screw

inside the center of the bone.

And you leave it there for a few months.

In that period,

the bone cells grow around the titanium implant,

and then you place a portion of the implant

that comes out through the skin,

and that's where you're gonna connect your prosthesis.

[Narrator] Implanting into a residual limb

that has been amputated below the elbow

has its own challenges,

because there are two smaller bones,

the radius and the ulna,

and they move independently from each other.

So they will move like this. They will move like this.

And they will also move in their own axis.

So we developed an artificial joint

that allows for those movements to take place

while preserving a natural orbit for the movement.

[Narrator] Now, the next step

is to surgically implant the electrodes inside the body.

We will place electrodes

in the muscles and the nerves around the residual limb.

Electrodes on the surface of the skin

are susceptible to electromagnetic interference.

[Narrator] Stuff all around us like tools or computers

can create noise interference radiating to the electrodes

if they're merely sitting on the surface of the skin.

This will cause the prosthetic to become uncontrollable.

Even just moving your arm around

can throw off a conventional sensor.

If it lifts a little bit,

it generates what's called a motion artifact.

If you move too fast, if you sweat,

the prosthesis become less controllable.

[Narrator] With electrodes implanted

directly inside muscles on nerves,

you don't have any of those problems.

If you have an amputation where the hand is gone,

you have many muscles here

that help you to control the fingers of the hand.

So there's a lot of sources that you can use

to drive the prosthesis.

[Narrator] But in the case of an amputation

above the elbow,

you don't have as much to work with.

So the team has to get creative

and rejigger the body's original biological wiring.

You have the biceps, and the biceps has two heads.

So it's not enough information

that we can extract from the muscles

to drive all the missing joints.

So a solution for that is

you can take a nerve that used to go to the hand

and then you transfer it into one head of the biceps.

So then when the patient thinks about closing the hand,

this part of the muscle will contract the short head.

[Narrator] There are three big nerves in the arm,

the radial, the ulnar, and the median,

which allows you to control these three fingers.

Basically, a nerve is a collection of axons

which are bundled into fascicles.

So if you think about my fingers

as the bundles of the nerve, what you can do is split them.

You take one of those bundles

to connect with that muscle that is available there.

And then for the other ones,

we can borrow a piece of muscle from the legs.

It's called a free muscle graft.

And then we transfer that to the arm

to connect to one of these fascicles.

[Narrator] Next, you insert a metal electrode

into the muscle,

and connect that to the connector

inside the titanium implants.

[Dr. Catalan] So it has a wire,

and that wire is covered

by materials that are biocompatible,

meaning that they're well taken by the body.

[Narrator] The signals that come from your brain

to control your limbs

travel through nerves,

but these organic signals are relatively weak,

about 10,000 times smaller

than the strength of the signals

generated by these new electric plugs.

So in a way, the implanted electrodes

use the muscle like a loudspeaker,

amplifying the signals from the brain to the muscles,

and to the prosthesis.

[Dr. Catalan] The implantable part has no batteries.

All the power happens in the prosthesis.

You can think about it as a USB port

into the nervous system.

[Narrator] The next step involves

training the AI in the bionic hand's CPU

to understand what the signals from the brain mean.

The way we control our limbs is by electric signals

coming down through the nerves to the muscles,

and these come in the form of electric impulses.

Those signals are captured

by the electronics of the prosthesis.

So they travel down to the prosthesis

where the brain of the prosthesis

understands what those signals are.

[Narrator] But that CPU in the prosthesis

doesn't automatically know

what those patterns of activation mean.

The AI needs to be trained.

[Dr. Catalan] What we do is we tell the patient

try to close your hand,

and then we record signals.

And then we say try to open your hand,

and we record the signals.

[Narrator] And then,

the team labels that action for the AI,

translating neural signals from the brain into code

that is now understood by the tiny computer

in the prosthetic arm,

which then engages its robotic motors

to move in specific ways.

The next step involves training the patient using software.

This was actually the first time

we saw the patient after the surgery.

We connected it to a virtual reality system.

[Narrator] Those two cables coming out of the implants

are sending signals wirelessly to the computer,

where they are interpreted

and used to control a virtual limb.

This trains the muscles

and makes the signals more distinct and reliable

in preparation for when the patient gets their bionic limb.

But, this training also addresses another challenge

that arises from amputation.

After you have an amputation, there's pain that remains

from something called phantom limb pain.

[Narrator] Which is caused by the brain getting confused

and imagining that the missing limb

is frozen or twisting in awkward ways.

So I developed some technologies

to treat phantom limb pain.

We couple those with virtual augmented reality

so the patient can engage the same neural resources

that were used to control the hand.

This helps them reduce their pain.

[Narrator] This training is useful

in fine tuning the algorithms

that will drive the robotic motors.

But working in the virtual world is one thing.

Without their bionic arm attached,

patients will do relatively well because there's no load.

So, the final step involves fitting the prosthesis

and testing in the real world.

Patients come into the lab, put on their bionic arm,

and perform daily tasks like packing a suitcase

or picking up small objects.

[Dr. Catalan] These are tasks that can tell you

a little bit about the function

the patient has with the prosthesis.

[Narrator] The team then makes adjustments

and runs further tests that evaluate and help improve

one of the most jaw dropping features of the prosthetic,

its ability to feel objects in its grasp.

When the prosthesis make contact with the object,

there are sensors in the fingertips,

and then the brain of the prosthesis

has also a neuro stimulator,

which delivers electrical pulses to the nerves.

[Narrator] And because the brain

receives this data from a nerve

that used to be connected to the biological limb,

it will interpret it as coming from the bionic hand.

If I have a biological receptor in my index finger,

that has a nerve that goes all the way up to my brain.

If I put an electrode along that nerve,

it doesn't matter where I stimulate,

the brain will create a sensation

as coming from the fingertip,

it's an automatic sensation that rises in consciousness.

[Narrator] The bionic hand uses sensors

in its thumb and index finger

to send an electrical signal through the prosthetic

and then along the original severed nerves

straight to the brain.

But the information from the fingertips

is not as nuanced as what a biological fingertip feels.

For us, we have hundreds of sensors

that travel in hundreds of neurons.

Today, we don't have that resolution

at the neural interface.

[Narrator] We're still a long way off

from the type of sensation

seen in the artificial limbs in Star Wars.

This hand only provides rough sensations,

but they're still useful,

because now a patient can feel

when there's an object in their hand

and if that object is slipping away.

But what about batteries?

They have to power the CPU and the motors

that drive the prosthesis, right?

You can have interchangeable batteries,

and whenever the prosthesis run out of battery,

they just switch it.

A battery will normally last a full day.

It's very much like our phones.

Everything is self-contained.

[Narrator] So the days of patients

carrying heavy backpacks

full of computers or bulky batteries are gone.

These days, Dr. Ortiz Catalan

really only sees patients a couple times a year

when something breaks or if he needs to fine tune anything.

But these high-end bionic hands

can come with a price tag of over $10,000.

[Dr. Catalan] But hopefully, like any other technology,

more it's available, the less the cost will be.

We created a human machine interface,

which means we can connect the prosthesis

or we can connect to your steering wheel of your car

and you can drive it

by thinking about movement of the wrist.

You can integrate it to whatever your imagination wants.

[Narrator] Cool.

So, can we make humans stronger, cyborg style?

[Dr. Catalan] There will be companies

that think about human augmentation,

making a human jump higher, run faster, carry higher loads.

You can have one prosthesis

that's much stronger than a human hand,

but you cannot have a prosthesis

that is as dexterous as a human hand.

That's something that we haven't achieved

from the robotic side.

Personally, I got involved in prosthetic devices

because I wanted to solve problems.

And I'm in the business of bionic medicine.

There's so many problems out there

that have not been solved when it comes to disabilities

that I feel that is more important that we focus on that.

[ringing music]