You’ve surely controlled a smartphone with your voice or gaze, but what if a gadget on your wrist could read your gestures literally from muscle impulses? Apple’s smartwatch can already recognize some gestures, but the company is working on the next step. It sounds fantastic, but Apple has published a study in which an AI model recognizes previously unseen hand gestures for the first time using muscle signals captured from wearable sensors.
Apple taught AI to recognize new gestures on your wearable devices
What Is Electromyography and Why Is It Needed in Gadgets
Apple published a new study on its Machine Learning Research blog. The paper is called EMBridge: Enhancing Gesture Generalization from EMG Signals through Cross-Modal Representation Learning and will be presented at the ICLR 2026 conference in April.
At the center of the research is EMG, or electromyography technology. Simply put, it’s a method of measuring the electrical activity of muscles during contraction. In practice, EMG has long been used in medical diagnostics, physical therapy, and prosthetic limb control.
But recently, this technology has been increasingly studied in the context of wearable devices and AR/VR systems. For example, Meta Ray-Ban Display glasses use EMG technology in the form of a so-called Neural Band — a wrist bracelet that, according to Meta, “interprets your muscle signals to navigate device features.”
It turns out Apple is moving in a similar direction. And, judging by the research, very confidently.
EMG signals are already used to control prosthetics, and soon they may control your watch as well.
What Data Apple Used to Train the Model
Interestingly, in Apple’s research, EMG signals were not captured from a wrist bracelet but rather using two large datasets. The first — emg2pose — is a large-scale open database containing 370 hours of EMG signal recordings with synchronized hand position data. It involved 193 users performing 29 different movement groups — from making a fist to counting on fingers. The total number of hand position labels exceeds 80 million, which is comparable to the largest datasets in computer vision.
Each finger movement is an entire symphony of electrical impulses in the forearm muscles
The second dataset — NinaPro DB2 — includes paired EMG and hand position data from 40 subjects across 49 different gestures: basic finger bends, functional grips, and combined movements. EMG signals were recorded from 12 electrodes on the forearm at a sampling rate of 2 kHz.
For comparison: 80 million hand position labels is as if every resident of Germany showed a gesture, and each one was recorded and annotated.
How EMBridge Works and Why It’s a Breakthrough
Apple has long used AI not only in products but also in manufacturing itself — robots at Apple factories are already assembling iPhones using neural networks, and that’s just the beginning. Now, the main development of the study is the EMBridge framework. Its task is to “bridge the gap” between real muscle signals and structured hand position data.
The model was first trained on EMG data and hand pose data separately. Then the researchers aligned the two types of representations so that the EMG encoder could learn from the pose encoder. In other words, the model learned to understand gestures through the lens of muscle signals.
Next, the team applied so-called masked pose reconstruction: part of the hand position data was hidden, and the model had to reconstruct it using exclusively information from EMG signals. It’s like being asked to guess a word with half the letters covered, but you can hear how it’s pronounced.
The model literally learns to “translate” from the language of muscles to the language of gestures — even if it sees this specific gesture for the first time.
To reduce training errors where similar gestures were accidentally perceived as completely different, the researchers taught the model to recognize similar hand configurations and generate “soft labels” for them instead of hard categories. This significantly improved the system’s ability to generalize gestures.
The result is impressive. According to the authors: “EMBridge is the first cross-modal representation learning framework capable of zero-shot gesture classification from EMG signals on wearable devices.” Simply put, the model can recognize gestures that were not part of its training dataset at all. And it does this using only 40% of the training data compared to existing methods.
Why Apple Needs This and What It Means for Users
The study itself, of course, does not mention specific future Apple products. But the authors directly point to practical applications: wearable devices for human-computer interaction. The paper discusses VR/AR scenarios and prosthetic control, where “a wrist-worn device must continuously determine hand gestures from EMG to control a virtual avatar or robotic arm.”
It’s not hard to imagine how such technology could be implemented in future Apple Watch models or other wearable devices for controlling Apple Vision Pro, Mac, iPhone, and even smart glasses, which are rumored to be in development. For example, Apple’s smartwatch already measures body parameters, and the accuracy of this data continues to grow.
In practice, this opens the path not only to new ways of interacting with devices but also to serious improvements in accessibility. For people with disabilities, this is especially important — accessibility features on iPhone already can do things many people don’t realize, and EMG control could be the next step.
However, there is a limitation. The authors note that the model still depends on specialized datasets containing both EMG signals and synchronized hand position data simultaneously. Collecting such data is difficult and expensive.
Perhaps very soon, to control a computer, it will be enough to simply wiggle your fingers.
The EMBridge study is yet another confirmation that Apple is seriously investing in technologies that could change the very essence of human-technology interaction. Gestures the model has never seen, it already recognizes. And that means the future where your watch understands the language of your muscles is closer than it seems.
