In mental task-based brain-computer interfaces (MT-BCIs), motor imagery represents the most commonly employed control paradigm; however, classification accuracy substantially deteriorates when discriminating between different movements within the same limb. The conventional solution to this problem has been to remap intended actions to imagery of different limbs, thereby undermining the inherent advantage of intuitive control in MT-BCIs. This study proposes a multimodal imagery approach that combines motor imagery with speech imagery of Japanese onomatopoeia. Leveraging the sound-symbolic properties of onomatopoeia, which enable intuitive representation of movement characteristics, we employed “teku-teku” for slow walking and “suta-suta” for fast walking to facilitate discrimination of movements within the same limb. We conducted a comparative experiment with 11 participants under two conditions: motor imagery only (MI condition) and motor imagery combined with onomatopoeia (MMI condition), evaluating both BCI performance and user experience. For slow-fast classification distinguishing movement speeds within the same limb, the MMI condition achieved 67.59% accuracy, demonstrating a significant improvement over the MI condition (61.73%, p = 0.0053). The online task success rate was also approximately 10.84 percentage points higher in the MMI condition (66.42%) compared to the MI condition (55.58%, p = 0.0208). Subjective assessments revealed that the MMI condition significantly reduced overall cognitive load as measured by the NASA-TLX weighted score (p = 0.0097), with particular improvements in mental demand, self-assessed performance, and frustration scores. Furthermore, the MMI condition enhanced imagery vividness and consistency, and strengthened the sense of agency.
Reo Hirano and Keita Watanabe. 2026. Motor Imagery BCI Extension Using Onomatopoeia Speech Imagery: Discriminating Walking Speeds within the Same Limb. In Proceedings of the Augmented Humans International Conference 2026 (AHs '26). Association for Computing Machinery, New York, NY, USA, 29–40. https://doi.org/10.1145/3795011.3795046
