In freehand mid-air manipulation within VR environments, users’ arms and hands can provide tactile cues for input operations. Conventional mid-air input lacks haptic feedback, which poses challenges for precise manipulation and continuous value control. This study proposes a novel approach that focuses on pressure variations during pinch gestures between the thumb and index finger, which are commonly used in freehand interactions. Specifically, we developed an efficient self-haptics method called NailHaptic for user interfaces (UI) with continuous value variables, by identifying the phenomenon where nail color changes according to applied pressure using RGB cameras. This system achieves the identification of three different levels of pressure applied to the nails through machine learning approaches. By integrating the machine learning model into the system, real-time pressure level determination becomes possible, demonstrating applications to intuitive UI in 3D environments.
MORIMOTO, Kosuke; HIRANO, Reo; WATANABE, Keita. NailHaptic: Continuous Value Control in VR through Nail Color Change Detection for Self-Haptics Interaction. In: Adjunct Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology. 2025. p. 1-3.