Fiber Bragg Grating-Based Multimodal Sensing for Human Respiration, Heartbeat, and Voice With CNN-Driven Speech Recognition

With the rapid development of wearable health monitoring technology, the integration of simultaneous multimodal physiological signal monitoring and voice interaction has emerged as a critical research frontier in smart healthcare and human–computer interaction. Fiber Bragg grating (FBG) sensing technology offers innovative solutions for wearable health monitoring due to its advantages of high sensitivity, electromagnetic immunity, and biocompatibility. In this work, a contact multimodal physiological monitoring system based on FBG laryngeal sensing is proposed. The system converts the microdisplacements of the laryngeal skin induced by physiological activities, such as respiration, heartbeat, and vocalization, into the intensity variations of FBG reflection spectra, which are subsequently mapped into the fluctuations of the electrical signal amplitude. By analyzing the electrical signal, physiological parameters, including respiratory rate and heartbeat rate, can be resolved, and reconstructed speech signals can also be derived. In addition, a dual-channel feature fusion convolutional neural network (CNN) model adapted to the FBG laryngeal sensing system is further constructed to enable efficient speech recognition. A concept verification experiment is carried out. The maximum absolute measurement errors of respiratory and heartbeat rates are 1 rpm and 2 bpm, respectively, during a monitoring time of 60 s. Concurrently, the average recognition accuracy of the proposed fusion model reaches 96.38% in a speech recognition task with 20 words.