Human Activity Recognition with Consumer Devices and Real-Life Perspectives
During the last decade, research on human activity recognition (HAR) has been consistently growing due to its broad range of applications such as surveillance, exercise monitoring or healthcare monitoring systems. For instance, many researchers are putting their efforts into applying HAR for fall detection applications targeting elderly people. The main drawback of most of the existing research is the requirements in terms of sensing devices – cost, amount, placement – which hamper the applicability of the proposed systems in real-life scenarios. Therefore, a lot of research is needed to address this issue so that it can have a real impact on the population.
In this doctoral dissertation, the abovementioned issue is addressed by focusing the research on globally available consumer devices – smartphones and smartwatches – and cheap microcontrollers to study their feasibility and applicability in real-life HAR applications. During the thesis, two sensing techniques are explored: an inertial-based approach with smartphones and smartwatches, and a Wi-Fi CSI-based approach with microcontrollers.
For the inertial-based approach, reliable data collection libraries for smartphones and smartwatches are firstly developed to support researchers and developers in creating advanced HAR applications. Secondly, a dataset with inertial measurements from smartphones and smartwatches for HAR is collected for five distinctive activities: SEATED, STANDING_UP, WALKING, TURNING and SITTING_DOWN. The dataset differentiates from other available datasets by its heterogeneity and balance in terms of participants’ age and gender. Thirdly, the collected dataset is employed in a three-way analysis where the impact on the recognition performance of the selected model – MLP, CNN, LSTM and CNN-LSTM –, the employed data source – smartphone, smartwatch or both – and the amount of data available for training is studied. Fourthly, based on the previous results, an mHealth system is developed to automate the computation of the results of a well-known mobility test, the Timed Up & Go (TUG) test, which demonstrates the feasibility of consumer devices for real-life HAR applications.
For the Wi-Fi CSI-based approach, an experiment with ESP32 microcontrollers is conducted to determine the feasibility of using the CSI data source for HAR real-life systems. Five activities (i.e., SEATED, STANDING_UP, WALKING, TURNING and SITTING_DOWN) and their relative position regarding the environment (i.e., towards the transmitter or receiver device) were considered. Preliminary results, supported by further experiments, determined that this data source might be unstable over time, making it unfeasible for use in a real-life scenario.
Human activity recognition, Real-life applications, Inertial measurement units, Channel state information, Machine learning, Deep learning, Smartphones, Smartwatches, ESP32 microcontrollers, Timed Up & Go test.
Funding
This thesis is funded by the Spanish Ministry of Universities with a predoctoral grant (FPU19/05352) and a research stay grant (EST23/00320).
Financial support for derived activities of this dissertation was received from the SyMptOMS-ET project (PID2020-120250RB-I00), funded by MICIU/AEI/10.13039/501100011033.
License
Human Activity Recognition with Consumer Devices and Real-Life Perspectives Copyright © 2024 Miguel Matey Sanz. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
