Imu Based Fall Detection

Optimize Sensor Configuration

Abstract—Falls represent a significant cause of injury among the elderly population. Extensive research has been devoted to the utilization of wearable IMU sensors in conjunction with machine learning techniques for fall detection. To address the challenge of acquiring costly training data, this paper presents a novel method that generates a substantial volume of synthetic IMU data with minimal real fall experiments. First, unmarked 3D motion capture technology is employed to reconstruct human movements. Subsequently, utilizing the biomechanical

Simulation Platform Opensim And Forward Kinematic

methods, an ample amount of training data from various body segments can be custom generated. An LSTM model is trained, achieving testing accuracies of 91.99% and 86.62% on two distinct datasets of actual fall-related IMU data,

Demonstrated The Comparable Performance Of Models

trained using genuine IMU data. Building upon the simulation framework, this paper further optimized the single IMU attachment position and multiple IMU combinations on fall detection. The proposed method simplifies fall detection data acquisition experiments, provides novel venue for generating low cost synthetic data in scenario where acquiring data for machine learning is challenging and paves the way for customizing machine learning configurations.

Index Terms— Fall Detection; Imu; Opensim; Data

augmentation.

I. Introduction

ALLING is one of the main causes of injuries among the elderly, exerting profound impacts on their health and quality of life. It is estimated that approximately 684,000 cases of fatal fall injuries occur annually, positioning them as the second leading cause of unintentional injury- related deaths, following closely behind road traffic injuries.

Manuscript received XX, 2023; This work was supported by the National Natural Science Foundation of China under Grant 61825303, 52105033, and 62088101; in part by Shanghai Municipal Science and Technology Shanghai Sailing Program under Grant 21YF1450400 and Major Project under Grant 2021SHZDZX0100; in part by Chenguang Program by Shanghai Municipal Education Commission under Grant

21Cga23. (*Corresponding Authors: Shuo Jiang.)

Jie Tang, Bin He, Zhipeng Wang, Yanmin Zhou and Shuo Jiang are with the College of Electronics and Information Engineering, Tongji Center for Intelligent Autonomous Systems, Shanghai 200120, China

(E-Mail:

Notably, within all fall-related incidents, individuals aged 60 and above exhibit the highest mortality rates. Elderly individuals face the greatest risk of death or severe injury due to falls, with this risk escalating as age advances. For instance, in the United States, 20% to 30% of elderly individuals involved in fall accidents endure moderate to severe injuries, encompassing abrasions, hip fractures, and cranial trauma, among others . Consequently, the pursuit of fall detection research and its practical implementation hold crucial socio- medical significance, offering the potential to furnish comprehensive care and assurance for the well-being of the elderly.

A diverse array of existing methodologies exists within the realm of fall detection. From the sensing perspective, these can be broadly classified into two categories: camera-based and wearable Inertial Measurement Unit (IMU) based approaches . In contrast to camera-based fall detection systems, wearable IMU-based systems are characterized by their ease of deployment and immunity to occlusion challenges. Early implementations of fall detection systems utilizing IMU sensors employed simplistic threshold-based detection algorithms, whereas the feasibility of machine learning techniques in enhancing the recognition of fall movements has been demonstrated . Wearable devices, particularly those incorporating IMUs, in combination with machine learning methodologies, play a pivotal role in fall detection. Queralta et al. established a fall detection system encompassing IMU sensor nodes, edge gateways, cloud services, and end-user applications . Liu et al. fused IMU sensor networks with Doppler radar systems, resulting in a significant reduction in false positive rates within fall detection systems . He et al.

integrated IMU sensors into a vest tailored for the elderly, Junkai Xu is with the State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong

(E-Mail:

Mentions of supplemental materials and animal/human rights statements can be included here. Color versions of one or more of the figures in this article are available

2

transmitting data via Bluetooth to a mobile smartphone running a fall detection program based on the k-Nearest Neighbors (KNN) algorithm, thereby enabling automatic alerts upon fall incidents . Xu et al. introduced a fall detection algorithm founded on a CNN-LSTM (Convolutional Neural Network- Long Short-Term Memory) composite network. This approach leverages neural networks to automatically extract features and conduct classification, obviating intricate preprocessing procedures .

The utilization of wearable IMUs in conjunction with machine learning for fall detection necessitates a substantial quantity of IMU data encompassing both fall and non-fall activities. However, the cost of acquiring fall detection data is high. Conventional data collection approaches typically involve tracking the IMU data of elderly individuals or recruiting younger participants to perform motion experiments for data collection. Lisowska et al. collected data from 20 volunteers wearing Silmee devices during simulated falls and daily activities . Some researchers have publicly released experimental datasets, such as the MobiFall dataset and the SisFall dataset . Bagala et al. established the FARSEEING database, noting that a comprehensive shared real-world fall database could enhance the understanding of fall processes .

Pre-Experiment

considerations: experimental protocol design and IMU sensor placement. Once the experimental protocol is established, post- experiment parameter adjustments become challenging, often requiring costly reiterations of the experiments. Despite comprehensive experimental protocol designs being proposed to contrast fall and non-fall movements , variations in action designs persist across experiments and datasets. This diversity may stem from considerations of experimental complexity or differing research foci. Özdemir et al. included 20 fall movements and 16 non-fall movements in their experiments , whereas Alfred Wertner's research examined only 4 fall and 10 non-fall movements . Wang et al.

highlights that the datasets utilized in most papers often fail to encompass all types of falls found in real-world scenarios . Variations in IMU quantities and placement positions are also evidently different across different datasets. Tong et al. propose that the wrist, hip, and thigh are not suitable positions for accelerometer-based devices due to their high frequency and complexity of motion. Instead, they suggest that regions below the neck and above the waist, specifically the upper torso, are more appropriate for distinguishing fall-related characteristics using acceleration data . Nukala et al. designed an artificial neural network for fall detection using IMU data from the waist and back, achieving a sensitivity of 97.0% and specificity of 97.2% . In the experiments conducted by Ojetola et al., two SHIMMER sensors were attached to the participant's chest and thigh . Yan et al. proposed a fall detection method based on multi-inertial sensor skeletons using ST-GCN and employed the Up-Fall dataset , which places IMUs on the neck, waist, thigh, wrist, and ankle for model training and testing . In summary, the requirement for participants to wear IMU devices during motion experiments for data collection is associated with both high costs and limited flexibility. Furthermore, the integration of these datasets presents challenges due to variations in the experimental designs across different studies, potentially resulting in a diminution of data set quality.

To address data limitations, researchers have proposed various approaches: (1) Data Augmentation Techniques: Researchers have employed techniques like rotation to augment limited fall-related IMU datasets. Theodoridis et al. augmented training data by rotating the original data around random angles within the range of [-10, 10] degrees, enhancing the model's robustness to IMU rotations . Yhdego et al. inverted the sign of sensor data across all three axes to simulate sensor inversion . (2) Transfer Learning Methods: Butt et al.

utilized transfer learning by retraining a VGG-16 convolutional neural network (CNN), initially trained on a large-scale image dataset, to learn features relevant to human actions for fall detection tasks . Yhdego et al. repurposed the AlexNet architecture of deep convolutional neural networks, modifying and retraining the last three layers of the AlexNet architecture.

They employed wavelet-based time-frequency analysis to represent sensor data through scaleograms, which were then fed

Into The Alexnet Framework . (3) Biomechanical

Simulation Tools for Synthetic Data: Mastorakis et al. utilized Opensim simulation software to generate synthetic data. They extracted a person's height and body orientation before a fall from depth camera-captured videos. These pieces of information are utilized to tailor the height of skeletal models and the falling motion, thereby generating vertical acceleration at the center of mass .

Among these approaches, the biomechanical simulation-based method has gained increasing attention from researchers and has seen successful applications in other research domains. Recinos et al. utilized an integrated framework of Opensim and Unity to estimate ground reaction forces in a simulation environment using IMU data collected from real motion experiments .

Jiang et al. used fatigue detection motion experiment data from a limited number of participants to drive model movements in Opensim. Simulated individuals with different physiological attributes executed movements, thereby enhancing the IMU dataset. Their research demonstrated that this model-based data augmentation approach outperformed traditional methods such as rotation, random noise, amplitude warping, time warping, scaling, and proposed methods . Uhlenberg et al. introduced a joint simulation framework consisting of a biomechanical human model and wearable inertial sensor models. A total of 960 inertial sensors were virtually connected to the lower limbs and shoe model of the human body in Opensim. They simulated gait patterns of hemiparetic patients using motion capture data, generating synthetic accelerations and angular velocities based on the inertial sensor model, enabling comprehensive analysis of gait events . The aforementioned studies have substantiated the feasibility of employing simulation techniques to synthesize IMU data. This approach holds potential significance for addressing fall detection challenges in scenarios characterized by

3

substantial data paucity. As per our investigation, existing real IMU datasets in the field of fall detection suffer from issues such as limited data volume and inconsistent experimental paradigms. In contrast, an abundance of video datasets capturing falls is available, offering the potential to replicate more realistic fall scenarios.

Additionally, the feasibility of utilizing biomechanical simulations to generate synthetic IMU data has been established and met with successful application, while its potential remains underutilized in the domain of fall detection. Considering the above factors collectively, the integration of vision-based 3D motion reconstruction with Opensim biomechanical simulation enables cross-modal data generation (from video to IMU). This methodology holds promise within the domain of fall detection, as it effectively addresses the challenge of scarce fall-specific data availability. However, research on this approach remains scarce.

To address the challenges of acquiring costly fall training data, optimizing sensor positions and combinations, this paper proposes a methodology for fall detection that leverages biomechanical simulation software, such as Opensim, to generate synthetic IMU data for training machine learning models. The resulting models exhibit favorable performance on real fall experiment IMU datasets. The contributions of this

Paper Are Summarized As Follows:

1. Cost-effective Data Generation: In response to the high cost of fall-related data, this paper employs Opensim to

Reconstructed From Video. The Dataset Encompasses

various fall and non-fall movements. By manipulating

Methodology Enhances Synthetic Data Through Data

augmentation, yielding a robust synthetic dataset. 2. Customized IMU positions: Through simulation, this study customizes the generation of synthetic IMU data from distinct body regions, investigating the impact of IMU placement on fall detection. The effectiveness of the proposed methodology is validated from both signal and model perspectives.

3. Optimized the Combination of Multiple IMUs: Based on simulation, the combination of synthetic IMU data from multiple body regions has been optimized to provide a comprehensive representation of falling actions.

A. Overview

The system is broadly divided into three main components: driving Opensim skeletal models, generating synthetic IMU data, and training/testing machine learning models (Fig. 1).

OpenSim is a sophisticated biomechanical modeling and simulation software platform that enables the creation and analysis of musculoskeletal models to study human movement and biomechanics . Initially, to drive Opensim skeletal models, multiple cameras capture human fall actions, followed by OpenPose detecting key body points in the videos. The three-dimensional coordinates of these key points are calculated using the triangulation algorithm. Subsequently, human movements are reconstructed via Opensim's marker-based inverse kinematics. Simulated IMUs can be placed freely on the skeletal model, and thus the synthetic IMU raw data can be extracted during model movements. Following this, multiple machine learning models, such as LSTM networks, are trained using the synthetic IMU data and tested on both publicly available IMU datasets and experimentally collected IMU datasets to demonstrated the feasibility of the proposed methods.

B. 2D Pose Estimation

Traditional methods for reconstructing human motion based on optical markers rely on expensive optical motion capture equipment. To achieve markerless motion reconstruction without specialized equipment, this study employs multiple cameras to record human motion videos. Human body pose estimation is performed on the recorded videos from each Fig. 1. Experimental framework of the proposed system.

4

camera, yielding two-dimensional pixel coordinates of human keypoints. OpenPose, a widely used 2D human pose estimation algorithm, accomplishes pose detection by analyzing human keypoints within the images . It employs a multi-stage deep learning architecture, encompassing a convolutional neural network (CNN) stage for body part detection and a pose association stage for linking body parts. In the initial stage, CNN is employed to detect body parts in the images, such as the head, hands, and feet. The network employs convolution, pooling, and fully connected layers to extract features, generating confidence maps and positional information associated with each keypoint. In the subsequent stage, keypoint confidence maps generated in the preceding stage are utilized to associate body parts. This stage involves generating part affinity fields to describe the associational information connecting various body parts. By detecting associations between connected body parts, the algorithm discerns the structure of the human pose. Upon obtaining preliminary keypoint and association information, refinement and filtering of detected poses ensue. Precise estimation of body part positions is achieved by leveraging associational information based on connected body parts, followed by thresholding and non-maximum suppression to eliminate redundant keypoints.

The ultimate output comprises coordinates and confidences of detected human keypoints, alongside associated connection information, which collectively represent the pose of the human body in the image.

Fig. 2. 2D human pose estimation in video. The Body_25B model, employed for human pose detection, encompasses a total of 25 distinct keypoints of the human body.

C. 3D Coordinate Calculation

The marker-based inverse kinematics solving algorithm of Opensim requires the three-dimensional world coordinates of all marker points. This necessitates camera calibration to determine the camera's pose relative to the ground.

Subsequently, the pixel coordinates of key points in camera videos are converted to world coordinates. This paper employed the classical chessboard calibration method . Pose recognition is initially performed on the human motion videos recorded by each camera. Leveraging the multi-camera intrinsic and extrinsic parameters obtained during camera calibration, a triangulation method converts the 2D coordinates of human key points from different camera views captured at the same moment in a video frame into 3D coordinates in the world coordinate system. Finally, a Gaussian filter is applied to smooth the 3D coordinates of all key points, aiming to refine the motion obtained through the inverse kinematics calculation step.

Direct linear transformation is a classic triangulation algorithm, and incorporating weighted confidence of each keypoint provides an effective enhancement to this method

Be The Three-Dimensional

homogeneous coordinates of a keypoint in the world coordinate

Two-Dimensional

homogeneous coordinates in the pixel coordinate system,

Is The Scale Factor, And

c is the confidence of the keypoints. According to camera geometry theory, the following equation can be deduced:

(1)

Multiplying both sides of the equation by , we get:

(2)

Utilizing the confidence of keypoints for weighting:

(3)

For N cameras, there are 2N equations, which can be

(4)

Using the Singular Value Decomposition (SVD) of matrix A to solve for the least squares solution of matrix Q:

D. Inverse Kinematics Calculation

This step aims to drive the model to perform human actions of interest. Specifically, it involves deducing joint movements that can generate a given motion trajectory. This problem is formulated as an optimization task, where joint angles are adjusted to minimize the discrepancy between simulated and measured trajectories, as depicted in (6). Due to the highly nonlinear nature of this optimization problem, numerical optimization methods are employed for its solution. For solving this least squares problem, a general quadratic programming solver is utilized with a convergence criterion set at 0.0001 and a maximum iteration limit of 1000 iterations .

Describing the motion of Opensim skeletal models involves employing joint angles and translational transformations,

Represents The

three-dimensional coordinates of the marker points obtained

Signifies The Three-

dimensional coordinates of simulated marker points at the given

;

denotes the generalized coordinate values obtained from

Represents The Weights Assigned To The

marker points. Therefore, the problem addressed by the inverse kinematics computation is a weighted least squares problem:

(6)

Fig. 3. a. The Body_25B skeletal model of the human body. The pink spheres at the joints in the figure serve as both key points for human pose detection and virtual markers for inverse kinematics calculations.

b. Simulated IMU placement. The orange cuboids rigidly attached to the skeleton represent the simulated IMUs.

E. Forward Kinematic Analysis

Forward kinematic analysis is a method used to determine the motion of specific segments within a biomechanical model given certain joint angles. In this analysis, the focus is on calculating the positions, velocities, and accelerations of various segments of the model during motion, starting from an initial state. Initially, the joint angles and translational transformations at the initial moment are provided, determining the model's initial state. Subsequently, based on the joint angles and the model's geometric structure, the position, angular velocity, and acceleration of the segments of interest are solved using the Newton-Euler equations. Over multiple time steps, the calculations are iterated from the initial pose, ultimately providing the kinematic information of the segments for the given joint angles throughout the entire motion.

Taking a simple two-link system as an example, the angular velocity of the next link is the sum of the angular velocity of the previous link and the angular velocity introduced by the joint's

Be The Position Vector Between The Links. The

angular velocity and acceleration transmission formula for the

(7)

To calculate acceleration, angular velocity, and other relevant information of various body segments during human motion, Opensim employs a more refined forward kinematic analysis algorithm. By customizing the placement of simulated IMUs, it is possible to obtain three-axis acceleration and angular velocity data for that position during the motion of the skeletal model.

F. Fall Detection Model Training

After obtaining the required synthetic IMU data, the generated synthetic dataset is used for model training. Considering the sequential nature of IMU outputs such as angular velocity and acceleration data, models like Recurrent Neural Network (RNN) are well-suited, particularly recurrent neural networks (RNNs) like LSTM networks. Additionally, this study explored other models such as K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Convolutional Neural Networks (CNN), among others. Among all these models, LSTM demonstrated the highest accuracy and robustness.

Lstm, A Variant Of Rnn, Is Designed For Modeling

sequential data and addresses the vanishing gradient problem in traditional RNN. The fundamental structure of LSTM includes the cell state, input gate, forget gate, and output gate .

Firstly, the cell state is at the core of the LSTM network and is responsible for carrying long-term information. It can be thought of as a channel that runs through the entire network. At each time step, LSTM updates the cell state using the input data and the output from the previous time step through the input gate and the forget gate.

Secondly, The input gate decides which information to add to the cell state. It first computes a candidate value based on the input data and the output from the previous time step. Then, it uses logistic regression to determine which parts of the cell state should be updated. The formula for the input gate is as follows:

(8)

Following that, the forget gate decides which information in the cell state to forget. Similar to the input gate, the forget gate employs logistic regression to determine the degree of forgetting for each part. The formula for the forget gate is as

(9)

The cell state is updated by multiplying the candidate value from the input gate and the forgotten parts of the cell state, and then adding new information. The formula for updating the cell

(10)

Lastly, the output gate decides which information will be used for the current time step's output. Like the input gate and forget gate, the output gate employs logistic regression to determine which parts of the output to consider. The formula

(11)

The output state is the final output of the LSTM and is a transformation of the cell state based on the output gate's information and the cell state's transformation. The formula for

(12)

Through these gating mechanisms and state updates, LSTM

Related Journal Articles & DOI Links

Selected peer-reviewed publications relevant to 12 Lead ECG Acquisition. Click the DOI to access the full paper (may require institutional access).

Why Choose Us?

Bangalore guidance for robotics, Spectre and autonomous systems projects.