Neonatal Vital Signs Monitor

Continuous Non-Contact Vital Sign Monitoring Of

neonates in intensive care units using RGB-D cameras Silas Ruhrberg Est´evez1,*, Alex Grafton1, Lynn Thomson2, Joana Warnecke1, Kathryn

Abstract

Neonates in intensive care require continuous monitoring. Current measurement devices are limited for long-term use due to the fragility of newborn skin and the interference of wires with medical care and parental interactions. Camera-based vital sign monitoring has the potential to address these limitations and has become of considerable interest in recent years due to the absence of physical contact between the recording equipment and the neonates, as well as the introduction of low-cost devices. We present a novel system to capture vital signs while offering clinical insights beyond current technologies using a single RGB-D camera. Heart rate and oxygen saturation were measured using colour and infrared signals with mean average errors (MAE) of 7.69 bpm and 3.37%, respectively. Using the depth signals, an MAE of 4.83 breaths per minute was achieved for respiratory rate. Tidal volume measurements were obtained with a MAE of 0.61 mL. Flow-volume loops can also be calculated from camera data, which have applications in respiratory disease diagnosis. Our system demonstrates promising capabilities for neonatal monitoring, augmenting current clinical recording techniques to potentially improve outcomes for neonates.

Introduction

Each year there are approximately 135 million live births globally . It is estimated that around 3 −10% of neonates, depending on their gestational age, receive some level of care on a neonatal intensive care unit

Arxiv:2412.06012V1 [Eess.Iv] 8 Dec 2024

(NICU) , . Common reasons for admission to a NICU include respiratory distress syndrome, bradycar- dia, and infection . Preterm birth, defined as birth occurring before 37 weeks of gestation, is a significant risk factor for NICU admission. It occurs in up to 10% of all births and is responsible for approximately 1 million neonatal deaths worldwide each year . Monitoring on the NICU can be achieved both using biosensors and examinations by healthcare professionals. For detailed evaluations, quantitative assessments of vital signs are often performed .

Vital signs reflect changes in physiological functions and standard clinical care in NICUs involves monitoring of heart rate, respiratory rate, body temperature, blood glucose levels, and oxygen saturation . Notably, bradycardia, characterised by a significantly reduced heart rate, is predictive of sepsis , while prolonged hyperventilation has been associated with adverse clinical outcomes , . Abnormal respiratory rates are also linked to increased risks of cardiac arrest, metabolic acidosis, and blood gas imbalances . Normal oxygen saturation target values for newborns range from 90% to 95% . Values higher than this range are harmful, particularly in preterm infants, as they can lead to increased generation of oxygen free radicals and ischaemia–reperfusion injury . Lower saturation levels indicate a need for oxygen supplementation to ensure adequate oxygen delivery to tissues. Continuous monitoring of vital signs allows for early detection of complications and timely intervention to improve neonatal outcomes in the NICU setting .

Respiratory function can be characterised by various metrics, ranging from simple respiratory rate to sophis- ticated assessments using spirometry, which measures airflow dynamics during breathing cycles to quantify volume and flow changes . In clinical settings, rapid assessment of neonatal breathing can be achieved us- ing the Silverman-Anderson and Downes’ score . However, both methods require clinicians to observe the patients and are thus subject to individual biases. During intensive care unit stays, thoracic impedance is often used to measure respiratory rate. This approach involves passing a small electrical current through surface electrodes placed on the chest and measuring the resulting voltage changes, which correlate with chest expansion during breathing . However, non-respiratory movements can interfere with the measurement by altering the voltage, leading to contamination of the respiratory rate data at an amplitude much larger than the respiratory signal of interest. In adults, a common method for monitoring respiratory function involves constructing flow-volume curves through maximum effort spirometry , . However, this is not a feasible monitoring option for neonates who cannot perform maximum effort breaths on command. Instead, tidal volumes from normal breaths serve as an approximate metric .

Neonatal skin is exceptionally delicate, and prolonged electrode attachment can lead to iatrogenic skin in-

2

juries . Furthermore, the presence of monitoring equipment may impede routine clinical care and hinder parent-infant interactions . Balancing accurate monitoring and potential risks to neonatal skin integrity and parent-child bonding remains a challenge in neonatal intensive care settings. Various sensing devices ne- cessitate separate attachments, each presenting unique limitations. Heart rate is commonly measured using either an ECG or a pulse oximeter. Although ECGs offer higher accuracy, they require attaching electrodes to the skin for extended periods. These electrodes can also be used for thoracic impedance measurements.

Oxygen saturation is estimated from a clip-on pulse oximeter that records the absorbance of light of different wavelengths which depends on the amount of oxygenated haemoglobin present in arterial blood. In the past 15 years, various non-contact approaches have been explored for measuring vital signs in NICUs.

RGB cameras have been utilised for heart rate monitoring , , , and oxygen saturation . RGB-D cameras have been trialled for respiratory rate and volume measurements , . RGB-D cameras capture video streams containing colour images (red, green, and blue), infrared images, and depth images, the latter of which measures the distance of objects in the scene from the camera for each pixel using time- of-flight technology. The increased usage of cameras is facilitated by recent improvements in image quality for portable low-cost solutions. Despite promising results from non-contact methods, comprehensive frame- works for monitoring all vital signs simultaneously remain elusive in clinical settings. Many studies have been limited by small sample sizes, the availability of accurate ground truth signals, or were conducted under controlled conditions that may not fully represent unaltered and real-world NICU environments. Further research is needed to address these limitations and develop non-contact monitoring systems capable of accu- rately and reliably tracking all vital signs in neonatal intensive care settings.

We conducted a clinical study at the Rosie Hospital in Cambridge to evaluate the feasibility of using contin- uous non-contact monitoring for infants in a real-world NICU setting. The primary objective of this study is to determine whether RGB-D cameras can accurately measure vital signs, such as heart rate, respiratory rate, and oxygen saturation, which are typically recorded using sensors attached to the skin of the neonates.

Importantly, our method achieves this without physical contact or modifications to the clinical environment. Additionally, we aim to measure other critical parameters, including tidal volume and flow-volume dynamics, which cannot be recorded with standard NICU equipment. Furthermore, we will provide a publicly available dataset of the pose estimations with the resulting depth and colour signals along with the measured ground truth data and the analysis code to facilitate the development of new algorithms in this field.

To achieve these objectives, we utilised a single RGB-D camera (Microsoft Azure Kinect) mounted on the

3

incubator in a manner that did not disrupt clinical procedures (see Fig. 1). Ground truth signals from standard NICU equipment were employed to validate the vital signs derived from the camera data. For respiratory monitoring, infants receiving mechanical ventilation were selected as ground truth due to the recognised inaccuracies associated with thoracic impedance measurements .

(D) Depth Image

Figure 1: Recording setup used in the clinical study and resulting images. (a) Mounting of the Azure Kinect camera to the incubator using a flexible arm. A doll is used for visualisation purposes. (b) RGB image recorded by the Azure Kinect camera. (c) Infrared image of the same neonate. (d) Depth image derived from the infrared image using time-of-flight. In the RGB and infrared images, the facial region of the baby and other identifiable objects are obscured by a black square. The pixel values of the infrared and depth images have been scaled for visualisation purposes, where darker regions correspond to distances closer to the camera. Black pixels in both the infrared and depth images indicate areas where insufficient infrared reflection occurred. Additionally, a phototherapy mask is applied to the left foot to shield the pulse oximeter from infrared radiation.

Results

included in the vital sign monitoring study. The study population consisted of 11 males and 3 females, with 8 infants from white backgrounds and 6 from non-white backgrounds. Respiratory ground truth measure- ments from ventilators were available for 3 neonates. Each neonate was recorded for approximately 1 h with

4

the Azure Kinect RBG-D camera while simultaneously measuring vital signs using the standard equipment available on the NICU. A summary of patient demographics is provided in Table 1. The collected valid vital sign data is depicted in Figure 2, demonstrating consistency with anticipated neona- tal physiological parameters. All video sequences were also manually examined to exclude segments where the neonates were covered during clinical interventions. Regions of interest were then identified using a top- down pose estimation to locate the position of hips and shoulders . For colour signals, non-skin pixels were excluded using a skin mask . Representative examples showing the performance of the methods are illustrated in Figure 3. The skin segmentation was found to work in ambient light conditions only. However, the heart rate and oxygen saturation algorithms assume ambient lighting and were found to produce inaccu- rate readings in very low light and phototherapy conditions when no skin segmentation was used. Therefore, skin segmentation may serve as a safety feature, alerting to conditions where measurement reliability may be compromised. Notably, respiratory monitoring remains robust under low light, facilitated by infrared illumi- nation from the camera for time-of-flight recordings, with pose detection maintaining acceptable performance even in challenging lighting environments.

(D) Tidal Volume

Figure 2: Characterisation of ground truth data set. (a) Heart rate from ECG, (b) oxygen saturation from the pulse oximeter, (c) respiratory rate from ventilator, and (d) tidal volume from ventilator.

Respiratory Rate And Volume

To obtain accurate ground truth data for both respiratory rate and tidal volume, mechanically ventilated babies were included in our study. The collected RGB-D videos were processed to extract frequency and volume measurements. It is shown that estimating frequency using a Fourier transform of the signal was more accurate than counting the number of peaks (see Table 2). On the dataset of 3 ventilated babies, this method achieves an MAE of 4.83 breaths per minute. A representative example of tidal volume and respiratory rate

(C)

Figure 3: Region of interest identification in images. (a) Representative examples of hip and shoulder identification using the pose estimation algorithms for 6 samples. (b) Resulting ROIs using hip and shoulder key-points. (c) Subsequent skin segmentation for colour signals. Blue light phototherapy (2) and very low light conditions (4) result in no skin being identified by the segmentation.

monitoring is shown in Figure 4, along with a detailed statistical analysis of the performance of the Fourier transform based respiratory rate estimation in comparison to ventilator ground truth. Tidal volumes were derived from the depth signal by analysing peak-to-valley differences. The MAE com- pared to the ventilator’s best estimate was found to be 0.85 mL (17.46%). Considering potential inaccuracies in the ventilator measurements to define upper and lower bounds on the ground truth based on inspiratory and expiratory volumes, the MAE decreased to 0.61 mL (12.81%) (see Table 3). The tidal volume measure- ment accuracy is analysed in Figure 5.

Due to limited availability of ventilated babies in the NICU, our algorithms were also tested on neonates not receiving respiratory support. For these babies, established ground truth values were not available. Instead,

(F) Respiratory Rate Cp

Figure 4: Respiratory rate analysis. (a) Representative 1 h recording of respiratory rate. (b) Represen- tative 1 h recording of tidal volume, illustrating the ventilator ground truth for tidal volumes with the best estimate and associated upper and lower bounds derived from inspiratory and expiratory volumes. Tempo- rary deviations from the ground truth, likely due to infant hand movements, are observed. (c) Bland-Altman plot showing respiratory rate errors. (d) Correlation between camera-derived and ventilator-derived respira- tory rates. For clarity, only one sample per minute is displayed in the correlation and Bland-Altman plots.

(e) Histogram of error distribution for the camera measurements compared to the ground truth. (f) Coverage Probability (CP) plot showing the proportion of camera-derived values lying within specific absolute error thresholds relative to the ground truth.

the 9 neonates recorded for approximately 1 h each were divided into two groups based on their clinical outcomes after their NICU stay. The poor outcome group comprised babies who either required supplemen- tary oxygen at home or unfortunately did not survive until 36 weeks. Conversely, all babies in the normal breathing group survived and did not require supplementary oxygen after 36 weeks. Babies who exhibited normal breathing at 36 weeks were found to have significantly higher tidal volumes (p = 0.016) than their counterparts using a Mann-Whitney U-test (see Figure 5). However, the tidal volume per kg of body-weight standardisation for neonates, did not differ significantly between groups (p = 0.905). This suggests that the algorithms are capable of identifying the higher tidal volumes in the normal group.

(F) Tidal Volume Cp

Figure 5: Tidal volume analysis. (a) The normal breathing group has significantly higher (p = 0.016) raw tidal volumes than the poor outcome group. (b) Corrected tidal volumes are not significantly different (p = 0.905). (c) Bland-Altman plot showing tidal volume errors. (d) Correlation between camera-derived and ventilator-derived tidal volumes. For clarity, only one sample per minute is displayed in the correlation and Bland-Altman plots. (e) Histogram of error distribution for the camera measurements compared to the ground truth. (f) Coverage Probability (CP) plot showing the proportion of camera-derived values lying within specific absolute error thresholds relative to the ground truth.

Respiratory Dynamics

Flow-volume loops were successfully constructed both from camera and ventilator data. A representative example is shown in Figure 6. The loops from the two modalities show similar characteristics. Small differ- ences in shape are likely attributed to air leakage in the ventilator-constructed loop. As seen in Figure 4, the magnitude of this leakage is approximately 0.5 mL. During examination of single breaths, it was noted that the camera system was sensitive enough to capture subtle variations in breathing such as interrupted breaths.

Interruptions occur when a neonate attempts to breathe during a ventilator-induced breath, which is then terminated prematurely. Examples of a representative normal breath and a prematurely terminated breath followed by a second autonomous breath are shown in Supplementary Information Figure 1 & 2. The flow- volume construction was also trialled on non-ventilated neonates, revealing more variable loops characterised by a less smooth pattern (see Figure 6). In comparing the loops obtained in non-ventilated neonates of the normal breathing group with those of the poor outcome group, the former exhibited larger loops, consistent with the observed differences in tidal volumes. The presented loop from the normal breathing group also has a shape resembling convex expiratory loops documented in the literature. This loop form has been observed in about 1 in 4 neonates that are healthy or suffer from chronic lung disease .

Using the spatial information gained from the imaging system, it was also possible to examine regional vari- ations in breathing. For this the tidal volumes were calculated using the spatial average of the defined region only. The sensitivity of this approach was demonstrated by showing that the measured regional tidal volumes are higher in uncovered regions than regions partially occluded by medical equipment (see Supplementary Information Figure 3).

Heart Rate

Five different methods for estimating heart rate from generated colour signals using the CHROM and POS algorithms were trialled (see Table 4). The signals generated using each algorithm were analysed for fre- quency using both peak counting and Fourier analysis of the frequency spectrum. Fourier analysis was found to be more accurate than peak counting. Fourier-based methods achieved good accuracy for both the POS and CHROM algorithms, but the best performance was achieved by averaging the heart rate estimations of the two methods. A representative 1 h recording and estimates from all methods are presented in Figure 7.

Across our dataset of 11 neonates the average of the CHROM and POS algorithms with Fourier analysis achieved a MAE of 7.69 bpm. A detailed statistical analysis of the heart rate estimates is provided in Figure 7.

(D) Camera Abnormal

Figure 6: Flow-volume loops. (a) Flow-volume loops for a representative neonate generated using the RGB-D camera, illustrating the flow and volume dynamics during individual breaths. (b) Corresponding flow-volume loops generated from ventilator-derived data for the same neonate. Each loop represents a single breath, showcasing the close correspondence between camera-based and ventilator-based measurements. (c) Flow-volume loops from a non-ventilated neonate in the normal breathing group, demonstrating typical respiratory patterns. (d) Flow-volume loops from a non-ventilated neonate in the poor breathing group are visible smaller.

(F) Heart Rate Cp

Figure 7: Heart rate analysis. (a) Representative 1 h recording of heart rate monitoring using CHROM and POS signals with peak counting. (b) Representative 1 h recording of heart rate monitoring using Fourier analysis of the CHROM and POS signals. (c) Bland-Altman plot showing heart rate errors. (d) Correlation between camera-derived and pulse oximeter-derived tidal volumes. For clarity, only one sample per minute is displayed in the correlation and Bland-Altman plots. (e) Histogram of error distribution for the camera measurements compared to the ground truth. (f) Coverage Probability (CP) plot showing the proportion of camera-derived values lying within specific absolute error thresholds relative to the ground truth.

Oxygen Saturation

For our study we evaluated four different non-contact algorithms that have been reported in the literature. Among these methods, the infrared-based algorithm demonstrated the highest accuracy, as detailed in Ta- ble 5. A representative 1h recording comparing all methods is shown in Figure 8. Across our dataset of 7 neonates, a MAE of 3.37% and a MSE of 15.89% were achieved. Notably, the YCgCr colour method also achieved a MAE below 4%, and unlike the infrared-based method, it can be achieved using regular RGB cameras. The RGB method performed slightly worse, likely due to the challenging lighting conditions. The calibration-free method was found to introduce significant errors into the measurements. A detailed statisti- cal analysis of the infrared-based method is presented in Figure 8.

(F) Spo2 Cp

Figure 8: Oxygen saturation. (a) Representative 1 h recording of oxygen saturation monitoring using the infrared and RGB algorithms. (b) Representative 1 h recording of oxygen saturation monitoring using the YCgCr and calibration free algorithms. (c) Bland-Altman plot showing oxygen saturation errors. (d) Correlation between camera-derived and pulse oximeter-derived tidal volumes. For clarity, only one sample per minute is displayed in the correlation and Bland-Altman plots. (e) Histogram of error distribution for the camera measurements compared to the ground truth. (f) Coverage Probability (CP) plot showing the proportion of camera-derived values lying within specific absolute error thresholds relative to the ground truth.

Discussion

This study introduces the first system for continuous non-contact estimation of all vital signs considered standard of care. Importantly, unlike Villaroel et al. , our method does not require any permanent mod- ification to the incubator. Monitoring was undertaken using a single low-cost RGB-D camera placed above the incubators. The camera is able to record respiratory rate, heart rate and oxygen saturation in line with clinical standards. For the recorded tidal volumes and flow dynamics there are no clinical standards to the best of our knowledge but our results are in good agreement with ground truth. All algorithms were validated in a clinical study in a NICU.

Over the past decade, there have been multiple attempts to develop a non-contact RGB-D camera-based res- piratory monitoring system for neonates. Cenci et al. used RGB-D cameras to monitor respiratory rates of NICU patients and compared these values against reference standards. Their study involved 3 infants, each of which was recorded for a total of 150 s. Respiratory rate was extracted from manually defined ROIs by calculating the time differences between adjacent peaks. Although a detailed performance analysis was not provided, the study reports a high correlation between the respiratory rates obtained from the camera and those derived from a cardiomonitor .

Kyrollos et al. conducted a more sophisticated study involving the application of deep learning techniques for respiratory monitoring using RGB-D camera data. Their approach entailed training a deep learning net- work to accurately identify facial and chest regions within the images. Subsequently, respiratory rate was estimated based on signal changes in both RGB and depth data obtained from these identified regions. The algorithm was only validated on a single patient, for which data was recorded for 20 min. Although a compre- hensive performance analysis was not provided, the study reports that the measurement error for respiratory rate estimation was below 3.5 breaths per minute for approximately 69% and 67% of the recording duration, when derived from RGB and depth image data respectively .

Villarroel et al. conducted an extensive study involving 30 subjects with recordings spanning over 400 h to monitor respiratory rate. They trained a deep neural network to identify whether babies are present in the image and segment visible skin regions. However, due to the inaccuracies of thoracic impedance measure- ments only 44% of the recording was included in the analysis. The authors performed a more detailed analysis of their proposed algorithms which achieved a MAE of 3.5 breaths/min for 82% of the recording time they considered their ground truth and camera data valid. Khanam et. al also used deep learning techniques

13

to estimate respiratory rate from 10 s recordings of neonates using two RGB cameras. After exclusion of invalid segments, they achieved a MAE of 2.13 breaths per minute. Defining accurate requirements for respiratory rate and tidal volume measurements in neonatal monitoring presents challenges due to the absence of consensus on gold standard measurement devices. Currently, there is no universally agreed upon standard for assessing tidal volume measurements. However, a recent report by UNICEF indicated that achieving a MAE of 5 breaths per minute would be desirable for low cost infant mon- itoring . The true performance of previously developed monitoring systems remains unclear due to several factors. Many studies have provided only superficial analyses, lacking comprehensive validation against es- tablished benchmarks , . Many studies also only recorded under non-natural conditions

,

with modifications to the clinical environment or for very short time segments . Additionally, some studies only included segments of data where their algorithms perform well in the analysis , . The accuracy of previous studies in monitoring respiratory parameters is also severely limited by the accuracy of gold standard signals from thoracic impedance measurements . A more accurate respiratory signal can be obtained from mechanically ventilated neonates, where breath timings are precisely controlled by the ven- tilator. Usually, when the infants are mechanically ventilated, the impedance recordings are not performed.

For one neonate in our study, both impedance and ventilator data were available, suggesting a MAE of 18.5 breaths per minute. MAEs between thoracic impedance and breathing rates counted manually by clinical staff of over 10 breaths per minute have been reported in the literature . Consultant neonatologists in the Rosie hospital in Cambridge have also expressed reservations about using impedance-derived respiratory measurements to guide clinical decisions due to concerns about accuracy.

We demonstrate that reliable respiratory rates estimates can be obtained from depth signals. On the limited data available, the RGB-D camera outperforms reported accuracies of currently employed thoracic impedance systems. These results align well with previously reported accuracies in the field. Our system has a slightly lower accuracy over the entire dataset than some of the other studies. This discrepancy is anticipated as we did not exclude large segments of data where the system exhibited lower accuracy, opting instead to provide a comprehensive assessment of performance. Unlike thoracic impedance systems, our approach is also capable of measuring tidal volumes which can highlight breathing abnormalities.

The studies discussed thus far have primarily focused on respiratory rate as a key parameter for assessing respiratory function. However, other parameters such as tidal volumes and flow dynamics are important as

14

well. Tidal volumes in ventilated babies over 4.5 mL, for instance, have been shown to be a predictor of successful extubation . A study by Rehouma et al. measured both respiratory rate and tidal volumes in a paediatric ICU setting . However, they only imaged manikins with test lungs connected to ventilators rather than actual infants. In a subsequent study by the same group, actual ventilated infants were included, but only one infant was imaged for a total duration of 5 min . The baby was 4 months postpartum and had a tidal volume of around 40 mL, significantly larger than the neonates in our study, which they could measure with a mean relative error of 9.17%. In our dataset, which included three mechanically ventilated premature infants, we observed a slightly higher error rate of 12.81%. This increase is anticipated, as larger tidal volumes facilitate the separation of breathing movements from measurement noise. Our recording also encompassed periods of neonatal movement, which further impacted measurement accuracy but is expected in real-world applications. Furthermore, we demonstrate the non-invasive construction of flow-volume loops that visually closely resemble those recorded by ventilators, a similarity confirmed by consultant neonatol- ogists. In non-ventilated babies, significantly higher tidal volumes are recorded in neonates with normal breathing compared to those with poor outcomes. The resulting flow-volume loops in the poor outcome group are also smaller.

Additionally, we anticipate that the camera could provide valuable insights into respiratory dynamics by capturing spatial variations in breathing efforts and enabling the construction of flow-volume loops. These insights may offer clinicians valuable information to better understand and manage respiratory dynamics in neonates. Tidal flow-volume loops can reveal differences between healthy infants and those with chronic lung diseases . Asymmetry in chest wall movements is an early predictor of pneumothorax in neonates, as demonstrated by Wasiman et al. . Using a sensor attached to the chest, the authors detected changes in chest movement 31 min before a clinical diagnosis of pneumothorax could be confirmed. Another condition that can present with tidal volume asymmetry is congenital scoliosis, as reported by Redding et al. . In the future, there is potential to use RGB-D cameras to verify correct endotracheal tube placement in neonates.

Endotracheal tube misplacement can have serious consequences including atelectasis (lung collapse) of the non-ventilated lung, pneumothorax and even death . Currently, x-rays are used if tube misplacement is suspected . By monitoring changes in spatial airflow distribution, particularly asymmetries that may indicate inadequate ventilation of one lung, RGB-D camera-based systems could offer a real-time method to assess endotracheal tube position without the need for invasive or radiation-based procedures.

An application of RGB cameras for heart rate monitoring in a NICU setting was done by Chen et al. . Heart rate was estimated using Fourier analysis of the green channel. On a clinical study of 5 babies they

15

achieved a MAE of 7.4 bpm. Paul et al. also used RGB cameras with manual ROI selection. They reported errors of less than 3 bpm in infants with baseline heart rates of about 120 bpm during high-quality recordings, but this level of accuracy was achievable for less than 40% of the recording time. A study by Svoboda et al. has achieved high accuracy in heart rate measurement compared to a pulse oximeter, but this is not as accurate as an ECG . The previously discussed study conducted by Villaroel et al. also examined heart rate, achieving a MAE of 2.3 bpm over 76% of valid data. Unlike the thoracic impedance measurement, ECG provides robust measurements, of which the authors deemed 91.2% to be valid .

Heart rate estimation from colour signals is shown to be feasible for continuous monitoring using our RGB-D camera. The American National Standards Institute specifies an acceptable range of error for heart rate measurements as ±10% or ±5 bpm, whichever is greater . However, higher accuracy has been achieved in the literature using deep learning approaches that are less sensitive to variations in lighting conditions and more expensive cameras. This difference is likely attributed to the high noise inherent in colour signals, in contrast to the more robust depth measurements obtained using built-in infrared lighting. On a similar dataset, Grafton et al.

have achieved an MAE of 3.83 bpm with deep learning methods. A large scale study conducted by Huang et al. used deep learning techniques for estimation of heart rate in neonates, achieving a MAE of 3.97 bpm. The authors from the latter study also directly compared their method to es- tablished heart rate estimates and concluded that it outperformed the CHROM and POS algorithms slightly on their dataset. However, their recording session only lasted a maximum of 1 min, leaving uncertainty about the system’s accuracy for long-term monitoring applications.

Various established techniques for measuring oxygen saturation using cameras have been documented in the literature , , , . In laboratory settings they have been reported to give good accuracy but there is limited evidence on their performance in challenging clinical environments. One of the first applications of non-contact monitoring on neonates was performed by Ye et al. . They managed to record oxygen saturation from 22 infants on the NICU with a MAE of less than 4%. Villaroel et al. previously demon- strated the capture of a bradycardia episode resulting in desaturation using an RGB camera. They, however, did not provide any statistical analysis of their method in the study.

For our study, we evaluated four different non-contact algorithms that have been reported in the literature. The superior performance of the infrared based method is not surprising given that traditional pulse oximeters have the same principle of operation. The infrared result is compliant with UK regulations that mandate the MSE to be below 16% . The RGB method performed slightly worse, likely due to the challenging lighting

16

conditions. The calibration-free method was found to introduce significant errors into the measurements. This is likely due to the inadequacy of the shallow and deep layer model for neonates or the inaccurate mod- elling of our camera under the assumption of absorbance at a single wavelength. The model is also based on the absorption characteristics of adult haemoglobin Hb A. Neonates, however, mainly have Hb F, which was produced during the pregnancy and remains the dominant form until 6 −12 months post delivery . De- spite structural disparities, studies have indicated negligible differences in oxygen absorption values between fetal and adult haemoglobin molecules . A limitation of our study is that we only compared the SpO2 to devices used in clinical practice. For full validation, establishing ground truth using blood gas samples would be necessary. However, this procedure is highly invasive, only provides intermitted measurements, and falls beyond the scope of our study.

In contrast to previous studies, performance of the developed algorithms was evaluated on the entire recorded dataset where the baby is visible in the image rgardless of the baby’s position and valid ground truth data was available. Additionally, we have created a fully anonymised dataset of the pre-processed data that can be made publicly available along with the code used in the analysis. This may facilitate the development of novel non-contact monitoring algorithms, addressing the current limitation of publicly available neonatal data for testing such methods.

Future efforts will focus on validating the algorithms on a larger study population. Specifically, inclusion of neonates with respiratory pathologies will be crucial to verify the diagnostic potential of the system. By providing additional clinical insights, currently not available to clinicians, the system could assist in early recognition of pathological changes, with the potential to improve patient outcomes. The non-contact vital sign monitoring system may integrate into a broader camera system for neonatal activity and physiology monitoring currently under development.

Clinical Study

Foundation Trust Rosie Hospital, involving the monitoring of neonates using RGB-D cameras to assess phys- ical activity and vital functions. The study was reviewed by the Research Ethics Committee (North-West,

17

Preston) with reference number 21/NW/0194 and IRAS ID 285615. Ground truth measurements were con- currently recorded using gold standard devices in the ICU including a pulse oximeter, ventilator, ECG, and thoracic impedance. All collected data underwent anonymisation following the extraction of relevant clinical information from hospital records to protect patient privacy. Informed written consent was obtained from all parents for the recording and subsequent analysis of the data. Participants had the right to withdraw from the study at any time and request deletion of their collected data.

The study was designed to minimise interference with clinical interventions and parental interactions with the babies. The recording setup was non-invasive and did not require any permanent changes to the incubator, ensuring it could be quickly removed during emergencies. Throughout the study, the clothing and positioning of the infants remained unchanged. For respiratory monitoring we specifically targeted babies on mechan- ical ventilation. During data collection, ventilated babies were recorded for up to 1 h, while non-ventilated babies were monitored for up to 72 h for physical activity monitoring. For this analysis, only infants who were nursed naked were included as visible skin on the chest area is required for the developed algorithms.

Neonates receiving mechanical ventilation were only included in the study if it was clinically appropriate to record them in a supine position.

Rgb-D Camera

The RGB-D camera used in this study was the Azure Kinect DK (Microsoft Corporation, USA), equipped with a 1MP Time-of-flight (ToF) depth camera and a 12MP CMOS RGB camera . Colour images with a resolution of 1280 × 720 were captured with 4 : 2 : 2 chroma sub-sampling and stored in the MJPEG format.

The depth image acquisition relied on the Amplitude Modulated Continuous Wave (AMCW) time-of-flight principle which involves illuminating the scene with near-infrared light emitted by laser diodes . The depth image is calculated by measuring the time difference between the start of illumination and the return of the infrared light from specific objects. Additionally, an infrared image is obtained by measuring the intensity of the reflected light. The infrared and depth images were upsampled from 640 × 576 to match the dimensions of the RGB image. The Azure Kinect offers both narrow and wide field of view options, with the narrow field of view mode selected for this study.

Related Journal Articles & DOI Links

Selected peer-reviewed publications relevant to 12 Lead ECG Acquisition. Click the DOI to access the full paper (may require institutional access).

Why Choose Us?

Bangalore guidance for robotics, Spectre and autonomous systems projects.