Camera Based Rppg Heart Rate

Abstract

This report introduces VitalLens 2.0, a new deep learning model for estimating physiological signals from face video. This new model demonstrates a significant leap in accuracy for remote photoplethysmography (rPPG), enabling the robust estimation of not only heart rate (HR) and respiratory rate (RR) but also Heart Rate Variability (HRV) metrics. This advance is achieved through a combination of a new model architecture and a substantial increase in the size and diversity of our training data, now totaling 1,413 unique individuals. We evaluate VitalLens 2.0 on a new, combined test set of 422 unique individuals from four public and private datasets. When averaging results by individual, VitalLens 2.0 achieves a Mean Absolute Error (MAE) of 1.57 bpm for HR, 1.08 bpm for RR, 10.18 ms for HRV-SDNN, and 16.45 ms for HRV-RMSSD. These results represent a new state-of-the-art, significantly outperforming previous methods. This model is now available to developers via the VitalLens API at https://rouast.com/api.

Introduction

Remote Photoplethysmography (rPPG) harnesses signals from standard video to estimate physio- logical information, offering immense potential for non-invasive health monitoring . Our initial release, VitalLens 1.0, provided real-time estimation of heart rate (HR) and respiratory rate (RR) .

This report introduces VitalLens 2.0, the next generation of our rPPG model. The primary goal is to move beyond simple rate estimation to achieve high-fidelity physiological waveform reconstruction. This moves the challenge from simple rate estimation (i.e., finding a dominant frequency) to high- fidelity waveform reconstruction, which requires the precise, sub-second temporal accuracy of Inter-Beat Intervals (IBIs) to be robust.

This leap in performance was achieved through these two key developments: 1. An expanded and meticulously curated training dataset. We combined our in-house data with the Vital Videos public dataset, then manually curated all samples to ensure high-quality video and labels. The final set totals 1,413 unique individuals.

2. A new model architecture and training methodology, designed specifically to capture the subtle inter-beat variations necessary for accurate HRV analysis. To validate these improvements, we establish a new, large-scale combined test set. This set is comprised of 1,081 video samples from 422 unique individuals, combining our in-house test set (PROSIT) with several publicly available datasets (Vital Videos [9, 10], UBFC-Phys , UBFC- rPPG ). All video chunks are processed to be 20-60 seconds in duration, making them suitable for time-domain HRV analysis. For our model development purposes we created our own strict, participant-disjoint training, validation, and test sets to ensure a robust, unbiased evaluation of generalization.

Arxiv:2510.27028V1 [Cs.Cv] 30 Oct 2025

• VitalLens 2.0. We introduce a new rPPG model capable of accurately estimating HRV metrics from face video, in addition to HR and RR. • Comprehensive evaluation. We benchmark VitalLens 2.0 on a large, diverse test set of 422 individuals, demonstrating its superior accuracy.

• State-of-the-art performance. We show that VitalLens 2.0 significantly outperforms handcrafted algorithms (e.g., POS ), other learning-based methods (e.g., MTTS-CAN ), and our previous VitalLens 1.0 architecture.

This new model is the engine behind the VitalLens API, enabling developers to integrate robust, real- time HRV analysis into their applications. For more information, visit https://rouast.com/api.

Architecture

The VitalLens 2.0 model is an end-to-end deep convolutional neural network, building upon an EfficientNet-based backbone . The architecture incorporates novel temporal-attentive mechanisms optimized for extracting high-fidelity physiological waveforms. This design is specifically focused on minimizing signal noise and preserving the precise temporal location of systolic peaks, which is the critical prerequisite for reliable IBI extraction and HRV analysis.

The model takes a sequence of video frames of a person’s face as input and outputs two continuous time-series signals: the pulse (PPG) waveform and the respiration (RESP) waveform. From these waveforms, downstream metrics such as Heart Rate (HR), Respiratory Rate (RR), and HRV metrics (e.g., SDNN, RMSSD, LF/HF) are derived using industry-standard signal processing techniques.

Ppg

Figure 1: Visual comparison of estimated waveforms from a sample handheld video segment.1 Top: VitalLens 2.0 PPG vs. Ground Truth. Middle: VitalLens 2.0 Respiration vs. Ground Truth. Bottom: VitalLens 1.0 and POS PPG vs. Ground Truth. VitalLens 2.0 achieves higher fidelity, accurately reconstructing the precise timing of the systolic peaks.

Figure 1 provides a visual comparison of the high-fidelity waveforms generated by VitalLens 2.0 against the ground truth and other models for a sample handheld video segment. 1The source video and ground truth vitals are publicly available as sample_video_1 at: https://github.

com/Rouast-Labs/vitallens-python/tree/main/examples

Datasets

The development of VitalLens 2.0 utilized a large-scale, multi-source dataset, combining our in-house PROSIT dataset with several publicly available datasets, including the Vital Videos (VV) collection [9, 10], UBFC-Phys , and UBFC-rPPG .

Training Dataset. The training dataset combines the training splits of PROSIT and the Vital Videos datasets we have access to. This results in a total training set of 1,413 unique individuals, a significant increase in size and diversity over the data used for VitalLens 1.0 . Beyond demographics, this diversity includes a wide variety of real-world filming locations, diverse lighting conditions, cameras, and varied backgrounds, and unscripted participant behavior, including significant camera motion from handheld devices. The composition of this training set is detailed in Table 1.

(C)

Figure 2: Participant demographics in the training dataset. (a) Age, (b) Gender, (c) Skin type. Figures 2 and 3 detail the demographic and physiological composition of this new training set. Data is reported per-individual. It is well-balanced in gender and includes a comprehensive representation across all six Fitzpatrick skin types. The vitals distributions cover a wide range of physiological states, including a broad spectrum of heart rate variability.

Sdnn (Ms)

Figure 3: Distributions of per-individual average vitals in the combined training dataset. (a) Heart Rate, (b) Respiratory Rate, (c) HRV-SDNN.

3

Test Dataset. To validate VitalLens 2.0, we constructed a new, combined test set. All individuals in this test set are disjoint from the training and validation sets. Chunks were re-processed to have durations between 20 and 60 seconds, enabling the calculation of time- and frequency-domain HRV metrics. The composition of this test set is detailed in Table 2. While all test samples have a ground truth signal for PPG, some (VV in part and both UBFC-rPPG and UBFC-Phys) do not have a ground truth signal for RESP. In addition, samples with known label-quality or synchronization issues (e.g., in parts of UBFC-Phys) were excluded to ensure a fair and reliable benchmark.

Table 2: VitalLens 2.0 Combined Test Set Composition

Methodology

Model Training and Validation. We follow a strict participant-disjoint methodology for training, validation, and testing. The 1,413 individuals in the training set were used to optimize model parameters. A separate validation set, also participant-disjoint from the training set, was used to monitor for overfitting and to perform hyperparameter tuning and model selection. The final VitalLens 2.0 model chosen for evaluation is the one that demonstrated the best performance on this validation set.

Models Compared. We benchmark VitalLens 2.0 against a comprehensive set of methods. These include traditional handcrafted algorithms (G , CHROM , and POS ) and several prominent learning-based methods (DeepPhys , MTTS-CAN , and EfficientPhys ). To ensure a fair and direct comparison, all learning-based baselines were re-trained from scratch on the exact same 1,413-participant training dataset used for VitalLens 2.0.

We Also Include Two Internal Baselines:

• VitalLens 1.0*: The original model as presented in , trained on the smaller, original dataset. • VitalLens 1.1: The original VitalLens 1.0 architecture re-trained on our new, larger 1,413- participant dataset.

This comparison allows us to isolate performance gains from architectural improvements (VitalLens 2.0 vs. 1.1) versus data improvements (VitalLens 1.1 vs. other baselines and 1.0*). Evaluation Metrics.

We evaluate performance at both the waveform and vital sign levels. For waveforms, we report Pearson Correlation (r) and Signal-to-Noise Ratio (SNR). For vital signs, we report Mean Absolute Error (MAE) for HR, RR, and the HRV metrics SDNN, RMSSD, and LF/HF.

HRV Calculation Pipeline. The estimation of HRV metrics from the predicted PPG waveform follows a multi-stage signal processing pipeline. First, cardiac cycles are identified by detecting valid peaks in the waveform, considering signal prominence, width, and periodicity relative to a rolling frequency estimate. These detections are filtered to retain only high-confidence peaks. From the resulting peak train, Inter-Beat Intervals (IBIs) are calculated. This IBI time-series is then cleaned by interpolating outlier intervals that fall outside a physiologically plausible range (e.g., due to missed or false detections). Finally, standard time-domain (SDNN, RMSSD) and frequency-domain (LF/HF) metrics are computed from the cleaned IBI sequence, contingent on meeting minimum duration (e.g., ≥20s for SDNN) and beat count thresholds.

Related Journal Articles & DOI Links

Selected peer-reviewed publications relevant to 12 Lead ECG Acquisition. Click the DOI to access the full paper (may require institutional access).

Why Choose Us?

Bangalore guidance for robotics, Spectre and autonomous systems projects.