Simulia Living Heart Device

On-Device Federated Learning With Flower

Akhil Mathur 1 2 Daniel J. Beutel 1 3 Pedro Porto Buarque de Gusm˜ao 1 Javier Fernandez-Marques 4 Taner Topal 1 3 Xinchi Qiu 1 Titouan Parcollet 5 Yan Gao 1 Nicholas D. Lane 1

Abstract

Federated Learning (FL) allows edge devices to collaboratively learn a shared prediction model while keeping their training data on the device, thereby decoupling the ability to do machine learning from the need to store data in the cloud. Despite the algorithmic advancements in FL, the support for on-device training of FL algorithms on edge devices remains poor. In this paper, we present an exploration of on-device FL on various smartphones and embedded devices using the Flower framework. We also evaluate the system costs of on-device FL and discuss how this quantification could be used to design more efficient FL algorithms.

Figure: System Model & Simulation Flow for Simulia Living Heart Device

Introduction

We have seen remarkable progress in enabling the execu- tion of deep learning models on mobile and embedded devices to infer user contexts and behaviors (Warden & Situnayake, 2019; Fromm et al., 2018; Chowdhery et al., 2019; Malekzadeh et al., 2019; Lee et al., 2019). This has been powered by the increasing computational abilities of edge devices as well as novel software optimizations to enable cloud-scale models to run on resource-constrained devices. However, when it comes to the training of these edge-focused models, a working assumption has been that the models will be trained centrally in the cloud, using train- ing data aggregated from several users.

Figure: System Model & Simulation Flow for Simulia Living Heart Device

Federated Learning (FL) (McMahan et al., 2017) aims to enable distributed edge devices (or users) to collaboratively train a shared prediction model while keeping their personal data private. At a high level, this is achieved by repeating three basic steps: i) local parameters update to a shared prediction model on each edge device, ii) sending the local parameter updates to a central server for aggregation, and iii) receiving the aggregated model back for the next round of local updates.

A major bottleneck to FL research is the paucity of frame- works that support federated training of workloads on mo- bile and embedded devices. While several frameworks in- cluding Tensorflow Federated (Google, 2020; Abadi et al., 2016) and LEAF (Caldas et al., 2018) enable simulation of FL clients, they cannot be used to understand the training dy- namics and compute the system costs of FL on edge devices.

Proceedings of the 3 rd MLSys Conference, Austin, TX, USA, 2020. Copyright 2020 by the author(s). Edge devices exhibit significant heterogeneity in their soft- ware stack, compute capabilities, and network bandwidth.

All these system-related factors, in combination with the choice of the FL clients and parameter aggregation algo- rithms, can impact the accuracy and training time of models trained in a federated setting.

In this paper, we present our exploration of on-device train- ing of FL workloads on Android smartphones and Nvidia Jetson series embedded devices, using the Flower frame- work (Beutel et al., 2020). Flower offers a stable implemen- tation of the core components of an FL system, and provides higher-level abstractions to enable researchers to experiment and implement new ideas on top of a reliable stack. We first demonstrate how we use the language-, platform- and ML framework-agnostic capabilities of Flower to support on-device training of FL workloads on edge devices with heterogeneous hardware and software stacks. We then de- ploy these FL clients on various embedded devices as well as on Android smartphones hosted in the Amazon AWS De- vice Farm (https://aws.amazon.com/device-farm/). Finally, we present an evaluation to compute various system-related metrics of FL and highlight how this quantification could lead to the design of more efficient FL algorithms.

Related Work

McMahan et al. (2017) introduced the basic federated av- eraging (FedAvg) algorithm and evaluated it in terms of communication efficiency. The optimization of distributed training with and without federated concepts has been cov- ered from many angles (Jia et al., 2018; Chahal et al., 2018). Bonawitz et al. (2019) detail the design of a large- scale Google-internal FL system. TFF (Google, 2020), PySyft (Ryffel et al., 2018), LEAF (Caldas et al., 2018), FedML (He et al., 2020) are other open-source frameworks that support research and experimentation of FL workloads.

On-Device Federated Learning With Flower

Figure 1. Flower framework architecture. Given their relative hardware limitations, most of the works involving machine learning on mobile devices were origi- nally aimed at adapting existing models to specific latency and storage constraints. For this purpose, optimized ver- sions of TensorFlow and PyTorch were developed (David et al., 2020; Paszke et al., 2019) and can now be consid- ered mainstream. Federated Learning, on the other hand, shifts the training burden from server to the client, which in turn creates the need for developing the adequate sup- porting back-ends (e.g. back-propagation) for low-power hardware as well. Recent works towards this goal include using low-precision training (Sun et al., 2020), controlled updates of biases to reduce memory (Cai et al., 2020), and early-exit models that provide a trade-off between accuracy and compute (Leontiadis et al., 2021). In time, as hardware capabilities in edge devices improve, we expect to see these advances being deployed by mainstream frameworks.

Primer On Flower

Flower is a novel client-agnostic federated learning frame- work. One of the underlying design goals of Flower is to enable integrating with an inherently heterogeneous and ever-evolving edge device landscape. There are multiple dimensions of on-device heterogeneity, amongst them are operating systems, machine learning frameworks, program- ming languages, connectivity, and hardware accelerators.

The Flower core framework, shown in Figure 1, implements the infrastructure to run these heterogeneous workloads at scale. On the server side, there are three major components involved: the FL loop, the RPC server, and a (user cus- tomizable) Strategy. Strategy here refers to the federated averaging algorithms (e.g., FedAvg) used for aggregating the model parameters across clients. Clients connect to the RPC server which is responsible for monitoring these connections and for sending and receiving Flower Protocol messages. The FL loop is at the heart of the FL process: it orchestrates the learning process and ensures that progress is made. It does not, however, make decisions about how to proceed, those decisions are delegated to the currently configured Strategy implementation.

A distinctive property of this architecture is a server which Figure 2. Flower Android client architecture. is unaware of the nature of connected clients. This allows to train models across heterogeneous client platforms and implementations, including workloads comprised of differ- ent client-side ML frameworks. Furthermore, on-device training and evaluation can be implemented in different programming languages, a property especially important for research on mobile and embedded platforms. These platforms often do not support Python, but rely on specific languages (Java on Android, Swift on iOS) for idiomatic development, or native C/C++ for some embedded devices.

Flower achieves a fully language-agnostic interface by offer- ing protocol-level integration. The Flower Protocol defines core server-side messages such as fit and evaluate, which include the (serialized) global model parameters and expect return messages from the client that return either updated model parameters (or gradients) or evaluation results. Each message contains additional user-customizable metadata that allows the server to control on-device hyper-parameters, for example, the number of on-device training epochs. Due to space constraints, we refer the reader to (Beutel et al., 2020) and https://flower.dev/ for more details on Flower.

Training

In this section, we describe two instances of on-device fed- erated learning with Flower. First, we present how Flower clients can be developed in Java and deployed on Android phones in the AWS Device Farm for federated model train- ing. Next, we discuss the implementation of Flower clients in Python and their deployment on heterogeneous embedded devices such as Nvidia Jetson series and Raspberry Pi.

Java Flower Clients For Android Smartphones

By design, Flower is language-agnostic and can work with any ML framework on the FL client, which maximizes its ability to federate existing training pipelines. However, it also means Flower inherits the limitations of these frame- works that currently offer limited support for on-device training on Android devices.

In the absence of a full-fledged model training library for

Related Journal Articles & DOI Links

Selected peer-reviewed publications relevant to 12 Lead ECG Acquisition. Click the DOI to access the full paper (may require institutional access).

Why Choose Us?

Bangalore guidance for robotics, Spectre and autonomous systems projects.