Enquire Now
Machine Learning · Python · Data Science · Devops Automation Of Software Deployment Pipelin · 2026

Devops Validating Webhook Policy

Data Preparation · Feature Engineering · Model Training · Evaluation — A practical machine-learning project concept for building, comparing and validating models using reproducible data-science workflows. Suitable for final-year, BE/BTech, ME/MTech and research-oriented project implementation.

7
Abstract Sections
7+
Tools & Platforms
3
Pipeline Stages

Devops Validating Webhook Policy

Devops Automation Of Software Deployment Pipelin · ML Project

Python · Data Preprocessing · Model Development · Evaluation

Project focus: anomaly detection / classification using IoT sensor streams and timestamped device measurements.

Embedded DevOps: A Survey on the Application of

DevOps Practices in Embedded Software and

Firmware Development

Parthiv Katapara Anand Sharma

ECE Dept. ECE Dept.

Institute of Technology Institute of Technology

Nirma University Nirma University

Ahmedabad 3824 Ahmedabad 3824

Email: [email protected] Email: [email protected]

Abstract—The adoption of DevOps practices in embedded on empirical and technical evidence derived from a review of

systems and firmware development is emerging as a response 2 primary research papers and industry whitepapers. These to the growing complexity of modern hardware–software co- documents span academic investigations, industry case studies, designed products. Unlike cloud-native applications, embedded systems introduce challenges such as hardware dependency, real- toolchain evaluations, and architectural patterns related to time constraints, and safety-critical requirements. This literature Embedded DevOps.

review synthesizes findings from 2 academic and industrial We structure our analysis around key DevOps prac- sources to examine how DevOps principles—particularly continu- tices—Continuous Integration (CI), Continuous Delivery ous integration, continuous delivery, and automated testing—are adapted to embedded contexts. We categorize efforts across (CD), test automation, deployment strategies, digital twin tooling, testing strategies, pipeline automation, and security usage, and pipeline orchestration—highlighting where em-

practices. The review highlights current limitations in deployment bedded constraints necessitate deviation or augmentation of workflows and observability, proposing a roadmap for future conventional methods , , . research. This work offers researchers and practitioners a consol- In doing so, this survey addresses the following key research idated understanding of Embedded DevOps, bridging fragmented literature with a structured perspective. questions:

Index Terms—Embedded DevOps, Continuous Integration, • RQ1: What are the unique challenges to applying De-

Continuous Delivery, Firmware Development, Cyber-Physical vOps in embedded software and firmware development?

Systems, Automation, Embedded Systems Testing. • RQ2: What adaptations and tooling strategies have been

I. I NTRODUCTION reported to overcome these challenges?

• RQ3: How do CI/CD, testing, and deployment pipelines

The adoption of DevOps in embedded systems development

manifest differently in embedded contexts compared to has emerged as a response to the increasing complexity and in- cloud-native ones? tegration demands of modern software-hardware co-designed • RQ4: What are the research and tooling gaps in current products. Embedded systems, especially those part of cyber-

Embedded DevOps practices?

physical systems (CPS), are evolving rapidly with stringent demands on safety, real-time responsiveness, and hardware- Through this grounded synthesis, we aim to provide practi- software synchronization , , . tioners with verified insights and researchers with a structured While DevOps has shown success in traditional IT and agenda for further investigation. cloud-native applications, its application in embedded sys- tems faces structural and technological barriers. These in- II. L ITERATURE S URVEY

clude hardware-dependent development environments, lack of automated deployment infrastructure, fragmented toolchains, This section presents a rigorous review of 2 academic and safety certification constraints , , . Furthermore, and industrial publications that explore how DevOps practices real-time and resource-constrained execution environments are being introduced, adapted, or challenged within embedded often prevent seamless integration of continuous deployment software and firmware development. The works span empir-

workflows that are standard in web or enterprise software ical studies, tooling evaluations, automation pipelines, and development , , . security-integrated DevOps models in CPS and embedded do- This survey aims to present a comprehensive and criti- mains. For clarity, the discussion is grouped into foundational cal synthesis of how DevOps principles are being applied, empirical works, tooling and automation pipelines, CI/CD

tailored, or challenged in embedded software and firmware integration strategies, testing frameworks, and security-aware development. Unlike position or vision papers, our focus is DevOps adaptations.

A. Foundational Works and Empirical Background This demonstrates DevOps alignment with Scrum-based em-

Lwakatare et al. conducted a comprehensive multi- bedded firmware workflows. case study involving four Finnish companies and identified The whitepaper by Parasoft presented commercial tool- how traditional DevOps transformations struggle in embedded ing for CI/CD pipelines in safety-critical firmware develop- software due to delayed hardware feedback loops, testing ment. Emphasis was placed on test harness generation (e.g., for infrastructure gaps, and safety-critical deployment pipelines. C/C++), remote execution on constrained devices, and trace-

ability between tests, source code, and requirements—vital for

Wijaya et al. provided an empirical prototype of

DevOps adoption in embedded systems using a Spark Ignition

Engine model. The study proposed a lightweight DevOps Hoang et al. described an automation framework inte- grating Jenkins and Maven for embedded builds. The approach architecture integrating CI with GitHub, but highlighted major reduced human intervention in firmware CI workflows and challenges like physical hardware access, emulator infidelity, and real-time test constraints. provided interfaces for monitoring and recovery from deploy-

Alias Robotics introduced DevSecOps principles in robotic

Kumar et al. discussed the use of GoogleTest, CppUnit, systems where embedded components dominate . They and static code analyzers in embedded CI contexts. The emphasize shifting security left in embedded DevOps pipelines integration of these tools with cross-compilation toolchains through continuous security testing, threat modeling, and pre- allowed faster code validation without disrupting constrained deployment verification. devices.

Zampetti et al. interviewed practitioners across ten

Adhil et al. explored deployment automation in reg-

CPS organizations and systematically reported major barri-

ulated industrial settings using hybrid pipelines (Jenkins + ers to DevOps adoption. These included CI failures due to custom bash + hardware trigger scripts). While facing hard- hardware/software integration delays, flakiness in testing HiL ware access delays, they implemented rollback safety nets setups, and the need for simulators as a bridging mechanism. and digital signatures to maintain production stability during

CIMdata discussed industrial drivers like vehicle au-

firmware updates. tonomy, real-time control, and hardware-software co-design, arguing that embedded systems must embrace DevOps to meet C. Testing and Validation in Embedded DevOps increasing demands for velocity and quality in smart product Garousi et al. conducted a comprehensive systematic evolution. literature mapping on embedded software testing, analyzing Milićević et al. emphasized the role of DevOps in over 3 papers. Their findings revealed dominant practices

education, proposing a layered curriculum to teach continu- including Hardware-in-the-Loop (HiL), Model-in-the-Loop ous development concepts—including embedded system rele- (MiL), Software-in-the-Loop (SiL), and simulation-based test vance—through integrated pipelines and automation tools. environments, all of which are essential for integrating testing in DevOps pipelines for embedded systems. They also identi-

B. Tooling and Automation in Embedded DevOps

fied industry trends towards automated test generation, non- Menon et al. proposed an automated CI/CD work- functional validation, and safety standard compliance (e.g., flow tailored for embedded firmware, deploying source code ISO 26262, DO-178C). through GitHub pipelines and executing tests on embedded Barbie et al. highlighted the role of digital twins in targets via Raspberry Pi acting as sandbox servers. Their archi- enabling test-driven development cycles in embedded appli-

tecture uses a custom OS image, Docker-based toolchains, and cations. Digital Twin Prototypes (DTPs) allow decoupled CI remote monitoring using Node.js, demonstrating functional pipelines by replicating physical sensor behavior in simulation, DevOps loops in embedded environments. supporting automated test execution without real hardware.

Barbie et al. introduced the concept of Digital Twin Nguyen’s implementation incorporated GoogleTest and

Prototypes (DTPs) for CI testing in embedded oceanographic PlatformIO into CI workflows for embedded C/C++ projects. systems. By simulating sensor and actuator inputs, DTPs These tools were integrated with Azure DevOps to trigger tests enable full software validation in virtual environments before on each code commit, enforcing continuous testing. deployment to physical hardware—enabling fast, reproducible The Parasoft whitepaper emphasized continuous test-

CI even without hardware access. ing frameworks tailored for embedded targets, highlighting

Engblom proposed simulation-based CI using virtual minimal-footprint test harnesses, cross-compiler compatibility, platforms. The method replaces physical hardware with ac- and the importance of gathering test coverage and traceability curate models that support unit, subsystem, and integration data from constrained systems. testing. This was particularly useful for regression automation Zampetti et al. reported organizational reliance on HiL and fault injection in safety-critical systems. and simulators in CPS pipelines, with multiple companies

Nguyen implemented a CI pipeline for embedded en- suffering from mismatched behavior between physical and vironments using Azure DevOps. The workflow integrated simulated environments. They observed delays in CI due to Git, Azure Boards, VSCode, and test tools to automate hardware lockouts and emphasized the necessity of robust test build–test–merge processes linked to product backlog tasks. abstraction layers.

TechRxiv’s digital twin paper presented test frame- response automation, these are rarely mirrored in embedded works for embedded marine systems, where data-driven sim- environments due to hardware isolation, real-time constraints, ulations enabled full-stack integration testing across complex and certification bottlenecks. sensor arrays. The work argued for virtualization as a corner- stone of scalable embedded test pipelines.

E. Comparative Summary of Embedded DevOps Literature

Kumar et al. and IJCRT both stressed the need for

static analysis, code instrumentation, and automated regression Table I summarizes the key contributions, focus areas, testing integrated into CI/CD flows. These practices are essen- and limitations across the 2 reviewed papers. The papers tial in embedded domains due to strict timing, memory, and are grouped by primary theme (empirical, tooling, testing, certification constraints. security, etc.) and evaluated across DevOps-specific criteria.

Dakkak et al. advocated for testing-as-a-service (TaaS)

in embedded product–service systems. Their DevServOps

III. D ISCUSSION AND S YNTHESIS

model proposed combining continuous delivery of firmware with continuous monitoring of service behavior to close the The surveyed literature highlights both promising devel- loop between device and cloud analytics. opments and persistent challenges in applying DevOps to

D. Security and DevSecOps in Embedded Systems embedded software and firmware development. This section

synthesizes cross-cutting themes, contrasts embedded DevOps Alias Robotics proposed one of the few comprehen- with its cloud-native counterpart, and identifies pressing re- sive frameworks applying DevSecOps principles to robotic search and tooling gaps. systems, which inherently depend on embedded firmware. Their approach integrates security scanning, static analysis, and vulnerability modeling directly into the CI/CD pipeline. A. Key Observations Across Studies

By treating firmware as a primary attack vector, the study

1) Hardware-Centric Constraints: Across nearly all empir- advocates for security validation as a first-class citizen during ical and implementation studies , , , hardware access development rather than post-deployment. was the single largest blocker in achieving continuous delivery.

Dakkak et al. introduced DevServOps, a concept fo-

Unlike cloud-native systems, embedded code must often be

cused on product-oriented service systems where embedded compiled, flashed, and validated on physical devices—many firmware and cloud services evolve together. Their model of which are inaccessible during development cycles. While embeds threat detection and resilience monitoring into the simulation-based strategies (e.g., , ) show promise, their product-service lifecycle, combining telemetry feedback with fidelity varies across domains. staged firmware rollouts. 2) CI Is Achievable; CD Is Rare: Most papers demon-

Ebert and Hochstein discuss the importance of inte-

strated reasonably mature CI workflows for embedded soft- grating quality assurance, operational metrics, and continu- ware using tools like Jenkins, Azure DevOps, GitHub Actions, ous verification into modern DevOps pipelines. Though not and custom shell scripts , , . However, only a embedded-specific, they stress that in domains with real- minority achieved continuous delivery due to OTA deployment time constraints and physical control systems—hallmarks of complexity, limited rollback mechanisms, and compliance

embedded environments—cross-functional teams must embed overheads. security into all delivery phases. Menon et al. briefly mention the absence of secure over- 3) Simulation and Digital Twins Are Gaining Ground: the-air (OTA) deployment standards as a limiting factor in Simulation-driven testing (e.g., Digital Twin Prototypes in ) embedded DevOps maturity. Their pipeline emphasizes artifact and virtual testbeds are increasingly being used to de- validation, implying a need for stronger cryptographic signing couple test execution from physical devices. This has enabled

and verification in firmware releases. more reproducible and scalable CI workflows, especially when Garousi et al. indirectly raise security concerns through used alongside automated test harnesses . the lens of test adequacy and completeness in regulated 4) Security Integration Is Underdeveloped: Only a few domains. Their findings reinforce the need for traceable test studies, such as , , deeply engaged with DevSecOps artifacts and secure configuration management to meet regu- principles in embedded environments. Most pipelines lacked

latory mandates like IEC 615 or ISO 26262. static security scanning, firmware signing, or secure deliv- Zampetti et al. report that few organizations have ery mechanisms, despite the known vulnerability surface of embedded formal security testing in their embedded DevOps firmware-centric systems. pipelines. The lack of security-centric stages is attributed to 5) Testing Practices Are Fragmented: While several papers complexity, tool immaturity, and organizational silos between implemented GoogleTest, CppUnit, or custom test harnesses

firmware teams and IT security groups. , the overall testing stack remains fragmented. Testing Overall, security integration in embedded DevOps remains levels (unit, integration, system) are often inconsistently ap- a nascent discipline. While cloud DevOps has mature tooling plied, and regression pipelines are weakly linked to coverage for secret management, attack surface reduction, and incident or safety requirements .

TABLE I

C OMPARATIVE S UMMARY OF K EY L ITERATURE ON E MBEDDED D EVO PS (A LL 2 PAPERS )

Paper Domain / Focus CI/CD Strategy Tooling Used Test Automation Security / Gaps

Lwakatare et al. (2016) Empirical study on em- Partial CI; No CD Manual pipelines, Limited automation Hardware bedded DevOps Jenkins dependency, ops misalignment Wijaya et al. (2019) Spark ignition controller GitHub-based CI GitHub + Arduino + Manual + unit test Lack of secure OTA, prototype Bash scripts emulator fidelity is-

sues Alias Robotics (2021) Robotics / DevSecOps CI/CD + Security Custom DevSecOps Security test inte- Threat modeling, gate stack grated vulnerability scanning Zampetti et al. (2023) CPS industry interviews CI with HiL, mini- Jenkins + internal HiL + sim + unit test Testing flakiness, mal CD tools lack of secure

rollback CIMdata (2022) Industrial automation CD vision for em- Not specified Conceptual only Hardware-software trends bedded co-evolution risks Milićević et al. (2021) DevOps in education for CI lab setup GitHub Actions + Simulation-based CI Not addressed embedded Docker

Menon et al. (2021) Embedded automation Full CI on Pi targets Docker, GitHub Ac- Remote embedded Secure OTA missing pipeline tions, Node.js testing Barbie et al. (2021) Ocean DTP systems CI via Digital Twin Python + custom sim Simulation-driven Secure config sync

Prototypes stack test not discussed

Engblom (2015) CI via simulation in em- Virtual CI environ- WindRiver + QEMU Fault injection, full No security enforce- bedded ments CI ment Nguyen (2022) Azure DevOps for em- Git-based CI/CD Azure DevOps, Plat- GoogleTest + custom No secure build or bedded firmware formIO, VS Code CI release validation

Parasoft (2022) Safety-critical CI + gated delivery Parasoft C/C++test, Test harness, trace- Compliance-focused, embedded CI/CD Jenkins ability, regression cryptographic integrity IJCRT (2018) Jenkins-based CI for Basic CI/CD Jenkins, Maven, cus- C/C++ test pipelines No formal threat embedded tom bash model

Kumar et al. (2021) Continuous testing em- Unit test driven CI GoogleTest, Unit + regression + Not covered bedded apps CppUnit, TICS coverage Adhil et al. (2020) Industrial DevOps inte- Jenkins-based hybrid Bash scripts + Git + Semi-automated test Emphasis on roll- gration CI/CD sensors loops back over security

Dakkak et al. (2023) DevServOps (product- Firmware+Cloud Service monitors, Product–cloud feed- Monitoring service systems) CI/CD OTA updates back loop integrated for threat response Ebert & Hochstein (2023) General DevOps in soft- Continuous feedback Conceptual (Netflix Performance Highlights need for ware engineering loops example) and availability early security inte-

monitoring gration Garousi et al. (2018) Survey on embedded Not DevOps specific Comprehensive test- MiL, SiL, HiL, Implied for regulated software testing ing SLR model-based safety assurance TechRxiv (2021) Digital twin for marine CI via DTPs Sim stack + CI Automated CI in No hardware con- systems hooks + Git simulation trol integration men-

tioned Milicevic (2021) DevOps education + End-to-end course Docker + Git + Jenk- Hands-on student Not discussed tooling CI/CD ins testing Nguyen Thesis (2022) Practical pipeline imple- Scrum-linked CI/CD Azure Boards + Git Build–test–merge Security not mentation + DevOps automation enforced

B. Contrasting Traditional and Embedded DevOps pipelines depend heavily on vendor-specific compilers, simulators, and debuggers. • Pipeline Continuity: Cloud-native systems allow fully • Security Enforcement: Modern cloud DevOps integrates automated deploy-test-feedback loops. In embedded sys- secrets management, SBOMs, and CI/CD security scans. tems, delivery is often stalled at hardware programming In embedded, such integrations are sparse or ad hoc.

and certification. • Feedback Latency: Embedded workflows experience C. Emerging Trends and Research Gaps higher feedback latency due to hardware queuing, lack of remote debugging, and manual flashing steps. The following research and tooling gaps were consistently • Toolchain Heterogeneity: Unlike the standardized cloud surfaced:

DevOps stacks (e.g., Docker, Kubernetes), embedded • Standardized DevOps Toolchains: There is no unified

pipeline framework for embedded development compara- V. C ONCLUSION ble to Jenkins-X or GitLab-CI in cloud-native contexts.

This survey paper has systematically reviewed 2 signif-

• OTA Deployment Models: Secure, incremental, and icant contributions addressing the application of DevOps in traceable OTA firmware deployment remains underex- embedded software and firmware development. Our analysis plored. revealed that while continuous integration (CI) is increasingly • Hardware–Simulation Bridging: Dynamic switching feasible—especially through containerized environments, sim- between HiL and simulation modes based on test case ulation, and digital twin techniques—continuous delivery (CD)

type could enable adaptive pipelines. remains rare due to physical hardware constraints, certifica- • Security Validation: DevSecOps integration must in- tion requirements, and limited test automation maturity. The clude firmware signing, secure update chains, and static surveyed works show a fragmented tooling ecosystem, with vulnerability analysis embedded in CI. most organizations relying on custom pipelines and ad hoc • Observability and Telemetry: Post-deployment moni- scripts. Testing strategies span from unit testing on emulators

toring is rarely integrated with delivery pipelines; em- to complex HiL setups, but a consistent test orchestration bedded systems lack mature observability patterns. framework for embedded domains remains elusive. Further- more, DevSecOps is in its infancy in this space, with only a

IV. F UTURE W ORK few papers integrating security validations or secure firmware

signing. Overall, the field is at a transitional point. Embedded

DevOps has moved beyond conceptual discussions and into

Building on the gaps identified across literature, the fol-

early-stage practical deployments, but broad, standardized lowing future directions emerge as high-priority research and adoption remains limited. engineering challenges: • Secure and Scalable OTA Pipelines: There is a pressing R EFERENCES need for well-architected, secure Over-The-Air (OTA)

L. E. Lwakatare, P. Kuvaja, and M. Oivo, “An Exploratory Study

update mechanisms that integrate cryptographic signing, of DevOps: Extending the Dimensions of DevOps with Practices,” in delta updates, rollback safety, and real-time verification Proceedings of the 49th Hawaii International Conference on System into the DevOps loop. Sciences (HICSS), 2016, pp. 5462–5471.

Alias Robotics, “Embedding the Principles and Practices of DevOps into

• Standardized Embedded DevOps Frameworks: Un- the Cybersecurity of Robotic Systems,” DevSecOps Whitepaper, 2021. like cloud-native software, embedded development lacks F. Zampetti, D. Tamburri, S. Panichella, and others, “Continuous Integra- plug-and-play CI/CD tools. Future work should aim tion and Delivery Practices for Cyber-Physical Systems: An Interview-

Based Study,” ACM Transactions on Software Engineering and Method-

to create unified frameworks that combine cross- ology, vol. 32, no. 3, pp. 1–44, 2023. compilation, hardware flashing, testing, monitoring, and CIMdata, “Accelerating Innovation in Embedded Systems by Adopting delivery stages. DevOps,” CIMdata Commentary, 2022.

J. Milićević, T. Yemane, and M. Radenković, “Embedding the Principles

• Simulation-First Verification Models: Digital Twin Pro- and Practices of DevOps into the IT Education,” European Project totypes and high-fidelity simulators must be matured fur- Management Journal, vol. 11, no. 2, pp. 3–10, 2021. ther to support pre-deployment regression testing that is A. Menon, “DevOps for Embedded: Automating Embedded Software hardware-agnostic yet reliable enough to replace physical Development,” Trenser Whitepaper, 2021.

A. Barbie, W. Hasselbring, and N. Pech, “Continuous Integration Testing

test beds. of Embedded Software with Digital Twin Prototypes,” TechRxiv Preprint, • Integrated DevSecOps Toolchains: DevOps pipelines 2021. for embedded firmware must integrate vulnerability scan- J. Engblom, “Continuous Integration for Embedded Systems Using Sim- ulation,” in Embedded World Conference, 2015. ning, threat modeling, SBOM generation, and build-time Q. H. Nguyen, “Continuous Integration for Embedded Environment,”

validation of secure artifacts as part of their workflow. Bachelor’s Thesis, Vaasan University of Applied Sciences, 2022. • End-to-End Observability and Feedback: Research Parasoft, “The Ultimate Guide to CI/CD for Embedded Software Sys- tems,” 2022. [Online]. Available: https://www.parasoft.com/resources should explore how real-time telemetry, fault diagnostics, A. Kumar and H. Singh, “Automation of DevOps in Embedded System and usage patterns can be fed back into development Development: Tools, Challenges, and Solution,” International Journal of

pipelines from deployed embedded devices to close the Creative Research Thoughts (IJCRT), vol. 6, no. 2, 2018.

R. Kumar and N. Mehta, “Continuous Integration and Testing of

loop. Embedded C/C++ Firmware,” IEEE Software, 2021. • Certification-Aware DevOps: For domains like automo- T. Adhil and others, “Applied Industrial DevOps for Embedded System tive, medical, and aerospace, DevOps models must evolve Integration,” in Industrial DevOps Workshop, 2020.

A. Dakkak, J. Bosch, and H. Holmstrom Olsson, “DevServOps: DevOps

to support traceable, certifiable workflows that comply For Product-Oriented Product Service Systems,” in Proceedings of SEAA with standards such as ISO 26262, DO-178C, and IEC 2023, 2023.

61508. C. Ebert and L. Hochstein, “DevOps in Practice,” IEEE Software, vol.

40, no. 1, pp. 94–100, 2023. Addressing these areas would bridge the gap between cur- V. Garousi, M. Felderer, Ç. M. Karapıçak, and U. Yılmaz, “Testing Em- rent DevOps practice and the unique constraints of embedded bedded Software: A Survey of the Literature,” Information and Software Technology, vol. 104, pp. 14–45, 2018. systems, enabling more secure, agile, and reliable firmware A. Barbie, W. Hasselbring, and N. Pech, “Continuous Integration o

Continued Discussion and Extended Analysis (Part 2)

Embedded DevOps: A Survey on the Application of

DevOps Practices in Embedded Software and

Firmware Development

Parthiv Katapara Anand Sharma

ECE Dept. ECE Dept.

Institute of Technology Institute of Technology

Nirma University Nirma University

Ahmedabad 3824 Ahmedabad 3824

Email: [email protected] Email: [email protected]

Abstract—The adoption of DevOps practices in embedded on empirical and technical evidence derived from a review of

systems and firmware development is emerging as a response 2 primary research papers and industry whitepapers. These to the growing complexity of modern hardware–software co- documents span academic investigations, industry case studies, designed products. Unlike cloud-native applications, embedded systems introduce challenges such as hardware dependency, real- toolchain evaluations, and architectural patterns related to time constraints, and safety-critical requirements. This literature Embedded DevOps.

review synthesizes findings from 2 academic and industrial We structure our analysis around key DevOps prac- sources to examine how DevOps principles—particularly continu- tices—Continuous Integration (CI), Continuous Delivery ous integration, continuous delivery, and automated testing—are adapted to embedded contexts. We categorize efforts across (CD), test automation, deployment strategies, digital twin tooling, testing strategies, pipeline automation, and security usage, and pipeline orchestration—highlighting where em-

practices. The review highlights current limitations in deployment bedded constraints necessitate deviation or augmentation of workflows and observability, proposing a roadmap for future conventional methods , , . research. This work offers researchers and practitioners a consol- In doing so, this survey addresses the following key research idated understanding of Embedded DevOps, bridging fragmented literature with a structured perspective. questions:

Index Terms—Embedded DevOps, Continuous Integration, • RQ1: What are the unique challenges to applying De-

Continuous Delivery, Firmware Development, Cyber-Physical vOps in embedded software and firmware development?

Systems, Automation, Embedded Systems Testing. • RQ2: What adaptations and tooling strategies have been

I. I NTRODUCTION reported to overcome these challenges?

• RQ3: How do CI/CD, testing, and deployment pipelines

The adoption of DevOps in embedded systems development

manifest differently in embedded contexts compared to has emerged as a response to the increasing complexity and in- cloud-native ones? tegration demands of modern software-hardware co-designed • RQ4: What are the research and tooling gaps in current products. Embedded systems, especially those part of cyber-

Embedded DevOps practices?

physical systems (CPS), are evolving rapidly with stringent demands on safety, real-time responsiveness, and hardware- Through this grounded synthesis, we aim to provide practi- software synchronization , , . tioners with verified insights and researchers with a structured While DevOps has shown success in traditional IT and agenda for further investigation. cloud-native applications, its application in embedded sys- tems faces structural and technological barriers. These in- II. L ITERATURE S URVEY

clude hardware-dependent development environments, lack of automated deployment infrastructure, fragmented toolchains, This section presents a rigorous review of 2 academic and safety certification constraints , , . Furthermore, and industrial publications that explore how DevOps practices real-time and resource-constrained execution environments are being introduced, adapted, or challenged within embedded often prevent seamless integration of continuous deployment software and firmware development. The works span empir-

workflows that are standard in web or enterprise software ical studies, tooling evaluations, automation pipelines, and development , , . security-integrated DevOps models in CPS and embedded do- This survey aims to present a comprehensive and criti- mains. For clarity, the discussion is grouped into foundational cal synthesis of how DevOps principles are being applied, empirical works, tooling and automation pipelines, CI/CD

tailored, or challenged in embedded software and firmware integration strategies, testing frameworks, and security-aware development. Unlike position or vision papers, our focus is DevOps adaptations.

A. Foundational Works and Empirical Background This demonstrates DevOps alignment with Scrum-based em-

Lwakatare et al. conducted a comprehensive multi- bedded firmware workflows. case study involving four Finnish companies and identified The whitepaper by Parasoft presented commercial tool- how traditional DevOps transformations struggle in embedded ing for CI/CD pipelines in safety-critical firmware develop- software due to delayed hardware feedback loops, testing ment. Emphasis was placed on test harness generation (e.g., for infrastructure gaps, and safety-critical deployment pipelines. C/C++), remote execution on constrained devices, and trace-

ability between tests, source code, and requirements—vital for

Wijaya et al. provided an empirical prototype of

DevOps adoption in embedded systems using a Spark Ignition

Engine model. The study proposed a lightweight DevOps Hoang et al. described an automation framework inte- grating Jenkins and Maven for embedded builds. The approach architecture integrating CI with GitHub, but highlighted major reduced human intervention in firmware CI workflows and challenges like physical hardware access, emulator infidelity, and real-time test constraints. provided interfaces for monitoring and recovery from deploy-

Alias Robotics introduced DevSecOps principles in robotic

Kumar et al. discussed the use of GoogleTest, CppUnit, systems where embedded components dominate . They and static code analyzers in embedded CI contexts. The emphasize shifting security left in embedded DevOps pipelines integration of these tools with cross-compilation toolchains through continuous security testing, threat modeling, and pre- allowed faster code validation without disrupting constrained deployment verification. devices.

Zampetti et al. interviewed practitioners across ten

Adhil et al. explored deployment automation in reg-

CPS organizations and systematically reported major barri-

ulated industrial settings using hybrid pipelines (Jenkins + ers to DevOps adoption. These included CI failures due to custom bash + hardware trigger scripts). While facing hard- hardware/software integration delays, flakiness in testing HiL ware access delays, they implemented rollback safety nets setups, and the need for simulators as a bridging mechanism. and digital signatures to maintain production stability during

CIMdata discussed industrial drivers like vehicle au-

firmware updates. tonomy, real-time control, and hardware-software co-design, arguing that embedded systems must embrace DevOps to meet C. Testing and Validation in Embedded DevOps increasing demands for velocity and quality in smart product Garousi et al. conducted a comprehensive systematic evolution. literature mapping on embedded software testing, analyzing Milićević et al. emphasized the role of DevOps in over 3 papers. Their findings revealed dominant practices

education, proposing a layered curriculum to teach continu- including Hardware-in-the-Loop (HiL), Model-in-the-Loop ous development concepts—including embedded system rele- (MiL), Software-in-the-Loop (SiL), and simulation-based test vance—through integrated pipelines and automation tools. environments, all of which are essential for integrating testing in DevOps pipelines for embedded systems. They also identi-

B. Tooling and Automation in Embedded DevOps

fied industry trends towards automated test generation, non- Menon et al. proposed an automated CI/CD work- functional validation, and safety standard compliance (e.g., flow tailored for embedded firmware, deploying source code ISO 26262, DO-178C). through GitHub pipelines and executing tests on embedded Barbie et al. highlighted the role of digital twins in targets via Raspberry Pi acting as sandbox servers. Their archi- enabling test-driven development cycles in embedded appli-

tecture uses a custom OS image, Docker-based toolchains, and cations. Digital Twin Prototypes (DTPs) allow decoupled CI remote monitoring using Node.js, demonstrating functional pipelines by replicating physical sensor behavior in simulation, DevOps loops in embedded environments. supporting automated test execution without real hardware.

Barbie et al. introduced the concept of Digital Twin Nguyen’s implementation incorporated GoogleTest and

Prototypes (DTPs) for CI testing in embedded oceanographic PlatformIO into CI workflows for embedded C/C++ projects. systems. By simulating sensor and actuator inputs, DTPs These tools were integrated with Azure DevOps to trigger tests enable full software validation in virtual environments before on each code commit, enforcing continuous testing. deployment to physical hardware—enabling fast, reproducible The Parasoft whitepaper emphasized continuous test-

CI even without hardware access. ing frameworks tailored for embedded targets, highlighting

Engblom proposed simulation-based CI using virtual minimal-footprint test harnesses, cross-compiler compatibility, platforms. The method replaces physical hardware with ac- and the importance of gathering test coverage and traceability curate models that support unit, subsystem, and integration data from constrained systems. testing. This was particularly useful for regression automation Zampetti et al. reported organizational reliance on HiL and fault injection in safety-critical systems. and simulators in CPS pipelines, with multiple companies

Nguyen implemented a CI pipeline for embedded en- suffering from mismatched behavior between physical and vironments using Azure DevOps. The workflow integrated simulated environments. They observed delays in CI due to Git, Azure Boards, VSCode, and test tools to automate hardware lockouts and emphasized the necessity of robust test build–test–merge processes linked to product backlog tasks. abstraction layers.

TechRxiv’s digital twin paper presented test frame- response automation, these are rarely mirrored in embedded works for embedded marine systems, where data-driven sim- environments due to hardware isolation, real-time constraints, ulations enabled full-stack integration testing across complex and certification bottlenecks. sensor arrays. The work argued for virtualization as a corner- stone of scalable embedded test pipelines.

E. Comparative Summary of Embedded DevOps Literature

Kumar et al. and IJCRT both stressed the need for

static analysis, code instrumentation, and automated regression Table I summarizes the key contributions, focus areas, testing integrated into CI/CD flows. These practices are essen- and limitations across the 2 reviewed papers. The papers tial in embedded domains due to strict timing, memory, and are grouped by primary theme (empirical, tooling, testing, certification constraints. security, etc.) and evaluated across DevOps-specific criteria.

Dakkak et al. advocated for testing-as-a-service (TaaS)

in embedded product–service systems. Their DevServOps

III. D ISCUSSION AND S YNTHESIS

model proposed combining continuous delivery of firmware with continuous monitoring of service behavior to close the The surveyed literature highlights both promising devel- loop between device and cloud analytics. opments and persistent challenges in applying DevOps to

D. Security and DevSecOps in Embedded Systems embedded software and firmware development. This section

synthesizes cross-cutting themes, contrasts embedded DevOps Alias Robotics proposed one of the few comprehen- with its cloud-native counterpart, and identifies pressing re- sive frameworks applying DevSecOps principles to robotic search and tooling gaps. systems, which inherently depend on embedded firmware. Their approach integrates security scanning, static analysis, and vulnerability modeling directly into the CI/CD pipeline. A. Key Observations Across Studies

By treating firmware as a primary attack vector, the study

1) Hardware-Centric Constraints: Across nearly all empir- advocates for security validation as a first-class citizen during ical and implementation studies , , , hardware access development rather than post-deployment. was the single largest blocker in achieving continuous delivery.

Dakkak et al. introduced DevServOps, a concept fo-

Unlike cloud-native systems, embedded code must often be

cused on product-oriented service systems where embedded compiled, flashed, and validated on physical devices—many firmware and cloud services evolve together. Their model of which are inaccessible during development cycles. While embeds threat detection and resilience monitoring into the simulation-based strategies (e.g., , ) show promise, their product-service lifecycle, combining telemetry feedback with fidelity varies across domains. staged firmware rollouts. 2) CI Is Achievable; CD Is Rare: Most papers demon-

Ebert and Hochstein discuss the importance of inte-

strated reasonably mature CI workflows for embedded soft- grating quality assurance, operational metrics, and continu- ware using tools like Jenkins, Azure DevOps, GitHub Actions, ous verification into modern DevOps pipelines. Though not and custom shell scripts , , . However, only a embedded-specific, they stress that in domains with real- minority achieved continuous delivery due to OTA deployment time constraints and physical control systems—hallmarks of complexity, limited rollback mechanisms, and compliance

embedded environments—cross-functional teams must embed overheads. security into all delivery phases. Menon et al. briefly mention the absence of secure over- 3) Simulation and Digital Twins Are Gaining Ground: the-air (OTA) deployment standards as a limiting factor in Simulation-driven testing (e.g., Digital Twin Prototypes in ) embedded DevOps maturity. Their pipeline emphasizes artifact and virtual testbeds are increasingly being used to de- validation, implying a need for stronger cryptographic signing couple test execution from physical devices. This has enabled

and verification in firmware releases. more reproducible and scalable CI workflows, especially when Garousi et al. indirectly raise security concerns through used alongside automated test harnesses . the lens of test adequacy and completeness in regulated 4) Security Integration Is Underdeveloped: Only a few domains. Their findings reinforce the need for traceable test studies, such as , , deeply engaged with DevSecOps artifacts and secure configuration management to meet regu- principles in embedded environments. Most pipelines lacked

latory mandates like IEC 615 or ISO 26262. static security scanning, firmware signing, or secure deliv- Zampetti et al. report that few organizations have ery mechanisms, despite the known vulnerability surface of embedded formal security testing in their embedded DevOps firmware-centric systems. pipelines. The lack of security-centric stages is attributed to 5) Testing Practices Are Fragmented: While several papers complexity, tool immaturity, and organizational silos between implemented GoogleTest, CppUnit, or custom test harnesses

firmware teams and IT security groups. , the overall testing stack remains fragmented. Testing Overall, security integration in embedded DevOps remains levels (unit, integration, system) are often inconsistently ap- a nascent discipline. While cloud DevOps has mature tooling plied, and regression pipelines are weakly linked to coverage for secret management, attack surface reduction, and incident or safety requirements .

TABLE I

C OMPARATIVE S UMMARY OF K EY L ITERATURE ON E MBEDDED D EVO PS (A LL 2 PAPERS )

Paper Domain / Focus CI/CD Strategy Tooling Used Test Automation Security / Gaps

Lwakatare et al. (2016) Empirical study on em- Partial CI; No CD Manual pipelines, Limited automation Hardware bedded DevOps Jenkins dependency, ops misalignment Wijaya et al. (2019) Spark ignition controller GitHub-based CI GitHub + Arduino + Manual + unit test Lack of secure OTA, prototype Bash scripts emulator fidelity is-

sues Alias Robotics (2021) Robotics / DevSecOps CI/CD + Security Custom DevSecOps Security test inte- Threat modeling, gate stack grated vulnerability scanning Zampetti et al. (2023) CPS industry interviews CI with HiL, mini- Jenkins + internal HiL + sim + unit test Testing flakiness, mal CD tools lack of secure

rollback CIMdata (2022) Industrial automation CD vision for em- Not specified Conceptual only Hardware-software trends bedded co-evolution risks Milićević et al. (2021) DevOps in education for CI lab setup GitHub Actions + Simulation-based CI Not addressed embedded Docker

Menon et al. (2021) Embedded automation Full CI on Pi targets Docker, GitHub Ac- Remote embedded Secure OTA missing pipeline tions, Node.js testing Barbie et al. (2021) Ocean DTP systems CI via Digital Twin Python + custom sim Simulation-driven Secure config sync

Prototypes stack test not discussed

Engblom (2015) CI via simulation in em- Virtual CI environ- WindRiver + QEMU Fault injection, full No security enforce- bedded ments CI ment Nguyen (2022) Azure DevOps for em- Git-based CI/CD Azure DevOps, Plat- GoogleTest + custom No secure build or bedded firmware formIO, VS Code CI release validation

Parasoft (2022) Safety-critical CI + gated delivery Parasoft C/C++test, Test harness, trace- Compliance-focused, embedded CI/CD Jenkins ability, regression cryptographic integrity IJCRT (2018) Jenkins-based CI for Basic CI/CD Jenkins, Maven, cus- C/C++ test pipelines No formal threat embedded tom bash model

Kumar et al. (2021) Continuous testing em- Unit test driven CI GoogleTest, Unit + regression + Not covered bedded apps CppUnit, TICS coverage Adhil et al. (2020) Industrial DevOps inte- Jenkins-based hybrid Bash scripts + Git + Semi-automated test Emphasis on roll- gration CI/CD sensors loops back over security

Dakkak et al. (2023) DevServOps (product- Firmware+Cloud Service monitors, Product–cloud feed- Monitoring service systems) CI/CD OTA updates back loop integrated for threat response Ebert & Hochstein (2023) General DevOps in soft- Continuous feedback Conceptual (Netflix Performance Highlights need for ware engineering loops example) and availability early security inte-

monitoring gration Garousi et al. (2018) Survey on embedded Not DevOps specific Comprehensive test- MiL, SiL, HiL, Implied for regulated software testing ing SLR model-based safety assurance TechRxiv (2021) Digital twin for marine CI via DTPs Sim stack + CI Automated CI in No hardware con- systems hooks + Git simulation trol integration men-

tioned Milicevic (2021) DevOps education + End-to-end course Docker + Git + Jenk- Hands-on student Not discussed tooling CI/CD ins testing Nguyen Thesis (2022) Practical pipeline imple- Scrum-linked CI/CD Azure Boards + Git Build–test–merge Security not mentation + DevOps automation enforced

B. Contrasting Traditional and Embedded DevOps pipelines depend heavily on vendor-specific compilers, simulators, and debuggers. • Pipeline Continuity: Cloud-native systems allow fully • Security Enforcement: Modern cloud DevOps integrates automated deploy-test-feedback loops. In embedded sys- secrets management, SBOMs, and CI/CD security scans. tems, delivery is often stalled at hardware programming In embedded, such integrations are sparse or ad hoc.

and certification. • Feedback Latency: Embedded workflows experience C. Emerging Trends and Research Gaps higher feedback latency due to hardware queuing, lack of remote debugging, and manual flashing steps. The following research and tooling gaps were consistently • Toolchain Heterogeneity: Unlike the standardized cloud surfaced:

DevOps stacks (e.g., Docker, Kubernetes), embedded • Standardized DevOps Toolchains: There is no unified

pipeline framework for embedded development compara- V. C ONCLUSION ble to Jenkins-X or GitLab-CI in cloud-native contexts.

This survey paper has systematically reviewed 2 signif-

• OTA Deployment Models: Secure, incremental, and icant contributions addressing the application of DevOps in traceable OTA firmware deployment remains underex- embedded software and firmware development. Our analysis plored. revealed that while continuous integration (CI) is increasingly • Hardware–Simulation Bridging: Dynamic switching feasible—especially through containerized environments, sim- between HiL and simulation modes based on test case ulation, and digital twin techniques—continuous delivery (CD)

type could enable adaptive pipelines. remains rare due to physical hardware constraints, certifica- • Security Validation: DevSecOps integration must in- tion requirements, and limited test automation maturity. The clude firmware signing, secure update chains, and static surveyed works show a fragmented tooling ecosystem, with vulnerability analysis embedded in CI. most organizations relying on custom pipelines and ad hoc • Observability and Telemetry: Post-deployment moni- scripts. Testing strategies span from unit testing on emulators

toring is rarely integrated with delivery pipelines; em- to complex HiL setups, but a consistent test orchestration bedded systems lack mature observability patterns. framework for embedded domains remains elusive. Further- more, DevSecOps is in its infancy in this space, with only a

IV. F UTURE W ORK few papers integrating security validations or secure firmware

signing. Overall, the field is at a transitional point. Embedded

DevOps has moved beyond conceptual discussions and into

Building on the gaps identified across literature, the fol-

early-stage practical deployments, but broad, standardized lowing future directions emerge as high-priority research and adoption remains limited. engineering challenges: • Secure and Scalable OTA Pipelines: There is a pressing R EFERENCES need for well-architected, secure Over-The-Air (OTA)

L. E. Lwakatare, P. Kuvaja, and M. Oivo, “An Exploratory Study

update mechanisms that integrate cryptographic signing, of DevOps: Extending the Dimensions of DevOps with Practices,” in delta updates, rollback safety, and real-time verification Proceedings of the 49th Hawaii International Conference on System into the DevOps loop. Sciences (HICSS), 2016, pp. 5462–5471.

Alias Robotics, “Embedding the Principles and Practices of DevOps into

• Standardized Embedded DevOps Frameworks: Un- the Cybersecurity of Robotic Systems,” DevSecOps Whitepaper, 2021. like cloud-native software, embedded development lacks F. Zampetti, D. Tamburri, S. Panichella, and others, “Continuous Integra- plug-and-play CI/CD tools. Future work should aim tion and Delivery Practices for Cyber-Physical Systems: An Interview-

Based Study,” ACM Transactions on Software Engineering and Method-

to create unified frameworks that combine cross- ology, vol. 32, no. 3, pp. 1–44, 2023. compilation, hardware flashing, testing, monitoring, and CIMdata, “Accelerating Innovation in Embedded Systems by Adopting delivery stages. DevOps,” CIMdata Commentary, 2022.

J. Milićević, T. Yemane, and M. Radenković, “Embedding the Principles

• Simulation-First Verification Models: Digital Twin Pro- and Practices of DevOps into the IT Education,” European Project totypes and high-fidelity simulators must be matured fur- Management Journal, vol. 11, no. 2, pp. 3–10, 2021. ther to support pre-deployment regression testing that is A. Menon, “DevOps for Embedded: Automating Embedded Software hardware-agnostic yet reliable enough to replace physical Development,” Trenser Whitepaper, 2021.

A. Barbie, W. Hasselbring, and N. Pech, “Continuous Integration Testing

test beds. of Embedded Software with Digital Twin Prototypes,” TechRxiv Preprint, • Integrated DevSecOps Toolchains: DevOps pipelines 2021. for embedded firmware must integrate vulnerability scan- J. Engblom, “Continuous Integration for Embedded Systems Using Sim- ulation,” in Embedded World Conference, 2015. ning, threat modeling, SBOM generation, and build-time Q. H. Nguyen, “Continuous Integration for Embedded Environment,”

validation of secure artifacts as part of their workflow. Bachelor’s Thesis, Vaasan University of Applied Sciences, 2022. • End-to-End Observability and Feedback: Research Parasoft, “The Ultimate Guide to CI/CD for Embedded Software Sys- tems,” 2022. [Online]. Available: https://www.parasoft.com/resources should explore how real-time telemetry, fault diagnostics, A. Kumar and H. Singh, “Automation of DevOps in Embedded System and usage patterns can be fed back into development Development: Tools, Challenges, and Solution,” International Journal of

pipelines from deployed embedded devices to close the Creative Research Thoughts (IJCRT), vol. 6, no. 2, 2018.

R. Kumar and N. Mehta, “Continuous Integration and Testing of

loop. Embedded C/C++ Firmware,” IEEE Software, 2021. • Certification-Aware DevOps: For domains like automo- T. Adhil and others, “Applied Industrial DevOps for Embedded System tive, medical, and aerospace, DevOps models must evolve Integration,” in Industrial DevOps Workshop, 2020.

A. Dakkak, J. Bosch, and H. Holmstrom Olsson, “DevServOps: DevOps

to support traceable, certifiable workflows that comply For Product-Oriented Product Service Systems,” in Proceedings of SEAA with standards such as ISO 26262, DO-178C, and IEC 2023, 2023.

61508. C. Ebert and L. Hochstein, “DevOps in Practice,” IEEE Software, vol.

40, no. 1, pp. 94–100, 2023. Addressing these areas would bridge the gap between cur- V. Garousi, M. Felderer, Ç. M. Karapıçak, and U. Yılmaz, “Testing Em- rent DevOps practice and the unique constraints of embedded bedded Software: A Survey of the Literature,” Information and Software Technology, vol. 104, pp. 14–45, 2018. systems, enabling more secure, agile, and reliable firmware A. Barbie, W. Hasselbring, and N. Pech, “Continuous Integration o

Continued Discussion and Extended Analysis (Part 3)

Embedded DevOps: A Survey on the Application of

DevOps Practices in Embedded Software and

Firmware Development

Parthiv Katapara Anand Sharma

ECE Dept. ECE Dept.

Institute of Technology Institute of Technology

Nirma University Nirma University

Ahmedabad 3824 Ahmedabad 3824

Email: [email protected] Email: [email protected]

Abstract—The adoption of DevOps practices in embedded on empirical and technical evidence derived from a review of

systems and firmware development is emerging as a response 2 primary research papers and industry whitepapers. These to the growing complexity of modern hardware–software co- documents span academic investigations, industry case studies, designed products. Unlike cloud-native applications, embedded systems introduce challenges such as hardware dependency, real- toolchain evaluations, and architectural patterns related to time constraints, and safety-critical requirements. This literature Embedded DevOps.

review synthesizes findings from 2 academic and industrial We structure our analysis around key DevOps prac- sources to examine how DevOps principles—particularly continu- tices—Continuous Integration (CI), Continuous Delivery ous integration, continuous delivery, and automated testing—are adapted to embedded contexts. We categorize efforts across (CD), test automation, deployment strategies, digital twin tooling, testing strategies, pipeline automation, and security usage, and pipeline orchestration—highlighting where em-

practices. The review highlights current limitations in deployment bedded constraints necessitate deviation or augmentation of workflows and observability, proposing a roadmap for future conventional methods , , . research. This work offers researchers and practitioners a consol- In doing so, this survey addresses the following key research idated understanding of Embedded DevOps, bridging fragmented literature with a structured perspective. questions:

Index Terms—Embedded DevOps, Continuous Integration, • RQ1: What are the unique challenges to applying De-

Continuous Delivery, Firmware Development, Cyber-Physical vOps in embedded software and firmware development?

Systems, Automation, Embedded Systems Testing. • RQ2: What adaptations and tooling strategies have been

I. I NTRODUCTION reported to overcome these challenges?

• RQ3: How do CI/CD, testing, and deployment pipelines

The adoption of DevOps in embedded systems development

manifest differently in embedded contexts compared to has emerged as a response to the increasing complexity and in- cloud-native ones? tegration demands of modern software-hardware co-designed • RQ4: What are the research and tooling gaps in current products. Embedded systems, especially those part of cyber-

Embedded DevOps practices?

physical systems (CPS), are evolving rapidly with stringent demands on safety, real-time responsiveness, and hardware- Through this grounded synthesis, we aim to provide practi- software synchronization , , . tioners with verified insights and researchers with a structured While DevOps has shown success in traditional IT and agenda for further investigation. cloud-native applications, its application in embedded sys- tems faces structural and technological barriers. These in- II. L ITERATURE S URVEY

clude hardware-dependent development environments, lack of automated deployment infrastructure, fragmented toolchains, This section presents a rigorous review of 2 academic and safety certification constraints , , . Furthermore, and industrial publications that explore how DevOps practices real-time and resource-constrained execution environments are being introduced, adapted, or challenged within embedded often prevent seamless integration of continuous deployment software and firmware development. The works span empir-

workflows that are standard in web or enterprise software ical studies, tooling evaluations, automation pipelines, and development , , . security-integrated DevOps models in CPS and embedded do- This survey aims to present a comprehensive and criti- mains. For clarity, the discussion is grouped into foundational cal synthesis of how DevOps principles are being applied, empirical works, tooling and automation pipelines, CI/CD

tailored, or challenged in embedded software and firmware integration strategies, testing frameworks, and security-aware development. Unlike position or vision papers, our focus is DevOps adaptations.

A. Foundational Works and Empirical Background This demonstrates DevOps alignment with Scrum-based em-

Lwakatare et al. conducted a comprehensive multi- bedded firmware workflows. case study involving four Finnish companies and identified The whitepaper by Parasoft presented commercial tool- how traditional DevOps transformations struggle in embedded ing for CI/CD pipelines in safety-critical firmware develop- software due to delayed hardware feedback loops, testing ment. Emphasis was placed on test harness generation (e.g., for infrastructure gaps, and safety-critical deployment pipelines. C/C++), remote execution on constrained devices, and trace-

ability between tests, source code, and requirements—vital for

Wijaya et al. provided an empirical prototype of

DevOps adoption in embedded systems using a Spark Ignition

Engine model. The study proposed a lightweight DevOps Hoang et al. described an automation framework inte- grating Jenkins and Maven for embedded builds. The approach architecture integrating CI with GitHub, but highlighted major reduced human intervention in firmware CI workflows and challenges like physical hardware access, emulator infidelity, and real-time test constraints. provided interfaces for monitoring and recovery from deploy-

Alias Robotics introduced DevSecOps principles in robotic

Kumar et al. discussed the use of GoogleTest, CppUnit, systems where embedded components dominate . They and static code analyzers in embedded CI contexts. The emphasize shifting security left in embedded DevOps pipelines integration of these tools with cross-compilation toolchains through continuous security testing, threat modeling, and pre- allowed faster code validation without disrupting constrained deployment verification. devices.

Zampetti et al. interviewed practitioners across ten

Adhil et al. explored deployment automation in reg-

CPS organizations and systematically reported major barri-

ulated industrial settings using hybrid pipelines (Jenkins + ers to DevOps adoption. These included CI failures due to custom bash + hardware trigger scripts). While facing hard- hardware/software integration delays, flakiness in testing HiL ware access delays, they implemented rollback safety nets setups, and the need for simulators as a bridging mechanism. and digital signatures to maintain production stability during

CIMdata discussed industrial drivers like vehicle au-

firmware updates. tonomy, real-time control, and hardware-software co-design, arguing that embedded systems must embrace DevOps to meet C. Testing and Validation in Embedded DevOps increasing demands for velocity and quality in smart product Garousi et al. conducted a comprehensive systematic evolution. literature mapping on embedded software testing, analyzing Milićević et al. emphasized the role of DevOps in over 3 papers. Their findings revealed dominant practices

education, proposing a layered curriculum to teach continu- including Hardware-in-the-Loop (HiL), Model-in-the-Loop ous development concepts—including embedded system rele- (MiL), Software-in-the-Loop (SiL), and simulation-based test vance—through integrated pipelines and automation tools. environments, all of which are essential for integrating testing in DevOps pipelines for embedded systems. They also identi-

B. Tooling and Automation in Embedded DevOps

fied industry trends towards automated test generation, non- Menon et al. proposed an automated CI/CD work- functional validation, and safety standard compliance (e.g., flow tailored for embedded firmware, deploying source code ISO 26262, DO-178C). through GitHub pipelines and executing tests on embedded Barbie et al. highlighted the role of digital twins in targets via Raspberry Pi acting as sandbox servers. Their archi- enabling test-driven development cycles in embedded appli-

tecture uses a custom OS image, Docker-based toolchains, and cations. Digital Twin Prototypes (DTPs) allow decoupled CI remote monitoring using Node.js, demonstrating functional pipelines by replicating physical sensor behavior in simulation, DevOps loops in embedded environments. supporting automated test execution without real hardware.

Barbie et al. introduced the concept of Digital Twin Nguyen’s implementation incorporated GoogleTest and

Prototypes (DTPs) for CI testing in embedded oceanographic PlatformIO into CI workflows for embedded C/C++ projects. systems. By simulating sensor and actuator inputs, DTPs These tools were integrated with Azure DevOps to trigger tests enable full software validation in virtual environments before on each code commit, enforcing continuous testing. deployment to physical hardware—enabling fast, reproducible The Parasoft whitepaper emphasized continuous test-

CI even without hardware access. ing frameworks tailored for embedded targets, highlighting

Engblom proposed simulation-based CI using virtual minimal-footprint test harnesses, cross-compiler compatibility, platforms. The method replaces physical hardware with ac- and the importance of gathering test coverage and traceability curate models that support unit, subsystem, and integration data from constrained systems. testing. This was particularly useful for regression automation Zampetti et al. reported organizational reliance on HiL and fault injection in safety-critical systems. and simulators in CPS pipelines, with multiple companies

Nguyen implemented a CI pipeline for embedded en- suffering from mismatched behavior between physical and vironments using Azure DevOps. The workflow integrated simulated environments. They observed delays in CI due to Git, Azure Boards, VSCode, and test tools to automate hardware lockouts and emphasized the necessity of robust test build–test–merge processes linked to product backlog tasks. abstraction layers.

TechRxiv’s digital twin paper presented test frame- response automation, these are rarely mirrored in embedded works for embedded marine systems, where data-driven sim- environments due to hardware isolation, real-time constraints, ulations enabled full-stack integration testing across complex and certification bottlenecks. sensor arrays. The work argued for virtualization as a corner- stone of scalable embedded test pipelines.

E. Comparative Summary of Embedded DevOps Literature

Kumar et al. and IJCRT both stressed the need for

static analysis, code instrumentation, and automated regression Table I summarizes the key contributions, focus areas, testing integrated into CI/CD flows. These practices are essen- and limitations across the 2 reviewed papers. The papers tial in embedded domains due to strict timing, memory, and are grouped by primary theme (empirical, tooling, testing, certification constraints. security, etc.) and evaluated across DevOps-specific criteria.

Dakkak et al. advocated for testing-as-a-service (TaaS)

in embedded product–service systems. Their DevServOps

III. D ISCUSSION AND S YNTHESIS

model proposed combining continuous delivery of firmware with continuous monitoring of service behavior to close the The surveyed literature highlights both promising devel- loop between device and cloud analytics. opments and persistent challenges in applying DevOps to

D. Security and DevSecOps in Embedded Systems embedded software and firmware development. This section

synthesizes cross-cutting themes, contrasts embedded DevOps Alias Robotics proposed one of the few comprehen- with its cloud-native counterpart, and identifies pressing re- sive frameworks applying DevSecOps principles to robotic search and tooling gaps. systems, which inherently depend on embedded firmware. Their approach integrates security scanning, static analysis, and vulnerability modeling directly into the CI/CD pipeline. A. Key Observations Across Studies

By treating firmware as a primary attack vector, the study

1) Hardware-Centric Constraints: Across nearly all empir- advocates for security validation as a first-class citizen during ical and implementation studies , , , hardware access development rather than post-deployment. was the single largest blocker in achieving continuous delivery.

Dakkak et al. introduced DevServOps, a concept fo-

Unlike cloud-native systems, embedded code must often be

cused on product-oriented service systems where embedded compiled, flashed, and validated on physical devices—many firmware and cloud services evolve together. Their model of which are inaccessible during development cycles. While embeds threat detection and resilience monitoring into the simulation-based strategies (e.g., , ) show promise, their product-service lifecycle, combining telemetry feedback with fidelity varies across domains. staged firmware rollouts. 2) CI Is Achievable; CD Is Rare: Most papers demon-

Ebert and Hochstein discuss the importance of inte-

strated reasonably mature CI workflows for embedded soft- grating quality assurance, operational metrics, and continu- ware using tools like Jenkins, Azure DevOps, GitHub Actions, ous verification into modern DevOps pipelines. Though not and custom shell scripts , , . However, only a embedded-specific, they stress that in domains with real- minority achieved continuous delivery due to OTA deployment time constraints and physical control systems—hallmarks of complexity, limited rollback mechanisms, and compliance

embedded environments—cross-functional teams must embed overheads. security into all delivery phases. Menon et al. briefly mention the absence of secure over- 3) Simulation and Digital Twins Are Gaining Ground: the-air (OTA) deployment standards as a limiting factor in Simulation-driven testing (e.g., Digital Twin Prototypes in ) embedded DevOps maturity. Their pipeline emphasizes artifact and virtual testbeds are increasingly being used to de- validation, implying a need for stronger cryptographic signing couple test execution from physical devices. This has enabled

and verification in firmware releases. more reproducible and scalable CI workflows, especially when Garousi et al. indirectly raise security concerns through used alongside automated test harnesses . the lens of test adequacy and completeness in regulated 4) Security Integration Is Underdeveloped: Only a few domains. Their findings reinforce the need for traceable test studies, such as , , deeply engaged with DevSecOps artifacts and secure configuration management to meet regu- principles in embedded environments. Most pipelines lacked

latory mandates like IEC 615 or ISO 26262. static security scanning, firmware signing, or secure deliv- Zampetti et al. report that few organizations have ery mechanisms, despite the known vulnerability surface of embedded formal security testing in their embedded DevOps firmware-centric systems. pipelines. The lack of security-centric stages is attributed to 5) Testing Practices Are Fragmented: While several papers complexity, tool immaturity, and organizational silos between implemented GoogleTest, CppUnit, or custom test harnesses

firmware teams and IT security groups. , the overall testing stack remains fragmented. Testing Overall, security integration in embedded DevOps remains levels (unit, integration, system) are often inconsistently ap- a nascent discipline. While cloud DevOps has mature tooling plied, and regression pipelines are weakly linked to coverage for secret management, attack surface reduction, and incident or safety requirements .

TABLE I

C OMPARATIVE S UMMARY OF K EY L ITERATURE ON E MBEDDED D EVO PS (A LL 2 PAPERS )

Paper Domain / Focus CI/CD Strategy Tooling Used Test Automation Security / Gaps

Lwakatare et al. (2016) Empirical study on em- Partial CI; No CD Manual pipelines, Limited automation Hardware bedded DevOps Jenkins dependency, ops misalignment Wijaya et al. (2019) Spark ignition controller GitHub-based CI GitHub + Arduino + Manual + unit test Lack of secure OTA, prototype Bash scripts emulator fidelity is-

sues Alias Robotics (2021) Robotics / DevSecOps CI/CD + Security Custom DevSecOps Security test inte- Threat modeling, gate stack grated vulnerability scanning Zampetti et al. (2023) CPS industry interviews CI with HiL, mini- Jenkins + internal HiL + sim + unit test Testing flakiness, mal CD tools lack of secure

rollback CIMdata (2022) Industrial automation CD vision for em- Not specified Conceptual only Hardware-software trends bedded co-evolution risks Milićević et al. (2021) DevOps in education for CI lab setup GitHub Actions + Simulation-based CI Not addressed embedded Docker

Menon et al. (2021) Embedded automation Full CI on Pi targets Docker, GitHub Ac- Remote embedded Secure OTA missing pipeline tions, Node.js testing Barbie et al. (2021) Ocean DTP systems CI via Digital Twin Python + custom sim Simulation-driven Secure config sync

Prototypes stack test not discussed

Engblom (2015) CI via simulation in em- Virtual CI environ- WindRiver + QEMU Fault injection, full No security enforce- bedded ments CI ment Nguyen (2022) Azure DevOps for em- Git-based CI/CD Azure DevOps, Plat- GoogleTest + custom No secure build or bedded firmware formIO, VS Code CI release validation

Parasoft (2022) Safety-critical CI + gated delivery Parasoft C/C++test, Test harness, trace- Compliance-focused, embedded CI/CD Jenkins ability, regression cryptographic integrity IJCRT (2018) Jenkins-based CI for Basic CI/CD Jenkins, Maven, cus- C/C++ test pipelines No formal threat embedded tom bash model

Kumar et al. (2021) Continuous testing em- Unit test driven CI GoogleTest, Unit + regression + Not covered bedded apps CppUnit, TICS coverage Adhil et al. (2020) Industrial DevOps inte- Jenkins-based hybrid Bash scripts + Git + Semi-automated test Emphasis on roll- gration CI/CD sensors loops back over security

Dakkak et al. (2023) DevServOps (product- Firmware+Cloud Service monitors, Product–cloud feed- Monitoring service systems) CI/CD OTA updates back loop integrated for threat response Ebert & Hochstein (2023) General DevOps in soft- Continuous feedback Conceptual (Netflix Performance Highlights need for ware engineering loops example) and availability early security inte-

monitoring gration Garousi et al. (2018) Survey on embedded Not DevOps specific Comprehensive test- MiL, SiL, HiL, Implied for regulated software testing ing SLR model-based safety assurance TechRxiv (2021) Digital twin for marine CI via DTPs Sim stack + CI Automated CI in No hardware con- systems hooks + Git simulation trol integration men-

tioned Milicevic (2021) DevOps education + End-to-end course Docker + Git + Jenk- Hands-on student Not discussed tooling CI/CD ins testing Nguyen Thesis (2022) Practical pipeline imple- Scrum-linked CI/CD Azure Boards + Git Build–test–merge Security not mentation + DevOps automation enforced

B. Contrasting Traditional and Embedded DevOps pipelines depend heavily on vendor-specific compilers, simulators, and debuggers. • Pipeline Continuity: Cloud-native systems allow fully • Security Enforcement: Modern cloud DevOps integrates automated deploy-test-feedback loops. In embedded sys- secrets management, SBOMs, and CI/CD security scans. tems, delivery is often stalled at hardware programming In embedded, such integrations are sparse or ad hoc.

and certification. • Feedback Latency: Embedded workflows experience C. Emerging Trends and Research Gaps higher feedback latency due to hardware queuing, lack of remote debugging, and manual flashing steps. The following research and tooling gaps were consistently • Toolchain Heterogeneity: Unlike the standardized cloud surfaced:

DevOps stacks (e.g., Docker, Kubernetes), embedded • Standardized DevOps Toolchains: There is no unified

pipeline framework for embedded development compara- V. C ONCLUSION ble to Jenkins-X or GitLab-CI in cloud-native contexts.

This survey paper has systematically reviewed 2 signif-

• OTA Deployment Models: Secure, incremental, and icant contributions addressing the application of DevOps in traceable OTA firmware deployment remains underex- embedded software and firmware development. Our analysis plored. revealed that while continuous integration (CI) is increasingly • Hardware–Simulation Bridging: Dynamic switching feasible—especially through containerized environments, sim- between HiL and simulation modes based on test case ulation, and digital twin techniques—continuous delivery (CD)

type could enable adaptive pipelines. remains rare due to physical hardware constraints, certifica- • Security Validation: DevSecOps integration must in- tion requirements, and limited test automation maturity. The clude firmware signing, secure update chains, and static surveyed works show a fragmented tooling ecosystem, with vulnerability analysis embedded in CI. most organizations relying on custom pipelines and ad hoc • Observability and Telemetry: Post-deployment moni- scripts. Testing strategies span from unit testing on emulators

toring is rarely integrated with delivery pipelines; em- to complex HiL setups, but a consistent test orchestration bedded systems lack mature observability patterns. framework for embedded domains remains elusive. Further- more, DevSecOps is in its infancy in this space, with only a

IV. F UTURE W ORK few papers integrating security validations or secure firmware

signing. Overall, the field is at a transitional point. Embedded

DevOps has moved beyond conceptual discussions and into

Building on the gaps identified across literature, the fol-

early-stage practical deployments, but broad, standardized lowing future directions emerge as high-priority research and adoption remains limited. engineering challenges: • Secure and Scalable OTA Pipelines: There is a pressing R EFERENCES need for well-architected, secure Over-The-Air (OTA)

L. E. Lwakatare, P. Kuvaja, and M. Oivo, “An Exploratory Study

update mechanisms that integrate cryptographic signing, of DevOps: Extending the Dimensions of DevOps with Practices,” in delta updates, rollback safety, and real-time verification Proceedings of the 49th Hawaii International Conference on System into the DevOps loop. Sciences (HICSS), 2016, pp. 5462–5471.

Alias Robotics, “Embedding the Principles and Practices of DevOps into

• Standardized Embedded DevOps Frameworks: Un- the Cybersecurity of Robotic Systems,” DevSecOps Whitepaper, 2021. like cloud-native software, embedded development lacks F. Zampetti, D. Tamburri, S. Panichella, and others, “Continuous Integra- plug-and-play CI/CD tools. Future work should aim tion and Delivery Practices for Cyber-Physical Systems: An Interview-

Based Study,” ACM Transactions on Software Engineering and Method-

to create unified frameworks that combine cross- ology, vol. 32, no. 3, pp. 1–44, 2023. compilation, hardware flashing, testing, monitoring, and CIMdata, “Accelerating Innovation in Embedded Systems by Adopting delivery stages. DevOps,” CIMdata Commentary, 2022.

J. Milićević, T. Yemane, and M. Radenković, “Embedding the Principles

• Simulation-First Verification Models: Digital Twin Pro- and Practices of DevOps into the IT Education,” European Project totypes and high-fidelity simulators must be matured fur- Management Journal, vol. 11, no. 2, pp. 3–10, 2021. ther to support pre-deployment regression testing that is A. Menon, “DevOps for Embedded: Automating Embedded Software hardware-agnostic yet reliable enough to replace physical Development,” Trenser Whitepaper, 2021.

A. Barbie, W. Hasselbring, and N. Pech, “Continuous Integration Testing

test beds. of Embedded Software with Digital Twin Prototypes,” TechRxiv Preprint, • Integrated DevSecOps Toolchains: DevOps pipelines 2021. for embedded firmware must integrate vulnerability scan- J. Engblom, “Continuous Integration for Embedded Systems Using Sim- ulation,” in Embedded World Conference, 2015. ning, threat modeling, SBOM generation, and build-time Q. H. Nguyen, “Continuous Integration for Embedded Environment,”

validation of secure artifacts as part of their workflow. Bachelor’s Thesis, Vaasan University of Applied Sciences, 2022. • End-to-End Observability and Feedback: Research Parasoft, “The Ultimate Guide to CI/CD for Embedded Software Sys- tems,” 2022. [Online]. Available: https://www.parasoft.com/resources should explore how real-time telemetry, fault diagnostics, A. Kumar and H. Singh, “Automation of DevOps in Embedded System and usage patterns can be fed back into development Development: Tools, Challenges, and Solution,” International Journal of

pipelines from deployed embedded devices to close the Creative Research Thoughts (IJCRT), vol. 6, no. 2, 2018.

R. Kumar and N. Mehta, “Continuous Integration and Testing of

loop. Embedded C/C++ Firmware,” IEEE Software, 2021. • Certification-Aware DevOps: For domains like automo- T. Adhil and others, “Applied Industrial DevOps for Embedded System tive, medical, and aerospace, DevOps models must evolve Integration,” in Industrial DevOps Workshop, 2020.

A. Dakkak, J. Bosch, and H. Holmstrom Olsson, “DevServOps: DevOps

to support traceable, certifiable workflows that comply For Product-Oriented Product Service Systems,” in Proceedings of SEAA with standards such as ISO 26262, DO-178C, and IEC 2023, 2023.

61508. C. Ebert and L. Hochstein, “DevOps in Practice,” IEEE Software, vol.

40, no. 1, pp. 94–100, 2023. Addressing these areas would bridge the gap between cur- V. Garousi, M. Felderer, Ç. M. Karapıçak, and U. Yılmaz, “Testing Em- rent DevOps practice and the unique constraints of embedded bedded Software: A Survey of the Literature,” Information and Software Technology, vol. 104, pp. 14–45, 2018. systems, enabling more secure, agile, and reliable firmware A. Barbie, W. Hasselbring, and N. Pech, “Continuous Integration o

FAQ

Typical stacks include Git, a CI engine (Jenkins/GitHub Actions/GitLab CI), Docker, Kubernetes or a cloud PaaS, infrastructure-as-code (Terraform/Ansible), and monitoring (Prometheus/Grafana). Exact tools for Devops Validating Webhook Policy depend on the chosen cloud and delivery model.
It can be delivered as a fully documented simulation on local/kind clusters or as a guided deployment on a cloud free tier. Both approaches are acceptable for academic evaluation when metrics, logs and diagrams are captured.
Pipeline screenshots, successful/failed run history, deployment frequency charts, rollback demos, monitoring dashboards, IaC plans/applies, and a short viva demo script are commonly included.