Devops Vercel Deployment Automation

Devops Automation Of Software Deployment Pipelin · ML Project

Python · Data Preprocessing · Model Development · Evaluation

Project focus: anomaly detection / classification using IoT sensor streams and timestamped device measurements.

A Systematic Literature Review on Continuous Integration and

Deployment (CI/CD) for Secure Cloud Computing

Sabbir M. Saleh a

, Nazim Madhavji b and John Steinbacher c

Department of Computer Science, University of Western Ontario, London, Ontario, Canada

IBM Canada Lab, Markham, Ontario, Canada

[email protected], [email protected], [email protected]

Keywords: Continuous Integration, Continuous Deployment, CI/CD, Cloud, Security, Systematic Literature Review.

Abstract: As cloud environments become widespread, cybersecurity has emerged as a top priority across areas such as

networks, communication, data privacy, response times, and availability. Various sectors, including industries, healthcare, and government, have recently faced cyberattacks targeting their computing systems. Ensuring secure app deployment in cloud environments requires substantial effort. With the growing interest in cloud security, conducting a systematic literature review (SLR) is critical to identifying research gaps. Continuous

Software Engineering, which includes continuous integration (CI), delivery (CDE), and deployment (CD), is

essential for software development and deployment. In our SLR, we reviewed 6 papers, summarising tools, approaches, and challenges related to the security of CI/CD in the cloud. We addressed key aspects of cloud security and CI/CD and reported on tools such as Harbor, SonarQube, and GitHub Actions. Challenges such as image manipulation, unauthorised access, and weak authentication were highlighted. The review also uncovered research gaps in how tools and practices address these security issues in CI/CD pipelines, revealing

a need for further study to improve cloud-based security solutions.

1 INTRODUCTION software to an environment cloud, while CDE takes it

further by managing updates (Humble and Farley, Cloud computing has become the go-to method for 2010). Automating these processes makes the process software deployment because it offers clear more efficient and improves software quality (Weber advantages over traditional setups. These include et al., 2016) while reducing risks (Bar et al., 2013). flexible infrastructure, accessible data storage and While automation helps in many ways, it also

sharing, less administrative hassle, and access from brings certain security risks. Vulnerabilities such as anywhere. Continuous Integration (CI), originating Regular Expression Denial of Service (ReDoS) from Extreme Programming (XP) (Newkirk, 2002), (Saboor et al., 2022) can open cloud services to is an Agile method where team members regularly attacks such as Log4j, SolarWinds, and CodeCov. integrate code changes, which results in faster Of the 5 articles we reviewed, 6 met our

production, better product quality, and a more selection criteria (see Section 3.3). These articles effective team overall (Fitzgerald and Stol, 2017). helped us explore the following research questions: Automation plays a crucial role in CI, especially RQ1. What tools and methods are available for in testing and development. It boosts efficiency, securely implementing CI/CD in the cloud? improves teamwork among developers, and leads to RQ2. What solutions have been suggested for

more predictable releases (Leppänen et al., 2015; maintaining secure CI/CD pipelines in cloud

Ståhl and Bosch, 2014; Fitzgerald and Stol, 2014). CI, environments?

along with Continuous Delivery (CDE) and RQ3. What are the main challenges when securing

Continuous Deployment (CD), are core parts of cloud-based CI/CD pipelines?

DevOps (Lacoste, 2009). CD is about deploying

a https://orcid.org/0000-0001-9944-26 b https://orcid.org/0009-0006-5207-32 c https://orcid.org/0009-0001-6572-63

This study reviewed the current tools (Section Shahin et al. (2021) analysed DevOps forums to 4.1), proposed solutions (Section 4.2), and challenges identify architecture design issues, noting that (Section 4.3) regarding secure CI/CD pipelines over deployment, security, and testing were the most the cloud platform. challenging during DevOps transitions.

To identify the challenges (Section 4.3) that Faustino et al. (2022) reviewed DevOps

prevent practitioners from adopting solutions, leading scenarios, noting faster delivery and increased to security vulnerabilities. automation. However, security issues have yet to be The rest of the paper is structured as follows: discussed. Section 2 looks at related work, including review Rajapakse et al. (2022) identified challenges and method and the possible research gaps (Sections 2.1 solutions for adopting DevSecOps, focusing on

and 2.2) identified from our RQ findings. Section 3 collaboration, insider threats, and limitations of explains the SLR method, covering RQs (Section SAST and Dynamic Application Security Testing 3.1), search strategy (Section 3.2), data sources (DAST) tools. They aimed to understand the (Section 3.3), inclusion/exclusion criteria (Section difficulties in adopting DevSecOps. 3.4), and the SLR steps (Figure 2), along with how we Shahin et al. (2019) proposed a framework to re-

extracted and synthesised the data (Section 3.5). architect CD with goals for Operational Aspects (e.g.,

Section 4 presents the results, demographic data development settings, stakeholders’ requirements)

(Figure 3), and findings for each RQ (Sections 4.1, and Quality Attributes (e.g., resilience, modifiability, 4.2, 4.3). We follow this with analysis and deployability, etc.). discussions in Section 5. Threats to validity are Shahin et al. (2017a) discussed issues in adopting covered in Section 6, and Section 7 wraps things up CI/CD/CDE, such as coordination, skills, and tools. with conclusions and future work. They also noted a need for more research on pipeline

Table 1 presents the area between our SLR and the

2 RELATED WORKS existing work.

Table 1: Summarising the Focused Area.

During our SLR, we identified literature reviews, survey papers, and systematic literature reviews. Publications Focused Areas

Shahin et al. 2017b Automation of CD

These addressed various aspects of CI/CD. Zhang et al. 20 IDE for SAST

Shahin et al. (2017b) surveyed CI/CD and Waseem et al. 2021, 20 Microservice

DevOps practitioners, highlighting deficiencies in Zampetti et al. 20 Collaboration of SW and HW

Shahin et al. 20 Architectural issues in DevOps

automated testing, rigid deployment methods, and Faustino et al. 20 Benefits of DevOps security awareness. They aimed to categorise Rajapakse et al. 20 Adoption of DevSecOps

Shahin et al. 20 Architectural issues in CD

elements influencing CD practice adoption, such as Shahin et al. 2017a Adoption of CI/CD/CDE better tools and management support. This SLR Security of CI/CD over the Cloud

Zhang et al. (2018) detailed practitioners'

struggles with containerising CD and identifying 2.1 Review Methodology prerequisites and challenges before establishing CI- based Workflow (CIW) and Docker Hub auto-builds In software engineering (SE), conducting multiple Workflow (DHW). They noted trade-offs in stability reviews on a single topic is common (Shahin et al., and simplicity and the need for better security and 2017a). Since the introduction of Evidence-Based access controls. An IDE model for cloud-based Static Software Engineering (Kitchenham et al., 2004,

Application Security Testing (SAST) tools was 2006, 2022a), systematic literature reviews (SLRs)

implemented but did not significantly enhance fixing have become a key research method (Zhang et al., insecure code. 2011). However, reviewing secure CI/CD in the cloud Waseem et al. (2021, 2023) discussed the security requires a more focused approach (Düllmann et al., vulnerabilities in microservices developed with 2018).

Docker that are open to cyberattacks and highlighted

the need to focus on pipeline security over the cloud. 2.2 Research Gaps

Zampetti et al. (2023) emphasised that combining

hardware and software expertise can overcome CI There is a growing need for research to improve and CDE implementation challenges in Cyber- security in containerised applications. This includes Physical Systems (CPS), focusing on SW and HW refining tools such as seccomp profiles for Docker, component interactions.

AppArmor, SELinux, and content trust (Garg and challenges (e.g., gaps, integration, performance, etc.)

Stavik, 2019; Le et al., 2023; Lopes et al., 2020). in maintaining CI/CD pipelines in the cloud. Low-code platforms present security challenges, mainly due to weak authentication and cybercrime 3.2 Search Strategy (Rafi et al., 2022). GitHub Actions has security concerns that require Specific search phrases were created to find relevant further study (Decan et al., 2022; Koishybayev et al., studies based on the guidelines from Zhang et al. 2022; Hilton et al., 2017; Benedetti et al., 2022a). (2011) and Kitchenham et al. (2022a). This task faced

Research into architectural challenges, such as challenges because many papers used synonyms like

deployment, security, and testing, is also important. "cloud security" and "cybersecurity." To enhance our

Principles like shift-left security, compliance with search, we employed snowballing (Wohlin, 2014) by

standards (OWASP, NIST), and zero-trust examining citations in the studies and conducted a architecture can make systems more resilient (Shahin manual search as recommended by Zhang et al. et al., 2017; Zhang et al., 2018; Shahin et al., 2021). (2011). This established a Quasi-Gold Standard

Finally, there is potential for new automated (QGS), identifying 9 relevant papers. The initial

Software Supply Chain (SSC) solutions to detect search string was: vulnerabilities and enhance the security of CI/CD pipelines (Enck and Williams, 2022; Byrne et al., 2020; Karl et al., 2022).

3 RESEARCH METHOD Figure 1: Search String of the initial search for SLR.

We conducted an SLR, which combines available 3.3 Data Collection Sources

research relevant to a focused area of interest and specific RQs. By following the guidelines of The automatic search was carried out across six Kitchenham, B. et al., 2022a, our research method digital libraries: Scopus, ACM, IEEE Xplore, Wiley, consists of planning, conducting, and reporting with Springer Link (SL), and ScienceDirect (SD) (Chen et the specification of the RQs, identifying research by al., 2010). generating a search strategy, selecting primary studies CiteSeerX and AIS eLibrary have complex search

through inclusion and exclusion criteria, and data functions and lack post-query refinements (Li & extraction and synthesis. Rainer, 2022; Brereton et al., 2007). Kluwer has merged with and is indexed by Springer Link

3.1 Goal, Question, Metric (GQM) (Gusenbauer and Haddaway, 2020; Maplesden et al.,

2015). Additionally, Inspec overlaps with Scopus The Goal of this SLR is to analyse and synthesise (Maplesden et al., 2015). In contrast, Google Scholar tools and approaches for securing CI/CD pipelines on yields results with less than 1% accuracy for cloud services, highlight the challenges of existing systematic searches (Gusenbauer and Haddaway, solutions, and answer the RQs. 2020; Chen et al., 2010; Boeker et al., 2013).

We prepared our RQs according to the criteria of

the PICOC by Mark, and Helen (2008) – Population 3.4 Study Selection Criteria (a deployment area, e.g., the cloud), Intervention (technologies to perform specific tasks, e.g., tools), We established inclusion and exclusion criteria to Comparison (with which the intervention is being identify studies relevant to our research questions, compared, e.g., the practitioners), Outcomes considering these criteria might be adjusted as we (findings, e.g., existing approaches, the challenges, moved through the search process (Staples and Niazi,

and the practices to the goal {secure the CI/CD 2007). pipeline over the cloud}), and Context (in which the Inclusion Criteria: analogy will take place, e.g., the industry). - Full-text (Brereton et al., 2007) peer-reviewed The identified Metrics for this SLR are: papers published in English. Identifying existing and proposed methods, - Address CI/CD security in the cloud. technologies, and practices for secure CI/CD - Empirical research (Kitchenham et al., 2022b).

maintenance. Classifying and enumerating security Exclusion Criteria: - Abstracts, conference info, news, and videos.

- Earlier versions of papers by the same authors appeared in conferences, with 4 papers (62.12%), when more recent versions are available (e.g., followed by 1 journal articles (22.73%) and 1 conference vs. journal publications). workshop papers (15.15%). - Duplicate studies from digital libraries.

3.5 Data Extraction and Synthesis

We read the full text of the selected papers for review

and reporting, applying the inclusion and exclusion criteria.

Figure 3: Demographic Data of Relevant Studies.

4.1 Findings of RQ

We present the tools, approaches, and frameworks

identified in our review with short descriptions (Table 2). We compiled information on 6 tools and eight distinct approaches and frameworks.

4.2 Findings of RQ

Here, we list the proposed tools and approaches with

Figure 2: Steps of the Study Selection for SLR. short descriptions (Table 3) retrieved from the papers.

We compiled information on five tools and twelve

We passed the subsequent steps for this SLR: approaches/frameworks. Step 1: We started with 4,8 articles based on the Some recommended practices (findings of RQ2) search criteria. for organisations to address CI/CD pipeline security Step 2: We screened the titles, keywords, and issues: abstracts to narrow it down to 5 papers. Of these, Trust developers: If they can make deployment

4 directly met our criteria, and an additional 9 decisions, it may facilitate the continuous deployment

were found using the Snowballing method. process (Shahin et al., 2017b).

Step 3: We reviewed the introductions and Increase collaboration between operations and

conclusions of the 5 papers, selecting those development teams: This may help complete complex relevant to our study. After thoroughly reviewing the tasks effectively (Shahin et al., 2021). full articles, 6 were included in our final selection. Invest in automated testing and quality assurance for continuous delivery (Shahin et al., 2017b).

Securing a software supply chain requires

4 RESULTS transparency, validity, and separation between

activities and components (Okafor et al., 2022).

This section summarises the research questions' Providing access to developers from tool builders

findings (sections 4.1, 4.2, 4.3) by synthesising and of Jenkins, CircleCI, TravisCI, etc., helps to provide analysing the extracted data. Figure 3 displays the better feedback (Hilton et al., 2017). publication demographics, showing that from 20 to Limiting the CI/CD access may protect the 2023, 4 of the 6 relevant papers (over 60%) were pipeline from tampering (Pecka et al., 2022). published, emphasising the recent focus on CI/CD A solid engineering culture can emphasise quality

security in the cloud. Most of these publications where employees can become experts (Dursun, 2023).

Table 2: Existing Methods (approaches and frameworks) and Tools.

Name Description Reference

Docker Bench for Security Tool for enforcing security best practices for Docker images/containers.

Garg and Stavik, 20

Docker Trusted Registry Secure storage and deployment of Docker images/containers.

CodeShip SaaS for logging CD workflow failures. Zhang et al., 20

Garg and Stavik, 2019; Brandy et al., 2020; Mahboob

CoreOSs Clair, OpenSCAP, Anchore

Vulnerability scanners using NVD and CVEs data. and Coffman, 2021; Throner et al., 2021; Nadgowda

Engine, Trivy

and Luan, 20 Abhishek and Rao, 2021; Athamnah M. et al., 2021; SonarQube, SonarCloud Tools for detecting security issues and maintaining code quality in CI/CD. Luo L. et al., 2021; Romero E. at al., 2022; Leite et al., 20 Scans dependencies to ensure trust in the Software Supply Chain (SSC) within Throner et al., 2021; Bass et al., 2015; Alfadel et al., Snyk

CI/CD. 20

Alfadel et al., 2023, Okafor, C et al., 2022, Pan, Z. et

CodeQL (Code Analysis Platform) An automation tool for identifying security vulnerabilities

al., 20 Super-Linter A repository with multiple linter tools Cankar et al., 2023, Chhillar and Sharma, 20

Mega-Linter Tool to analyse CI/CD consistency Cankar et al., 20

Prisma Compute (Twistlock), Prisma Container security tools for vulnerability scanning, runtime protection, and

Athamnah M. et al., 2021, Le et al., 20

Cloud, Aqua blocking unsafe builds. Analizo, Code Climate Source code analysers are used to identify vulnerabilities and bug risks. Prometheus, Zabbix, Nagios Incident management and monitoring tools. Leite et al., 20 Graylog, Logstash Log management tools for security and reliability. Splunk, DynaTrace, Dapper,

Monitoring tools for detecting and blocking security threats. Bennett and Barrett, 20

AppDynamics Veracode, LGTM, Checkmarx, CodeGuru Reviewer, FindBugs,

SAST tools to detect vulnerabilities early in SDLC. Luo et al., 20

IntelliJ, Coverity Scan

IBM UrbanCode Deploy, Microsoft ARA (Application Release Automation) tools for identifying bugs, memory

Révész and Pataki (2017, 2019)

Visual Studio Release Management leaks, and code smells. Debricked, NSP, Sonatype,

CI tools for scanning commits/PRs and automating vulnerability detection. Alfadel et al., 20

vuln-regex-detector Cijitter, CijScan CI tools for defending against cryptojacking. Alfadel et al., 2023, Li Z et al., 20 Garg and Stavik, 2019, Le et al., 2023, Lopes et al., AppArmor, SELinux Docker security tools for defence layers. Seccomp Restricts app access to ensure security. Le et al., 2023, Lopes et al., 20 Spire, Dependabot, tekton-chain,

DevSecOps solutions play a critical role in CI/CD security. Nadgowda and Luan, 20

Code Risk Analyzer, Mend

Chef OSS is used to configure and secure DevOps in the cloud. Alonso et al., 20

ART Autonomous real-time testing for CI/CD (DevTstOps). Fehlmann and Kranich (2021)

Asylo Development framework ensuring privacy through TEEs. Mahboob and Coffman, 20

STRIDE Microsoft’s threat modelling framework (Spoofing, Tampering, etc.). Davis et al., 20

Signature-based, Anomaly-based Approaches for monitoring containers and securing CI/CD pipelines. Jyothsna et al., 2011; Kumar and Sangwan, 20 Harbor Blocks deployment of unscanned Docker images. Mahboob and Coffman, 2021, Throner et al., 20

VirusTotal Scans Docker images for malicious content. Abhishek and Rao (2021)

GitHub Actions (GHA) Automates CI/CD and mitigates security risks. Okafor, C et al., 2022, Tu et al., 20

Table 3: Proposed Methods and Tools.

Name Description Reference

ACT Testbot Automated bot for continuous testing, defect analysis, reporting, and management in CI/CD

Chhillar and Sharma, 20

Automated Continuous Testing builds. UBCIS Benchmarks vulnerabilities in container scanning tools (e.g., Debian, Ubuntu, Alpine). Berkovich et al., 20 Scans GitHub Actions workflows for security weaknesses, auto-notifies for protection Koishybayev et al. 2022, Benedetti et al.,

GHAST, GWChecker

against SSC attacks. 2022a CIAnalyser Removes malicious code from OSS CI/CD scripts/pipelines. Pan, Z. et al., 20

Framework for preventing Docker image vulnerabilities, with scanners at each pipeline

Multi-layered security Brandy et al., 20 layer.

DIVA

Detects and evaluates security issues in Docker images. Shu et al., 20

Docker Image Vulnerability Analysis

CloudInspector Provides real-time, auditable security information in a CI/CD pipeline. Flittner et al., 20 Cluster-Scoped-CICD Kubernetes CI/CD pipeline with privacy guarantees using Asylo. Mahboob and Coffman, 20

Automated DevSecOps framework for addressing security risks with a defense-in-depth

ADOC Kumar and Goyal, 20 strategy.

DVE (Deliberated Vulnerable Stores and auto-processes exploited scripts and vulnerability data for cloud-native

Huang et al., 20

Environment) applications. Buildwatch Monitors pipeline dependencies to detect security risks. Ohm et al., 20 SUNSET Identifies and evaluates software supply chain security risks. Benedetti et al., 2022b

A call-aware container scheduler secures CI/CD by blocking unsafe builds and scanning for

SySched Le et al., 20 known CVEs. Tapiserí Visionary DevSecOps design for certification and introspection of a pipeline. Nadgowda and Luan, 20

Enhances pipeline security, transparency, traceability, and tamper-proofing through

Blockchain Technology Akbar et al., 2022, Bankar and Shah 20 blockchain. Supervised Learning Machine learning is used to automate tests in CI/CD to mitigate attacks. Drees et al., 20

4.3 Findings of RQ 3 GitHub environment, generating issues related to

privileges, permissions, and secrets (Koishybayev et Below, we report the challenges in existing tools and al., 2022; Hilton et al., 2017; Benedetti et al., 2022). approaches, including practices that raise security Existing DevSecOps Practices. Security issues issues within cloud-based CI/CD pipelines. related to encryption, image signing, and

Authorisation. Trusted Execution Environments vulnerability scanning remain in open-source

(TEE) can enhance security, but Dev resources may DevSecOps environments (Kumar and Goyal, 2020). be at risk if hackers can access Harbor (Mahboob and The SolarWinds incident showed that practices need

Coffman, 2021). Inadequate authorisation can result more standard recommendations (Nadgowda and

in pipeline security issues (Throner et al., 2021). Luan, 2021; Williams, 2022). This can lead to

Vulnerabilities Assessment. This happens pre- incomplete toolsets and compromised software

deployment, leaving post-deployment updates designs. unchecked and insecure (Huang et al., 2020). Due to Low-code Platforms. Integrating low-code the complexities of Infrastructure IaC, inspecting platforms such as PowerApps, AppSheet, and workflows for security flaws is challenging (Cankar KiSSFLOW in DevOps may introduce security issues et al., 2023; Alonso et al., 2022). (Rafi et al., 2022).

Tools Integration. Tools such as Clair, Software Supply Chain (SSC). The unified

SonarQube, GoKart, etc. should be rapidly integrated design of the CI server in a CD pipeline poses security into cloud platforms, though they require long-term challenges, as attackers can compromise the entire commitments (Garg and Stavik, 2019; Abhishek & system by altering one part (Throner et al., 2021; Bass Rao, 2021; Christakis et al., 2022). The disconnection et al., 2015; Ullah et al., 2017; Hilton et al., 2017). of tools such as Coverity Scan, LGTM, and Automated SSCs can propagate human errors, such

Checkmarx from IDEs can render scanning results as not updating vulnerable dependencies, leading to

obsolete if the code is updated during the scan (Luo et pipeline breaks, for example, the Log4j attack (Enck al., 2021). and Williams, 2022; Byrne et al., 2020; Williams,

Third-Party and OSS Tools. Choosing consistent 2022). Securing the build process is crucial since

tools is crucial due to vulnerabilities in third-party tools such as Tekton, Jenkins, GHA, Travis CI, and software and OSS (Kumar and Goyal, 2020; Berkovich AWS Code Deploy are widely used (Enck and et al., 2020). Integrating these tools faces challenges Williams, 2022; Karl et al., 2022). Failure to with security boundaries, upgrade complexities, and promptly update and address risks can result in practitioner reluctance to update, leading to outdated intrusions, such as the SolarWinds attacks

dependencies and security issues such as lack of (Nadgowda and Luan, 2021; Williams, 2022). authentication (Zampetti et al., 2023, Pan et al., 2023; Zhu et al., 2023; Benedetti et al., 2022b).

Layer of Defence. Regular updates are essential 5 ANALYSIS AND DISCUSSIONS

(e.g., for Seccomp) to prevent DoS attacks, but determining necessary updates is complex and time- The provided list (RQ1) encompasses a diverse range consuming, hindering practitioner approval (Lopes et of tools and technologies to enhance the security al., 2020). posture of CI/CD pipelines, primarily focusing on

Architectural Design Issues. Deployment, Docker-based cloud environments. This includes

security, and testing are challenging (Shahin et al., security scanning tools, automated testing 2017). Developers and customers have concerns frameworks, monitoring solutions, and vulnerability about existing tools and need help with cloud assessment and remediation tools, contributing to a deployment (Shahin et al., 2021). To address robust and secure software development lifecycle. developers' pain points, better testing support and Integrating these tools and technologies within

automatic security upgrades in CD workflows are CI/CD pipelines significantly enhances security by required (Zhang et al., 2018). addressing vulnerabilities, ensuring code quality, and

GitHub Actions (GHA). While GHA can proactively monitoring and responding to security

potentially reduce CI/CD pipeline security issues by threats. For instance, tools such as Docker Bench for recommending specific commits, it faces low Security and SonarQube help identify and rectify adoption and has security concerns such as PR security issues early in development. Meanwhile, manipulation and bypassing code reviews (Decan et monitoring tools such as Prometheus and Nagios al., 2022; Saroar & Nayebi, 2023; Benedetti et al., provide real-time insights into the deployed

2022). GitHub CI combines CI workflows with the applications' operational status and security posture.

The proposed (RQ2) tools and practices aim to technologies (RQ1) for maintaining security in the bolster CI/CD pipeline security. Tools cover code CI/CD pipeline over the cloud platforms. analysis, dependency scanning, and runtime To keep up with the continually updating protection, while practices emphasise collaboration, environment, practitioners and researchers should automated testing, and secure software supply chains. stay updated on the latest advancements where future

Implementing these measures may enhance security, research is needed. streamline processes, and mitigate risks in CI/CD We have uncovered various tools, frameworks, pipelines; however, accurate tests are needed on and practices (RQ2) proposed by researchers to cloud platforms. fortify security in the CI/CD pipeline. With cloud

The excerpt (RQ3) provides a comprehensive platforms ubiquitous, these findings suggest

overview of the security challenges inherent in cloud- significant insights for practitioners and future

based CI/CD pipelines, summarised below: - researchers aiming to stay at the cutting edge of  Installation and updating issues, secure DevOps practices.  Practitioners and developers’ issues, Finally, we have reported the challenges and  Organisational issues, issues that arise when dealing with security  Difficulties with third-party and OSS tools. considerations in cloud-based CI/CD pipelines

(RQ3). These issues involved container vulnerability, lack of integration between security tools and IDEs,

6 THREATS TO VALIDITY and dependency on third-party software and OSS

tools. Close cooperation between practitioners,

In our systematic literature review (SLR), we security specialists, and researchers is needed to

mitigate the research gaps. identified potential threats to validity across several

We aim to apply Topic Modeling (an

areas, including search strategy, data collection, study unsupervised ML technique (Sefara and Rangata, selection, and synthesis. We conducted automated searches using diverse terminology to accommodate 2023) that uses Natural Language Processing) various taxonomies, though some digital libraries methods such as Latent Semantic Analysis (LSA), were excluded due to complex search strings and Probabilistic Latent Semantic Analysis (pLSA), and

Latent Dirichlet Allocation (LDA) effectively applied

irrelevant results. Our study selection process adhered to analyse scattered and fragmented security-related to established guidelines from Zhang et al. (2011), Kitchenham et al. (2022), Brereton et al. (2007), and text data (for example, plain text, lack of integration,

Wohlin (2014). disorganised contents, lack of contexts such as partial

incident reports, truncated logs, or isolated pieces of Based on Runeson and Höst’s (2009) framework, information, etc. which can be derived from grey we identified the following threats: literature, and the industries).

Internal Validity: Potential data extraction errors

We also aim to propose a blockchain-based

were mitigated by thorough double-checking. External Validity: Strict criteria may have led to a solution (Akbar et al., 2022; Bankar and Shah, 2020) higher exclusion rate, potentially introducing (an advanced database mechanism for maintaining selection bias, but they were essential for relevance. data privacy) for addressing the insufficient container security (for example, beyond 80% of Docker hub

Comprehensive search techniques helped minimise

the risk of missing significant studies. images contain one high level of vulnerability

Construct Validity: Standardization efforts discovered by researchers after scanning 300,0

addressed inconsistencies in study definitions. images in 85,0 repositories) (Zhang et al., 2018;

Reliability: Variability in study design and quality Shu et al., 2017), insecure deployment environments

was a concern, though we aimed to include a diverse (such as updating vulnerable dependencies, a human range of studies to reduce the impact of publication error which leads to cyberattacks such as Log4j, bias. SolarWinds, CodeCov etc.) (Benedetti et al., 2022b;

Enck and Williams, 2022; Byrne et al., 2020; Karl et

al., 2022), etc. Before this, we also aim to conduct a literature review on blockchain-based solutions for

7 CONCLUSIONS AND FUTURE securing the CI/CD pipeline.

WORK In conclusion, this SLR gave us an understanding

of CI/CD security and plans for future works,

Our systematic literature review provided valuable combining methodologies and technologies to fortify

insights into the existing methods, tools, and

the foundations of secure software integration and Bennett, B. T., & Barrett, M. L. (2018). Incorporating deployment in cloud platforms. devops into undergraduate software engineering courses: A suggested framework. Journal of Computing Sciences in Colleges, 34(2), 180-187. Berkovich, S., Kam, J., & Wurster, G. (2020). {UBCIS}:

REFERENCES Ultimate benchmark for container image scanning. In

13th USENIX Workshop on Cyber Security Á. Révész, and N. Pataki, “Containerized A/B Testing,” Experimentation and Test (CSET 20). Proc. of the Sixth Workshop on Software Quality Billawa, P., Bambhore Tukaram, A., Díaz Ferreyra, N. E., Analysis, Monitoring, Improvement, and Applications Steghöfer, J. P., Scandariato, R., & Simhandl, G. (2022, (Belgrade, Serbia, September 11-13, 2017) August). Sok: Security of microservice applications: A

SQAMIA’17, 2017, pp. 14(1)-14(8). practitioners’ perspective on challenges and best

Abhishek, M. K., & Rao, D. R. (2021, July). Framework to practices. In Proceedings of the 17th International secure docker containers. In 20 Fifth World Conference on Availability, Reliability and Security Conference on Smart Trends in Systems Security and (pp. 1-10).

Sustainability (WorldS4) (pp. 152-156). IEEE. Boeker, M., Vach, W., & Motschall, E. (2013). Google

Akbar, M. A., Mahmood, S., & Siemon, D. (2022, June). Scholar as replacement for systematic literature

Toward effective and efficient DevOps using searches: good relative recall and precision are not

blockchain. In Proceedings of the 26th International enough. BMC Medical Research Methodology, 13, 1- Conference on Evaluation and Assessment in Software 12. Engineering (pp. 421-427). Brady, K., Moon, S., Nguyen, T., & Coffman, J. (2020, Alfadel, M., Nagy, N. A., Costa, D. E., Abdalkareem, R., & January). Docker container security in cloud

Shihab, E. (2023). Empirical analysis of security- computing. In 20 10th Annual Computing and

related code reviews in npm packages. Journal of Communication Workshop and Conference (CCWC) Systems and Software, 203, 111752. (pp. 0975-0980). IEEE. Alonso, J., Piliszek, R., & Cankar, M. (2022). Embracing Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M.,

IaC through the DevSecOps philosophy: Concepts, & Khalil, M. (2007). Lessons from applying the

challenges, and a reference framework. IEEE Software, systematic literature review process within the software 40(1), 56-62. engineering domain. Journal of systems and software, Athamnah, M., Hussain, M. F., & Hasan, S. S. (2021, 80(4), 571-583. November). Impact of Running Dynamic/Static Scans Byrne, A., Nadgowda, S., & Coskun, A. K. (2020, on the Performance of an App Running in a GKE December). Ace: Just-in-time serverless software

Clusters. In 20 Second International Conference on component discovery through approximate concrete

Intelligent Data Science Technologies and Applications execution. In Proceedings of the 20 Sixth

(IDSTA) (pp. 46-53). IEEE. International Workshop on Serverless Computing (pp. Bankar, S., & Shah, D. (2020, November). DevOps project 37-42). artifacts management using blockchain technology. In Cankar, M., Petrovic, N., Pita Costa, J., Cernivec, A., Antic,

ECAI&ML international conference (pp. 115-120). J., Martincic, T., & Stepec, D. (2023, April). Security

Bar, P., Benfredj, R., Marks, J., Ulevinov, D., Wozniak, B., in DevSecOps: Applying Tools and Machine Learning Casale, G., & Knottenbelt, W. J. (2013, April). Towards to Verification and Monitoring Steps. In Companion of a monitoring feedback loop for cloud applications. In the 20 ACM/SPEC International Conference on Proceedings of the 20 international workshop on Performance Engineering (pp. 201-205). Multi-cloud applications and federated clouds (pp. 43- Chen, L., Babar, M. A., & Zhang, H. (2010, April).

44). Towards an evidence-based understanding of electronic Bass, L., Holz, R., Rimba, P., Tran, A. B., & Zhu, L. (2015, data sources. At the 14th International Conference on

May). Securing a deployment pipeline. In 20 Evaluation and Assessment in Software Engineering

IEEE/ACM 3rd International Workshop on Release (EASE), BCS Learning & Development.

Engineering (pp. 4-7). IEEE. Chhillar, D., & Sharma, K. (2019, February). ACT Testbot

Benedetti, G., Verderame, L., & Merlo, A. (2022, and 4S Quality Metrics in XAAS Framework. In 20

November). Automatic security assessment of github International Conference on Machine Learning, Big

actions workflows. In Proceedings of the 20 ACM Data, Cloud and Parallel Computing (COMITCon) (pp. Workshop on Software Supply Chain Offensive 503-509). IEEE. Research and Ecosystem Defenses (pp. 37-45). Christakis, M., Cottenier, T., Filieri, A., Luo, L., Mansur, Benedetti, G., Verderame, L., & Merlo, A. (2022, M. N., Pike, L., ... & Visser, W. (2022, November).

September). Alice in (software supply) chains: risk Input splitting for cloud-based static application

identification and evaluation. In International security testing platforms. In Proceedings of the 30th

Conference on the Quality of Information and ACM Joint European Software Engineering

Communications Technology (pp. 281-295). Cham: Conference and Symposium on the Foundations of

Springer International Publishing. Software Engineering (pp. 1367-1378).

Davis, J. C., Amusuo, P., & Bushagour, J. R. (2022, May). Garg, S., & Garg, S. (2019, March). Automated cloud

A first offering of software engineering. In Proceedings infrastructure, continuous integration and continuous

of the First International Workshop on Designing and delivery using docker with robust container security. In

Running Project-Based Courses in Software 20 IEEE Conference on Multimedia Information

Engineering Education (pp. 5-9). Processing and Retrieval (MIPR) (pp. 467-470). IEEE. Decan, A., Mens, T., Mazrae, P. R., & Golzadeh, M. (2022, Gruhn, V., Hannebauer, C., & John, C. (2013, August). October). On the use of github actions in software Security of public continuous integration services. In development repositories. In 20 IEEE International Proceedings of the 9th International Symposium on Conference on Software Maintenance and Evolution open collaboration (pp. 1-10).

(ICSME) (pp. 235-245). IEEE. Gusenbauer, M., & Haddaway, N. R. (2020). Which Drees, J. P., Gupta, P., Hüllermeier, E., Jager, T., Konze, academic search systems are suitable for systematic A., Priesterjahn, C., ... & Somorovsky, J. (2021, reviews or meta ‐ analyses? Evaluating retrieval

November). Automated detection of side channels in

qualities of Google Scholar, PubMed, and 2 other cryptographic protocols: DROWN the ROBOTs!. In resources. Research synthesis methods, 11(2), 181-217.

Proceedings of the 14th ACM Workshop on Artificial

Hilton, M., Nelson, N., Tunnell, T., Marinov, D., & Dig, D. Intelligence and Security (pp. 169-180). (2017, August). Trade-offs in continuous integration: Düllmann, T. F., Paule, C., & van Hoorn, A. (2018, May). assurance, security, and flexibility. In Proceedings of

Exploiting devops practices for dependable and secure

the 20 11th Joint Meeting on Foundations of continuous delivery pipelines. In Proceedings of the 4th Software Engineering (pp. 197-207).

International Workshop on Rapid Continuous Software

Huang, M., Fan, W., Huang, W., Cheng, Y., & Xiao, H. Engineering (pp. 27-30). (2020, June). Research on building exploitable

Dursun, H. (2023, June). Full Spec Software via Platform

vulnerability database for cloud-native app. In 20

Engineering: Transition from Bolting-on to Building-

IEEE 4th Information Technology, Networking, in. In Proceedings of the 27th International Conference

Electronic and Automation Control Conference

on Evaluation and Assessment in Software Engineering (ITNEC) (Vol. 1, pp. 758-762). IEEE. (pp. 172-175). Hudic, A., Flittner, M., Lorünser, T., Radl, P. M., & Bless,

El Khairi, A., Caselli, M., Knierim, C., Peter, A., &

R. (2016, August). Towards a unified secure cloud

Continella, A. (2022, November). Contextualizing

service development and deployment life-cycle. In system calls in containers for anomaly-based intrusion 20 11th International Conference on Availability, detection. In Proceedings of the 20 on Cloud Reliability and Security (ARES) (pp. 428-436). IEEE. Computing Security Workshop (pp. 9-21). Humble, J., & Farley, D. (2010). Continuous delivery:

Enck, W., & Williams, L. (2022). Top five challenges in

reliable software releases through build, test, and software supply chain security: Observations from 3 deployment automation. Pearson Education.. industry and government organizations. IEEE Security

Jamshidi, P., Pahl, C., Mendonça, N. C., Lewis, J., &

Tilkov, S. (2018). Microservices: The journey so far

Faustino, J., Adriano, D., Amaro, R., Pereira, R., & da

and challenges ahead. IEEE Software, 35(3), 24-35.

Silva, M. M. (2022). DevOps benefits: A systematic

Kang, H., Le, M., & Tao, S. (2016, April). Container and

literature review. Software: Practice and Experience, microservice driven design for cloud infrastructure 52(9), 1905-1926. devops. In 20 IEEE International Conference on Fehlmann, T., & Kranich, E. (2021). ART for Agile: Cloud Engineering (IC2E) (pp. 202-211). IEEE.

Autonomous Real-Time Testing in the Product

Karl, M., Musch, M., Ma, G., Johns, M., & Lekies, S.

Development Cycle. In Systems, Software and Services

(2022, October). No keys to the kingdom required: a Process Improvement: 28th European Conference, comprehensive investigation of missing authentication EuroSPI 2021, Krems, Austria, September 1–3, 2021, vulnerabilities in the wild. In Proceedings of the 22nd

Proceedings 2 (pp. 377-390). Springer International

ACM Internet Measurement Conference (pp. 619-632). Publishing.

Kitchenham, B. (2004). Procedures for performing

Fitzgerald, B., & Stol, K. J. (2014, June). Continuous

systematic reviews. Keele, UK, Keele University, software engineering and beyond: trends and 33(2004), 1-26. challenges. In Proceedings of the 1st International

Kitchenham, B. (2006). Evidence-based software

Workshop on rapid continuous software engineering

engineering and systematic literature reviews. In (pp. 1-9).

Product-Focused Software Process Improvement: 7th

Fitzgerald, B., & Stol, K. J. (2017). Continuous software

International Conference, PROFES 2006, Amsterdam, engineering: A roadmap and agenda. Journal of The Netherlands, June 12-14, 2006. Proceedings 7 (pp. Systems and Software, 123, 176-189. 3-3). Springer Berlin Heidelberg. Flittner, M., Balaban, S., & Bless, R. (2016, April). Kitchenham, B. A., Dyba, T., & Jorgensen, M. (2004,

Cloudinspector: A transparency-as-a-service solution

May). Evidence-based software engineering. In

for legal issues in cloud computing. In 20 IEEE

Proceedings. 26th International Conference on

International Conference on Cloud Engineering

Software Engineering (pp. 273-281). IEEE. Workshop (IC2EW) (pp. 94-99). IEEE.

Kitchenham, B., Madeyski, L., & Budgen, D. (2022). How

should software engineering secondary studies include

grey material?. IEEE Transactions on Software Newkirk, J. (2002, May). Introduction to agile processes

Engineering, 49(2), 872-882. and extreme programming. In Proceedings of the 24th

Kitchenham, B., Madeyski, L., & Budgen, D. (2022). international conference on Software engineering (pp. SEGRESS: Software engineering guidelines for 695-696). reporting secondary studies. IEEE Transactions on Ohm, M., Sykosch, A., & Meier, M. (2020, August).

Software Engineering, 49(3), 1273-1298. Towards detection of software supply chain attacks by

Koishybayev, I., Nahapetyan, A., Zachariah, R., Muralee, forensic artifacts. In Proceedings of the 15th

S., Reaves, B., Kapravelos, A., & Machiry, A. (2022). international conference on availability, reliability and Characterizing the security of github {CI} workflows. security (pp. 1-6). In 31st USENIX Security Symposium (USENIX Okafor, C., Schorlemmer, T. R., Torres-Arias, S., & Davis,

Security 22) (pp. 2747-2763). J. C. (2022, November). Sok: Analysis of software

Kumar, R., & Goyal, R. (2020). Modeling continuous supply chain security by establishing secure design security: A conceptual model for automated properties. In Proceedings of the 20 ACM Workshop

DevSecOps using open-source software over cloud on Software Supply Chain Offensive Research and

(ADOC). Computers & Security, 97, 101967. Ecosystem Defenses (pp. 15-24). Lacoste, F. J. (2009, August). Killing the gatekeeper: Pan, Z., Shen, W., Wang, X., Yang, Y., Chang, R., Liu, Y., Introducing a continuous integration system. In 20 ... & Ren, K. (2023). Ambush From All Sides: agile conference (pp. 387-392). IEEE. Understanding Security Threats in Open-Source Le, M. V., Ahmed, S., Williams, D., & Jamjoom, H. (2023, Software CI/CD Pipelines. IEEE Transactions on

July). Securing container-based clouds with syscall- Dependable and Secure Computing, 21(1), 403-418. aware scheduling. In Proceedings of the 20 ACM Pashchenko, I., Scandariato, R., Sabetta, A., & Massacci, F. Asia Conference on Computer and Communications (2021, May). Secure software development in the era of

Security (pp. 812-826). fluid multi-party open software and services. In 20

Leite, L., Rocha, C., Kon, F., Milojicic, D., & Meirelles, P. IEEE/ACM 43rd International Conference on Software (2019). A survey of DevOps concepts and challenges. Engineering: New Ideas and Emerging Results (ICSE- ACM Computing Surveys (CSUR), 52(6), 1-35. NIER) (pp. 91-95). IEEE. Leppänen, M., Mäkinen, S., Pagels, M., Eloranta, V. P., Pecka, N., Ben Othmane, L., & Valani, A. (2022, May). Itkonen, J., Mäntylä, M. V., & Männistö, T. (2015). The Privilege escalation attack scenarios on the devops

highways and country roads to continuous deployment. pipeline within a kubernetes environment. In

Ieee software, 32(2), 64-72. Proceedings of the International Conference on

Li, Z., & Rainer, A. (2022, November). Academic search Software and System Processes and International engines: constraints, bugs, and recommendations. In Conference on Global Software Engineering (pp. 45- Proceedings of the 13th International Workshop on 49). Automating Test Case Design, Selection and Petticrew, M., & Roberts, H. (2008). Systematic reviews in

Evaluation (pp. 25-32). the social sciences: A practical guide. John Wiley &

Lopes, N., Martins, R., Correia, M. E., Serrano, S., & Sons. Nunes, F. (2020, December). Container hardening Rafi, S., Akbar, M. A., Sánchez-Gordón, M., & Colomo- through automated seccomp profiling. In Proceedings Palacios, R. (2022, September). Devops practitioners’ of the 20 6th International Workshop on Container perceptions of the low-code trend. In Proceedings of the

Technologies and Container Clouds (pp. 31-36). 16th ACM/IEEE International Symposium on

Luo, L., Schäf, M., Sanchez, D., & Bodden, E. (2021, Empirical Software Engineering and Measurement (pp. August). Ide support for cloud-based static analyses. In 301-306). Proceedings of the 29th ACM Joint meeting on Rajapakse, R. N., Zahedi, M., Babar, M. A., & Shen, H. european software engineering conference and (2022). Challenges and solutions when adopting symposium on the foundations of software engineering DevSecOps: A systematic review. Information and

(pp. 1178-1189). software technology, 141, 106700. Mahboob, J., & Coffman, J. (2021, January). A kubernetes Révész, Á., & Pataki, N. (2019, March). Continuous A/B ci/cd pipeline with asylo as a trusted execution testing in containers. In Proceedings of the 20 2nd environment abstraction framework. In 20 IEEE 11th International Conference on Geoinformatics and Data Annual Computing and Communication Workshop and Analysis (pp. 11-14).

Conference (CCWC) (pp. 0529-0535). IEEE. Romero, E. E., Camacho, C. D., Montenegro, C. E., Acosta, Maplesden, D., Tempero, E., Hosking, J., & Grundy, J. C. Ó. E., Crespo, R. G., Gaona, E. E., & Martínez, M. H. (2015). Performance analysis for object-oriented (2022). Integration of DevOps practices on a noise software: A systematic mapping. IEEE Transactions on monitor system with CircleCI and Terraform. ACM

Software Engineering, 41(7), 691-710. Transactions on Management Information Systems

Nadgowda, S., & Luan, L. (2021, December). tapiserí: (TMIS), 13(4), 1-24. Blueprint to modernize DevSecOps for real world. In Runeson, P., & Höst, M. (2009). Guidelines for conducting

Proceedings of the Seventh International Workshop on and reporting case study research in software

Container Technologies and Container Clouds (pp. 13- engineering. Empirical software engineering, 14, 131-

18). 164.

Saboor, A., Hassan, M. F., Akbar, R., Susanto, E., Shah, S. International Conference on Service-Oriented System N. M., Siddiqui, M. A., & Magsi, S. A. (2022). Root- Engineering (SOSE) (pp. 134-143). IEEE. Of-Trust for Continuous Integration and Continuous Torkura, K. A., Sukmana, M. I., & Meinel, C. (2017,

Deployment Pipeline in Cloud Computing. Computers, December). Integr

Continued Discussion and Extended Analysis (Part 2)

A Systematic Literature Review on Continuous Integration and

Deployment (CI/CD) for Secure Cloud Computing

Sabbir M. Saleh a

, Nazim Madhavji b and John Steinbacher c

Department of Computer Science, University of Western Ontario, London, Ontario, Canada

IBM Canada Lab, Markham, Ontario, Canada

[email protected], [email protected], [email protected]

Keywords: Continuous Integration, Continuous Deployment, CI/CD, Cloud, Security, Systematic Literature Review.

Abstract: As cloud environments become widespread, cybersecurity has emerged as a top priority across areas such as

Software Engineering, which includes continuous integration (CI), delivery (CDE), and deployment (CD), is

a need for further study to improve cloud-based security solutions.

1 INTRODUCTION software to an environment cloud, while CDE takes it

more predictable releases (Leppänen et al., 2015; maintaining secure CI/CD pipelines in cloud

Ståhl and Bosch, 2014; Fitzgerald and Stol, 2014). CI, environments?

along with Continuous Delivery (CDE) and RQ3. What are the main challenges when securing

Continuous Deployment (CD), are core parts of cloud-based CI/CD pipelines?

DevOps (Lacoste, 2009). CD is about deploying

a https://orcid.org/0000-0001-9944-26 b https://orcid.org/0009-0006-5207-32 c https://orcid.org/0009-0001-6572-63

To identify the challenges (Section 4.3) that Faustino et al. (2022) reviewed DevOps

extracted and synthesised the data (Section 3.5). architect CD with goals for Operational Aspects (e.g.,

Section 4 presents the results, demographic data development settings, stakeholders’ requirements)

Table 1 presents the area between our SLR and the

2 RELATED WORKS existing work.

Table 1: Summarising the Focused Area.

During our SLR, we identified literature reviews, survey papers, and systematic literature reviews. Publications Focused Areas

Shahin et al. 2017b Automation of CD

These addressed various aspects of CI/CD. Zhang et al. 20 IDE for SAST

Shahin et al. (2017b) surveyed CI/CD and Waseem et al. 2021, 20 Microservice

DevOps practitioners, highlighting deficiencies in Zampetti et al. 20 Collaboration of SW and HW

Shahin et al. 20 Architectural issues in DevOps

automated testing, rigid deployment methods, and Faustino et al. 20 Benefits of DevOps security awareness. They aimed to categorise Rajapakse et al. 20 Adoption of DevSecOps

Shahin et al. 20 Architectural issues in CD

elements influencing CD practice adoption, such as Shahin et al. 2017a Adoption of CI/CD/CDE better tools and management support. This SLR Security of CI/CD over the Cloud

Zhang et al. (2018) detailed practitioners'

Application Security Testing (SAST) tools was 2006, 2022a), systematic literature reviews (SLRs)

Docker that are open to cyberattacks and highlighted

the need to focus on pipeline security over the cloud. 2.2 Research Gaps

Zampetti et al. (2023) emphasised that combining

AppArmor, SELinux, and content trust (Garg and challenges (e.g., gaps, integration, performance, etc.)

Research into architectural challenges, such as challenges because many papers used synonyms like

deployment, security, and testing, is also important. "cloud security" and "cybersecurity." To enhance our

Principles like shift-left security, compliance with search, we employed snowballing (Wohlin, 2014) by

Finally, there is potential for new automated (QGS), identifying 9 relevant papers. The initial

Software Supply Chain (SSC) solutions to detect search string was: vulnerabilities and enhance the security of CI/CD pipelines (Enck and Williams, 2022; Byrne et al., 2020; Karl et al., 2022).

3 RESEARCH METHOD Figure 1: Search String of the initial search for SLR.

We conducted an SLR, which combines available 3.3 Data Collection Sources

3.1 Goal, Question, Metric (GQM) (Gusenbauer and Haddaway, 2020; Maplesden et al.,

We prepared our RQs according to the criteria of

maintenance. Classifying and enumerating security Exclusion Criteria: - Abstracts, conference info, news, and videos.

3.5 Data Extraction and Synthesis

We read the full text of the selected papers for review

and reporting, applying the inclusion and exclusion criteria.

Figure 3: Demographic Data of Relevant Studies.

4.1 Findings of RQ

We present the tools, approaches, and frameworks

identified in our review with short descriptions (Table 2). We compiled information on 6 tools and eight distinct approaches and frameworks.

4.2 Findings of RQ

Here, we list the proposed tools and approaches with

Figure 2: Steps of the Study Selection for SLR. short descriptions (Table 3) retrieved from the papers.

We compiled information on five tools and twelve

4 directly met our criteria, and an additional 9 decisions, it may facilitate the continuous deployment

were found using the Snowballing method. process (Shahin et al., 2017b).

Step 3: We reviewed the introductions and Increase collaboration between operations and

Securing a software supply chain requires

4 RESULTS transparency, validity, and separation between

activities and components (Okafor et al., 2022).

This section summarises the research questions' Providing access to developers from tool builders

security in the cloud. Most of these publications where employees can become experts (Dursun, 2023).

Table 2: Existing Methods (approaches and frameworks) and Tools.

Name Description Reference

Docker Bench for Security Tool for enforcing security best practices for Docker images/containers.

Garg and Stavik, 20

Docker Trusted Registry Secure storage and deployment of Docker images/containers.

CodeShip SaaS for logging CD workflow failures. Zhang et al., 20

Garg and Stavik, 2019; Brandy et al., 2020; Mahboob

CoreOSs Clair, OpenSCAP, Anchore

Vulnerability scanners using NVD and CVEs data. and Coffman, 2021; Throner et al., 2021; Nadgowda

Engine, Trivy

CI/CD. 20

Alfadel et al., 2023, Okafor, C et al., 2022, Pan, Z. et

CodeQL (Code Analysis Platform) An automation tool for identifying security vulnerabilities

al., 20 Super-Linter A repository with multiple linter tools Cankar et al., 2023, Chhillar and Sharma, 20

Mega-Linter Tool to analyse CI/CD consistency Cankar et al., 20

Prisma Compute (Twistlock), Prisma Container security tools for vulnerability scanning, runtime protection, and

Athamnah M. et al., 2021, Le et al., 20

Monitoring tools for detecting and blocking security threats. Bennett and Barrett, 20

AppDynamics Veracode, LGTM, Checkmarx, CodeGuru Reviewer, FindBugs,

SAST tools to detect vulnerabilities early in SDLC. Luo et al., 20

IntelliJ, Coverity Scan

IBM UrbanCode Deploy, Microsoft ARA (Application Release Automation) tools for identifying bugs, memory

Révész and Pataki (2017, 2019)

Visual Studio Release Management leaks, and code smells. Debricked, NSP, Sonatype,

CI tools for scanning commits/PRs and automating vulnerability detection. Alfadel et al., 20

DevSecOps solutions play a critical role in CI/CD security. Nadgowda and Luan, 20

Code Risk Analyzer, Mend

Chef OSS is used to configure and secure DevOps in the cloud. Alonso et al., 20

ART Autonomous real-time testing for CI/CD (DevTstOps). Fehlmann and Kranich (2021)

Asylo Development framework ensuring privacy through TEEs. Mahboob and Coffman, 20

STRIDE Microsoft’s threat modelling framework (Spoofing, Tampering, etc.). Davis et al., 20

VirusTotal Scans Docker images for malicious content. Abhishek and Rao (2021)

GitHub Actions (GHA) Automates CI/CD and mitigates security risks. Okafor, C et al., 2022, Tu et al., 20

Table 3: Proposed Methods and Tools.

Name Description Reference

ACT Testbot Automated bot for continuous testing, defect analysis, reporting, and management in CI/CD

Chhillar and Sharma, 20

GHAST, GWChecker

against SSC attacks. 2022a CIAnalyser Removes malicious code from OSS CI/CD scripts/pipelines. Pan, Z. et al., 20

Framework for preventing Docker image vulnerabilities, with scanners at each pipeline

Multi-layered security Brandy et al., 20 layer.

DIVA

Detects and evaluates security issues in Docker images. Shu et al., 20

Docker Image Vulnerability Analysis

Automated DevSecOps framework for addressing security risks with a defense-in-depth

ADOC Kumar and Goyal, 20 strategy.

DVE (Deliberated Vulnerable Stores and auto-processes exploited scripts and vulnerability data for cloud-native

Huang et al., 20

A call-aware container scheduler secures CI/CD by blocking unsafe builds and scanning for

SySched Le et al., 20 known CVEs. Tapiserí Visionary DevSecOps design for certification and introspection of a pipeline. Nadgowda and Luan, 20

Enhances pipeline security, transparency, traceability, and tamper-proofing through

Blockchain Technology Akbar et al., 2022, Bankar and Shah 20 blockchain. Supervised Learning Machine learning is used to automate tests in CI/CD to mitigate attacks. Drees et al., 20

4.3 Findings of RQ 3 GitHub environment, generating issues related to

Authorisation. Trusted Execution Environments vulnerability scanning remain in open-source

Coffman, 2021). Inadequate authorisation can result more standard recommendations (Nadgowda and

in pipeline security issues (Throner et al., 2021). Luan, 2021; Williams, 2022). This can lead to

Vulnerabilities Assessment. This happens pre- incomplete toolsets and compromised software

Tools Integration. Tools such as Clair, Software Supply Chain (SSC). The unified

Checkmarx from IDEs can render scanning results as not updating vulnerable dependencies, leading to

obsolete if the code is updated during the scan (Luo et pipeline breaks, for example, the Log4j attack (Enck al., 2021). and Williams, 2022; Byrne et al., 2020; Williams,

Third-Party and OSS Tools. Choosing consistent 2022). Securing the build process is crucial since

dependencies and security issues such as lack of (Nadgowda and Luan, 2021; Williams, 2022). authentication (Zampetti et al., 2023, Pan et al., 2023; Zhu et al., 2023; Benedetti et al., 2022b).

Layer of Defence. Regular updates are essential 5 ANALYSIS AND DISCUSSIONS

Architectural Design Issues. Deployment, Docker-based cloud environments. This includes

automatic security upgrades in CD workflows are CI/CD pipelines significantly enhances security by required (Zhang et al., 2018). addressing vulnerabilities, ensuring code quality, and

GitHub Actions (GHA). While GHA can proactively monitoring and responding to security

2022). GitHub CI combines CI workflows with the applications' operational status and security posture.

The excerpt (RQ3) provides a comprehensive platforms ubiquitous, these findings suggest

overview of the security challenges inherent in cloud- significant insights for practitioners and future

(RQ3). These issues involved container vulnerability, lack of integration between security tools and IDEs,

6 THREATS TO VALIDITY and dependency on third-party software and OSS

tools. Close cooperation between practitioners,

In our systematic literature review (SLR), we security specialists, and researchers is needed to

mitigate the research gaps. identified potential threats to validity across several

We aim to apply Topic Modeling (an

Latent Dirichlet Allocation (LDA) effectively applied

Wohlin (2014). disorganised contents, lack of contexts such as partial

Internal Validity: Potential data extraction errors

We also aim to propose a blockchain-based

Comprehensive search techniques helped minimise

the risk of missing significant studies. images contain one high level of vulnerability

Construct Validity: Standardization efforts discovered by researchers after scanning 300,0

addressed inconsistencies in study definitions. images in 85,0 repositories) (Zhang et al., 2018;

Reliability: Variability in study design and quality Shu et al., 2017), insecure deployment environments

Enck and Williams, 2022; Byrne et al., 2020; Karl et

al., 2022), etc. Before this, we also aim to conduct a literature review on blockchain-based solutions for

7 CONCLUSIONS AND FUTURE securing the CI/CD pipeline.

WORK In conclusion, this SLR gave us an understanding

of CI/CD security and plans for future works,

Our systematic literature review provided valuable combining methodologies and technologies to fortify

insights into the existing methods, tools, and

REFERENCES Ultimate benchmark for container image scanning. In

SQAMIA’17, 2017, pp. 14(1)-14(8). practitioners’ perspective on challenges and best

Sustainability (WorldS4) (pp. 152-156). IEEE. Boeker, M., Vach, W., & Motschall, E. (2013). Google

Akbar, M. A., Mahmood, S., & Siemon, D. (2022, June). Scholar as replacement for systematic literature

Toward effective and efficient DevOps using searches: good relative recall and precision are not

Shihab, E. (2023). Empirical analysis of security- computing. In 20 10th Annual Computing and

IaC through the DevSecOps philosophy: Concepts, & Khalil, M. (2007). Lessons from applying the

Clusters. In 20 Second International Conference on component discovery through approximate concrete

Intelligent Data Science Technologies and Applications execution. In Proceedings of the 20 Sixth

ECAI&ML international conference (pp. 115-120). J., Martincic, T., & Stepec, D. (2023, April). Security

44). Towards an evidence-based understanding of electronic Bass, L., Holz, R., Rimba, P., Tran, A. B., & Zhu, L. (2015, data sources. At the 14th International Conference on

May). Securing a deployment pipeline. In 20 Evaluation and Assessment in Software Engineering

IEEE/ACM 3rd International Workshop on Release (EASE), BCS Learning & Development.

Engineering (pp. 4-7). IEEE. Chhillar, D., & Sharma, K. (2019, February). ACT Testbot

Benedetti, G., Verderame, L., & Merlo, A. (2022, and 4S Quality Metrics in XAAS Framework. In 20

November). Automatic security assessment of github International Conference on Machine Learning, Big

September). Alice in (software supply) chains: risk Input splitting for cloud-based static application

identification and evaluation. In International security testing platforms. In Proceedings of the 30th

Conference on the Quality of Information and ACM Joint European Software Engineering

Communications Technology (pp. 281-295). Cham: Conference and Symposium on the Foundations of

Springer International Publishing. Software Engineering (pp. 1367-1378).

Davis, J. C., Amusuo, P., & Bushagour, J. R. (2022, May). Garg, S., & Garg, S. (2019, March). Automated cloud

A first offering of software engineering. In Proceedings infrastructure, continuous integration and continuous

of the First International Workshop on Designing and delivery using docker with robust container security. In

Running Project-Based Courses in Software 20 IEEE Conference on Multimedia Information

November). Automated detection of side channels in

qualities of Google Scholar, PubMed, and 2 other cryptographic protocols: DROWN the ROBOTs!. In resources. Research synthesis methods, 11(2), 181-217.

Proceedings of the 14th ACM Workshop on Artificial

Exploiting devops practices for dependable and secure

the 20 11th Joint Meeting on Foundations of continuous delivery pipelines. In Proceedings of the 4th Software Engineering (pp. 197-207).

International Workshop on Rapid Continuous Software

Huang, M., Fan, W., Huang, W., Cheng, Y., & Xiao, H. Engineering (pp. 27-30). (2020, June). Research on building exploitable

Dursun, H. (2023, June). Full Spec Software via Platform

vulnerability database for cloud-native app. In 20

Engineering: Transition from Bolting-on to Building-

IEEE 4th Information Technology, Networking, in. In Proceedings of the 27th International Conference

Electronic and Automation Control Conference

on Evaluation and Assessment in Software Engineering (ITNEC) (Vol. 1, pp. 758-762). IEEE. (pp. 172-175). Hudic, A., Flittner, M., Lorünser, T., Radl, P. M., & Bless,

El Khairi, A., Caselli, M., Knierim, C., Peter, A., &

R. (2016, August). Towards a unified secure cloud

Continella, A. (2022, November). Contextualizing

Enck, W., & Williams, L. (2022). Top five challenges in

reliable software releases through build, test, and software supply chain security: Observations from 3 deployment automation. Pearson Education.. industry and government organizations. IEEE Security

Jamshidi, P., Pahl, C., Mendonça, N. C., Lewis, J., &

Tilkov, S. (2018). Microservices: The journey so far

Faustino, J., Adriano, D., Amaro, R., Pereira, R., & da

and challenges ahead. IEEE Software, 35(3), 24-35.

Silva, M. M. (2022). DevOps benefits: A systematic

Kang, H., Le, M., & Tao, S. (2016, April). Container and

Autonomous Real-Time Testing in the Product

Karl, M., Musch, M., Ma, G., Johns, M., & Lekies, S.

Development Cycle. In Systems, Software and Services

Proceedings 2 (pp. 377-390). Springer International

ACM Internet Measurement Conference (pp. 619-632). Publishing.

Kitchenham, B. (2004). Procedures for performing

Fitzgerald, B., & Stol, K. J. (2014, June). Continuous

systematic reviews. Keele, UK, Keele University, software engineering and beyond: trends and 33(2004), 1-26. challenges. In Proceedings of the 1st International

Kitchenham, B. (2006). Evidence-based software

Workshop on rapid continuous software engineering

engineering and systematic literature reviews. In (pp. 1-9).

Product-Focused Software Process Improvement: 7th

Fitzgerald, B., & Stol, K. J. (2017). Continuous software

Cloudinspector: A transparency-as-a-service solution

May). Evidence-based software engineering. In

for legal issues in cloud computing. In 20 IEEE

Proceedings. 26th International Conference on

International Conference on Cloud Engineering

Software Engineering (pp. 273-281). IEEE. Workshop (IC2EW) (pp. 94-99). IEEE.

Kitchenham, B., Madeyski, L., & Budgen, D. (2022). How

should software engineering secondary studies include

grey material?. IEEE Transactions on Software Newkirk, J. (2002, May). Introduction to agile processes

Engineering, 49(2), 872-882. and extreme programming. In Proceedings of the 24th

Software Engineering, 49(3), 1273-1298. Towards detection of software supply chain attacks by

Koishybayev, I., Nahapetyan, A., Zachariah, R., Muralee, forensic artifacts. In Proceedings of the 15th

Security 22) (pp. 2747-2763). J. C. (2022, November). Sok: Analysis of software

Kumar, R., & Goyal, R. (2020). Modeling continuous supply chain security by establishing secure design security: A conceptual model for automated properties. In Proceedings of the 20 ACM Workshop

DevSecOps using open-source software over cloud on Software Supply Chain Offensive Research and

Security (pp. 812-826). fluid multi-party open software and services. In 20

highways and country roads to continuous deployment. pipeline within a kubernetes environment. In

Ieee software, 32(2), 64-72. Proceedings of the International Conference on

Evaluation (pp. 25-32). the social sciences: A practical guide. John Wiley &

Technologies and Container Clouds (pp. 31-36). 16th ACM/IEEE International Symposium on

Software Engineering, 41(7), 691-710. Transactions on Management Information Systems

Nadgowda, S., & Luan, L. (2021, December). tapiserí: (TMIS), 13(4), 1-24. Blueprint to modernize DevSecOps for real world. In Runeson, P., & Höst, M. (2009). Guidelines for conducting

Proceedings of the Seventh International Workshop on and reporting case study research in software

Container Technologies and Container Clouds (pp. 13- engineering. Empirical software engineering, 14, 131-

18). 164.

Deployment Pipeline in Cloud Computing. Computers, December). Integr

FAQ

Typical stacks include Git, a CI engine (Jenkins/GitHub Actions/GitLab CI), Docker, Kubernetes or a cloud PaaS, infrastructure-as-code (Terraform/Ansible), and monitoring (Prometheus/Grafana). Exact tools for Devops Vercel Deployment Automation depend on the chosen cloud and delivery model.

It can be delivered as a fully documented simulation on local/kind clusters or as a guided deployment on a cloud free tier. Both approaches are acceptable for academic evaluation when metrics, logs and diagrams are captured.

Pipeline screenshots, successful/failed run history, deployment frequency charts, rollback demos, monitoring dashboards, IaC plans/applies, and a short viva demo script are commonly included.