Cloud File Integrity Monitoring Tool Using Aws Lambda — Complete Documentation & Simulation

Overview & System Context

Cloud File Integrity Monitoring Tool Using Aws Lambda: Comprehensive Web-Sourced Technical Investigation Exhaustive Research Dossier · Pure Web-Scraped Content: ~5460 Words · Real Software Screenshots: 3 1. Executive Overview & Technical Background PostgreSQL ( POHST-gres-kew-EL), also known as Postgres, is a free and open-source relational database management system (RDBMS) emphasizing extensibility and SQL compliance. PostgreSQL features transactions with atomicity, consistency, isolation, durability (ACID) properties, automatically updatable views, materialized views, triggers, foreign keys, and stored procedures. It is supported on all major operating systems, including Windows, Linux, macOS, FreeBSD, and OpenBSD, and handles a range of workloads from single machines to data warehouses, data lakes, or web services with many concurrent users. The PostgreSQL Global Development Group focuses only on developing a database engine and closely related components. This core is, technically, what comprises PostgreSQL itself, but there is an extensive developer community and ecosystem that provides other important feature sets that might, traditionally, be provided by a proprietary software vendor. These include special-purpose database engine features, like those needed to support a geospatial or temporal database or features which emulate other database products. Also available from third parties are a wide variety of user and machine interface features, such as graphical user interfaces or load balancing and high availability toolsets. The large third-party PostgreSQL support network of people, companies, products, and projects, even though not part of The PostgreSQL Development Group, are essential to the PostgreSQL database engine's adoption and use and make up the PostgreSQL ecosystem writ large. PostgreSQL was originally named POSTGRES, referring to its origins as a successor to the Ingres database developed at the University of California, Berkeley. In 1996, the project was renamed PostgreSQL to reflect its support for SQL. After a review in 2007, the development team decided to keep the name PostgreSQL and the alias Postgres.

Slony-I Londiste, part of SkyTools (developed by Skype) Bucardo multi-master replication (developed by Backcountry.com) SymmetricDS multi-master, multi-tier replication Expression indexes can be created with an index of the result of an expression or function, instead of simply the value of a column. Partial indexes, which only index part of a table, can be created by adding a WHERE clause to the end of the CREATE INDEX statement. This allows for a smaller index to be created. The planner is able to use multiple indexes together to satisfy complex queries, using temporary in-memory bitmap index operations (useful for data warehouse applications for joining a large fact table to smaller dimension tables such as those arranged in a star schema). k-nearest neighbors (k-NN) indexing (also referred to KNN-GiST) provides efficient searching of "closest values" to that specified, useful to finding similar words, or close objects or locations with geospatial data. This is achieved without exhaustive matching of values. Index-only scans often allow the system to fetch data from indexes without ever having to access the main table. Block Range Indexes (BRIN).

Boolean Arbitrary-precision numerics Character (text, varchar, char) Binary Date/time (timestamp/time with/without time zone, date, interval) Money Enum Bit strings Text search type Composite HStore, an extension enabled key–value store within PostgreSQL Arrays (variable-length and can be of any data type, including text and composite types) up to 1 GB in total storage size Geometric primitives IPv4 and IPv6 addresses Classless Inter-Domain Routing (CIDR) blocks and MAC addresses XML supporting XPath queries Universally unique identifier (UUID) JavaScript Object Notation (JSON), and a faster binary JSONB (not the same as BSON) In addition, users can create their own data types which can usually be made fully indexable via PostgreSQL's indexing infrastructures – GiST, GIN, SP-GiST. Examples of these include the geographic information system (GIS) data types from the PostGIS project for PostgreSQL. There is also a data type called a domain, which is the same as any other data type but with optional constraints defined by the creator of that domain. This means any data entered into a column using the domain will have to conform to whichever constraints were defined as part of the domain.

A data type that represents a range of data can be used which are called range types. These can be discrete ranges (e.g. all integer values 1 to 10) or continuous ranges (e.g., any time between 10:00 am and 11:00 am). The built-in range types available include ranges of integers, big integers, decimal numbers, time stamps (with and without time zone) and dates. Custom range types can be created to make new types of ranges available, such as IP address ranges using the inet type as a base, or float ranges using the float data type as a base. Range types support inclusive and exclusive range boundaries using the [] and () characters respectively. (e.g., [4,9) represents all integers starting from and including 4 up to but not including 9.) Range types are also compatible with existing operators used to check for overlap, containment, right of etc.

Figure 1: Highly scalable IT monitoring with Checkmk Cloud Casts Conversions Data types Data domains Functions, including aggregate functions and window functions Indexes including custom indexes for custom types Operators (existing ones can be overloaded) Procedural languages Plain SQL (safe). Simpler SQL functions can get expanded inline into the calling (SQL) query, which saves function call overhead and allows the query optimizer to "see inside" the function. Procedural Language/PostgreSQL (PL/pgSQL) (safe), which resembles Oracle's Procedural Language for SQL (PL/SQL) procedural language and SQL/Persistent Stored Modules (SQL/PSM). C (unsafe), which allows loading one or more custom shared library into the database. Functions written in C offer the best performance, but bugs in code can crash and potentially corrupt the database. Most built-in functions are written in C. In addition, PostgreSQL allows procedural languages to be loaded into the database through extensions. Three language extensions are included with PostgreSQL to support Perl, Tcl, and Python. For Python, the current Python 3 is used, and the discontinued Python 2 is no longer supported as of PostgreSQL 15. Both were supported previously, defaulting to Python 2, while old and new versions couldn't be used in the same session. External projects provide support for many other languages, including PL/Java, JavaScript (PL/V8), PL/Julia, PL/R, PL/Ruby, and others.

Password: either SCRAM-SHA-256, MD5 or plain-text Generic Security Services Application Program Interface (GSSAPI) Security Support Provider Interface (SSPI) Kerberos ident (maps O/S user-name as provided by an ident server to database user-name) Peer (maps local user name to database user name) Lightweight Directory Access Protocol (LDAP) Active Directory (AD) RADIUS Certificate Pluggable authentication module (PAM) The GSSAPI, SSPI, Kerberos, peer, ident and certificate methods can also use a specified "map" file that lists which users matched by that authentication system are allowed to connect as a specific database user. These methods are specified in the cluster's host-based authentication configuration file (pg_hba.conf), which determines what connections are allowed. This allows control over which user can connect to which database, where they can connect from (IP address, IP address range, domain socket), which authentication system will be enforced, and whether the connection must use Transport Layer Security (TLS).

More recently, PostgreSQL 18's asynchronous I/O subsystem has been reported to improve read performance by up to 3x for I/O-bound workloads such as sequential scans and vacuum operations, compared to prior versions. Many informal performance studies of PostgreSQL have been done. Performance improvements aimed at improving scalability began heavily with version 8.1. Simple benchmarks between version 8.0 and version 8.4 showed that the latter was more than ten times faster on read-only workloads and at least 7.5 times faster on both read and write workloads. The first industry-standard and peer-validated benchmark was completed in June 2007, using the Sun Java System Application Server (proprietary version of GlassFish) 9.0 Platform Edition, UltraSPARC T1-based Sun Fire server and PostgreSQL 8.2. This result of 778.14 SPECjAppServer2004 JOPS@Standard compares favourably with the 874 JOPS@Standard with Oracle 10 on an Itanium-based HP-UX system. In August 2007, Sun submitted an improved benchmark score of 813.73 SPECjAppServer2004 JOPS@Standard. With the system under test at a reduced price, the price/performance improved from $84.98/JOPS to $70.57/JOPS. The default configuration of PostgreSQL uses only a small amount of dedicated memory for performance-critical purposes such as caching database blocks and sorting. This limitation is primarily because older operating systems required kernel changes to allow allocating large blocks of shared memory. PostgreSQL.org provides advice on basic recommended performance practice in a wiki. In April 2012, Robert Haas of EnterpriseDB demonstrated PostgreSQL 9.2's linear CPU scalability using a server with 64 cores. Matloob Khushi performed benchmarking between PostgreSQL 9.0 and MySQL 5.6.15 for their ability to process genomic data. In his performance analysis he found that PostgreSQL extracts overlapping genomic regions eight times faster than MySQL using two datasets of 80,000 each forming random human DNA regions. Insertion and data uploads in PostgreSQL were also better, although general searching ability of both databases was almost equivalent.

Figure 2: File integrity monitoring with AWS Systems Manager and Amazon Security ... psql The primary front-end for PostgreSQL is the psql command-line program, which can be used to enter SQL queries directly, or execute them from a file. In addition, psql provides a number of meta-commands and various shell-like features to facilitate writing scripts and automating a wide variety of tasks; for example tab completion of object names and SQL syntax.

pgAdmin The pgAdmin package is a free and open-source graphical user interface (GUI) administration tool for PostgreSQL, which is supported on many computer platforms. The program is available in more than a dozen languages. The first prototype, named pgManager, was written for PostgreSQL 6.3.2 from 1998, and rewritten and released as pgAdmin under the GNU General Public License (GPL) in later months. The second incarnation (named pgAdmin II) was a complete rewrite, first released on January 16, 2002. The third version, pgAdmin III, was originally released under the Artistic License and then released under the same license as PostgreSQL. Unlike prior versions that were written in Visual Basic, pgAdmin III is written in C++, using the wxWidgets framework allowing it to run on most common operating systems. The query tool includes a scripting language called pgScript for supporting admin and development tasks. In December 2014, Dave Page, the pgAdmin project founder and primary developer, announced that with the shift towards web-based models, work has begun on pgAdmin 4 with the aim to facilitate cloud deployments. In 2016, pgAdmin 4 was released. The pgAdmin 4 backend was written in Python, using Flask and the Qt framework. phpPgAdmin phpPgAdmin is a web-based administration tool for PostgreSQL written in PHP and based on the popular phpMyAdmin interface originally written for MySQL administration.

PostgreSQL Studio PostgreSQL Studio allows users to perform essential PostgreSQL database development tasks from a web-based console. PostgreSQL Studio allows users to work with cloud databases without the need to open firewalls. TeamPostgreSQL AJAX/JavaScript-driven web interface for PostgreSQL. Allows browsing, maintaining and creating data and database objects via a web browser. The interface offers tabbed SQL editor with autocompletion, row editing widgets, click-through foreign key navigation between rows and tables, favorites management for commonly used scripts, among other features. Supports SSH for both the web interface and the database connections. Installers are available for Windows, Macintosh, and Linux, and a simple cross-platform archive that runs from a script. LibreOffice, OpenOffice.org LibreOffice and OpenOffice.org Base can be used as a front-end for PostgreSQL. pgBadger The pgBadger PostgreSQL log analyzer generates detailed reports from a PostgreSQL log file. pgDevOps pgDevOps is a suite of web tools to install & manage multiple PostgreSQL versions, extensions, and community components, develop SQL queries, monitor running databases and find performance problems. Adminer Adminer is a simple web-based administration tool for PostgreSQL and others, written in PHP. pgBackRest pgBackRest is a backup and restore tool for PostgreSQL that provides support for full, differential, and incremental backups. pgaudit pgaudit is a PostgreSQL extension that provides detailed session and/or object audit logging via the standard logging facility provided by PostgreSQL. WAL-E WAL-E is a backup and restore tool for PostgreSQL that provides support for physical (WAL-based) backups, written in Python. DBeaver DBeaver is a free and open source GUI administration tool for PostgreSQL, it has Visual Entity Diagrams and Intellisense features. It also has a commercial PRO license. Postgresus Postgresus is an open source backup tool with GUI for scheduled backups with support of external sources (S3, NAS, FTP, Google Drive, Google Cloud, etc.) and notifications to external services (Slack, Discord, Telegram, SMTP, etc.). VisuaLeaf VisuaLeaf is a multi database workspace that allows users to work with PostgreSQL and other databases visually. Users can visualize their data, create queries visually, design database schemas, and write SQL scripts.

VisuaLeaf also provides features such as RBAC management, query building, data visualization, and database administration tools. A number of companies offer proprietary tools for PostgreSQL. They often consist of a universal core that is adapted for various specific database products. These tools mostly share the administration features with the open source tools but offer improvements in data modeling, importing, exporting or reporting.

Microsoft, used for a petabyte-scale “Release Quality View” (RQV) analytics dashboard, which tracks quality of Windows updates analyzing 20K types of metrics from over 800M Windows devices. In 2009, the social-networking website Myspace used Aster Data Systems's nCluster database for data warehousing, which was built on unmodified PostgreSQL. Geni.com uses PostgreSQL for their main genealogy database. OpenStreetMap, a collaborative project to create a free editable map of the world. Afilias, domain registries for .org, .info and others. Sony Online multiplayer online games. BASF, shopping platform for their agribusiness portal. Reddit social news website. Skype VoIP application, central business databases. Sun xVM, Sun's virtualization and datacenter automation suite. MusicBrainz, open online music encyclopedia. The International Space Station – to collect telemetry data in orbit and replicate it to the ground. MyYearbook social-networking site. Instagram, a mobile photo-sharing service. Disqus, an online discussion and commenting service. TripAdvisor, travel-information website of mostly user-generated content. Yandex, a Russian internet company switched its Yandex.Mail service from Oracle to Postgres.

Amazon Redshift, part of AWS, a columnar online analytical processing (OLAP) system based on ParAccel's Postgres modifications. National Oceanic and Atmospheric Administration's (NOAA) National Weather Service (NWS), Interactive Forecast Preparation System (IFPS), a system that integrates data from the NEXRAD weather radars, surface, and hydrology systems to build detailed localized forecast models. United Kingdom's national weather service, Met Office, has begun swapping Oracle for PostgreSQL in a strategy to deploy more open source technology. WhitePages.com had been using Oracle and MySQL, but when it came to moving its core directories in-house, it turned to PostgreSQL. Because WhitePages.com needs to combine large sets of data from multiple sources, PostgreSQL's ability to load and index data at high rates was a key to its decision to use PostgreSQL. FlightAware, a flight tracking website. Grofers, an online grocery delivery service. The Guardian migrated from MongoDB to PostgreSQL in 2018. YugabyteDB implements the PostgreSQL query layer as its default SQL mode OpenAI uses PostgreSQL as part of its primary API service - including for ChatGPT.

Heroku, a platform as a service provider, has supported PostgreSQL since the start in 2007. They offer value-add features like full database roll-back (ability to restore a database from any specified time), which is based on WAL-E, open-source software developed by Heroku. In January 2012, EnterpriseDB released a cloud version of both PostgreSQL and their own proprietary Postgres Plus Advanced Server with automated provisioning for failover, replication, load-balancing, and scaling. It runs on Amazon Web Services. Since 2015, Postgres Advanced Server has been offered as ApsaraDB for PPAS, a relational database as a service on Alibaba Cloud. VMware has offered vFabric Postgres (also termed vPostgres) for private clouds on VMware vSphere since May 2012. The company announced End of Availability (EOA) of the product in 2014. In November 2013, Amazon Web Services announced the addition of PostgreSQL to their Relational Database Service offering. In November 2016, Amazon Web Services announced the addition of PostgreSQL compatibility to their cloud-native Amazon Aurora managed database offering. In May 2017, Microsoft Azure announced Azure Databases for PostgreSQL. In May 2019, Alibaba Cloud announced PolarDB for PostgreSQL. Jelastic Multicloud Platform as a Service has provided container-based PostgreSQL support since 2011. It also offers automated asynchronous master-slave replication of PostgreSQL. In June 2019, IBM Cloud announced IBM Cloud Hyper Protect DBaaS for PostgreSQL. In December 2022, Google Cloud Platform announced general availability of AlloyDB as fully managed PostgreSQL cloud service. In April 2024, Google Cloud Platform announced general availability of AlloyDB Omni, a downloadable version of AlloyDB designed to run on any infrastructure, including on-premises, other clouds, or edge environments.

Figure 3: Viewing CloudWatch logs for Lambda functions - AWS Lambda Greenplum Database: A massively parallel processing (MPP) data warehouse based on an older version of PostgreSQL, designed for large-scale analytics. TimescaleDB: A time-series database delivered as a PostgreSQL extension, optimized for handling fast ingest and complex queries of time-series data. Amazon Aurora: A cloud-native relational database offered by Amazon Web Services that provides a PostgreSQL-compatible edition.

Comparison of relational database management systems Database scalability List of databases using MVCC LLVM (llvmjit is the JIT engine used by PostgreSQL) Official website , and wiki A Software Catalog of related projects and products The official Main Source Code Repository (for browsing), and the Developer FAQ The official Reference for PostgreSQL Documentation Authors All official PostgreSQL Source Code Repositories PostgreSQL on GitHub Customers need solutions to track inventory data such as files and software across Amazon Elastic Compute Cloud (Amazon EC2) instances, detect unauthorized changes, and integrate alerts into their existing security workflows.

In this blog post, I walk you through a highly scalable serverless file integrity monitoring solution. It uses AWS Systems Manager Inventory to collect file metadata from Amazon EC2 instances. The metadata is sent through the Systems Manager Resource Data Sync feature to a versioned Amazon Simple Storage Service (Amazon S3) bucket, storing one inventory object for each EC2 instance. Each time a new object is created in Amazon S3, an Amazon S3 Event Notification triggers a custom AWS Lambda function. This Lambda function compares the latest inventory version with the previous one to detect file changes. If a file that isn’t expected to change has been created, modified, or deleted, the function creates an actionable finding in AWS Security Hub . Findings are then ingested by Amazon Security Lake in a standard OCSF format, which centralizes and normalizes the data. Finally, the data can be analyzed using Amazon Athena for one-time queries, or by building visual dashboards with Amazon QuickSight and Amazon OpenSearch Service . Figure 1 summarizes this flow: This integration offers an alternative to the default AWS Config and Security Hub integration, which relies on limited data (for example, no file modification timestamps). The solution presented in this post provides control and flexibility to implement custom logic tailored to your operational needs and support security-related efforts.

2. System Architecture & Core Working Principles This flexible solution can also be used with other Systems Manager Inventory metadata , such as installed applications, network configurations, or Windows registry entries, enabling custom detection logic across a wide range of operational and security use cases.

Before you get started, you need an AWS account with permissions to create and manage AWS resources such as Amazon EC2, AWS Systems Manager , Amazon S3, and Lambda. Start by launching an EC2 instance and creating a file that you will later modify to simulate an unauthorized change. Create an AWS Identity and Access Management (IAM) role to allow the EC2 instance to communicate with Systems Manager: If Security Hub and Security Lake are already enabled, you can skip to Step 3 . To start, enable Security Hub, which collects and aggregates security findings. AWS Security Hub CSPM adds continuous monitoring and automated checks against best practices.

Note: For this demo, you don’t need the Security standards options and can clear them. Next, activate Security Lake to start collecting actionable findings from Security Hub: With Security Hub and Security Lake enabled, the next step is to enable Systems Manager Inventory to collect file metadata and configure a Resource Data Sync to export this data to S3 for analysis.

Note: In production, enable S3 server access logging on the inventory bucket to keep an audit trail of access requests, enforce HTTPS-only access , and enable CloudTrail data events for S3 to record who accessed or modified inventory files. The next step is to enable Systems Manager Inventory and set up the resource data sync: Next, complete the setup to detect changes and create findings. Each time Systems Manager Inventory writes a new object to Amazon S3, an S3 Event Notification triggers a Lambda function that compares the latest and previous object versions. If it finds created, modified, or deleted files, it creates a security finding. To accomplish this, you will create the Lambda function, set its environment variables, add the helper layer, and attach the required permissions.

The following is an example finding generated in AWS Security Finding Format (ASFF) and sent to Security Hub. In this example, you see a notification about a file change on the EC2 instance listed under the Resources section. This function detects file changes, reports findings, and removes unused Amazon S3 object versions to reduce costs. Note : In production, set Lambda reserved concurrency to prevent unbounded scaling, configure a dead letter queue (DLQ) to capture failed invocations, and optionally attach the function to an Amazon VPC for network isolation .

Configure the two required environment variables in the Lambda console. These two variables (one for critical paths to monitor and one for security finding severity) must be set or the function will fail. For better modularity, add some helper functions to a Lambda layer . These functions are already referenced in the import section of the preceding Lambda function’s Python code. The helper functions check critical paths, load file metadata, compare modification times, and extract the EC2 instance ID.

Open AWS CloudShell from the top-right corner of the AWS console header, then copy and paste the following script and press Enter. It creates the helper layer and attaches it to your Lambda function. Finally, set up S3 Event Notifications to trigger the Lambda function when new inventory data arrives. 3. Software Implementation, Frameworks & Protocols In this example, inventory collection runs every 30 minutes (48 times each day) but can be adjusted based on security requirements to optimize costs. The Lambda function is triggered once for each instance whenever a new inventory object is created. You can further reduce event volume by filtering EC2 instances through S3 Event Notification prefixes , enabling focused monitoring of high-value instances.

Now that the EC2 instance is running and the sample configuration file /etc/paymentapp/config.yaml has been initialized, you’re ready to simulate an unauthorized change to test the file integrity monitoring setup. This simulates a configuration tampering event. During the next Systems Manager Inventory run, the updated metadata will be saved to Amazon S3.

While Security Hub provides a centralized view of findings, you can deepen your analysis using Amazon Athena to run SQL queries directly on the normalized Security Lake data in Amazon S3. This data follows the Open Cybersecurity Schema Framework (OCSF), which is a vendor-neutral standard that simplifies integration and analysis of security data across different tools and services.

Note: Be sure to adjust the FROM clause for other Regions. Security Lake processes findings before they appear in Athena, so expect a short delay between ingestion and data availability. You will see a similar result for the preceding query, shown in Figure 9: Security Lake classifies this finding as an OCSF 2004 Class, Detection Finding . You can explore the full schema definitions at OCSF Categories . For more query examples, see the Security Lake query examples . For visual exploration and real-time insights, you can integrate Security Lake with OpenSearch Service and QuickSight, both of which now offer extensive generative AI support. For a guided walkthrough using QuickSight, see How to visualize Amazon Security Lake findings with Amazon QuickSight .

After testing the step-by-step guide, make sure to clean up the resources you created for this post to avoid ongoing costs. In this post, you learned how to use Systems Manager Inventory to track file integrity, report findings to Security Hub, and analyze them using Security Lake. You can access the full sample code to set up this solution in the AWS Samples repository . While this post uses a single-account, single-Region setup for simplicity, Security Lake supports collecting data across multiple accounts and Regions using AWS Organizations . You can also use a Systems Manager resource data sync to send inventory data to a central S3 bucket.

Getting Started with Amazon Security Lake and Systems Manager Inventory provides guidance for enabling scalable, cloud-centric monitoring with full operational context. One of the biggest challenges in cloud security is ensuring that critical files remain authentic, traceable, and tamper-free after being uploaded into cloud environments.

In industries such as legal systems, healthcare, forensic investigations, and compliance-driven environments, even a small unauthorized file modification can create serious operational and security risks. How can you detect when a file has been silently modified while still maintaining a reliable audit trail and real-time operational visibility? To explore that problem, I designed and deployed Project Aegis — a serverless file integrity monitoring platform built on AWS.

Amazon S3 acts as the ingestion layer for uploaded files and automatically triggers the processing pipeline using event notifications. Amazon SNS sends real-time operational alerts whenever suspicious file modifications are detected. After initially building the project manually inside the AWS Console, I later automated the infrastructure using Terraform to improve: One of the most important parts of the project was designing reliable file tampering detection.

Instead of relying only on filenames, the platform generates SHA-256 hashes to compare actual file content integrity. 4. Practical Deployment, Analysis & Case Studies One of the most valuable parts of building Project Aegis was troubleshooting real operational and infrastructure problems while designing the platform.

Several engineering challenges forced me to think beyond simply connecting AWS services and instead focus on debugging, observability, system behavior, and operational reliability. Added proper IAM permissions ( sns:Publish ) and integrated SNS workflows directly into Lambda processing. It was initially difficult to verify whether Lambda executions completed successfully.

Used Amazon CloudWatch logs to trace execution flow, monitor failures, and debug event processing behavior. Implemented Infrastructure as Code using Terraform to automate AWS resource provisioning. The infrastructure can now be recreated consistently using repeatable deployment workflows. Building Project Aegis reinforced several important cloud engineering and operational concepts for me: One of the biggest mindset shifts from this project was realizing that cloud engineering is not simply about deploying services.

Project Aegis started as a cloud security learning project, but it ultimately became an exercise in operational thinking, infrastructure automation, observability, and system reliability. If you’ve worked on similar cloud security or serverless engineering projects, I’d genuinely enjoy connecting and learning from other approaches and ideas.

This CloudFormation template deploys a File Integrity Monitoring (FIM) solution using AWS Lambda, DynamoDB, and SNS. It is designed to monitor the integrity of files across specified EC2 instances and alert via email if any changes are detected. The template sets up the necessary AWS resources to monitor file integrity on EC2 instances. When a file change is detected, an alert is sent to the specified email addresses using Amazon Simple Notification Service (SNS).

Once deployed, the system will automatically monitor the specified files for any changes based on the provided FilesJson configuration. Notifications will be sent to the EmailRecipients if any discrepancies are detected. Files uploaded to S3 are automatically fingerprinted, compared against their last known state, and recorded in a tamper-evident, hash-linked audit trail — with a companion CLI for inspecting the results.

Once a file leaves your hands — uploaded to storage, shared, archived — you generally have no reliable way to know later whether it's still the same file. Manual checksums require someone to remember to run them, compare them, and keep a record. That doesn't scale, and it's easy to tamper with a single stored hash if there's nothing linking it to a verifiable history. This project turns an S3 bucket into a self-auditing evidence locker. Every file uploaded to incoming/ is automatically: No manual step is required — the moment a file lands in S3, the Lambda function processes it and the CLI can immediately report on it.

5. Performance Evaluation & Extended Technical Observations See docs/architecture.md for a full breakdown of the data flow, record format, and hash-chaining design. See deployment/README.md for full setup instructions, including required IAM permissions and how to reproduce this project in your own AWS account.

This project was originally built and demonstrated in an AWS Academy Learner Lab (sandbox) environment. Sandbox resources are temporary and tied to a training account, so this repository preserves the source code, architecture, and evidence permanently — it does not depend on the original sandbox to be understood or redeployed. A serverless file-integrity monitoring system using AWS S3, Lambda, SHA-256 fingerprinting, and cryptographic audit trails.

This article presents a serverless file integrity monitoring solution using AWS Systems Manager, Amazon S3, Lambda, Security Hub, and Security Lake to detect unauthorized file changes on EC2 instances. This solution provides a flexible, serverless approach to file integrity monitoring with centralized security findings and advanced analytics capabilities across AWS accounts and regions.

The AWS News Feed is currently looking for gold sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed. The AWS News Feed is currently looking for silver sponsors. If you want to support the AWS community and reach a large audience of AWS professionals, consider sponsoring the AWS News Feed.

This CloudFormation template deploys a File Integrity Monitoring (FIM) solution using AWS Lambda, DynamoDB, and SNS. It is designed to monitor the integrity of files across specified EC2 instances and alert via email if any changes are detected. The template sets up the necessary AWS resources to monitor file integrity on EC2 instances. When a file change is detected, an alert is sent to the specified email addresses using Amazon Simple Notification Service (SNS).

Once deployed, the system will automatically monitor the specified files for any changes based on the provided FilesJson configuration. Notifications will be sent to the EmailRecipients if any discrepancies are detected. This chapter covers security, governance, and compliance for AWS Lambda. The data protection, compliance, resilience, and infrastructure security guidance applies to all Lambda compute types. Access control and encryption have compute-type-specific details: for Lambda Web Functions, see Lambda Web Function permissions , Lambda Web Function endpoint access control , and Encryption at rest for Web Functions ; for Lambda MicroVMs, see Security and permissions .

The remaining topics in this chapter describe Lambda Event Functions. Cloud security at AWS is the highest priority. As an AWS customer, you benefit from a data center and network architecture that is built to meet the requirements of the most security-sensitive organizations. Security is a shared responsibility between AWS and you. The shared responsibility model describes this as security of the cloud and security in the cloud: Security of the cloud âII AWS is responsible for protecting the infrastructure that runs AWS services in the AWS Cloud.

AWS also provides you with services that you can use securely. Third-party auditors regularly test and verify the effectiveness of our security as part of the AWS compliance programs . To learn about the compliance programs that apply to AWS Lambda, see AWS services in Scope by Compliance Program . Security in the cloud âII Your responsibility is determined by the AWS service that you use. You are also responsible for other factors including the sensitivity of your data, your company's requirements, and applicable laws and regulations.

This documentation helps you understand how to apply the shared responsibility model when using Lambda. The following topics show you how to configure Lambda to meet your security and compliance objectives. You also learn how to use other AWS services that help you to monitor and secure your Lambda resources. Topics include compliance validation , data protection , identity and access management , interface VPC endpoints , and resilience .

For more information about applying security principles to Lambda applications, see Security in Serverless Land. Harvested Online Documentation & Reference Sources • https://en.wikipedia.org/wiki/PostgreSQL • https://aws.amazon.com/blogs/security/file-integrity-monitoring-with-aws-systems-manager-and-amazon-security-lake/ • https://dev.to/wilfriedbako/building-project-aegis-designing-a-serverless-file-integrity-monitoring-system-on-aws-4457 • https://github.com/digital-edge/aws-file-integrity-monitor • https://github.com/PranidhiBhatt/aws-digital-evidence-locker • https://aws-news.com/article/2026-01-27-file-integrity-monitoring-with-aws-systems-manager-and-amazon-security-lake • https://github.com/digital-edge/aws-file-integrity-monitor/blob/master/README.md • https://docs.aws.amazon.com/lambda/latest/dg/lambda-security.html • https://docs.aws.amazon.com/lambda/latest/dg/lambda-monitoring.html