Ai Bug Detection | AI Final Year Project Report

Complete Technical Paper Content — Ai Bug Detection

1. Executive Abstract, Problem Formulation & Background

DeepBugs: Deep Learning to Find Bugs DeepBugs is a framework for learning name-based bug detectors from an existing code corpus. See our OOPSLA'18 paper for a detailed description. Quick Start A quick and easy way to play with a simplified version of DeepBugs is a Jupyter notebook, which you can run on Google's Colaboratory . To use the full DeepBugs tool, read on. Overview All commands are called from the main directory. Python code (most of the implementation) and JavaScript code (for extracting data from .js files) are in the /python and /javascript directories. All data to learn from, e.g., .js files are expected to be in the /data directory. All data that is generated, e.g., intermediate representations, are written into the main directory. It is recommended to move them into separate directories. All generated data files have a timestamp as part of the file name. Below, all files are used with * . When running commands multiple times, make sure to use the most recent files. Requirements Node.js npm modules (install with npm install module_name ): acorn, estraverse, walk-sync Python 3 Python packages: keras, scipy, numpy, sklearn JavaScript Corpus The full corpus can be downloaded here and is expected to be stored in data/js/programs_all . It consists of 100.000 training files, listed in data/js/programs_training.txt , and 50.000 files for validation, listed in data/js/programs_eval.txt . This repository contains only a very small subset of the corpus. It is stored in data/js/programs_50 . Training and validation files for the small corpus are listed in data/js/programs_50_training.txt and data/js/programs_50_eval.txt . Learning a Bug Detector Creating a bug detector consists of two main steps: Extract positive (i.e., likely correct) and negative (i.e., likely buggy) training examples from code. Train a classifier to distinguish correct from incorrect code examples. Each bug detector addresses a particular bug pattern, e.g.: The SwappedArgs bug detector looks for accidentally swapped arguments of a function call, e.g., calling setPoint(y,x) instead of setPoint(x,y) . The BinOperator bug detector looks for incorrect operators in binary operations, e.g., i <= len instead of i < len . The IncorrectBinaryOperand bug detector looks for incorrect operands in binary operations, e.g., height - x instead of height - y . Step 1: Extract positive and negative training examples node javascript/extractFromJS.js calls --parallel 4 data/js/programs_50_training.txt data/js/programs_50 The --parallel argument sets the number of processes to run. programs_50_training.txt contains files to include (one file per line). To extract data for validation, run the command with data/js/programs_50_eval.txt . The last argument is a directory that gets recursively scanned for .js files, considering only files listed in the file provided as the second argument. The command produces calls_*.json files, which is data suitable for the SwappedArgs bug detector. For the other bug two detectors, replace calls with binOps in the above command. Step 2: Train a classifier to identify bugs A) Train and validate the classifier python3 python/BugLearnAndValidate.py --pattern SwappedArgs --token_emb token_to_vector.json --type_emb type_to_vector.json --node_emb node_type_to_vector.json --training_data calls_xx*.json --validation_data calls_yy*.json The first argument selects the bug pattern. The next three arguments are vector representations for tokens (here: identifiers and literals), for types, and for AST node types. These files are provided in the repository. The remaining arguments are two lists of .json files. They contain the training and validation data extracted in Step 1. After learning the bug detector, the command measures accurracy and recall w.r.t. seeded bugs and writes a list of potential bugs in the unmodified validation code (see poss_anomalies.txt ). B) Train a classifier for later use python3 python/BugLearn.py --pattern SwappedArgs --token_emb token_to_vector.json --type_emb type_to_vector.json --node_emb node_type_to_vector.json --training_data calls_xx*.json Optionally, pass --out some/dir to set the output directory for the trained model. Note that learning a bug detector from the very small corpus of 50 programs will yield a classifier with low accuracy that is unlikely to be useful. To leverage the full power of DeepBugs, you'll need a larger code corpus, e.g., the JS150 corpus mentioned above. Finding Bugs Finding bugs in one or more source files consists of these two steps: Extract code pieces Use a trained classifier to identify bugs Step 1: Extract code pieces node javascript/extractFromJS.js calls --files <list of files> contains one or more files to be examined. Code pieces can be extracted from any javascript file (.js) given with path specification relative to the main directory. The command produces calls_*.json files, which is data suitable for the SwappedArgs bug detector. For the other bug two detectors, replace calls with binOps in the above command. Step 2: Use a trained classifier to identify bugs python3 python/BugFind.py --pattern SwappedArgs --threshold 0.95 --model some/dir --token_emb token_to_vector.json --type_emb type_to_vector.json --node_emb node_type_to_vector.json --testing_data calls_xx*.json The first argument selects the bug pattern. 0.95 is the threshold for reporting bugs; higher means fewer warnings of higher certainty. --model sets the directory to load a trained model from. The next three arguments are vector representations for tokens (here: identifiers and literals), for types, and for AST node types. These files are provided in the repository. The remaining argument is a list of .json files. They contain the data extracted in Step 1. The command examines every code piece and writes a list of potential bugs with its probability of being incorrect Embeddings for Identifiers The above bug detectors rely on a vector representation for identifier names and literals. To use our framework, the easiest is to use the shipped token_to_vector.json file. Alternatively, you can learn the embeddings via Word2Vec as follows: Extract identifiers and tokens: node javascript/extractFromJS.js tokens --parallel 4 data/js/programs_50_training.txt data/js/programs_50 The command produces tokens_*.json files. Encode identifiers and literals with context into arrays of numbers (for faster reading during learning): python3 python/TokensToTopTokens.py tokens_*.json The arguments are the just created files. The command produces encoded_tokens_*.json files and a file token_to_number_*.json that assigns a number to each identifier and literal. Learn embeddings for identifiers and literals: python3 python/EmbeddingLearnerWord2Vec.py token_to_number_*.json encoded_tokens_*.json The arguments are the just created files. The command produces a file token_to_vector_*.json .

DeepBugs is a framework for learning name-based bug detectors from an existing code corpus. See our OOPSLA'18 paper for a detailed description.

A quick and easy way to play with a simplified version of DeepBugs is a Jupyter notebook, which you can run on Google's Colaboratory . To use the full DeepBugs tool, read on.

node javascript/extractFromJS.js calls --parallel 4 data/js/programs_50_training.txt data/js/programs_50

A) Train and validate the classifier python3 python/BugLearnAndValidate.py --pattern SwappedArgs --token_emb token_to_vector.json --type_emb type_to_vector.json --node_emb node_type_to_vector.json --training_data calls_xx*.json --validation_data calls_yy*.json

B) Train a classifier for later use python3 python/BugLearn.py --pattern SwappedArgs --token_emb token_to_vector.json --type_emb type_to_vector.json --node_emb node_type_to_vector.json --training_data calls_xx*.json

Note that learning a bug detector from the very small corpus of 50 programs will yield a classifier with low accuracy that is unlikely to be useful. To leverage the full power of DeepBugs, you'll need a larger code corpus, e.g., the JS150 corpus mentioned above.

python3 python/BugFind.py --pattern SwappedArgs --threshold 0.95 --model some/dir --token_emb token_to_vector.json --type_emb type_to_vector.json --node_emb node_type_to_vector.json --testing_data calls_xx*.json

2. Related Work, Literature Survey & Algorithmic Foundations

The above bug detectors rely on a vector representation for identifier names and literals. To use our framework, the easiest is to use the shipped token_to_vector.json file. Alternatively, you can learn the embeddings via Word2Vec as follows:

node javascript/extractFromJS.js tokens --parallel 4 data/js/programs_50_training.txt data/js/programs_50

python3 python/EmbeddingLearnerWord2Vec.py token_to_number_*.json encoded_tokens_*.json

Deep Learning for Contextual Bug Detection and Automated Fixes in Software Systems Published at the 2024 2nd International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT) , IEEE, pp. 624–629. Paper: IEEE Xplore · DOI: 10.1109/ICAICCIT64383.2024.10912101 This repository is the reference companion to the paper: what the work claims, how the method is put together, what the numbers were, and where the limits are. The full text lives on IEEE Xplore and is not redistributed here. The problem, in one paragraph Static analysers and code review look at code in pieces. They read a function, a file, a diff. A large share of production bugs are not in any one of those pieces. They live in the interaction: module A calls B, B quietly changed what it returns, and the failure surfaces somewhere neither author was looking. A tool that only ever sees isolated fragments cannot find that class of bug, because the evidence for it is not inside any single fragment. This work asks whether giving a model the surrounding context, meaning the project architecture, the dependency structure, and observed runtime behaviour, lets it catch defects that per-file analysis misses, and whether the same model can propose the patch. What was built Two branches over the same file, joined by one classifier. A transformer reads the token stream and the syntax tree. That covers what the code says: the syntactic and semantic view a language model is good at. A graph neural network reads a dependency graph whose nodes are modules and whose edges are the relationships between them. That covers who the code talks to. The two representations are concatenated and scored together. When the detector flags a file, a decoder conditioned on both representations generates a candidate fix, which is then validated by unit tests and runtime analysis rather than trusted on sight. The whole thing sits inside a CI/CD pipeline so detection and patch generation happen on the commit rather than in a review queue days later. The mechanism that matters is the metadata. Input is not the code alone but X = [C, M] , where M carries architecture, dependencies, and runtime conditions. That is the difference between a model that can only ask "is this function wrong" and one that can ask "is this function wrong given what it depends on and how it behaves in production ". Full method, with the equations as published: docs/method.md . Every number with what it does and does not establish: docs/results.md . Questions readers actually ask: docs/notes-for-readers.md . Results Reported in the paper. These are the paper's measurements, on the paper's evaluation setup. Detection quality Model Accuracy Precision Recall F1 Transformer 88.2 86.9 87.4 87.1 Graph neural network 85.7 84.2 85.1 84.6 Combined 91.4 90.8 91.1 91.0 The combination beats either branch by roughly three points of accuracy over the transformer and close to six over the graph model. That gap is the paper's central claim: the two views are complementary rather than redundant. Worth noting which branch is weaker alone. The graph model is the lower scorer on its own, which is what you would expect, since structure without content cannot tell a correct function from a broken one. Its contribution shows up only in combination. Time to detect and fix Method Detection (s) Fixing (s) Total (s) Traditional static analysis 25.4 8.3 19.2 Transformer 12.3 6.5 18.8 Combined 8.7 4.8 13.5 Inside a CI/CD pipeline Pipeline Average build (s) Bug detection rate Automated fix success Without model integration 45.7 65% 62% With model integration 32.2 91% 88% The build got faster, not slower, which is the counterintuitive result. Catching a defect during the build costs less than the rebuild it prevents. A note for anyone reading the tables closely: in the time table, the first row's total does not equal its two components, while the other two rows do. The paper's discussion section compares 13.5 s against 25.4 s for traditional analysis, so 25.4 is the figure the narrative treats as the static-analysis baseline. Flagging it here so nobody has to wonder whether they misread it. What the paper concludes Contextual information improves both the accuracy and the relevance of automated bug detection and repair. Models given architecture and runtime metadata found defects that per-fragment analysis missed, specifically the ones arising from component interaction and dynamic behaviour. Putting those models in a CI/CD pipeline made real-time detection and patch generation workable rather than theoretical, and cut detection and repair time substantially. The broader argument is about direction of travel: less manual intervention, better code dependability, and a realistic path toward software that repairs a meaningful fraction of its own defects. Where it does not work yet Stated plainly in the paper, and repeated here because a companion repo that only lists strengths is not worth reading. It needs a lot of labelled data. The method depends on a large corpus of code annotated with bugs and their corresponding fixes. Where that corpus is small or absent, performance degrades. This is the binding constraint on applying it to a new language or a private codebase. Generalisation is unproven. Results hold for the languages and settings tested. Whether they transfer across programming languages and frameworks needs further work, and the paper does not claim otherwise. Runtime overhead at scale. In very large projects with deep dependency graphs, collecting and processing the runtime data that makes the method work carries a cost that can offset the gains. Where the field was Twenty-two references, grouped by what they contribute to this work: docs/related-work.md . The short version. Prior program repair split three ways: edit-based methods that localise and patch directly, neural machine translation methods that treat repair as buggy-to-fixed sequence translation, and more recent LLM-based approaches. The NMT framing is easy to implement but has a known failure mode, since buggy and fixed code overlap heavily, models learn to copy rather than to edit. Edit-based methods avoid that but need fiddly multi-step machinery and often assume the broken code parses into a graph, which is exactly what a syntax error breaks. This work sits alongside the context-aware line of research, taking the position that the missing signal is environmental rather than architectural. Authors Kushal Gaddamwar , PDPM IIIT Jabalpur, Computer Science and Engineering, Madhya Pradesh, India Yatharth Srivastava, The LNM Institute of Information Technology, Jaipur Jyoti Parashar, ADGIPS, New Delhi Ritwij Aryan Parmar, Manipal Institute of Technology, Karnataka Aditya Kumar Pandey, Manipal Institute of Technology, Karnataka Virendra Singh Kushwah, VIT Bhopal University, School of Computing Science and AI, Sehore Citing this @inproceedings { gaddamwar2024contextual , title = { Deep Learning for Contextual Bug Detection and Automated Fixes in Software Systems } , author = { Gaddamwar, Kushal and Srivastava, Yatharth and Parashar, Jyoti and Parmar, Ritwij Aryan and Pandey, Aditya Kumar and Kushwah, Virendra Singh } , booktitle = { 2024 2nd International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT) } , pages = { 624--629 } , year = { 2024 } , publisher = { IEEE } , doi = { 10.1109/ICAICCIT64383.2024.10912101 } } A CITATION.cff is included, so GitHub's "Cite this repository" button produces the same thing. Keywords Context-aware bug detection · automated bug fixing · CI/CD integration · software development lifecycle · transformers · graph neural networks · automated program repair Licence Documentation in this repository is released under CC BY 4.0 . The paper itself is © 2024 IEEE; read it on IEEE Xplore .

Published at the 2024 2nd International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT) , IEEE, pp. 624–629.

This repository is the reference companion to the paper: what the work claims, how the method is put together, what the numbers were, and where the limits are. The full text lives on IEEE Xplore and is not redistributed here.

Static analysers and code review look at code in pieces. They read a function, a file, a diff. A large share of production bugs are not in any one of those pieces. They live in the interaction: module A calls B, B quietly changed what it returns, and the failure surfaces somewhere neither author was looking. A tool that only ever sees isolated fragments cannot find that class of bug, because the evidence for it is not inside any single fragment.

This work asks whether giving a model the surrounding context, meaning the project architecture, the dependency structure, and observed runtime behaviour, lets it catch defects that per-file analysis misses, and whether the same model can propose the patch.

3. Dataset Ingestion, Preprocessing & Stochastic Augmentations

A transformer reads the token stream and the syntax tree. That covers what the code says: the syntactic and semantic view a language model is good at.

A graph neural network reads a dependency graph whose nodes are modules and whose edges are the relationships between them. That covers who the code talks to.

The two representations are concatenated and scored together. When the detector flags a file, a decoder conditioned on both representations generates a candidate fix, which is then validated by unit tests and runtime analysis rather than trusted on sight. The whole thing sits inside a CI/CD pipeline so detection and patch generation happen on the commit rather than in a review queue days later.

The mechanism that matters is the metadata. Input is not the code alone but X = [C, M] , where M carries architecture, dependencies, and runtime conditions. That is the difference between a model that can only ask "is this function wrong" and one that can ask "is this function wrong given what it depends on and how it behaves in production ".

Full method, with the equations as published: docs/method.md . Every number with what it does and does not establish: docs/results.md . Questions readers actually ask: docs/notes-for-readers.md .

Reported in the paper. These are the paper's measurements, on the paper's evaluation setup.

The combination beats either branch by roughly three points of accuracy over the transformer and close to six over the graph model. That gap is the paper's central claim: the two views are complementary rather than redundant.

Worth noting which branch is weaker alone. The graph model is the lower scorer on its own, which is what you would expect, since structure without content cannot tell a correct function from a broken one. Its contribution shows up only in combination.

4. Neural Network Backbone, Layer Specifications & Loss Formulations

The build got faster, not slower, which is the counterintuitive result. Catching a defect during the build costs less than the rebuild it prevents.

A note for anyone reading the tables closely: in the time table, the first row's total does not equal its two components, while the other two rows do. The paper's discussion section compares 13.5 s against 25.4 s for traditional analysis, so 25.4 is the figure the narrative treats as the static-analysis baseline. Flagging it here so nobody has to wonder whether they misread it.

Contextual information improves both the accuracy and the relevance of automated bug detection and repair. Models given architecture and runtime metadata found defects that per-fragment analysis missed, specifically the ones arising from component interaction and dynamic behaviour. Putting those models in a CI/CD pipeline made real-time detection and patch generation workable rather than theoretical, and cut detection and repair time substantially.

The broader argument is about direction of travel: less manual intervention, better code dependability, and a realistic path toward software that repairs a meaningful fraction of its own defects.

Stated plainly in the paper, and repeated here because a companion repo that only lists strengths is not worth reading.

It needs a lot of labelled data. The method depends on a large corpus of code annotated with bugs and their corresponding fixes. Where that corpus is small or absent, performance degrades. This is the binding constraint on applying it to a new language or a private codebase.

Generalisation is unproven. Results hold for the languages and settings tested. Whether they transfer across programming languages and frameworks needs further work, and the paper does not claim otherwise.

Runtime overhead at scale. In very large projects with deep dependency graphs, collecting and processing the runtime data that makes the method work carries a cost that can offset the gains.

5. Training Protocol, Convergence Behavior & Experimental Evaluation

Twenty-two references, grouped by what they contribute to this work: docs/related-work.md .

The short version. Prior program repair split three ways: edit-based methods that localise and patch directly, neural machine translation methods that treat repair as buggy-to-fixed sequence translation, and more recent LLM-based approaches. The NMT framing is easy to implement but has a known failure mode, since buggy and fixed code overlap heavily, models learn to copy rather than to edit. Edit-based methods avoid that but need fiddly multi-step machinery and often assume the broken code parses into a graph, which is exactly what a syntax error breaks. This work sits alongside the context-aware line of research, taking the position that the missing signal is environmental rather than architectural.

Kushal Gaddamwar , PDPM IIIT Jabalpur, Computer Science and Engineering, Madhya Pradesh, India

Yatharth Srivastava, The LNM Institute of Information Technology, Jaipur Jyoti Parashar, ADGIPS, New Delhi Ritwij Aryan Parmar, Manipal Institute of Technology, Karnataka Aditya Kumar Pandey, Manipal Institute of Technology, Karnataka Virendra Singh Kushwah, VIT Bhopal University, School of Computing Science and AI, Sehore

@inproceedings { gaddamwar2024contextual , title = { Deep Learning for Contextual Bug Detection and Automated Fixes in Software Systems } , author = { Gaddamwar, Kushal and Srivastava, Yatharth and Parashar, Jyoti and Parmar, Ritwij Aryan and Pandey, Aditya Kumar and Kushwah, Virendra Singh } , booktitle = { 2024 2nd International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT) } , pages = { 624--629 } , year = { 2024 } , publisher = { IEEE } , doi = { 10.1109/ICAICCIT64383.2024.10912101 } }

A CITATION.cff is included, so GitHub's "Cite this repository" button produces the same thing.

Context-aware bug detection · automated bug fixing · CI/CD integration · software development lifecycle · transformers · graph neural networks · automated program repair