Legal Document Summarization | AI Final Year Project Report

Complete Technical Paper Content — Legal Document Summarization

1. Executive Abstract, Problem Formulation & Background

These datasets are used in machine learning (ML) research and have been cited in peer-reviewed academic journals. Datasets are an integral part of the field of machine learning. Major advances in this field can result from advances in learning algorithms (such as deep learning), computer hardware, and, less intuitively, the availability of high-quality training datasets. High-quality labeled training datasets for supervised and semi-supervised machine-learning algorithms are usually difficult and expensive to produce because of the large amount of time needed to label the data. Although they do not need to be labeled, high-quality unlabeled datasets for unsupervised learning can also be difficult and costly to produce. Many organizations, including governments, publish and share their datasets, often using common metadata formats (such as Croissant). The datasets are classified, based on the licenses, into two groups: open data and non-open data. The datasets from various governmental-bodies are presented in List of open government data sites. The datasets are ported on open data portals. They are made available for searching, depositing and accessing through interfaces like Open API. The datasets are made available as various sorted types and subtypes.

The data portal is classified based on its type of license. The open source license based data portals are known as open data portals which are used by many government organizations and academic institutions.

The data portal sometimes lists a wide variety of subtypes of datasets pertaining to many machine learning applications.

The data portals which are suitable for a specific subtype of machine learning application are listed in the subsequent sections.

**RTP set** – 35,087 stationary-point geometries (reactant, transition state and product) drawn from 11,961 elementary reactions, each labeled with density-functional energies, atomic forces and full Hessian matrices at the ωB97X-D/6-31G(d) level. **IRC set** – 34,248 structures along 600 minimum-energy reaction paths, used to test extrapolation beyond trained stationary points. **NMS set** – 62,527 off-equilibrium geometries generated by normal-mode sampling to probe model robustness under thermal perturbations. The collection underpins the study Does Hessian Data Improve the Performance of Machine Learning Potentials? and was used to train and benchmark the machine-learning interatomic potentials reported therein. The dataset itself is distributed under a CC licence via Figshare.

OpenML: Web platform with Python, R, Java, and other APIs for downloading hundreds of machine learning datasets, evaluating algorithms on datasets, and benchmarking algorithm performance against dozens of other algorithms. PMLB: A large, curated repository of benchmark datasets for evaluating supervised machine learning algorithms. Provides classification and regression datasets in a standardized format that are accessible through a Python API. Metatext NLP: https://metatext.io/datasets web repository maintained by community, containing nearly 1000 benchmark datasets, and counting. Provides many tasks from classification to QA, and various languages from English, Portuguese to Arabic. Appen: Off The Shelf and Open Source Datasets hosted and maintained by the company. These biological, image, physical, question answering, signal, sound, text, and video resources number over 250 and can be applied to over 25 different use cases.

2. Related Work, Literature Survey & Algorithmic Foundations

In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called bullshitting, confabulation, delusion, or mirage) is a response generated by AI that contains false or misleading information presented as fact. These terms draw a loose analogy with human psychology, where a hallucination typically involves false percepts. Chatbots powered by large language models (LLMs), like ChatGPT, may embed plausible-sounding random falsehoods within their generated content, such as fabricated citations. Detecting and mitigating errors and hallucinations pose significant challenges for practical deployment and reliability of LLMs in high-stakes scenarios, such as chip design, supply chain logistics, and medical diagnostics. Some software engineers and statisticians have criticized the specific term "AI hallucination" for unreasonably anthropomorphizing computers. Symbolic artificial intelligence models generally do not produce hallucinations, unlike large language models.

Uses, definitions and characterizations of the term hallucination in the context of LLMs include:

"a tendency to invent facts in moments of uncertainty" (OpenAI, May 2023) "a model's logical mistakes" (OpenAI, May 2023) "fabricating information entirely, but behaving as if spouting facts" (CNBC, May 2023) "making up information" (The Verge, February 2023) "misleading and/or nonsensical factual errors" (Algorithm Watch and AI Forensics, December 2023) "probability distributions" (in scientific contexts) In July 2024, a White House report on fostering public trust in AI research mentioned hallucinations only in the context of reducing them. Notably, when acknowledging David Baker's Nobel Prize-winning work with AI-generated proteins, the Nobel committee avoided it entirely, instead referring to "imaginative protein creation". Hicks, Humphries, and Slater, in their article in Ethics and Information Technology, argue that the output of LLMs is bullshit under Harry Frankfurt's definition, and that the models are "in an important way indifferent to the truth of their outputs", with true statements only accidentally true, and false ones accidentally false. Some researchers also call the uncritical use of AI botshit. The term mirage has been proposed as an alternative to the use of hallucination, framing false or misleading responses as predictable results of LLM data processing created by both AI training datasets and the rules engineered into the LLM rather than a mental breaking from reality. This shift in language aims to increase AI literacy by using terminology that avoids anthropomorphic implications that LLMs have conscious minds with their own intent, shows that untrue outputs come from patterns in training data, and indicates that outputs are novel syntheses of training data requiring critical human judgment.

Other examples involve baiting ChatGPT with a false premise to see if it embellishes upon the premise. When asked about "Harold Coward's idea of dynamic canonicity", ChatGPT fabricated that Coward wrote a book titled Dynamic Canonicity: A Model for Biblical and Theological Interpretation, arguing that religious principles are actually in a constant state of change. When pressed, ChatGPT continued to insist that the book was real. Asked for proof that dinosaurs built a civilization, ChatGPT claimed there were fossil remains of dinosaur tools and stated, "Some species of dinosaurs even developed primitive forms of art, such as engravings on stones". When prompted that "Scientists have recently discovered churros, the delicious fried-dough pastries ... (are) ideal tools for home surgery", ChatGPT claimed that a "study published in the journal Science" found that the dough is pliable enough to form into surgical instruments that can get into hard-to-reach places, and that the flavor has a calming effect on patients. By 2023, analysts considered frequent hallucination to be a major problem in LLM technology, with a Google executive identifying hallucination reduction as a "fundamental" task for ChatGPT competitor Google Gemini. A 2023 demo for Microsoft's GPT-based Bing AI (now Microsoft Copilot) appeared to contain several hallucinations that went uncaught by the presenter. In June 2023, Mark Walters, a gun rights activist and radio personality, sued OpenAI in a Georgia state court after ChatGPT mischaracterized a legal complaint in a manner alleged to be defamatory against Walters. The complaint in question was brought in May 2023 by the Second Amendment Foundation against Washington attorney general Robert W. Ferguson for allegedly violating their freedom of speech, whereas the ChatGPT-generated summary bore no resemblance and claimed that Walters was accused of embezzlement and fraud while holding a Second Amendment Foundation office post that he never held in real life. According to AI legal expert Eugene Volokh, OpenAI is likely not shielded against this claim by Section 230, because OpenAI likely "materially contributed" to the creation of the defamatory content. In May 2025, Judge Tracie Cason of Gwinnett County Superior Court ruled in favor of OpenAI, stating that the plaintiff had not shown he was defamed, as Walters failed to show that OpenAI's statements about him were negligent or made with "actual malice". In February 2024, Canadian airline Air Canada was ordered by the Civil Resolution Tribunal in Moffatt v. Air Canada to pay damages to a customer and honor a bereavement fare policy that was hallucinated by a support chatbot, which incorrectly stated that customers could retroactively request a bereavement discount within 90 days of the date the ticket was issued (the actual policy does not allow the fare to be requested after the flight is booked). The Tribunal rejected Air Canada's defense that the chatbot was a "separate legal entity that is responsible for its own actions". In October 2025, several hallucinations, including non-existent academic sources and a fake quote from a federal court judgement were discovered in an A$440,000 report written by Deloitte and submitted to the Australian government in July. The company later submitted a revised report with these errors removed, and will issue a partial refund to the government. The following month, in November 2025, The Independent, a news publication in Newfoundland and Labrador, Canada, discovered that Deloitte's CA$1.6 million Health Human Resources Plan for the Government of Newfoundland and Labrador commissioned in May 2025 contained at least four false citations to non-existent research papers.

In reality, such textures and outlines would not be represented by single nodes, but rather by associated weight patterns of multiple nodes.

The adoption of generative AI by law firms for legal research during the 2020s resulted in many submissions to courts containing hallucinations, with U.S. judges nationwide sanctioning or reprimanding lawyers. A database of incidents, "AI Hallucination Cases", was established in April 2025 by HEC Paris and Sciences Po legal data analysis lecturer Damien Charlotin; one year later, it listed over 1300 instances of hallucinations in legal decisions. In May 2023, it was discovered that Stephen Schwartz had submitted six fake case precedents generated by ChatGPT in his brief to the Southern District of New York on Mata v. Avianca, Inc., a personal injury case against the airline Avianca. Schwartz said that he had never previously used ChatGPT, that he did not recognize the possibility that ChatGPT's output could have been fabricated, and that ChatGPT continued to assert the authenticity of the precedents after their nonexistence was discovered. In response, Brantley Starr of the Northern District of Texas banned the submission of AI-generated case filings that have not been reviewed by a human, noting that:

3. Dataset Ingestion, Preprocessing & Stochastic Augmentations

Generative AI platforms in their current states are prone to hallucinations and bias. On hallucinations, they make stuff up—even quotes and citations. Another issue is reliability or bias. While attorneys swear an oath to set aside their personal prejudices, biases, and beliefs to faithfully uphold the law and represent their clients, generative artificial intelligence is the product of programming devised by humans who did not have to swear such an oath. As such, these systems hold no allegiance to any client, the rule of law, or the laws and Constitution of the United States (or, as addressed above, the truth). Unbound by any sense of duty, honor, or justice, such programs act according to computer code rather than conviction, based on programming rather than principle. On 23 June 2023, judge P. Kevin Castel dismissed the Mata case and issued a $5,000 fine to Schwartz and another lawyer—who had both continued to stand by the fictitious precedents despite Schwartz's previous claims—for bad faith conduct. Castel characterized numerous errors and inconsistencies in the opinion summaries, describing one of the cited opinions as "gibberish" and "[bordering] on nonsensical". In May 2025, high-profile pro se plaintiff Ashley Gjøvik, who has a Juris Doctor from Santa Clara University School of Law, was admonished in U.S. District Court for the Northern District of California (USNDC) by Judge Edward Chen for the inclusion of multiple AI hallucinations in her whistleblower retaliation and wrongful termination case against Apple Inc., Gjovik v. Apple Inc., warning that future instances would result in Gjøvik being held in contempt of court and her pro se privileges revoked. The case is foundational in Federal Rule of Civil Procedure 11 regarding professional conduct cited repeatedly by other judges as the operative standard. It has also been discussed by the American Bar Association in updating the model professional rules and cited in its opinion regarding the use of generative AI. In September 2025, USNDC Magistrate Judge Peter Kang sanctioned Webb Law attorney Katherine Cervantes for her submission of July 21 that included an AI hallucination which she stated had been obtained from Westlaw AI, the Thomson Reuters' artificial intelligence tool. On April 28, 2026, Judge Kang sanctioned and fined Webb Law Group managing partner Lenden Webb for insufficient supervision of Cervantes—a junior lawyer incorporating AI for the first time—therefore allowing the legal submission that included an AI hallucinated citation of July 21, 2025. On 18 March 2026, Clayton County, Georgia, Assistant District Attorney Deborah Leslie appeared before the Supreme Court of Georgia to deliver oral arguments regarding an order to deny an appeal for a murder conviction, which a Clayton County Superior Court judge used in denying the defendant's motion. During the session, Chief Justice Nels S. D. Peterson noted at least five fictitious citations within the filing, as well as five others that did not support the proposition for which they were cited. On 30 March 2026, Leslie admitted in an affidavit to not verifying AI-generated "expanded legal research" used to prepare the order, which District Attorney Tasha Mosley corroborated in a written apology to Peterson. On 5 May, the Supreme Court vacated the lower court's ruling and ordered for a new order with legitimate citations to be written. Leslie was also suspended from practicing before the Supreme Court for six months and was ordered to take additional education for ethics and proper use of artificial intelligence. In April 2026, Sullivan & Cromwell submitted an apology to the U.S. Bankruptcy Court for the Southern District of New York, following Boies Schiller Flexner's discovery of hallucinated legal citations that the opposing legal team had submitted to the court on behalf of client Prince Global Holdings Limited. On 28 April 2026, U.S. Magistrate Judge Robert T. Numbers formally reprimanded former assistant U.S. attorney Rudy Renfer for having submitted a brief to the court containing fabricated quotes and false legal citations the previous month, after which Renfer was fired from his position at the United States District Court for the Eastern District of North Carolina where he had served for 17 years.

Hybrid GRU-Based Pointer-Generator Network for Legal Clause Summarization Deep Learning-based Legal Text Summarization using Bidirectional GRU, Pointer-Generator Networks, Coverage Attention, and Beam Search Decoding. Overview Legal documents are often lengthy, complex, and difficult to analyze manually. This project presents an abstractive legal text summarization system designed to generate concise and coherent summaries of legal clauses and legislative documents. The proposed architecture combines a Bidirectional GRU Encoder, Pointer-Generator Network, Coverage Attention Mechanism, and Beam Search Decoding to improve factual consistency, handle out-of-vocabulary legal terminology, and reduce repetitive text generation. This work was developed as a research project and was presented at the IEEE International Conference on Innovative Trends in Communication and Computer Technology (I3CTCON 2026). Key Features Bidirectional GRU Encoder for contextual understanding Pointer-Generator Network for copying important legal terms Coverage Attention Mechanism to reduce repetition Coverage Loss Annealing during training Beam Search Decoding for improved summary quality ROUGE-based evaluation framework BillSum legal dataset integration End-to-end legal clause summarization pipeline Dataset BillSum Dataset BillSum is a publicly available corpus of United States Congressional and California state legislative bills paired with expert-written summaries. Dataset Split Samples Training Set 8,000 Test Set 500 The dataset contains long-form legal documents requiring abstractive summarization while preserving critical legal information. System Architecture Input Legal Document │ ▼ Bidirectional GRU Encoder │ ▼ Coverage Attention Layer │ ▼ Pointer-Generator Decoder │ ▼ Beam Search Decoding │ ▼ Generated Legal Summary Core Components Bidirectional GRU Encoder Processes input text in both forward and backward directions to capture long-range contextual dependencies. Coverage Attention Maintains a coverage vector that tracks previously attended words, significantly reducing repetitive phrase generation. Pointer-Generator Network Combines two mechanisms: Vocabulary-based word generation Direct copying of important source tokens This is particularly useful for handling legal terminology and rare entities. Beam Search Decoding Generates higher-quality summaries by exploring multiple candidate sequences during inference. Technologies Used Category Tools Programming Language Python Deep Learning PyTorch Data Processing Pandas, NumPy NLP NLTK Evaluation ROUGE Score Visualization Matplotlib Machine Learning Utilities Scikit-learn Training Configuration Parameter Configuration Encoder Bidirectional GRU Decoder Pointer-Generator GRU Attention Coverage Attention Decoding Strategy Beam Search Loss Function NLL Loss + Coverage Loss Framework PyTorch Results Performance was evaluated using standard ROUGE metrics. Metric Score ROUGE-1 0.504 ROUGE-2 0.285 ROUGE-L 0.363 Key Observations Coverage attention reduced repetitive phrase generation. Pointer-generator improved handling of legal terminology. Beam search produced more coherent summaries compared to greedy decoding. The model generated concise summaries while preserving important legal information. Repository Structure legal-clause-summarization/ │ ├── legal_summarization.ipynb ├── requirements.txt └── README.md Installation git clone https://github.com/AnaghaHegde05/legal-clause-summarization.git cd legal-clause-summarization pip install -r requirements.txt Running the Project jupyter notebook legal_summarization.ipynb Run the notebook sequentially to: Load and preprocess the BillSum dataset Train the Pointer-Generator model Evaluate using ROUGE metrics Generate summaries for unseen legal documents Research Contribution This work proposes a hybrid GRU-based Pointer-Generator architecture enhanced with Coverage Attention for legal clause summarization. Major contributions include: Reducing repetitive text generation through coverage mechanisms Improving handling of legal terminology using pointer networks Enhancing summary coherence through beam search decoding Evaluating performance on a real-world legal summarization dataset Future Work Transformer-based legal summarization models LegalBERT integration Multi-document legal summarization FastAPI deployment for real-time inference Web-based summarization interface Explainable AI techniques for summary interpretation Publication Hybrid GRU-Based Pointer-Generator Model for Legal Clause-Level Summarization Presented at: IEEE International Conference on Innovative Trends in Communication and Computer Technology (I3CTCON 2026) Author Anagha Hegde B.E. Computer Science Engineering KLE Technological University Interests: Artificial Intelligence Machine Learning Natural Language Processing Intelligent Systems

Deep Learning-based Legal Text Summarization using Bidirectional GRU, Pointer-Generator Networks, Coverage Attention, and Beam Search Decoding.

Legal documents are often lengthy, complex, and difficult to analyze manually. This project presents an abstractive legal text summarization system designed to generate concise and coherent summaries of legal clauses and legislative documents.

The proposed architecture combines a Bidirectional GRU Encoder, Pointer-Generator Network, Coverage Attention Mechanism, and Beam Search Decoding to improve factual consistency, handle out-of-vocabulary legal terminology, and reduce repetitive text generation.

This work was developed as a research project and was presented at the IEEE International Conference on Innovative Trends in Communication and Computer Technology (I3CTCON 2026).

4. Neural Network Backbone, Layer Specifications & Loss Formulations

BillSum is a publicly available corpus of United States Congressional and California state legislative bills paired with expert-written summaries.

The dataset contains long-form legal documents requiring abstractive summarization while preserving critical legal information.

Input Legal Document │ ▼ Bidirectional GRU Encoder │ ▼ Coverage Attention Layer │ ▼ Pointer-Generator Decoder │ ▼ Beam Search Decoding │ ▼ Generated Legal Summary

Processes input text in both forward and backward directions to capture long-range contextual dependencies.

Maintains a coverage vector that tracks previously attended words, significantly reducing repetitive phrase generation.

Generates higher-quality summaries by exploring multiple candidate sequences during inference.

5. Training Protocol, Convergence Behavior & Experimental Evaluation

legal-clause-summarization/ │ ├── legal_summarization.ipynb ├── requirements.txt └── README.md

git clone https://github.com/AnaghaHegde05/legal-clause-summarization.git cd legal-clause-summarization pip install -r requirements.txt

This work proposes a hybrid GRU-based Pointer-Generator architecture enhanced with Coverage Attention for legal clause summarization.

IEEE International Conference on Innovative Trends in Communication and Computer Technology (I3CTCON 2026)

Deep learning-based legal text summarization using GRU Encoder-Decoder, Pointer-Generator Network, Coverage Attention, and Beam Search on the BillSum dataset.

Project Title : Legal Document Summarization Overview This project introduces an automated solution for summarizing legal texts, leveraging machine learning techniques. In response to the growing complexity and volume of legal documents, there's a pressing need for efficient information extraction methods to support decision-making processes. Our approach employs natural language processing (NLP) techniques and deep learning architectures, including Transformer models, to develop a robust summarization model. By analyzing diverse legal documents, our system extracts key insights and condenses lengthy texts into concise summaries. Evaluation results confirm the effectiveness of our approach in accurately capturing essential information while maintaining readability. By automating the summarization process, our project contributes to streamlining legal workflows and enhancing productivity for legal professionals, researchers, and businesses. This project represents a significant advancement in the field of legal tech, offering a valuable tool for navigating the complexities of legal documentation. By leveraging innovation, we aim to revolutionize the way legal information is processed and utilized, benefiting legal professionals and the broader community alike. Setup Instructions Upload Files to Google Colab : Begin by uploading the Legal_text_summarization.ipynb and app.py files to Google Colab for seamless execution and collaboration. Generate Hugging Face Token : Log in to huggingface.com and navigate to Your Profile -> Access Tokens -> New Token. Generate a new token and name it for identification purposes. Copy the token value. Set up Colab Secrets : In Google Colab, navigate to the secrets tab and add a new secret named 'HF_TOKEN'. Paste the copied token value into the corresponding field. Run the Project : Execute each cell of the notebook sequentially to set up the environment, load necessary libraries, and process the legal documents. Access the Web Application : Upon completion, the notebook will generate a URL for the web application. Follow the instructions to access the Streamlit app. You will be prompted to enter the IPv4 address provided by running the command !wget -q -0 - ipv4.icanhazip.com in code cell. Paste this address into the security screen to gain access to the summarization app. Files and Resources fine_tuned_model : This folder is generated from the training and storing the T5 model which is trained on the dataset and being used to generate summary in the web application. app.py : The app.py is the file that has the code of the streamlit app that handles the queries and input pdfs for which model will be used to summarize it late on. Legal_text_summarization.ipynb : The Legal_text_summarization.ipynb file is the main file that contains all the dataset loading, preprocessing and model generation, training and model fine-tuning functions. Features Efficient Summarization : Utilizes state-of-the-art NLP T5 model to generate accurate and concise summaries of legal documents. User-Friendly Interface : The Streamlit web application provides an intuitive interface for users to interact with the summarization tool. Secure Access : Ensures security by requiring users to authenticate their access through the provided IPv4 address. Contributors [Shrey Patel]