Ai Contract Review | AI Final Year Project Report

Complete Technical Paper Content — Ai Contract Review

1. Executive Abstract, Problem Formulation & Background

Contract Review AI Agent (On-premise, Opensource) End-to-end, local-first contract review AI Agent that combines LLAMA 3.18B LLM (served via Ollama ) with a lightweight TF‑IDF + Logistic Regression text classifier for clause risk labeling. Architecture The agent orchestrates clause extraction and classification, calling a domain-tuned scikit‑learn model as a tool while using an on‑prem LLM (e.g., LLaMA 3 via Ollama ) for reasoning, clause extraction and control. Key Components (by folder) /contractreviewagent.py, /agentexecutor.py – Agent executor, prompt templates, chunking, and tool wiring. /tools ,/tools/regressionmodeltrainer/ – The scikit‑learn clause_text_classifier.joblib (TF‑IDF + Logistic Regression) , labelled datasets and training scripts. archdiagrams/ – Documentations, Architecture diagrams etc. How It Works Ingestion & Chunking – The agent/executor ingests contract text , normalizes text, and chunks content. Orchestration (LLM) – An on‑prem LLM (e.g., LLaMA 3 ) hosted with Ollama handles reasoning, tool selection, and flow control. Risk Classification (Tool) – The agent calls a local scikit‑learn classifier ( TF‑IDF + LogisticRegression ) to assign labels to clause text. Persistence – Structured outputs (JSON) are stored for downstream use (UI, search, analytics). HMI/UI – A simple UI or scripts consume the stored results for review. Quickstart Prerequisites Python 3.10+ Ollama installed locally, with a model pulled, e.g.: ollama pull llama3 (Optional) Poppler/Tesseract for PDF/OCR if needed by your pipeline. Setup git clone https://github.com/abinjaik/contractreviewaiagent.git cd contractreviewaiagent git checkout main python -m venv .venv && source .venv/bin/activate # (Windows: .venv\Scripts\activate) pip install -r requirements.txt Train (optional) & Load Classifier If you need to retrain the classifier: python /tools/regressionmodeltrainer/linearmodelclassification.py # expects msa_clauses_dataset_v3.csv The classifier model ( TfidfVectorizer + LogisticRegression ) is saved as clause_text_classifier.joblib . Run the Agent python agentexecutor.py --input CONTRACTTEXT This will save the output JSON inside extracted_clauses The LLM agent can call classify_clause_risk during execution to produce deterministic, low‑latency labels. Future Plans Set up and configure jobs to parse, extract the PDF, Word content and ingest it to the Agent Store the JSON output from Agent to a Object store and Map with the source document Seemless integration between Web Front end, and Object store Setup MLFlow - Tracebility, Auditing License Apache 3.0

End-to-end, local-first contract review AI Agent that combines LLAMA 3.18B LLM (served via Ollama ) with a lightweight TF‑IDF + Logistic Regression text classifier for clause risk labeling.

The agent orchestrates clause extraction and classification, calling a domain-tuned scikit‑learn model as a tool while using an on‑prem LLM (e.g., LLaMA 3 via Ollama ) for reasoning, clause extraction and control.

git clone https://github.com/abinjaik/contractreviewaiagent.git cd contractreviewaiagent git checkout main python -m venv .venv && source .venv/bin/activate # (Windows: .venv\Scripts\activate) pip install -r requirements.txt

python /tools/regressionmodeltrainer/linearmodelclassification.py # expects msa_clauses_dataset_v3.csv

The classifier model ( TfidfVectorizer + LogisticRegression ) is saved as clause_text_classifier.joblib .

The LLM agent can call classify_clause_risk during execution to produce deterministic, low‑latency labels.

2. Related Work, Literature Survey & Algorithmic Foundations

AI-powered contract review agent that extracts and classifies clauses from contracts using Llama 3 (Ollama) and a machine learning classifier. Features streaming extraction, FastAPI clause prediction, and modular Python code for legal tech automation and analysis.

AI Contract Risk Review Application - Enhanced with LLM Analysis A powerful Streamlit-based application that uses advanced LLM analysis to analyze contracts, identify risks, and suggest improvements. Built with OpenAI GPT models and Exa.ai search for comprehensive contract review. 🚀 Enhanced Features Advanced Document Processing Multi-Library PDF Extraction : PyMuPDF, pdfplumber, PyPDF2 with automatic fallback Robust DOCX Processing : python-docx and docx2txt with error handling Smart Text Cleaning : Automatic formatting cleanup and OCR error correction Markdown Conversion : Convert contracts to structured markdown format LLM-Powered Analysis (No Regex!) Intelligent Contract Type Detection : Uses GPT models instead of pattern matching Smart Governing Law Identification : LLM-based jurisdiction detection Advanced Clause Extraction : AI identifies and categorizes key contract clauses Risk Assessment : Individual clause risk analysis with detailed recommendations Features Multi-format Support : Upload PDF, DOCX, or TXT contract files LLM-Based Analysis : Uses OpenAI GPT models for intelligent contract understanding Comprehensive Clause Detection : Identifies termination, liability, confidentiality, and more Individual Clause Risk Assessment : Detailed risk analysis for each clause type Regulatory Research : Integrates Exa.ai for up-to-date legal information Visual Highlighting : Color-coded risk highlighting in original text Detailed Suggestions : AI-powered improvement recommendations Multiple Export Options : Download as JSON or Markdown Supported Contract Types The LLM can intelligently detect and analyze: Non-Disclosure Agreements (NDA) Master Service Agreements (MSA) Data Processing Agreements (DPA) Service Level Agreements (SLA) Employment Contracts License Agreements Purchase Agreements Lease Agreements General Commercial Contracts Installation Install required dependencies : pip install -r requirements.txt Set up environment variables : Copy .env.example to .env and add your API keys: OPENAI_API_KEY=your_openai_api_key_here EXA_API_KEY=your_exa_api_key_here Usage Start the application : streamlit run Home.py Open your browser and navigate to http://localhost:8501 Upload a contract using the file uploader Review the LLM analysis : Contract type and governing law detection Key clauses extraction with summaries Individual clause risk assessments Color-coded risk highlighting Download results as JSON or Markdown Project Structure contract_review/ ├── Home.py # Main Streamlit application ├── chain.py # Original LangChain integration ├── parsers_llm.py # Enhanced document parsing with LLM analysis ├── llm_analyzer.py # LLM-based contract analysis engine ├── exa_search.py # Exa.ai search integration ├── prompts/ # LLM prompts directory │ ├── __init__.py │ └── contract_analysis.py # Contract analysis prompts ├── requirements.txt # Dependencies (no version pinning) ├── test_functionality.py # Test script ├── sample_contract.txt # Sample contract for testing ├── .env.example # Environment variables template └── README.md # This file LLM Analysis Pipeline Document Processing : Multi-library text extraction with fallbacks Contract Type Detection : LLM analyzes content to determine contract category Governing Law Identification : AI extracts jurisdiction information Clause Extraction : LLM identifies and categorizes key contract provisions Risk Assessment : Individual analysis of each clause for potential issues Recommendation Generation : AI-powered suggestions for improvements API Requirements OpenAI API : For GPT-based contract analysis (gpt-4.1-mini supported) Exa.ai API : For regulatory and legal research Key Enhancements No More Regex! Pure LLM Analysis : Replaced all regex patterns with intelligent AI processing Context-Aware : Understands contract context and legal terminology Structured Output : JSON-formatted results for consistent processing Enhanced Document Processing Fallback Extraction : Multiple libraries ensure text extraction success Format Support : PDF, DOCX, TXT with intelligent handling Text Cleaning : Automatic cleanup of OCR artifacts and formatting issues Advanced UI Real-time Feedback : Progress indicators during LLM processing Detailed Previews : Text extraction and clause analysis previews Enhanced Visualization : Better formatting and risk color coding Testing Run the test script to verify all components are working: python test_functionality.py Disclaimer This application uses AI for contract analysis and should be used as a tool to assist legal review, not replace professional legal advice. All AI-generated analysis should be reviewed by qualified legal professionals. License This project is provided as-is for educational and demonstration purposes.

A powerful Streamlit-based application that uses advanced LLM analysis to analyze contracts, identify risks, and suggest improvements. Built with OpenAI GPT models and Exa.ai search for comprehensive contract review.

contract_review/ ├── Home.py # Main Streamlit application ├── chain.py # Original LangChain integration ├── parsers_llm.py # Enhanced document parsing with LLM analysis ├── llm_analyzer.py # LLM-based contract analysis engine ├── exa_search.py # Exa.ai search integration ├── prompts/ # LLM prompts directory │ ├── __init__.py │ └── contract_analysis.py # Contract analysis prompts ├── requirements.txt # Dependencies (no version pinning) ├── test_functionality.py # Test script ├── sample_contract.txt # Sample contract for testing ├── .env.example # Environment variables template └── README.md # This file

This application uses AI for contract analysis and should be used as a tool to assist legal review, not replace professional legal advice. All AI-generated analysis should be reviewed by qualified legal professionals.

AI-Powered Architecture Review: A Step-by-Step Guide Architecture reviews catch the most expensive bugs — the ones that require weeks of refactoring to fix after launch. But traditional architecture reviews are slow, inconsistent, and often skipped under deadline pressure. With AI coding assistants like Claude Code, you can run systematic architecture reviews in minutes instead of days. This guide walks through the complete process, from initial analysis to final recommendations. Why AI Architecture Reviews? Manual architecture reviews suffer from three problems: Inconsistency — Different reviewers focus on different aspects. Critical issues get missed. Speed — A thorough review of a complex system can take days of a senior engineer's time. Knowledge gaps — No single reviewer is an expert in every architectural pattern, framework, and anti-pattern. AI-powered reviews solve all three: they follow a consistent checklist, complete in minutes, and draw from broad training on architecture patterns across thousands of codebases. Prerequisites Before starting, make sure you have: An AI coding assistant (Claude Code, Copilot, Cursor, etc.) Access to the codebase you want to review Architecture documentation (if it exists — the AI can work without it) Step 1: Establish the System Context Start by giving your AI assistant the big picture. The quality of an architecture review depends heavily on understanding the system's purpose, constraints, and operating environment. Prompt: Analyze this codebase and provide a high-level architectural overview: 1. What is the system's primary purpose? 2. What architectural pattern does it follow (monolith, microservices, serverless, etc.)? 3. What are the main components/modules and their responsibilities? 4. What external dependencies does it rely on? 5. What data stores does it use and how does data flow between components? Focus on facts observable from the code, not assumptions. What to look for in the response: Does the AI's understanding match your mental model? Are there components the AI identified that you weren't aware of? Are dependencies accurately captured? If the AI's understanding diverges from reality, correct it before proceeding. Accurate context is the foundation of everything that follows. Step 2: Evaluate Structural Integrity Now dive into the architecture's structural soundness. This is where you catch the design-level issues that are hardest to fix later. Prompt: Evaluate the structural integrity of this architecture: ## Coupling Analysis - Identify tightly coupled components (high dependency between modules) - Flag circular dependencies - Check if dependency direction follows the Dependency Inversion Principle ## Cohesion Check - Are modules focused on a single responsibility? - Are there "god classes" or "god modules" that do too much? - Could any module be split for better maintainability? ## Layer Violations - Does the architecture have clear layers (presentation, business, data)? - Are there places where layers are bypassed? - Do lower layers depend on upper layers? (they shouldn't) ## Boundary Integrity - Are module boundaries well-defined? - Is there shared mutable state between modules? - Do modules expose implementation details they shouldn't? Rate each area: GOOD / NEEDS ATTENTION / CRITICAL Pro tip: Run this analysis on each major module independently, then again on the system as a whole. Module-level issues often reveal system-level patterns. Step 3: Assess Scalability and Performance Architecture decisions that work at 100 users often break at 10,000. This step identifies scaling bottlenecks before they become production incidents. Prompt: Analyze this architecture for scalability and performance: 1. **Bottleneck identification:** - What is the most likely performance bottleneck under 10x current load? - Are there single points of failure? - Which components cannot scale horizontally? 2. **Data layer scalability:** - Can the database handle 10x growth in data volume? - Are queries optimized for the expected data patterns? - Is there a caching strategy? Is it appropriate? 3. **Concurrency model:** - How does the system handle concurrent requests? - Are there race conditions or deadlock risks? - Is the async/sync boundary well-defined? 4. **Resource usage:** - Are there memory leaks or unbounded growth patterns? - Are connections pooled appropriately? - Are timeouts configured for all external calls? Provide specific code locations for each finding. Step 4: Security Architecture Review Security flaws at the architecture level — like missing authentication boundaries or improper data isolation — are far more dangerous than code-level bugs. Prompt: Perform a security architecture review: 1. **Authentication & Authorization:** - Is authentication enforced at every entry point? - Is authorization granular enough (RBAC, ABAC)? - Are there endpoints that bypass auth checks? 2. **Data Protection:** - Is sensitive data encrypted at rest and in transit? - Are secrets (API keys, passwords) stored securely? - Is PII handled according to data protection principles? 3. **Input Boundaries:** - Are all external inputs validated and sanitized? - Are there injection risks (SQL, command, template)? - Are file uploads restricted and validated? 4. **Attack Surface:** - What is the external attack surface? - Are unnecessary ports or endpoints exposed? - Is there proper rate limiting? 5. **Dependency Security:** - Are dependencies up to date? - Are there known vulnerabilities in current dependencies? Flag findings as: INFO / WARNING / CRITICAL Step 5: Evaluate Maintainability and Evolvability Architecture should support change, not resist it. This step assesses how well the system will handle future requirements. Prompt: Evaluate the long-term maintainability of this architecture: 1. **Change impact analysis:** - If we need to swap the database, how many files change? - If we need to add a new API endpoint, what is the minimum change set? - If we need to change the authentication provider, what breaks? 2. **Testing architecture:** - Can components be tested in isolation? - Are there proper integration test boundaries? - Is the test infrastructure maintainable? 3. **Configuration management:** - Is configuration separated from code? - Can the system run in different environments without code changes? - Are feature flags used for gradual rollouts? 4. **Documentation alignment:** - Does the code match the documented architecture? - Are architectural decisions recorded (ADRs)? - Can a new developer understand the system from docs + code? Provide a maintainability score (1-10) with justification. Step 6: Generate the Architecture Review Report Finally, compile all findings into a structured report that can be shared with the team and tracked over time. Prompt: Based on the analysis above, generate a structured Architecture Review Report: ## Executive Summary (2-3 sentences: overall health, biggest risk, top recommendation) ## Findings Table | ID | Area | Severity | Finding | Recommendation | |----|------|----------|---------|----------------| | A1 | ... | CRITICAL/WARNING/INFO | ... | ... | ## Top 3 Recommendations (Ordered by impact, with effort estimates) ## Architecture Debt Register (Issues to track and address over time) ## Positive Patterns (What the architecture does well — important for team morale) Best Practices for AI Architecture Reviews Do: Run reviews regularly — not just before launches. Monthly reviews catch drift early. Review in layers — system level, then module level, then integration points. Save review history — track architectural debt over time. Combine AI + human judgment — AI finds patterns; humans evaluate trade-offs. Don't: Don't skip context — garbage in, garbage out. Step 1 matters most. Don't accept everything — AI can hallucinate architectural issues. Verify critical findings. Don't review only new code — legacy systems benefit most from architecture reviews. Don't ignore positive findings — knowing what works well is as valuable as finding problems. Automating Architecture Reviews For teams that want to run architecture reviews automatically (e.g., on every PR that touches core modules), consider: MCP integration — Use an MCP server to make review prompts available directly in your AI assistant CI/CD hooks — Trigger architecture checks on PRs that modify module boundaries Scheduled reviews — Run full architecture reviews weekly using a pre-built prompt toolkit Going Further This guide covers the core architecture review workflow. For production use, you will want: Domain-specific prompts — tailored to your tech stack (microservices, serverless, monolith) Pattern libraries — reference architectures for comparison Customizable severity thresholds — aligned with your team's risk tolerance Integration templates — for CI/CD and automated review pipelines The AI Developer Toolkit includes 4 architecture-specific prompts covering system design review, microservices patterns, database architecture, and API design — each tested on real production codebases. Want More? These are samples from the AI Developer Toolkit — 32+ production-tested prompts for code review, security audits, architecture reviews, debugging, and test generation. Get the Full Toolkit ($39) | Free MCP Server

Architecture reviews catch the most expensive bugs — the ones that require weeks of refactoring to fix after launch. But traditional architecture reviews are slow, inconsistent, and often skipped under deadline pressure.

3. Dataset Ingestion, Preprocessing & Stochastic Augmentations

With AI coding assistants like Claude Code, you can run systematic architecture reviews in minutes instead of days. This guide walks through the complete process, from initial analysis to final recommendations.

AI-powered reviews solve all three: they follow a consistent checklist, complete in minutes, and draw from broad training on architecture patterns across thousands of codebases.

Start by giving your AI assistant the big picture. The quality of an architecture review depends heavily on understanding the system's purpose, constraints, and operating environment.

Analyze this codebase and provide a high-level architectural overview: 1. What is the system's primary purpose? 2. What architectural pattern does it follow (monolith, microservices, serverless, etc.)? 3. What are the main components/modules and their responsibilities? 4. What external dependencies does it rely on? 5. What data stores does it use and how does data flow between components? Focus on facts observable from the code, not assumptions.

If the AI's understanding diverges from reality, correct it before proceeding. Accurate context is the foundation of everything that follows.

Now dive into the architecture's structural soundness. This is where you catch the design-level issues that are hardest to fix later.

Evaluate the structural integrity of this architecture: ## Coupling Analysis - Identify tightly coupled components (high dependency between modules) - Flag circular dependencies - Check if dependency direction follows the Dependency Inversion Principle ## Cohesion Check - Are modules focused on a single responsibility? - Are there "god classes" or "god modules" that do too much? - Could any module be split for better maintainability? ## Layer Violations - Does the architecture have clear layers (presentation, business, data)? - Are there places where layers are bypassed? - Do lower layers depend on upper layers? (they shouldn't) ## Boundary Integrity - Are module boundaries well-defined? - Is there shared mutable state between modules? - Do modules expose implementation details they shouldn't? Rate each area: GOOD / NEEDS ATTENTION / CRITICAL

4. Neural Network Backbone, Layer Specifications & Loss Formulations

Pro tip: Run this analysis on each major module independently, then again on the system as a whole. Module-level issues often reveal system-level patterns.

Architecture decisions that work at 100 users often break at 10,000. This step identifies scaling bottlenecks before they become production incidents.

Analyze this architecture for scalability and performance: 1. **Bottleneck identification:** - What is the most likely performance bottleneck under 10x current load? - Are there single points of failure? - Which components cannot scale horizontally? 2. **Data layer scalability:** - Can the database handle 10x growth in data volume? - Are queries optimized for the expected data patterns? - Is there a caching strategy? Is it appropriate? 3. **Concurrency model:** - How does the system handle concurrent requests? - Are there race conditions or deadlock risks? - Is the async/sync boundary well-defined? 4. **Resource usage:** - Are there memory leaks or unbounded growth patterns? - Are connections pooled appropriately? - Are timeouts configured for all external calls? Provide specific code locations for each finding.

Security flaws at the architecture level — like missing authentication boundaries or improper data isolation — are far more dangerous than code-level bugs.

Perform a security architecture review: 1. **Authentication & Authorization:** - Is authentication enforced at every entry point? - Is authorization granular enough (RBAC, ABAC)? - Are there endpoints that bypass auth checks? 2. **Data Protection:** - Is sensitive data encrypted at rest and in transit? - Are secrets (API keys, passwords) stored securely? - Is PII handled according to data protection principles? 3. **Input Boundaries:** - Are all external inputs validated and sanitized? - Are there injection risks (SQL, command, template)? - Are file uploads restricted and validated? 4. **Attack Surface:** - What is the external attack surface? - Are unnecessary ports or endpoints exposed? - Is there proper rate limiting? 5. **Dependency Security:** - Are dependencies up to date? - Are there known vulnerabilities in current dependencies? Flag findings as: INFO / WARNING / CRITICAL

Architecture should support change, not resist it. This step assesses how well the system will handle future requirements.

Evaluate the long-term maintainability of this architecture: 1. **Change impact analysis:** - If we need to swap the database, how many files change? - If we need to add a new API endpoint, what is the minimum change set? - If we need to change the authentication provider, what breaks? 2. **Testing architecture:** - Can components be tested in isolation? - Are there proper integration test boundaries? - Is the test infrastructure maintainable? 3. **Configuration management:** - Is configuration separated from code? - Can the system run in different environments without code changes? - Are feature flags used for gradual rollouts? 4. **Documentation alignment:** - Does the code match the documented architecture? - Are architectural decisions recorded (ADRs)? - Can a new developer understand the system from docs + code? Provide a maintainability score (1-10) with justification.

5. Training Protocol, Convergence Behavior & Experimental Evaluation

Finally, compile all findings into a structured report that can be shared with the team and tracked over time.

Based on the analysis above, generate a structured Architecture Review Report: ## Executive Summary (2-3 sentences: overall health, biggest risk, top recommendation) ## Findings Table | ID | Area | Severity | Finding | Recommendation | |----|------|----------|---------|----------------| | A1 | ... | CRITICAL/WARNING/INFO | ... | ... | ## Top 3 Recommendations (Ordered by impact, with effort estimates) ## Architecture Debt Register (Issues to track and address over time) ## Positive Patterns (What the architecture does well — important for team morale)

For teams that want to run architecture reviews automatically (e.g., on every PR that touches core modules), consider:

This guide covers the core architecture review workflow. For production use, you will want:

The AI Developer Toolkit includes 4 architecture-specific prompts covering system design review, microservices patterns, database architecture, and API design — each tested on real production codebases.

These are samples from the AI Developer Toolkit — 32+ production-tested prompts for code review, security audits, architecture reviews, debugging, and test generation.

Learn techniques to verify and validate AI-generated code, and how Copilot Chat can help.