PAI PDF AI

Bharat Document Intelligence Platform

Independent R&D Project
  • Local LLM (Ollama)
  • Elasticsearch Vector RAG
  • FastAPI Backend
  • React & Framer Motion

An AI-powered document intelligence platform that helps users understand complex Indian legal and financial documents. It extracts information, cross-references against Indian laws and regulations via a local vector knowledge base, and presents risk insights with explicit proof chunks—all running entirely locally for absolute data privacy.

Interactive Animated Architecture Flow

Start with the end-to-end platform flow. This animation shows how domain knowledge, AI extraction, and proof-based answers connect across the full Bharat Document Intelligence Platform.

  Source Available on Request (Private Repo)

What it does

Millions of Indian documents (Property sale deeds, loan agreements, tax notices) exist only as unstructured PDFs. PAI PDF AI bridges the gap by transforming these PDFs into actionable information. By utilizing Retrieval-Augmented Generation (RAG) against domain knowledge, the AI explains technical jargon in plain English/Tamil, highlights clauses, and detects risks.

Bharat Document Intelligence Platform overview

Platform vision snapshot for the Bharat Document Intelligence Platform

PAI PDF AI Dashboard

PAI PDF AI - Proof-of-Concept Dashboard Interface

Key Features

  • Semantic Knowledge Search: Matches document clauses against national laws and state rules using Elasticsearch.
  • Bounding-Box Highlights: Instantly finds the exact paragraph in the PDF where the AI extracted a particular value.
  • Human Feedback Loop: Analysts can correct AI extractions, and the system instantly embeds and learns these new patterns.
  • Zero Data Leakage: Utilizes local Ollama LLMs (`llama3`, `nomic-embed-text`) ensuring sensitive documents never leave the machine.

Tech Stack

LayerTechnology
FrontendReact + Vite + Framer Motion
Backend APIFastAPI + Python
Vector StoreElasticsearch
LLM IntegrationOllama (`llama3` model)
EmbeddingsOllama (`nomic-embed-text`)
PDF ParsingPyMuPDF (`fitz`), Unstructured

System Architecture Flows

The platform operates in three distinct phases: Setting up the domain knowledge, AI extraction, and interactive Q&A. These processes ensure answers are always grounded in verifiable rules.

Phase 1: Knowledge Setup (Domain Rules)

graph TD A[Admin] -->|Upload Laws/Regulations PDF| B(PyMuPDF Parser) B -->|Extract Text| C{Text Splitter} C -->|Chunks| D["Ollama Embeddings: nomic-embed-text"] D -->|Vector Vectors| E[(Elasticsearch Vector DB)] classDef default fill:#1e293b,stroke:#3b82f6,stroke-width:2px,color:#fff; classDef db fill:#0f172a,stroke:#10b981,stroke-width:2px,color:#fff; class E db;
  • Admins upload reference materials (e.g., Transfer of Property Act, RBI circulars).
  • Documents are parsed, chunked, and embedded into vector space.
  • Stored in Elasticsearch to act as the Ground Truth for the AI engine.

Phase 2 & 3: PDF Ingestion & Cross-Reference Q&A

graph TD U[User] -->|Upload Sale Deed PDF| P(Backend Pipeline) P -->|Extract Key Entities| E[Extracted Clauses] E -->|Semantic Search| KB[(Elasticsearch DB)] KB -->|Retrieve Relevant Laws| Ctx[Context Rules] U -->|Ask Question| Q(Query Processor) Q -->|Search Text| KB Ctx --> Prompt Q --> Prompt E --> Prompt Prompt -->|Send to LLM| O["Ollama LLaMA-3"] O -->|Generated Answer| Res["Response + Bounding Box Proof"] Res --> U classDef default fill:#1e293b,stroke:#8b5cf6,stroke-width:2px,color:#fff; classDef db fill:#0f172a,stroke:#10b981,stroke-width:2px,color:#fff; classDef user fill:#334155,stroke:#f59e0b,stroke-width:2px,color:#fff; class KB db; class U user;
  • User uploads an unstructured document. The AI extracts entities and clauses.
  • Extracted clauses are verified against the Elasticsearch knowledge base.
  • When the user asks a question, the LLM generates an answer, and the UI highlights the exact source chunks (proof).

Continuous Learning: Semantic Validation & Human Reinforcement Layer

To continuously improve data extraction accuracy over time, the platform implements a Self-Learning Semantic Validation Engine:

graph TD H["Human Analyst"] -->|Corrects Bounding Box / Field| SE["Semantic Embedding (Ollama: nomic-embed-text)"] SE -->|768-dim Vector| ES[(Elasticsearch Store)] ES -->|Next Document Load| KNN{"k-NN Query Match?"} KNN -->|Yes: Semantic Match| HC["Retrieve Verified Coordinates"] KNN -->|No: New Layout| HD["Heuristic Fallback (evidence_knowledge.json)"] HD -->|Indian-context anchors| HF["Bilingual English/Tamil Heuristics"] classDef default fill:#1e293b,stroke:#a855f7,stroke-width:2px,color:#fff; classDef db fill:#0f172a,stroke:#10b981,stroke-width:2px,color:#fff; class ES db;
  • Semantic Embedding: Extracts text surrounding any analyst-corrected bounding box and generates vectors using local Ollama.
  • Elasticsearch Store & k-NN Retrieval: Saves coordinates tied to vector embeddings. Subsequent matching documents retrieve correct coordinates using k-Nearest Neighbors (k-NN) vector queries.
  • Heuristic Fallback: Brand new files utilize a bilingual (English/Tamil) anchor dictionary to dynamically locate values.

GitHub Project Planning & Iterations

I structured the development into structured sprints using GitHub Projects, tracking tasks via issues across the different phases.

GitHub project roadmap covering PAI PDF AI development phases
GitHub Project Roadmap
Phase planning across extraction, RAG, QA, and future intelligence layers

Development Roadmap & Phased Execution

The overall platform vision is structured into 8 progressive development phases, building up from parsing engines to a specialized multi-agent framework.

1 Phase 1: Document Extraction Engine

Goals: PDF parsing, OCR, document type classification, and entity extraction. Deliverable: Structured JSON extraction output.

2 Phase 2: RAG Knowledge Platform

Goals: Ingesting acts, rules, and regulations into Elasticsearch. Involves semantic retrieval over legal corpora. Deliverable: Question-answering engine over laws.

3 Phase 3: Cross-Reference Engine

Goals: Linking document content directly with relevant laws/regulations, and generating bounding box proofs. Deliverable: Source-backed QA explanations.

4 Phase 4: Risk Detection Engine

Goals: Build rule-based and AI-assisted engines for risk scoring and compliance validation. Deliverable: Automated risk assessment reports.

5 Phase 5: Tamil Nadu Property Intelligence

Goals: Sale deed analysis, Patta/Chitta parsing, and EC verification checks. Deliverable: Property due diligence assistant.

6 Phase 6: Banking & Loan Intelligence

Goals: Analyzing loans, mortgage agreements, and interest policies. Deliverable: Banking compliance document assistant.

7 Phase 7: Legal Notice Intelligence

Goals: Understanding legal threats, extracting deadlines, and citing protective statues. Deliverable: Proactive legal notice responder.

8 Phase 8: Multi-Agent System

Goals: Coordinate specialized agents (Document, Legal, Banking, Property, Research). Deliverable: Unified domain reasoning system.

Back to Portfolio