Document Intelligence with LLMs: Extracting Structure from Unstructured Data [2026]

Enterprises are drowning in documents. Financial statements arrive as PDFs. Contracts exist in scanned images. Invoices, forms, and reports accumulate in formats that resist automation. According to IDC, unstructured data accounts for 80-90% of all enterprise data, yet most of it remains locked away from analytical systems and business processes.

Traditional document processing relied on rigid templates and rule-based OCR. These approaches break the moment a vendor changes their invoice format or a new contract type arrives. Large language models with vision capabilities have fundamentally changed what's possible: systems that understand document structure, extract meaning from context, and adapt to new formats without reprogramming.

The Document Intelligence Challenge

Document processing isn't a new problem. What's changed is the sophistication of available tools and the expectations for accuracy and flexibility.

Why Traditional OCR Falls Short

Optical Character Recognition (OCR) converts images to text, but text alone doesn't capture document meaning. A bank statement is more than characters on a page: it has accounts, transactions, dates, and balances arranged in a specific logical structure.

Traditional approaches fail for several reasons:

• Layout blindness — OCR produces a stream of text without understanding columns, tables, or sections
• Template dependency — Rule-based extractors break when document formats change
• Context ignorance — Systems can't infer meaning from surrounding content
• No semantic understanding — "Total" near a number doesn't automatically link them

What LLMs Bring to Document Processing

Vision-capable LLMs (GPT-4 Vision, Claude, Gemini) can look at a document image and understand it much like a human would. They recognize:

• Document structure — Headers, sections, paragraphs, lists, tables
• Semantic relationships — Which values belong to which labels
• Context from content — Inferring document type and purpose from the text itself
• Flexibility across formats — Same model handles invoices, contracts, and forms

This doesn't mean traditional techniques are obsolete. The best pipelines combine specialized extraction tools with LLM understanding. For an overview of how LLMs work with external data sources, see our RAG guide.

Architecture Approaches

There's no single correct architecture for document intelligence. The right choice depends on your document types, accuracy requirements, and infrastructure constraints.

Vision-First Approach

The simplest architecture sends document images directly to a vision-capable LLM with instructions for what to extract. This works remarkably well for many use cases and requires minimal preprocessing.

Advantages:

• Handles any document layout without configuration
• Understands visual context (stamps, signatures, logos)
• Fast to implement and iterate

Disadvantages:

• Cost scales with page count
• Large documents may exceed token limits
• Less precise for dense tabular data

Text Extraction + LLM

Extract text first using OCR or PDF text extraction, then use an LLM to structure and interpret the content. Lower cost per document but loses spatial relationships.

Hybrid Pipeline

Production systems often combine approaches: specialized table extraction for financial data, layout analysis for document segmentation, and LLMs for semantic understanding and validation. This provides the best accuracy but requires more engineering investment.

Building an Extraction Pipeline

A production document intelligence pipeline typically includes these stages: ingestion, layout analysis, content extraction, LLM structuring, and validation. Let's walk through each with code examples.

Document Ingestion

The first step is loading documents and converting them to a processable format. For PDFs, this might mean extracting text, rendering pages as images, or both.

import fitz  # PyMuPDF
from pathlib import Path
from dataclasses import dataclass

@dataclass
class DocumentPage:
    page_number: int
    text: str
    image_path: Path | None = None

def load_pdf(pdf_path: str, render_images: bool = True) -> list[DocumentPage]:
    """Load PDF and extract text and optionally render page images."""
    doc = fitz.open(pdf_path)
    pages = []

    for page_num, page in enumerate(doc):
        # Extract text with layout preservation
        text = page.get_text("text")

        image_path = None
        if render_images:
            # Render page as image for vision models
            pix = page.get_pixmap(matrix=fitz.Matrix(2, 2))  # 2x zoom for quality
            image_path = Path(f"/tmp/page_{page_num}.png")
            pix.save(str(image_path))

        pages.append(DocumentPage(
            page_number=page_num + 1,
            text=text,
            image_path=image_path
        ))

    doc.close()
    return pages

Layout Analysis

Understanding document structure before extraction improves accuracy. Layout analysis identifies headers, paragraphs, tables, and other structural elements.

For complex documents, consider using specialized layout models like LayoutLM or document AI services that return bounding boxes and element classifications. For simpler cases, heuristics based on text positioning often suffice.

Table Extraction

Tables are notoriously difficult for general-purpose extraction. Purpose-built tools significantly outperform both OCR and vision LLMs for dense tabular data.

import pdfplumber
from typing import Any

def extract_tables(pdf_path: str) -> list[dict[str, Any]]:
    """Extract tables from PDF using pdfplumber."""
    tables = []

    with pdfplumber.open(pdf_path) as pdf:
        for page_num, page in enumerate(pdf.pages):
            page_tables = page.extract_tables()

            for table_idx, table in enumerate(page_tables):
                if not table or len(table) < 2:
                    continue

                # First row as headers, rest as data
                headers = [str(h).strip() if h else f"col_{i}"
                          for i, h in enumerate(table[0])]
                rows = []

                for row in table[1:]:
                    row_dict = {}
                    for i, cell in enumerate(row):
                        header = headers[i] if i < len(headers) else f"col_{i}"
                        row_dict[header] = str(cell).strip() if cell else ""
                    rows.append(row_dict)

                tables.append({
                    "page": page_num + 1,
                    "table_index": table_idx,
                    "headers": headers,
                    "rows": rows
                })

    return tables

LLM-Powered Structuring

The LLM's role is to interpret extracted content and produce structured output. Using structured output modes (JSON mode or function calling) ensures consistent, parseable results.

import anthropic
import base64
from pathlib import Path

def extract_with_vision(
    image_path: Path,
    extraction_prompt: str,
    schema_description: str
) -> dict:
    """Extract structured data from document image using Claude."""
    client = anthropic.Anthropic()

    # Load and encode image
    with open(image_path, "rb") as f:
        image_data = base64.standard_b64encode(f.read()).decode("utf-8")

    # Determine media type
    suffix = image_path.suffix.lower()
    media_type = {
        ".png": "image/png",
        ".jpg": "image/jpeg",
        ".jpeg": "image/jpeg"
    }.get(suffix, "image/png")

    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=4096,
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "image",
                        "source": {
                            "type": "base64",
                            "media_type": media_type,
                            "data": image_data
                        }
                    },
                    {
                        "type": "text",
                        "text": f"""{extraction_prompt}

Return the extracted data as JSON matching this schema:
{schema_description}

Respond with only valid JSON, no additional text."""
                    }
                ]
            }
        ]
    )

    import json
    return json.loads(response.content[0].text)

Validation with Pydantic

Validation is critical. LLMs can hallucinate fields, misformat dates, or return unexpected structures. Pydantic schemas enforce data contracts and provide clear error messages when extraction fails.

from pydantic import BaseModel, Field, field_validator
from datetime import date
from decimal import Decimal
from typing import Optional

class Transaction(BaseModel):
    date: date
    description: str
    amount: Decimal
    transaction_type: str = Field(pattern="^(credit|debit)$")

    @field_validator("amount", mode="before")
    @classmethod
    def parse_amount(cls, v):
        """Handle various amount formats."""
        if isinstance(v, str):
            # Remove currency symbols and commas
            cleaned = v.replace("$", "").replace(",", "").replace(" ", "")
            return Decimal(cleaned)
        return v

class BankStatement(BaseModel):
    account_number: str
    statement_period_start: date
    statement_period_end: date
    opening_balance: Decimal
    closing_balance: Decimal
    transactions: list[Transaction]

    @field_validator("account_number")
    @classmethod
    def validate_account(cls, v):
        """Ensure account number format."""
        cleaned = v.replace(" ", "").replace("-", "")
        if not cleaned.isdigit() or len(cleaned) < 8:
            raise ValueError("Invalid account number format")
        return cleaned

def validate_extraction(raw_data: dict, schema: type[BaseModel]) -> BaseModel:
    """Validate extracted data against Pydantic schema."""
    try:
        return schema.model_validate(raw_data)
    except Exception as e:
        # Log validation error for review
        print(f"Validation failed: {e}")
        raise

Complete Pipeline Example

Here's how these components fit together in a complete extraction pipeline:

from pathlib import Path
from dataclasses import dataclass
from typing import TypeVar, Generic
from pydantic import BaseModel

T = TypeVar("T", bound=BaseModel)

@dataclass
class ExtractionResult(Generic[T]):
    success: bool
    data: T | None
    errors: list[str]
    confidence: float
    raw_response: dict

class DocumentExtractor:
    def __init__(self, model: str = "claude-sonnet-4-20250514"):
        self.model = model

    def extract(
        self,
        pdf_path: str,
        schema: type[T],
        extraction_prompt: str
    ) -> ExtractionResult[T]:
        """Full extraction pipeline with validation."""
        errors = []

        # 1. Load document
        pages = load_pdf(pdf_path, render_images=True)

        # 2. Extract tables (for documents with tabular data)
        tables = extract_tables(pdf_path)

        # 3. Build context for LLM
        context = self._build_context(pages, tables)

        # 4. Extract with vision (using first page image)
        if pages and pages[0].image_path:
            try:
                raw_data = extract_with_vision(
                    pages[0].image_path,
                    extraction_prompt,
                    schema.model_json_schema()
                )
            except Exception as e:
                errors.append(f"Extraction failed: {e}")
                return ExtractionResult(
                    success=False,
                    data=None,
                    errors=errors,
                    confidence=0.0,
                    raw_response={}
                )

        # 5. Validate against schema
        try:
            validated = schema.model_validate(raw_data)
            return ExtractionResult(
                success=True,
                data=validated,
                errors=[],
                confidence=0.95,
                raw_response=raw_data
            )
        except Exception as e:
            errors.append(f"Validation failed: {e}")
            return ExtractionResult(
                success=False,
                data=None,
                errors=errors,
                confidence=0.0,
                raw_response=raw_data
            )

    def _build_context(self, pages, tables) -> str:
        """Combine page text and table data into context."""
        context_parts = []

        for page in pages:
            context_parts.append(f"--- Page {page.page_number} ---\n{page.text}")

        if tables:
            context_parts.append("\n--- Extracted Tables ---")
            for table in tables:
                context_parts.append(f"Table on page {table['page']}:")
                context_parts.append(str(table['rows'][:5]))  # First 5 rows

        return "\n\n".join(context_parts)

Handling Complex Document Types

Different document categories require different extraction strategies. Here's guidance for common enterprise document types.

Financial Statements

Bank statements, invoices, and financial reports share common challenges: dense tables, precise numeric requirements, and strict validation needs.

• Use specialized table extraction — pdfplumber or Camelot for transaction tables
• Validate totals — Sum extracted transactions and compare to stated totals
• Handle multi-page tables — Track headers across page breaks
• Currency parsing — Normalize formats (1,234.56 vs 1.234,56)

Contracts and Legal Documents

Legal documents prioritize text extraction accuracy and section identification over tabular data.

• Section detection — Identify clauses, parties, dates, and key terms
• Cross-reference handling — "As defined in Section 3.2"
• Signature detection — Identify signed vs unsigned documents
• Amendment tracking — Link amendments to original agreements

Forms and Applications

Structured forms with fields and checkboxes benefit from layout understanding.

• Field-value pairing — Match labels to their filled values
• Checkbox detection — Identify checked vs unchecked options
• Handwriting recognition — Vision models handle handwritten entries
• Required field validation — Ensure mandatory fields are populated

Production Considerations

Moving document intelligence from prototype to production requires attention to accuracy measurement, cost management, and data security.

Accuracy Measurement

You can't improve what you don't measure. Build evaluation into your pipeline from day one.

• Ground truth datasets — Manually label a representative sample of documents
• Field-level accuracy — Track accuracy per field, not just per document
• Confidence scoring — Flag low-confidence extractions for human review
• Error categorization — Distinguish OCR errors, schema mismatches, and hallucinations

Target accuracy depends on downstream use. Automated processing might require 99%+ accuracy, while human-in-the-loop workflows can tolerate lower thresholds with review queues.

Cost Optimization

Document processing costs can escalate quickly at scale. Optimize through:

• Tiered processing — Use cheaper models for simple documents, expensive models for complex ones
• Caching — Cache extractions for documents you've seen before
• Batching — Process multiple pages in single API calls where possible
• Resolution tuning — Lower image resolution for text-heavy documents

Typical costs range from $0.01-0.10 per page depending on complexity and model choice.

Security and Compliance

Documents often contain sensitive information. Address security early:

• Data residency — Ensure processing happens in appropriate jurisdictions
• PII detection — Identify and handle personal information appropriately
• Audit logging — Track what was extracted from which documents
• Retention policies — Define how long extracted data is stored

For GDPR and similar regulations, consider on-premises processing or providers with appropriate certifications.

Tools and Services Landscape

The document intelligence market includes cloud services, open-source tools, and specialized platforms.

Cloud Service Comparison

For applications requiring semantic search over extracted content, consider integrating with a vector database to enable retrieval across your document corpus.

Final Thoughts

Document intelligence has reached a practical inflection point. Vision-capable LLMs can now understand document layouts, extract semantic meaning, and handle format variations that would have required months of rule-based programming just a few years ago. The gap between what's technically possible and what's deployed in production is narrowing rapidly.

Success comes from combining the right tools: specialized extraction for tables and structured data, vision models for semantic understanding and flexibility, and rigorous validation to catch errors before they propagate. Pydantic schemas aren't just nice to have; they're essential for building systems you can trust with real business data.

Start with a focused use case and measurable accuracy targets. Build evaluation into your pipeline from day one. And remember that the goal isn't perfect extraction from any possible document; it's reliable extraction from your specific documents that integrates cleanly with your existing systems and workflows.