Home · About · Services · Industries · Projects · Products · Process · Blog · FAQ · Contact
LLM document processing: a practical extraction pipeline How to turn invoices, forms and letters into validated structured data with an LLM: ingestion, OCR, schema-bound extraction, checks, human review and…
A finance team receives a few hundred supplier invoices a week as PDF attachments. Someone opens each one, types the supplier, invoice number, dates, line items and totals into the accounting system, and checks the VAT or GST. It is slow, it is boring, and the errors it produces are the expensive kind: a duplicate payment, a wrong tax code, a missed due date.
LLMs now handle this job well. Not because the model reads perfectly, but because you can wrap it in a pipeline that checks its work.
The short answer: LLM document processing works as a pipeline. It ingests each file, converts it to text (using OCR where needed), asks the model to fill a strict JSON schema, then validates the result with ordinary code. Anything that fails a check or looks uncertain goes to a person for review. Only validated data reaches your system.
The shape of the pipeline Working document pipelines share the same stages, whatever the document type. The model is one stage of seven, and usually not the one that causes trouble.
Intake: collect files from a mailbox, an upload form, a shared drive or an API, and store the original untouched. Classify: work out what the document is (invoice, credit note, purchase order, ID proof, something irrelevant) so the right schema is used. Extract text: pull text and layout from digital PDFs directly, and run OCR on scans and photos. Structured extraction: send the text, and sometimes page images, to the model with a JSON schema it must follow. Validate: check types, totals, dates, formats and duplicates in code. Review: route failures and low-confidence results to a person with the source document side by side. Export: write approved records to the ERP, accounting package or database, and keep an audit trail. Keep each stage separate, with its own status in a database table. When a supplier changes their invoice layout, you want to rerun extraction on fifty documents, not re-upload them. A simple jobs table with a status column and a queue worker is enough for most volumes. We often build the orchestration in n8n or as a small FastAPI service with a background worker, depending on how much custom logic sits around it.
Getting good text out first Before tuning prompts, look at the text the model actually receives.
Digital PDFs, the ones exported from software, already contain a text layer. Libraries such as PyMuPDF or pdfplumber read it quickly and for free, including word positions. Scanned PDFs and phone photos have no text layer and need OCR. For those we use a layout-aware service such as Azure AI Document Intelligence, whose layout model returns text, tables and reading order, and can output Markdown that keeps table structure intact. Tesseract is a reasonable self-hosted option for clean scans, though it is weaker on tables and poor photos.
Tables matter most. If line items come through as one long run of numbers, the model has to guess which number belongs to which column, and it will sometimes guess wrong. Preserving rows and columns, as Markdown tables or as structured cells, removes most line-item errors before the model is involved.
Current multimodal models can also read page images directly. That helps with stamps, handwriting and odd layouts, but it costs more per page and is harder to debug. Our default is text first, with page images added only for document types where text extraction keeps failing.
Extraction with a strict schema Do not ask the model to "return JSON" and hope. Both OpenAI and Azure OpenAI support structured outputs, where the response is constrained to a JSON schema you supply. Define that schema once, in code, and use it for the request, the validation and the database write.
from openai import OpenAI
from pydantic import BaseModel
class LineItem(BaseModel):
description: str
quantity: str | None
unit_price: str | None
amount: str
class Invoice(BaseModel):
supplier_name: str
supplier_tax_id: str | None
invoice_number: str
invoice_date: str # ISO 8601, YYYY-MM-DD
due_date: str | None
currency: str # ISO 4217, e.g. INR, GBP
subtotal: str
tax_total: str
grand_total: str
line_items: list[LineItem]
client = OpenAI() # for Azure, point base_url at your resource's /openai/v1/ endpoint
def extract_invoice(doc_text: str, model: str) -> Invoice:
response = client.responses.parse(
model=model,
input=[
{"role": "system", "content": (
"Extract the invoice fields exactly as printed. "
"Use null when a field is not present. Never calculate or guess values. "
"Write dates as YYYY-MM-DD and amounts as plain numbers without separators."
)},
{"role": "user", "content": doc_text},
],
text_format=Invoice,
)
return response.output_parsedSome decisions in that code are deliberate. Amounts are strings, converted to Decimal during validation, so no floating-point rounding sneaks in. Missing fields are nullable rather than optional, because strict schemas require every key to be present and null is an honest answer. The instruction not to calculate matters: if the model derives a total it could not find, your validation can no longer catch a misread.
Version your prompts and schemas alongside the code, and store the model name and prompt version with every extracted record. When accuracy changes, you will want to know what changed.
Validation catches what the model cannot A schema guarantees shape, not truth. The model can return a perfectly formed invoice with a transposed digit in the total. Ordinary code catches that, and it is the part teams most often skip.
from decimal import Decimal, InvalidOperation
from datetime import date
def check_invoice(inv: Invoice, seen_numbers: set[tuple[str, str]]) -> list[str]:
problems = []
try:
lines = sum(Decimal(li.amount) for li in inv.line_items)
subtotal, tax, total = (Decimal(inv.subtotal), Decimal(inv.tax_total),
Decimal(inv.grand_total))
if abs(lines - subtotal) > Decimal("0.05"):
problems.append(f"line items {lines} != subtotal {subtotal}")
if abs(subtotal + tax - total) > Decimal("0.05"):
problems.append("subtotal + tax != grand total")
except InvalidOperation:
problems.append("an amount is not a valid number")
try:
if date.fromisoformat(inv.invoice_date) > date.today():
problems.append("invoice date is in the future")
except ValueError:
problems.append("invoice date is not YYYY-MM-DD")
if (inv.supplier_name.lower(), inv.invoice_number) in seen_numbers:
problems.append("possible duplicate invoice")
return problemsArithmetic checks are the most valuable single test, because a misread number almost always breaks a sum. Add checks specific to your documents: tax ID format (an Indian GSTIN has 15 characters with a fixed structure), supplier name matched against your vendor master, PO number present in the purchasing system, currency in an allowed list. Every rule you add moves work away from human reviewers.
Human review is part of the design No pipeline should write unchecked financial or medical data straight into a system of record. The goal is not zero human involvement. It is reducing a person's job from typing every field to confirming the few that need it.
Route a document to review when any validation fails, when the classifier is unsure of the document type, or when it comes from a supplier the system has not seen before. The review screen should show the original page next to the extracted fields, highlight the fields that failed checks, and let the reviewer correct and approve in one place. Save both the model's output and the corrected version. That pair of records is your best source of test data later.
Start with everything going through review, then relax the rules for document types and suppliers whose results have been consistently right. Auto-approval is something you earn with evidence, not a setting you turn on at launch.
Measuring accuracy and cost Before go-live, build a small evaluation set: 50 to 100 real documents with hand-checked correct values, covering your messiest suppliers and worst scans. Run the pipeline against it whenever you change the prompt, schema, OCR settings or model, and measure accuracy per field rather than per document. "Invoice number correct 99 times in 100, due date 91" tells you where to work. "Mostly fine" does not.
For cost, the main levers are the amount of text you send and the model you use. Trim boilerplate such as terms and conditions pages before extraction. Try a smaller model first, since for clean, well-structured documents it is often as accurate as a larger one, and keep the larger model for the document types that need it. Your evaluation set tells you which is which. Batch APIs, where your provider offers them, cut cost for work that does not need an answer in seconds.
Treat documents as the sensitive data they usually are. Use an API deployment whose data handling terms you have read, pick a region that suits your obligations, restrict who can see the review queue, and set a retention period for originals and extracted text.
A checklist before you build Pick one document type with real volume and a clear destination system. Do not start with "all our paperwork". Collect 50 to 100 real samples, including the bad scans, and write down the correct values. Decide the schema with the people who use the data, including which fields are mandatory. Store originals unchanged and track each document's status through every stage. Use structured outputs, amounts as strings or decimals, and a no-guessing instruction. Write arithmetic, format, lookup and duplicate checks before you tune prompts. Send everything to human review at first and save corrections as test data. Re-run the evaluation set on every prompt, schema or model change. If your documents are mostly clean digital PDFs from a handful of senders, a small team can build this in a few weeks. If you have handwritten forms, many layouts, or strict audit needs, plan for more time on OCR, review screens and evaluation than on the model itself. That is where most of the effort, and most of the accuracy, comes from.
Infigini IT Services · 11 Kanwali Road, Dehradun 248001, India · info@infigini.com · +91 95684 87789