Explainable Processing Atoms for Documents & Data

Context

Production systems for document reconciliation and OCR, built for clients such as a wholesale distributor and an accounting firm, left several codebases that each solved the same sub-problems (page routing, OCR tuning, LLM extraction, field comparison) with their own bugs. The atoms library consolidates them into one package. Its central question is the one behind the original systems: can a document check be automated so that a human can audit and contest every verdict? Extracting a field is not enough. The system must state whether two documents agree, why, and from what evidence.

Approach

Everything is built from atoms. An atom has one responsibility, typed inputs and outputs, no global state, and injected dependencies (LLM client, OCR engine, storage). A pipeline is a composition of atoms, either written in code or declared as a YAML recipe that is validated when it loads, before anything runs.

flowchart TB
    A["Input documents (PDF, scans, CSV, mail)"] --> B{"Page routing: native text or OCR?"}
    B -- "native" --> C["Positioned native text"]
    B -- "scan" --> D["PaddleOCR (production settings)"]
    C --> E["Segmentation & classification"]
    D --> E
    E --> F["LLM JSON extraction + verbatim grounding check"]
    F --> G["Cross-document reconciliation (invoice / delivery note / order)"]
    G --> H["Tri-state verdict + named reason + trace"]
    H --> I["Report / human review"]

The catalogue

The 459 atoms are grouped into 32 families, one Python package each, ordered along the processing chain: capture the world, understand it, reconcile and decide, deliver, then operate in production. Two further packages assemble them: steps (77 named, parameterised steps) and recipes (the YAML loader and validator).

Named recipes

Each of the 17 recipes is a YAML list of steps, named <family>.<atom>. Three examples:

• invoice_multi_doc: a batch of multi-document PDFs becomes checked logical documents. io.collect → pdf.read → segment.documents → classify.documents → llm.extract → validate.ids → validate.amounts → validate.decide → export.csv → google.sheets.upsert → connect.notify. Each document ends as valid, to check, or invalid.
• doc_reconciliation: a file of invoices, credit notes and delivery notes, checked against orders. Guarded LLM extraction (llm.extract_docs) is cross-checked with pattern extraction (extract.regex, compare.extractions, reconcile.fields), then check.document and check.completeness, lines against the order reference (compare.reference), the order → delivery → invoice chain (reconcile.nway), and a discrepancy file plus review queue (report.discrepancies, report.review). 17 steps.
• meeting_transcript: a meeting recording becomes a timestamped transcript with speakers. audio.decode → audio.vad → audio.transcribe → audio.diarize → audio.speakers → audio.segments → export.csv → export.transcript → connect.notify.

The others include delivery_vs_order, ledger_reconciliation, invoice_dedup, article_base_cleaning, stock_policy, mail_to_sheet, draft_mails, pdf_text_extract, rubric_eval, screen_session, camera_flow and process_from_transcript.

The Atlas

The Atlas is a single offline HTML page generated from the code itself (syntax trees and YAML), with no hand-written data. It draws the catalogue as a periodic table: each family is an element with a symbol, each module an atom, each recipe a molecule. It also flags atoms that lack documentation, tests or a usage example. The interface is in French.

The “Capture” series of the periodic table. Each box is a family, with its symbol, number of modules and size in thousands of lines; each tile below it is an atom.

The “Reconcile” series: the families that pair, compare, check and decide.

An atom’s page (compare.fields): what it does, taken from its docstring, then each public name with its full signature.

The invoice_multi_doc recipe as a molecule: 11 steps, the context keys that link them, and the services they need (dashed when optional).

Methods

• Tri-state results. Every comparison returns ok ∈ {True, False, None} (conforms / discrepancy / inconclusive) with a named reason code and a trace. A missing or unreadable value counts as inconclusive, never as an anomaly. Inconclusive results are still counted, not silently dropped.
• Hybrid native-text/OCR routing. Each PDF page is classified as native, OCR or empty, based on how much native text it holds and how much of it is covered by images. OCR runs only where it is needed.
• Grounded LLM extraction. Structured JSON extraction is validated against Pydantic schemas. Each extracted value is then checked verbatim against the source text, so altered or invented values are flagged.
• Cross-document reconciliation. Invoices, delivery notes and orders are compared line by line using named tolerances and dominance rules. For example, a matching line amount can override a price difference.
• Provenance. Every ported atom carries a Provenance: header and an entry tracing it to repo@commit:path:lines in the original production code. Bugs found while porting were fixed and each fix is covered by a non-regression test.
• Enforced architecture. Layering is checked by 30 import contracts: an atom imports only the core package. Strict typing, a ≥ 80 % coverage gate and a check that no client names appear in source code are also enforced.
• Device abstraction. Vision and audio atoms target cuda, mps or cpu via one setting.
• End-to-end evaluation. Tests run with real engines (PaddleOCR, Ollama qwen3:8b and qwen3-vl:8b) on data with known ground truth. Each metric is recorded against a threshold in a JSON/HTML report. Metrics that miss their threshold are kept and shown as failing.

Results

Codebase (current tree): 459 atom modules in 32 families, 77 parameterised steps, 17 YAML recipes, 30 import contracts. The suite collects 2,959 unit and recipe tests, plus 109 end-to-end tests that run separately. All 17 recipes have at least one end-to-end test, and 34 of the 35 Atlas elements are exercised end to end.

Selected end-to-end scores (synthetic documents with known ground truth, local 8B models):

Limitations & next steps

• Verbatim grounding cannot catch a value that exists in the document but was put in the wrong field: an absent field was wrongly filled in 1 of 8 cases.
• The 8B model judged a +50 % quantity gap a “minor” discrepancy, against the instruction. LLM verdicts are therefore normalised and kept downstream of deterministic rules.
• Scores come mostly from synthetic data. A real-document test set is being built outside the repository, and only cases that have been reviewed are scored.
• Next steps: measure the automatic-reconciliation rate and the false-positive rate on production data.

Stack

Python · Pydantic · PyYAML · PaddleOCR · Ollama (qwen3:8b, qwen3-vl:8b) · Mistral / Anthropic / OpenAI APIs · pytest · mypy · import-linter · ruff