Codex — High-Density Physical Archiving of Media on Paper

Context

Consumer digital media decay on the scale of years. Unpowered SSDs lose charge, optical discs delaminate, formats go out of use and accounts get closed. Paper can last for centuries, needs no electricity and can be checked by eye. The question is how much information an ordinary A4 sheet can hold if it must still be read back reliably with a standard flatbed scanner and no special hardware.

The problem is hard because the print-and-scan channel is noisy in several ways at once. Toner spreads and clumps, density varies across the page, and scanners add rotation, anisotropic scaling, uneven paper feed, blur, vignetting and gamma. Paper also stretches non-linearly: at the target density, 0.2 % of stretch shifts the grid by about three cells. Common 2D codes such as QR and DataMatrix are designed for robustness rather than density. A long-term archive also has to remain decodable when today’s software no longer exists.

Approach

flowchart TB
    A["Source media: photo, video, voice, text"] --> B["Format-conservative reduction
JPEG baseline · MJPEG/PCM AVI · PCM WAV · UTF-8"]
    B --> C["Archive: tar + zlib + SHA-256"]
    C --> D["Inter-page erasure code
Cauchy MDS over GF(256): any K of N pages"]
    D --> E["Per-block Reed–Solomon + CRC-32
page-wide interleaving, triplicated header"]
    E --> F["Multi-level cell rendering
1-bit A4 PDF at 1200 dpi"]
    F --> G["Print → scan
(simulated channel)"]
    G --> H["Fiducials → homography → local alignment markers"]
    H --> I["Grey-level calibration from the page itself"]
    I --> J["RS/CRC decode → erasure decode → SHA-256 check"]
    J --> K["Byte-exact files, or partial recovery"]
    F -.-> L["Printed appendix: format spec + decoder source in OCR-B with per-line CRC-16"]
    L -.-> J

Left: page 1 of 10 of the photo archive, as sent to the printer (A4, 1200 dpi, a grid of 1,145 × 1,576 cells), downscaled. Right: the boxed top-left corner at printer resolution (about 10 × 9 mm, each printer dot drawn as 2 × 2 pixels): the solid corner fiducial, the start of the header band, and the first data cells. Each cell is 8 × 8 dots (0.169 mm) and its number of inked dots encodes one of 4 levels. The square at the corner of the grid is an alignment marker.

A page of the printed appendix (page 11 of 30): the start of cdx_read.py (360 lines), set in OCR-B. Each line carries its number and its CRC-16, so a line misread by OCR can be found and checked by hand.

Methods

• Multi-level cells instead of binary modules. Each cell carries 2 bits. Cells are drawn as clustered dots at the printer’s native resolution, so the driver does no halftoning and produces no moiré.
• Geometry recovery in stages. Four corner fiducials, one of them hollow to fix orientation, give a global homography. Alignment markers spread across the page then absorb local, non-linear paper stretch.
• Self-calibrating reader. The decoder estimates the printed grey levels and spatial shading from calibration structures on each page, so it does not depend on the printer’s dot settings.
• Two-level error control, a product code. Reed–Solomon and CRC-32 inside each page correct random and burst errors such as stains and scratches. A Cauchy MDS erasure code across pages, decoded block by block, lets any K of N pages rebuild an archive.
• Graceful failure. If the SHA-256 check fails, the decoder still extracts every complete file that comes before the damaged region.
• Self-describing archive. A printed appendix contains the format specification, a reading procedure and the full decoder source in OCR-B, with a line number and CRC-16 on each line. The decoder needs only NumPy, Pillow and reedsolo. Text uses DEFLATE (zlib), which can be reimplemented in a few hundred lines, rather than the stronger LZMA.
• Earlier prototype (codex_proto/). This version explored several physical media: paper, laser-marked stainless steel, intaglio plates and microfilm frames. It used 2- to 16-level cells, fountain-style parity pages, MLP-based grey-level readers, a printed optics calculator for reading microfilm, and lossy media reduction (parametric voice coding, wavelet-residual photo coding).

Results

All figures below come from simulation only. The software models laser defects (spot blur, toner grain, dot gain, banding, non-uniform density, textured paper), scanner defects (±1.2° rotation, ±0.3 % x/y scale, feed wobble, blur, vignetting, noise, gamma 2.2), coffee stains, scratches, and one deliberately lost page per archive. No physical print-and-scan test has been run yet.

• Net payload: 383,469 bytes per A4 page, about 59 bits/mm².
• End-to-end: 46 of 50 pages were scanned and all 4 of 4 archives were restored byte-identical (photo, video, voice, text). No Reed–Solomon codeword failed, and no inter-page block was lost.
• Error rates: clean pages needed about 0.007–0.008 % of bytes corrected. Pages with stains and/or scratches needed at most 0.90 %.
• Archive sizes: photo 2.86 MB → 10 pages (8 data + 2 parity); video (800×600, 24 s, MJPEG + 16 kHz PCM) 8.30 MB → 25 pages; voice 0.91 MB → 5 pages; text 5.58 MB → 10 pages.
• Text: the public-domain Mémoires d’outre-tombe (5.43 M characters) would take 1,357 typeset A4 pages. Encoded, it takes 6 data pages, a ratio of about 226. zlib compresses this prose by 2.62 and LZMA by 3.58.
• Partial recovery: an archive truncated to a quarter of its length still returned 1.42 MB of the 5.58 MB of text.
• Decoder robustness: the decoder also worked on scans rotated 90° and 180°, on 600 dpi scans and on JPEG (q92) scans.
• Decoder size: 731 lines of Python for the printed decoder modules.

The same region of the photo page (380 × 300 dots, about 8 × 6.4 mm), enlarged. Left: as encoded. Right: after the simulated print-and-scan channel, resampled onto the page grid through the fiducial homography. Toner spread, blur and noise soften the cells, and the edge of a simulated coffee stain darkens the left part. The reader calibrates the grey levels locally, so it still separates the four levels. In the simulation run reported above, stained or scratched pages needed at most 0.90 % of bytes corrected.

The video archive: an underwater clip (school of fish over a reef) used as test footage. Top: the original 3840 × 2880 HEVC frames at 3.2 s, 12.8 s and 16.0 s, downscaled. Bottom: the same instants decoded from the simulated scans of the 25 pages (one deliberately lost). The paper channel itself loses nothing: the restored AVI is byte-identical to the archived one. What differs from the original is the reduction made before printing (800 × 600 MJPEG, 12.5 frames/s, 16 kHz mono sound) so that 24 s of video fit in 8.30 MB.

Close-up of the 12.8 s frame. Left: a 1152 × 864-pixel area of the original, downscaled. Right: the matching 240 × 180-pixel area of the restored frame, enlarged with no smoothing. The fish and the reef stay recognisable. Fine texture and colour detail are lost, and MJPEG block artefacts appear. This is the price of fitting the clip into 25 pages.

Limitations & next steps

• No physical test yet. The cell dot patterns were tuned in simulation. The next step is a real 1200 dpi print with a 1200 dpi scan to set them and measure the actual error margin.
• Small test set. The simulation covers one run over four archives. A sweep across defect severity, scan resolution and paper stock is still needed.
• Unmeasured physical properties. Long-term ageing of paper and toner, and how practical it is to OCR the decoder back from paper, have not been measured.
• Font dependency. The appendix body text relies on a system font.

Stack

Python · NumPy · SciPy · Pillow · reedsolo · ReportLab · soundfile · ffmpeg · Tesseract (OCR-B)