Deterministic Process Automation Under Strong Security Constraints

Mon, 01 Jun 2026 00:00:00 +0000

Context

A client operating under strong security constraints manages several hundred sites and runs four recurring back-office processes every day: maintaining field equipment, handling recurring spurious equipment events, writing intervention reports for its customers, and checking that each site’s periodic health check has arrived. The inputs are scattered across three systems with very different access paths: PDF reports delivered by e-mail, a business web portal with no public API, and Excel workbooks maintained by hand over many years.

The goal: automate these processes on an office workstation while keeping a human operator as the final authority on every customer message.

Design constraints

No language model at run time. Every decision comes from explicit rules and every sentence from a template. This makes the robot’s output reproducible: the same inputs always produce the same drafts, a design meant to allow word-for-word comparison of a new version against production; any change traces back to a rule or a data source. No customer data goes to an external model service. A vision-language model was used once, offline, during design, to map recorded operator screen sessions; it is not part of the deployed system.
Drafts, not sends. The robot only creates Outlook drafts; the operator reviews and sends them. A global kill switch forces draft mode regardless of per-process settings.
Local execution, no new infrastructure. Python on a Windows workstation, scheduled tasks, a shared NAS folder for state and configuration.
No plaintext secrets. Credentials and tokens live in the Windows Credential Manager (DPAPI); configuration is YAML the operator can edit without a rebuild.

Architecture

flowchart TB
 SCHED["Scheduled passes (morning / afternoon)"] --> ING
 MAIL["Mailbox (Microsoft Graph, read)"] --> ING["Ingestion PDF reports grouped by day"]
 ING --> PARSE["PDF parser (pypdf + regular expressions)"]
 PORTAL["Business web portal Playwright login → session token → JSON endpoints replayed"] --> RULES
 XLS["Reference & history workbooks (Excel)"] --> RULES
 PARSE --> RULES["Rule engine 4 processes, ordered rules & exclusions"]
 RULES --> TPL["Template composer French agreement, names, dates"]
 TPL --> DRAFTS["Outlook drafts (Graph, never sent)"]
 RULES --> TRACK["Tracking workbooks (atomic append)"]
 DRAFTS --> OP(["Operator reviews & sends"])
 OP -. "sent items: corrected recipients" .-> RULES
 NAS[("NAS: locks, heartbeats, journal, idempotence ledger")] --- RULES
 RULES --> HEALTH["Health alerts webhook + dead-man check"]

The four processes

Equipment maintenance. Fault lines parsed from the daily PDF reports are matched against each device’s battery and repair history. An ordered decision tree (first match wins) chooses between ignoring, waiting, battery replacement, device replacement or radio-range testing, and drafts a technician work order; each device on a multi-device order keeps its own decision.

Recurring spurious events. Event counts per site and per device over 24 hours are compared with thresholds to detect recurring spurious events. Portal data (an incident confirmed as real, the system being switched off minutes after a burst) overrides the counts; the outcome is an advisory e-mail to the customer, a work order, or nothing.

Intervention reports. For each on-site intervention, the robot rebuilds a single timeline from the portal’s event log, call log and field-staff records, handles events that straddle midnight, and drafts a customer report with the media files attached.

Periodic health checks. Sites whose periodic health check is missing are identified, the status of each connectivity channel is checked on the portal, and a customer e-mail, a technician work order or an internal note is drafted.

Composing an e-mail: from parameters to a draft

Each draft is the end of a long chain of explicit decisions. Counted in the source code (October 2026), the four rule engines have 53 terminal outcomes (each place where a rule returns a decision), mapped onto 37 decision types and 13 draft builders.

The customer advisory e-mail of the spurious-events process shows the scale. A site reaches it only after up to 14 ordered tests in the rule engine (16 terminal outcomes for this process alone). The composer then reads about twenty inputs — 17 fields of the site’s decision record and customer record, the recipient list, the attachments, and two learned resources (zone names from the portal and a spelling dictionary learned from the operator’s own history workbook). Between the decision and the final text there are about 100 conditional branch points (if statements, conditional expressions and boolean operands, counted on the syntax tree of the composer and the ten helpers it calls, before the HTML pass).

flowchart TB
 IN(["1 · Rule engine — 14 ordered tests, first match wins. Input: daily event reports · portal journal · customer record · learned site profile"]) --> T0

 T0["Blocking flag on the site record?"] -- yes --> X0["No e-mail"]
 T0 -- no --> T1["R6 · incident confirmed as real today?"]
 T1 -- yes --> X1["No e-mail"]
 T1 -- no --> T2["R17 · media events < 30 s apart across zones?"]
 T2 -- yes --> X2["Manual review"]
 T2 -- no --> T3["R12 · external cause confirmed on media?"]
 T3 -- no --> T4["R8 · ≥ 8 integrity events?"]
 T4 -- yes --> T4b["persisting ≥ 3 days?"]
 T4b -- yes --> X3["Technician work order"]
 T4b -- no --> X4["Keep watching"]
 T4 -- no --> T5["≥ 2 integrity events, ≥ half of the burst?"]
 T5 -- yes --> X5["No e-mail"]
 T5 -- no --> T6["R16 · ≥ 140 events, advisory ignored ≥ 4 weeks? (off by default)"]
 T6 -- yes --> X6["Billing notice"]
 T6 -- no --> T7["R15 · shared-building flag?"]
 T7 -- yes --> X7["Coordination e-mail"]
 T7 -- no --> T8["R5 · weekend + chronic-device flag?"]
 T8 -- yes --> X8["Temporary masking request"]
 T8 -- no --> T9["R1 · same device ≥ 7 events?"]
 T9 -- no --> T10["R2 · ≥ 5 devices and ≥ 8 events?"]
 T10 -- no --> T11["R4 · ≥ 8 events, ≥ 50 % 05:00–07:00?"]
 T11 -- no --> T12["R3 · ≥ 8 events, ≥ 60 % 12:00–18:00, ≥ 70 % in 4 h?"]
 T12 -- no --> T13["R1bis · site total ≥ 7 events?"]
 T13 -- no --> X9["No e-mail"]

 T3 & T9 & T10 & T11 & T12 & T13 -- yes --> ADV

 ADV(["2 · Composer — advisory e-mail, rule id carried along. Customer name: legal form removed, re-cased, acronyms kept"])
 ADV --> SUBJ["≥ 30 events?"]
 SUBJ -- yes --> S1["Subject with urgent prefix"]
 SUBJ -- no --> S2["Plain subject"]
 S1 & S2 --> GR["Contact cell"]
 GR -- "several people, or a role / organisation word" --> G1["Generic greeting"]
 GR -- "title + surname" --> G2["Greeting + normalised title (7 abbreviations) + SURNAME"]
 GR -- "surname only" --> G3["Greeting + SURNAME"]
 G1 & G2 & G3 --> DEV["Devices involved"]
 DEV -- one --> D1["Portal zone name if known, else cleaned label + jargon map + learned accents"]
 DEV -- several --> D2["Generic phrase + list of ≤ 6 by count, each with its device code if known"]
 D1 & D2 --> HND["System switched on/off within minutes of the burst? (R1–R4 only)"]
 HND -- "off just after" --> C1["Handling-error wording A"]
 HND -- "on just before" --> C2["Handling-error wording B"]
 HND -- no --> ACT["Site in activity? — all of: R1–R4 or R1bis · ≥ 2 devices · ≥ 10 events · no intervention in the window · professional site · ≥ 90 % in learned hours · media sensors only: ≥ 90 % while on"]
 ACT -- yes --> C3["Site-in-activity wording"]
 ACT -- no --> RUL["Rule that fired"]
 RUL -- "R1, R2" --> C4["Generic environmental cause"]
 RUL -- "R3, R4" --> C5["Staff-passage cause"]
 RUL -- R12 --> C6["External cause confirmed on media"]
 RUL -- R1bis --> C7["Cause to investigate on site"]
 C4 & C5 --> ATT["Media actually attached?"]
 ATT -- yes --> A1["+ reference to the attachments"]
 ATT -- no --> FLG
 A1 & C1 & C2 & C3 & C6 & C7 --> FLG["Site flag: frequent contractor works?"]
 FLG -- yes --> F1["Request: switch off before works"]
 FLG -- no --> F2["Request: treat the cause"]
 F1 & F2 --> REC["Addresses in the customer record?"]
 REC -- yes --> RC1["All addresses found"]
 REC -- no --> RC2["Internal fallback mailbox"]
 RC1 & RC2 --> DRAFT(["Plain text → Outlook HTML (unfolding, lists, fixed typography, signature) → drafts folder, never sent"])
 classDef q stroke-dasharray:4 3,stroke-width:2px;
 class T0,T1,T2,T3,T4,T4b,T5,T6,T7,T8,T9,T10,T11,T12,T13,SUBJ,GR,DEV,HND,ACT,RUL,ATT,FLG,REC q;

The wording space of this single e-mail: 2 subject forms, 3 greeting forms, 2 opening-sentence forms, 7 reachable cause wordings (two of them with or without a reference to attached media), 2 requested actions, and an optional device list of 2 to 6 lines. Most thresholds above live in the YAML configuration; a few (urgency, site-in-activity, 4-hour concentration) are still in code.

Parameter	Source	Effect on the e-mail
Event count over 24 h	daily reports	figure in the opening sentence; ≥ 30 adds an urgent prefix to the subject
Rule that fired	rule engine	selects the cause wording; gates the handling-error and site-in-activity checks
Devices involved (cleaned, de-duplicated)	daily reports	one named device, or a generic phrase plus a list of up to 6 lines ordered by count
Device code per report line	daily reports	appended in parentheses to each listed device, only when known
Zone name of the main device	portal	replaces the raw report label when available
Learned spellings	operator’s history workbook	restores accents and casing of device names
Title and contact-name cell	customer record	one of 3 greeting forms; several people, a role or an organisation name fall back to the generic form
Company name	customer record	legal form removed, re-cased with acronyms kept, used in subject and body
Site type, learned opening hours, interventions in the window	tracking workbook, portal journal	enable or suppress the site-in-activity wording
System switched on/off around the burst	portal journal	handling-error wording (2 variants), overriding every other cause
Site flags	customer record	choice between the 2 requested actions
Attachments present	portal media	reference to the attachments, or nothing
Recipient addresses	customer record	every address found; otherwise an internal fallback mailbox

The other drafts draw on the same shared French-language helpers: the preposition before a site name agrees with its first noun (6 forms, from chez to à l’ and aux); report titles agree in gender and number; dates are written in full French form; a “Contact” heading becomes plural when several contacts are listed; and an 8-entry map turns technical equipment jargon into customer vocabulary.

Engineering for reliability

Idempotence. A daily ledger keyed on site, message type and subject hash prevents duplicate drafts; a cross-day register prevents re-issuing a work order before the previous intervention has happened. Missed days are caught up automatically.
Concurrency on a shared NAS. Exclusive lock files (O_CREAT|O_EXCL) with a time-to-live for orphans; two workstations publish heartbeats, and the secondary takes over only when the primary has been silent for two minutes.
Safe spreadsheet writes. Writes go through a temporary file and an atomic replace, with retries when the workbook is open by a user; inserted rows shift footer formulas so totals stay correct.
Network robustness. Graph calls handle token refresh, HTTP 429 Retry-After and 5xx back-off.
Explicit failure. A crash wrapper with no dependency on the main package sends a redacted alert and a dead-man ping even when imports fail; a sentinel raises an alert when the portal returns no data at all instead of reporting success.
Shadow benchmarking. A sandbox build replays a day in read-only mode and diffs its drafts against production. The acceptance metric will be the share of drafts the operator sends without changing a single word; it has not been measured yet.

Results

1,766 automated tests collected in the repository (October 2026; pass status not logged), built on mock mail and portal adapters; tests that need real fixtures are skipped when those are absent.
9.5 % (132/1,383) of event lines over 12 days of real reports were silently dropped, found after a site was missing from a run: event codes and labels are concatenated in the extracted PDF text. The parser was rewritten to recognise the code forms actually emitted; after the rewrite, 0 lines are lost on 11 real reports.
Caching the fault-history workbook (1.49 MB, 15,533 rows) cut the equipment maintenance process from 161.1 s to 23.3 s on one replay of one day’s real reports (single run); workbook opens fell from 26 to 3.
First clean shadow day against production: 14 drafts on each side, same decisions and subjects, 0 missed, 0 added. Bodies have not yet been compared, and typical days have about 90 drafts.

Stack

Python 3.11 · Microsoft Graph · MSAL · httpx · Playwright · pypdf · openpyxl · pydantic · keyring · pytest · PyInstaller

Reliability on Boris Dupoux