📄 Invoice Intelligence Pipeline

PDF invoice in → validated, approved, and posted to QuickBooks — with a full audit trail, no human touch unless the document actually needs one.

Reference implementation / demo build. Not client work — built to demonstrate the architecture and engineering standard end-to-end, on a synthetic dataset.

A bookkeeper spends ~12 minutes on a single vendor invoice: reading the total, matching the vendor, keying in every line item, checking it against what's already been paid. At 200 invoices a month, that's 40 hours — a full work week — spent on data entry instead of the parts of the job that need a human.

An n8n pipeline that reads an incoming invoice PDF, extracts every line item (not just the total), validates it against real business rules, and routes on confidence — high-confidence invoices post straight to QuickBooks, everything else goes to a one-click Slack approval. Every step writes to Postgres, so there's a full audit trail of what happened, who approved what, and why a document was held.

[GIF / video goes here]

Six blocks, each an independent n8n workflow (workflows/*.json), Postgres as the single source of state and audit trail between them.

flowchart LR
    A["1. Ingest & Dedup<br/>00-ingest<br/>byte-hash dedup"] --> B["2. Extract<br/>10-extract<br/>Gemini vision"]
    B --> C["3. Validate & Score<br/>10-extract<br/>business dedup, outliers,<br/>confidence"]
    C -->|high confidence| E["5. Export<br/>30-export<br/>QuickBooks Bill"]
    C -->|low confidence| D["4. Approval<br/>15/16/20/21<br/>Slack or Telegram, one click"]
    D --> E
    C -.->|anything fails| F["6. Observability<br/>& Recovery<br/>99-error, 95-sweep-stuck<br/>dead_letter + Telegram"]
    B -.-> F

11/19 documents (58%) processed end-to-end with zero human review — day 1 of a multi-day run, n=19, full run of 50 in progress (Gemini's free tier caps at 20 requests/day — see Known limitations below). This is the number that maps directly to headcount saved: it's the share of invoices that never needed a Slack click.

The other 8/19 were held for a reason, not lost:

Extraction accuracy is the second number, not the headline — measured against a hand-built ground truth on a synthetic dataset (10 templates, 3 languages, 30% degraded scans):

High accuracy on a synthetic dataset reflects controlled input, not a promise about every vendor's invoice format in the wild.

Cost: $0.02 per document (n=19, real logged tokens at paid-tier Gemini rates, not an estimate) — against 12 minutes of manual entry.

• Idempotent exports — re-running the same invoice never creates a second QuickBooks Bill (UNIQUE(invoice_id, adapter), UPSERT)
• Two-layer duplicate detection — byte-identical resubmits caught before extraction; same-vendor-same-invoice-number duplicates caught after, even from a different scan
• Resume after failure — a document that errors mid-pipeline stays retryable, not silently dropped or falsely flagged as a duplicate on resubmission
• Dead letter queue — unprocessable documents (blank pages, unreadable scans) get a structured reason, not a garbage invoice row
• Dedicated error workflow — any node failure anywhere alerts to Telegram with the execution link, not silence
• Full audit trail — every approval decision (who, when, latency) and every pipeline run (tokens, cost, duration) is a row in Postgres
• Notification and approval layer is channel-agnostic — Slack and Telegram both support interactive approval through a shared decision-recording sub-workflow (common-record-approval-decision). Adding a channel means implementing one interface, not touching approval logic. Telegram inline keyboards hit an upstream n8n bug (#14775) that crashes workflow activation — worked around with a small self-hosted Bot API proxy (91-telegram-proxy) so both channels get real Approve/Reject buttons — see docs/findings.md

Bugs found and fixed while building this — the kind of detail that's easy to miss and expensive in production. Full write-ups in docs/findings.md.

• JSON key order silently changes an LLM's output. Storing responseSchema as JSONB let Postgres reorder its keys — and key order in the schema deterministically changed what Gemini extracted (reproduced: "Störungsticker" vs. the correct "Störungsticket", same input, only key order differed). Fixed by storing it as TEXT.
• A bulk INSERT ... RETURNING fanned out every downstream step. RETURNING invoice_id on a multi-row line-item insert returns one row per line item, and every node after it assumed exactly one row per document — a 5-line-item duplicate produced 5 identical duplicate-flag rows instead of one. Fixed by collapsing the insert into a CTE with a single summary row.
• A failed extraction made the byte-dedup permanently misfire. A document stuck mid-pipeline (status never left received) got classified as a duplicate on resubmission instead of a resume — the fix was quietly dropping every retry. Dedup now only matches terminal status; received triggers a resume instead.
• A bare $json reference breaks the moment any upstream node restructures the payload — not just when a node gets inserted mid-chain, but any node that replaces the shape wholesale, even one that was there from day one and only breaks once the real end-to-end path finally runs. Found four separate times; the fix is always the same explicit $('Node Name').json reference.
• QuickBooks' Realm ID appears nowhere in Intuit's UI — not the Developer Dashboard, not company settings. The only reliable source is the sandbox company console's page HTML via a companyId-anchored regex; a naive "any 15–16 digit number" search produces false positives that look exactly like a bad OAuth token until you cross-check.
• $credentials isn't a general expression variable, despite appearing in n8n's own credential source files. Tried building a Telegram API URL from a credential's value directly in an HTTP Request node's URL field ({{ $credentials.httpHeaderAuth.value }}) — it silently resolved to an empty string instead of erroring, confirmed by inspecting the actual outgoing request n8n logged. That expression only works inside a credential type's own definition file, not in a workflow node. Led to the 91-telegram-proxy design: the secret lives in a container env var a Code node reads directly, not in any expression field.

n8n (self-hosted) · Google Gemini (vision extraction, structured output) · Python 3.12 (dataset generator + accuracy harness) · PostgreSQL 17 / Neon · Slack (interactive approval) · Telegram (interactive approval, via a self-hosted Bot API proxy) · QuickBooks Online API

1. Database

   uv venv --python 3.12 && uv sync
   docker compose up -d              # local Postgres
   bash schema/apply.sh "$DATABASE_URL_LOCAL"
   uv run python -m seed.generate --seed 42 --count 50 --out data/

2. n8n

   • Import every file in workflows/*.json.
   • Activate all of them — including [DocIQ] 99-error. Nothing calls it directly; it's wired in via each other workflow's Settings → Error Workflow. Leave it inactive and failures go nowhere, silently.
   • Fill in credentials by their placeholder names in the JSON (<postgres credential>, <googlePalmApi credential>, <quickBooksOAuth2Api credential: ...>).

3. QuickBooks config — 30-export's QB Config node needs your own qb_realm_id and qb_default_account_id. Realm ID isn't in Intuit's UI anywhere (see Engineering findings above); the discovery method is in docs/findings.md.

4. Slack — Signing Secret on the HMAC SHA256 node in 20-approval-callback; Bot Token on Post to Slack in 15-request-approval; point your Slack App's Interactivity Request URL at https://<your-n8n-instance>/webhook/dociq-approval-callback.

5. Telegram (second interactive channel) — inline-keyboard buttons go through a small proxy (91-telegram-proxy), not the native Telegram node — see docs/findings.md for why.

   • Deploy 91-telegram-proxy first. It needs an httpHeaderAuth credential (any header name/value you generate — protects the proxy from being an open relay to your bot) and the bot token as a container environment variable on the n8n host, e.g. DOCIQ_TELEGRAM_BOT_TOKEN — not a credential, not .env, not the workflow JSON. On a shared n8n instance, prefix environment variables with the project name to avoid collisions with other workflows — a generic name like TELEGRAM_BOT_TOKEN can silently already belong to a different bot on the same host.
   • 16-request-approval-telegram: fill telegram_chat_id and telegram_proxy_url in its Telegram Config node (regular group vs. supergroup chat IDs differ — see docs/findings.md); attach the proxy's header credential to Send Telegram message.
   • 21-approval-callback-telegram: needs its own httpHeaderAuth credential for X-Telegram-Bot-Api-Secret-Token (generate a secret, this is what replaces Slack's HMAC check — see docs/findings.md). Call Telegram's setWebhook (through the proxy, same as any other Bot API call) with url: https://<your-n8n-instance>/webhook/dociq-telegram-callback and that same secret_token.

6. Environment — see .env.example for the full list (GEMINI_API_KEY, DATABASE_URL, etc).

Stated plainly — a buyer will ask:

• Synthetic dataset. 50 generated invoices, not real vendor documents. Accuracy is measured honestly against ground truth, but real-world vendor formats will vary.
• Single expense account for all Bills. Production needs category-based GL account mapping, not one account for everything.
• No currency conversion. The sandbox QuickBooks company is USD-denominated; source amounts export without FX conversion. Production requires multi-currency mapping.
• Free-tier API quota. Gemini's free tier caps at 20 requests/day/model — fine for building and demoing this, not for a production volume of invoices.

MIT — see LICENSE.