Add Azure Blob Storage archive & email safety net

- Add AzureBlobStorageService, DocumentArchiveSyncService, and related config for secondary PDF archiving of invoices/reminders - Add SQL procs and schema changes for archive backfill - Update invoice/reminder services to upload PDFs to blob storage - Add telemetry counters and unit tests for blob storage/archive logic - Add Fuchs:Email:OverrideRecipient config and enforce dev/test email redirect in ProcessWebComService, with tests - Improve JS date parsing (German formats), stricter JSON date detection - Increase widget SQL timeouts, update dependencies, docs, and project files
2026-07-03 10:15:04 +02:00
parent 3035ef1719
commit c3395bc83c
47 changed files with 1723 additions and 101 deletions
@@ -0,0 +1,359 @@
+# Invoice Lifecycle — From First Click to Customer Email
+
+Concept document describing the full, end-to-end invoicing process in the
+Fuchs Intranet: how an office user turns a completed service request into an
+invoice, how the invoice is drafted/previewed/edited, how it is finalised, and
+how it reaches the customer by email (including resend and reminders). This
+complements [`INVOICE_SET_PRICING.md`](INVOICE_SET_PRICING.md) (pricing/display
+rules for set items) and [`EVAL_live_invoice_editing.md`](EVAL_live_invoice_editing.md)
+(why the editor is stateless).
+
+> Scope note: this document describes the **implemented** flow in `Fuchs`
+> (ASP.NET Core MVC intranet, jQuery front-end). It does not cover the Razor
+> Pages areas of the workspace — the intranet/invoice module predates those and
+> is intentionally kept as-is (see `.github/copilot-instructions.md`).
+
+## 1. Actors & building blocks
+
+| Layer | Files | Responsibility |
+|---|---|---|
+| **Browser UI** | `js/intranet/modules/fis.req.js`, `fis.inv_shared.js`, `fis.inv.js` (bundled to `wwwroot/web/fis.req.de.js` / `fis.inv.de.js` via gulp `min:js`) | Request list, invoice editor dialog, PDF preview rendering, all user interaction |
+| **Controller** | `Controllers/IntranetController.Requests.cs`, `.Invoices.cs`, `.Invoices2.cs`, `.Reminder.cs` | Thin action dispatch (`req/*`, `inv/*`, `rem/*`), auth checks, request/response shaping |
+| **Services (DI)** | `Services/IInvoiceService` + `InvoiceService`, `Services/IComService` + `ProcessWebComService`, `Services/IPdfService` + `FuchsPdfService`, `Services/IReminderService` + `ReminderService` | Register/render/store invoices, send email, render PDF documents |
+| **Data model** | `code/FdsInvoiceData.cs` | Pure data holder: parses the posted `invc` JSON, builds SQL parameters (`BuildInvoiceParams`), exposes registration fields |
+| **PDF rendering** | `code/FuchsPdf.cs` (`ApplyInvoice`, `CreatePage_Letter`, `AddGirocode`), `code/InvoiceSetPricing.cs` | MigraDoc layout, draft watermark, GiroCode SEPA QR, set-pricing line transformation |
+| **Database** | `Fuchs_Database` SSDT project — `fds__prepInvoice`, `fds__createInvoice`, `fds__setInvoice`, `fds__createInvoice_Details`, `fds__setInvoiceFinal`, `fds__setInvoiceFile`, `fds__setInvoiceSent`, `fds__getInvoice`, `fds__newInvoiceId` | SQL-first persistence, invoice numbering, finalisation, sent/paid status |
+
+All backend I/O is SQL-first via OCORE helpers (`getSQLDataSet_async`,
+`setSQLValue_async`) and stored procedures — there is no EF Core in this path
+(see `.github/instructions/ocore.instructions.md`).
+
+## 2. High-level lifecycle
+
+```mermaid
+flowchart LR
+	A[Service request\ncompleted] --> B[Create invoice\nfrom request]
+	B --> C[Draft invoice\nregistered]
+	C --> D[Preview / Edit\nloop]
+	D -->|adjust items,\naddress, set-mode...| D
+	D -->|confirm| E[Finalise\ninvoice]
+	E --> F[Render + store\nfinal PDF]
+	F --> G[Email to\ncustomer]
+	G --> H[Mark as sent]
+	H -.optional.-> I[Resend email]
+	H -.optional.-> J[Reminder /\nMahnung]
+	E -.optional.-> K[Storno / Credit\nnote]
+```
+
+## 3. Entry point — from the request list
+
+Invoices are **not** created from a standalone "new invoice" wizard; they are
+always created *from* one or more completed service requests.
+
+1. The office user opens the **Aufträge** (requests) list, rendered by
+   `fis.req.js`. `$req.init2/init3` load the list from `req/reql`
+   (`HandleRequestList` → `fds__getRequests_list[2]`).
+2. Selecting a request row and clicking the invoice icon calls
+   `$inv.cInv` (bound in `fis.req.js` via `.click({ id: rw.Id }, $inv.cInv)`),
+   which — after an auth check (`fds_inv` level 2) — calls `$inv.cInv2({ id })`.
+3. `$inv.cInv2` posts to **`req/get`** (`HandleRequestGet` →
+   `fds__getRequest_details`, mode `r`) to load the request(s) plus any
+   already-linked invoice, then renders the invoice editor dialog
+   (`invoice_layout`) and wires `$inv.eM` (the contextual top menu: save,
+   set-mode switch, §13b toggle, contact-person edit).
+4. Multiple requests can be bundled onto a single invoice (the editor supports
+   several request "blocks", each becoming its own item group).
+
+```mermaid
+sequenceDiagram
+	participant U as User (browser)
+	participant JS as fis.req.js / fis.inv_shared.js
+	participant C as IntranetController
+	participant DB as SQL (fds__*)
+
+	U->>JS: click invoice icon on request row
+	JS->>JS: $inv.cInv -> $inv.cInv2({id})
+	JS->>C: POST req/get {id, mode:'r'}
+	C->>DB: EXEC fds__getRequest_details
+	DB-->>C: admin, requests, items, inv
+	C-->>JS: JSON {admin, requests, inv}
+	JS->>U: render invoice editor dialog (items, totals, address, email)
+```
+
+## 4. Drafting, preview and editing
+
+The invoice editor is **stateless**: the browser holds the working model
+(`table.invi` jQuery `.data()`), the server never caches a partial invoice
+between requests. Every preview/save round-trip posts the *entire* invoice
+payload; see `EVAL_live_invoice_editing.md` for the rationale.
+
+### 4.1 What the user can change
+- **Line items** — quantities, prices, notes, combine into one sum
+  (`$inv.rendersrq`, `$inv.quantChange`).
+- **Recipient fields** — invoice title, address, email, provision
+  location/period (inline edit fields, `fm(...)` helper in `fis.inv_shared.js`).
+- **§13b reverse-charge** toggle (`$inv.sp13b`) — suppresses VAT lines/columns.
+- **Set-pricing display mode** (`$inv.ssetmode` / `setSetmode`) — `SetPrice`
+  (default) / `ItemPrices` / `SetOnly`; see `INVOICE_SET_PRICING.md`. Purely
+  presentational — totals never change.
+- **Contact person** for the invoice (`$inv.sctp`, stored in `CustomValues`).
+
+All of this recalculates client-side totals live via the `fds.inv` event
+(`$inv.invSumUpdate`), which also builds the **backend item contract**
+(`$inv.itemToContract`: `{ id, type, title, desc, qty, price_net, total_net,
+vat }` plus set header/member tagging) that is posted to the server.
+
+### 4.2 Posting a draft / preview
+
+| User action | JS entry point | Endpoint | Controller handler | Effect |
+|---|---|---|---|---|
+| First save without preview | `$inv.ssave` | `req/save` | `Do_Process_Requests` case `save` | `RegisterInvoiceAsync(change: id present)` — creates or updates the draft row, **no PDF rendered** |
+| Preview a **new** invoice | `$inv.sprev(false)` | `req/sprep` | case `sprep` | Registers draft, then `GenerateInvoicePdf` → renders preview images |
+| Preview an **existing** draft | `$inv.sprev(true)` (aka `$inv.sedit`) | `req/sedit` | case `sedit` | Same as `sprep` but requires `id`, updates existing draft |
+| Cancel out of preview | dialog `cancel` handler | `req/sdel` | case `sdel` | `fds__remInvoice` — deletes the (unfinalised) draft row |
+
+Before posting, `$inv.invcPayload(d)` normalizes the editor's internal field
+names into the exact names `FdsInvoiceData.BuildInvoiceParams` reads (totals
+from `sms.ttn/ttb`, `invoicetitle→title`, `loc→provisionlocation`,
+`paymentterms→paymentterm`, etc. — see `INVOICE_SET_PRICING.md` for the full
+mapping table). The posted shape is always `{ admin, req, sms, new }`.
+
+On the server, `RegisterInvoiceAsync` (in `InvoiceService`) turns the posted
+JSON into SQL parameters and calls, in one batch:
+- **New invoice**: `fds__createInvoice` (allocates the `Id`, returns a fresh
+  row) → `fds__createInvoice_Details` (service net/VAT + `InvoiceOptions`,
+  e.g. `setmode:itemprices`, `§13b`).
+- **Existing draft**: `fds__setInvoice` (same parameter set, updates in place)
+  → `fds__createInvoice_Details` again.
+
+`GenerateInvoicePdf` (still in `InvoiceService`) builds `FuchsPdf.FdsTextBlocks`
+and calls `IPdfService.WriteLetterAsync` + `ApplyInvoice`, which internally
+uses `InvoiceSetPricing.Build(...)` to turn the posted item contract into
+ordered, priced/unpriced lines. While the invoice is **not yet final**
+(`IsDraft == true`), the rendered PDF carries a diagonal **"Entwurf"/draft
+overlay watermark** (`CreatePage_Letter(..., draft: true)` stamps
+`Data/overlay.png`) and — importantly — **no GiroCode payment QR** is added
+(`AddGirocode` is only called `if (!inv.IsDraft && payAmount > 0 ...)`).
+The preview images are returned as base64 (`DocToImageCollectionAsync`) and
+shown in a modal (`$ocms.dlg(... form:false, button:'Rechnung erstellen' ...)`).
+
+```mermaid
+sequenceDiagram
+	participant U as User
+	participant JS as fis.inv_shared.js
+	participant C as IntranetController.Requests
+	participant S as InvoiceService
+	participant DB as SQL
+
+	U->>JS: edit items / address / set-mode
+	JS->>JS: fds.inv event -> invSumUpdate (recalculate + build items[])
+	U->>JS: click "Vorschau" (preview)
+	JS->>C: POST req/sprep or req/sedit {invc: JSON, id?}
+	C->>S: RegisterInvoiceAsync(invoice, change, invId)
+	S->>DB: fds__createInvoice / fds__setInvoice + fds__createInvoice_Details
+	DB-->>S: registered invoice row (Id, ...)
+	S->>S: GenerateInvoicePdf (draft=true -> watermark, no GiroCode)
+	S-->>C: MigraDoc Document
+	C-->>JS: {id, img[] (base64 pages), total}
+	JS->>U: show PDF preview modal (confirm / cancel)
+```
+
+From the preview modal the user can loop back to editing (close and adjust),
+**cancel** (deletes the draft via `req/sdel`), or **confirm** to finalise.
+
+## 5. Finalisation ("Rechnung erstellen")
+
+Confirming the preview modal posts to **`req/sconf`**
+(`HandleRequestSconf`):
+
+1. `EXEC fds__setInvoiceFinal @Id, @authuser` — guarded by
+   `isFinal = 0 AND isSent = 0` so it can only fire once per draft. This is
+   also where the **real, sequential invoice number** is assigned:
+   `[InvoiceId] = fds__newInvoiceId(YEAR())` → format `R<year>-<0000>`
+   (the draft only ever had the internal numeric `Id`). `Version` is bumped
+   and any linked "replaces" bookkeeping (storno/credit chains) is updated.
+2. On success (`IsFinal == true` returned), the controller:
+   - Reloads the invoice via `LoadInvoiceAsync`.
+   - Calls `StoreInvoiceDocumentFileAsync` → renders the **final** PDF
+	 (`draft` now reflects `IsFinal`, so the watermark disappears and the
+	 GiroCode payment QR is added) → `fds__setInvoiceFile` persists the PDF
+	 bytes on the invoice row → `IBlobStorageService.UploadInvoicePdfAsync`
+	 archives a copy to blob storage.
+   - Re-fetches the invoice (`fds__getInvoice`) to get the final
+	 `SendToEmail` / `DocumentName` / `InvoiceBalance`.
+3. If a recipient email is present and the PDF rendered, the controller emails
+   it immediately (see §6) and marks the invoice `Sent` via
+   `fds__setInvoiceSent @auto=true`.
+4. The browser opens the stored PDF in a new tab (`req/idoc`), returns to the
+   request list, and reloads it (`$ocms.init('req')`, `$inv.rReload()`).
+
+```mermaid
+flowchart TD
+	A[POST req/sconf] --> B{fds__setInvoiceFinal\nisFinal=0 AND isSent=0?}
+	B -- no --> Z[500 - Aktion war nicht erfolgreich]
+	B -- yes --> C[Assign real InvoiceId\nR-yyyy-nnnn, bump Version]
+	C --> D[LoadInvoiceAsync]
+	D --> E[Render final PDF\n[no watermark, + GiroCode]]
+	E --> F[fds__setInvoiceFile\npersist PDF bytes]
+	F --> G[Upload to blob storage\nIBlobStorageService]
+	G --> H{SendToEmail set\nand PDF non-empty?}
+	H -- yes --> I[IComService.SendEmailAsync\nattach PDF inline]
+	I --> J[fds__setInvoiceSent auto=true]
+	H -- no --> K[Skip email - PDF only]
+	J --> L[Open PDF in new tab / reload request list]
+	K --> L
+```
+
+## 6. Sending the invoice by email
+
+Email delivery is handled by `IComService` / `ProcessWebComService`, which
+talks to the **ProcessWeb Mailer API** (`push_com`, `comType=email`).
+
+- `HandleRequestSconf` builds the HTML body via `BuildInvoiceBody(balance,
+  paymentTerms)` — a fixed German thank-you text, the amount (if non-zero),
+  and payment instructions (IBAN/BIC) with the invoice's payment term
+  (`fdInv.PaymentTerms`, e.g. `10wd` → "10 Werktagen").
+- The rendered PDF bytes are attached as `{ [DocumentName] = filebyte }` — a
+  single-entry dictionary of filename → bytes.
+- `SendEmailAsync(reference, subject, htmlBody, recipient, displayName,
+  attachments)` is called with `reference = "inv_<InvoiceId>"`,
+  `subject = "SanitärFuchs - <DocumentName>"`.
+- Inside `ProcessWebComService.SendEmailAsync`:
+  - The recipient email is validated; a configured
+	`Fuchs:Email:OverrideRecipient` (dev/test safety net, must stay empty in
+	Production) redirects **all** outbound mail to a single address.
+  - A signature is appended to the HTML body.
+  - If the mailer is disabled (`_settings.Enabled == false`), sending is a
+	documented no-op.
+  - Attachments are base64-encoded inline (`{ filename, mimeType,
+	contentBase64 }`) and POSTed to the Mailer API.
+- Only on a **successful** send does the controller call
+  `fds__setInvoiceSent @auto=true`, which sets `IsSent = 1` — so a failed
+  mailer call leaves the invoice finalised but *not marked sent* (visible via
+  the "sis" — "als versendet markieren" — manual action, see §7).
+
+```mermaid
+sequenceDiagram
+	participant C as IntranetController
+	participant Com as IComService (ProcessWebComService)
+	participant API as ProcessWeb Mailer API
+	participant DB as SQL
+
+	C->>C: BuildInvoiceBody(balance, paymentTerms)
+	C->>Com: SendEmailAsync("inv_<Id>", subject, body, email, "", {pdf})
+	Com->>Com: validate email, apply OverrideRecipient (dev), append signature
+	alt mailer enabled
+		Com->>API: POST push_com {comType:"email", attachments:[base64 pdf]}
+		API-->>Com: success/failure
+	else mailer disabled
+		Com-->>Com: no-op (logged)
+	end
+	Com-->>C: sent: true/false
+	opt sent == true
+		C->>DB: EXEC fds__setInvoiceSent @auto=true
+	end
+```
+
+## 7. After finalisation — manual actions
+
+Once an invoice is `IsFinal`, the request/invoice list menus
+(`$inv.iMn`/`$inv.iMnr` in `fis.inv_shared.js`) expose:
+
+| Action | JS | Endpoint | Notes |
+|---|---|---|---|
+| **Resend email** | `$inv.ccInv`/context menu → *(re-)send* → `HandleRequestResend` | `req/resend` | Re-renders the stored/registered PDF bytes (`RenderInvoicePdfBytesAsync`, no re-registration) and re-sends via `IComService`, **without** re-touching `IsSent` |
+| **Mark as sent manually** | `$inv.sis(id)` | `inv/sis` | `fds__setInvoiceSent @auto=false` — for invoices sent outside the system (e.g. printed/posted) |
+| **View / display PDF** | `$inv.disp(id, 'inv')` | `inv/rdoc` | Renders page images for on-screen display only |
+| **Download / open PDF** | `req/idoc` or `inv/pget`-derived | `req/idoc` | Serves the stored (or freshly rendered) PDF inline |
+| **Mark paid / unpaid** | `$inv.setPyd`/`setUpd` | `inv/setpyd` / `inv/setupd` | `fds__setInvoicePayed` / `fds__setInvoiceUNPayed` |
+| **Storno (cancel)** | `$inv.storno` → `$inv.cSt` | `inv/storno` (mode `simple`/`copy`) | `fds__createStorno_simple` / `fds__createStorno_copy`; the replaced invoice is cross-linked (`Replaces_InvId`) and cancelled once the storno itself is finalised |
+| **Credit note** | `$inv.credit` → `$inv.cSt` | `inv/credit` (mode `credit`) | `fds__createCredit_simple` — same preview/finalise/email loop as a normal invoice |
+| **Continue editing a draft** | `$inv.cntInv` / `clCntInv` | `inv/get` | Only available while `isFinal == false`; re-opens the same stateless editor |
+
+Storno/credit invoices are **new invoice drafts** created from the original
+one — they go through the exact same preview → finalise → email pipeline
+described in §4–§6, just seeded from `fds__createStorno_*`/`fds__createCredit_simple`
+instead of from a service request.
+
+## 8. Reminders (Mahnungen) — downstream of a sent, unpaid invoice
+
+Not part of the invoice creation flow itself, but the natural continuation
+once an invoice is sent and remains unpaid:
+
+- `$inv.ccRem(id, InvoiceId)` → `rem/lrem` (load prior reminder history) →
+  reminder editor (`$inv.ccRem_s2`) → `rem/get` → **preview** via `rem/prep`
+  (`$inv.rprev`, same stateless pattern as invoice preview) → **finalise**
+  via `rem/conf` → PDF opened, list reloaded.
+- `$inv.dspRem(id)` lists existing reminders (`inv/getrem`); each can be
+  resent (`rem/resend`, mirrors `req/resend`) or downloaded.
+- `$inv.srs(id)` → `rem/srs` marks a reminder as sent manually (mirrors
+  `inv/sis`).
+- Reminder PDFs use the same draft-watermark/GiroCode rule as invoices
+  (`FuchsPdf.ApplyReminder`, `rem.IsDraft`).
+
+This is handled by `IReminderService`/`ReminderService` and
+`IntranetController.Reminder.cs`, following the identical
+draft-preview-finalise-email shape as invoices — intentionally, so the two
+flows share the same mental model for the office user.
+
+## 9. End-to-end summary diagram
+
+```mermaid
+flowchart TD
+	subgraph Requests
+		R1[Service request completed] --> R2[Open request list - req/reql]
+		R2 --> R3[Select request(s), click invoice icon]
+	end
+
+	subgraph Draft & Edit
+		R3 --> D1[req/get - load request + existing invoice]
+		D1 --> D2[Edit items, address, email,\nset-mode, §13b, contact]
+		D2 --> D3[req/save - persist draft only]
+		D2 --> D4[req/sprep or req/sedit -\nregister + render draft PDF]
+		D4 --> D5[Preview modal\ndraft watermark, no GiroCode]
+		D5 -->|edit more| D2
+		D5 -->|cancel| D6[req/sdel - delete draft]
+	end
+
+	subgraph Finalise & Send
+		D5 -->|confirm| F1[req/sconf -\nfds__setInvoiceFinal]
+		F1 --> F2[Assign real InvoiceId\nR-yyyy-nnnn]
+		F2 --> F3[Render final PDF\nno watermark, + GiroCode]
+		F3 --> F4[fds__setInvoiceFile +\nblob storage upload]
+		F4 --> F5{SendToEmail set?}
+		F5 -- yes --> F6[IComService.SendEmailAsync\nProcessWeb Mailer API]
+		F6 --> F7[fds__setInvoiceSent auto=true]
+		F5 -- no --> F8[PDF stored, not emailed]
+	end
+
+	subgraph After Sending
+		F7 --> A1[req/resend - resend email]
+		F7 --> A2[inv/setpyd / setupd - payment status]
+		F7 --> A3[inv/storno or inv/credit -\nnew linked invoice draft]
+		F7 --> A4[rem/* - reminder / Mahnung flow]
+		A3 -.re-enters.-> D2
+	end
+```
+
+## 10. Key invariants worth remembering
+
+- **Stateless editor**: every preview/save/finalise call re-posts the full
+  `invc` JSON; the server never holds a partial invoice in memory or session
+  between requests (see `EVAL_live_invoice_editing.md`).
+- **Totals come from the registration, not the rendered lines**: `sms.ttn`
+  /`sms.ttb` (posted) become `InvoiceBalance`/`InvoiceBalance_net`; display
+  mode (set pricing) never changes what the customer owes.
+- **Numbering happens only at finalisation**: the draft has an internal `Id`
+  but no `InvoiceId` until `fds__setInvoiceFinal` assigns
+  `R<year>-<sequence>` — so previews never "burn" an invoice number.
+- **Draft vs. final changes the rendered PDF**: draft = watermark overlay, no
+  GiroCode; final = no watermark, GiroCode payment QR added when there's a
+  positive balance.
+- **Email is best-effort and tracked**: `IsSent` is only set `true`
+  automatically after a *successful* send; a failed send still leaves a
+  correctly finalised, stored invoice that staff can resend or mark sent
+  manually.
+- **Storno/credit/reminder all reuse the same pipeline**: they are not special
+  cases in the UI/backend contract — they are just differently-seeded drafts
+  going through the identical preview → finalise → email sequence.
+5