feat: Implement mfr truncation fix and enhance invoice handling
Playwright Tests / test (push) Canceled after 0s
Playwright Tests / test (push) Canceled after 0s
- Added a new SQL script to address the "String or binary data would be truncated" error in mfr sync by widening the Note fields in service objects and companies to NVARCHAR(MAX). - Updated the table types and associated stored procedures to accommodate the changes, ensuring compatibility with existing data. - Queued a re-sync for previously rejected rows due to truncation issues. - Created application logins for fuchs_fds and fuchs_web with placeholder passwords for initial setup. - Introduced functions for invoice backdating and invoice date retrieval to streamline invoice processing. - Developed a password generation function with customizable parameters for length and character types. - Added a random number generation function to support password creation.
This commit is contained in:
Stefan
@@ -0,0 +1,68 @@
|
||||
---
|
||||
status: Accepted
|
||||
date: 2026-09-30
|
||||
applyTo:
|
||||
- "Fuchs_Database/dbo/Functions/fds__fn_invoiceBackdate.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__setInvoiceFinal.sql"
|
||||
- "Fuchs/code/InvoiceDates.cs"
|
||||
- "Fuchs/Services/ERechnungMapper.cs"
|
||||
- "Fuchs/Services/InvoiceService.cs"
|
||||
- "Fuchs/Controllers/IntranetController.InvoiceDraft.cs"
|
||||
- "Fuchs/Controllers/IntranetController.Requests.cs"
|
||||
- "Fuchs/js/intranet/modules/fis.inv_shared.js"
|
||||
supersededBy: ""
|
||||
---
|
||||
|
||||
# 0032 — Invoices may be backdated to the last working day of the previous month (admins only)
|
||||
|
||||
## Context
|
||||
|
||||
Work done at the end of a month is often invoiced in the first days of the next month. The customer
|
||||
wants those invoices dated to the previous month. The invoice date was always the moment of
|
||||
finalisation (`fds__setInvoiceFinal`: `DateFinalized = GETUTCDATE()`), and the invoice number is
|
||||
drawn from the year of that date (`fds__newInvoiceId(Year(...))`).
|
||||
|
||||
Two invoice dates also existed side by side: the PDF printed `DateCreated` (draft creation) as
|
||||
"Rechnungsdatum", while the embedded ZUGFeRD/XRechnung issue date (BT-2) used `DateFinalized` — a
|
||||
visible date and an e-invoice date that could disagree.
|
||||
|
||||
## Decision
|
||||
|
||||
- **One invoice date.** PDF "Rechnungsdatum" and e-invoice issue date both come from
|
||||
`InvoiceDates.IssueDate`: `DateFinalized` (converted from the stored UTC to the German calendar
|
||||
day), falling back to `DateCreated` for drafts.
|
||||
- **Backdating option.** When finalising, a user with `fds_inv > 2` may date the invoice to the
|
||||
**last working day (Mon–Fri) of the previous month** — a fixed date, not a free choice. Public
|
||||
holidays are not excluded (possible later extension).
|
||||
- **Only while no invoice exists after that date.** Invoice numbers must ascend with invoice dates,
|
||||
so the option is offered and accepted only while no invoice has been finalized after the backdate
|
||||
day (a 2 h margin covers the UTC storage).
|
||||
- **Single source of truth:** `[dbo].[fds__fn_invoiceBackdate](@authuser)` returns the allowed date or
|
||||
NULL (authorisation + eligibility). `inv/dpreview` returns it so the preview dialog shows the
|
||||
per-invoice checkbox; `req/sconf` passes `@backdate = 1`, and `fds__setInvoiceFinal` re-evaluates
|
||||
the function and `THROW`s (60001) when it is no longer allowed — nothing is finalised then.
|
||||
- **Serialized finalisation.** `fds__setInvoiceFinal` runs check + numbering `UPDATE` in one
|
||||
transaction under an exclusive, transaction-owned app lock (`sp_getapplock`, resource
|
||||
`fds__setInvoiceFinal`) taken by every finalisation, so a concurrent normal finalisation cannot
|
||||
slip between the eligibility check and the backdated write and break the date/number order.
|
||||
- `req/sconf` maps only SQL error 60001 to the "no longer possible, create without backdating"
|
||||
message. A 0-row result (invoice already final/sent, e.g. a double submit) is reported as
|
||||
"already finalised" — never as a backdate rejection, which would invite a duplicate invoice.
|
||||
- A backdated invoice is stored with `DateFinalized = <day> 12:00` (UTC, stays on the same German
|
||||
day), its number comes from that day's year, and `DueDate` is recomputed from the payment term
|
||||
anchored on that day.
|
||||
## Consequences
|
||||
|
||||
- The PDF of every newly finalised invoice now shows the finalisation date instead of the draft
|
||||
creation date (intended: it now matches the e-invoice and the number sequence).
|
||||
- In January the option yields a December date and therefore a number from the previous year's range.
|
||||
- Lists, reports, DATEV and reminders follow the same invoice date — see ADR 0035.
|
||||
- Telemetry: `fuchs.invoices.backdated` (outcome ok / rejected / already_final / error); successful
|
||||
backdating logs EventId `InvoiceBackdated`.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Free date picker for admins** — rejected: breaks the date/number order too easily; the need is
|
||||
exactly "previous month's last working day".
|
||||
- **Global admin switch "backdate everything until the first invoice of the month"** — rejected by
|
||||
the customer in favour of an explicit per-invoice checkbox.
|
||||
@@ -0,0 +1,52 @@
|
||||
---
|
||||
status: Accepted
|
||||
date: 2026-09-30
|
||||
applyTo:
|
||||
- "Fuchs_Database/dbo/Tables/mfr__*.sql"
|
||||
- "Fuchs_Database/dbo/User Defined Types/mfr__tt__*.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/mfr__updt__*.sql"
|
||||
- "Fuchs_DataService/FdsMfrClient.cs"
|
||||
- "Fuchs_DataService/FdsColumnWidths.cs"
|
||||
supersededBy: ""
|
||||
---
|
||||
|
||||
# 0033 — mfr free-text columns are NVARCHAR(MAX); table, table type and copies stay aligned
|
||||
|
||||
## Context
|
||||
mfr publishes no length limits for its string properties (`MFR_RESTClient/Docs/mfr_interface_description.md`).
|
||||
`[mfr__serviceobjects].[Note]` and `[mfr__companies].[Note]` were `NVARCHAR(255)`, as were their
|
||||
merge types `mfr__tt__serviceobjects` / `mfr__tt__companies`. From 2026-09-25, users typed longer
|
||||
notes in mfr (ServiceObject up to 753 chars, Company up to 723). The `mfr__updt__*` merge proc copies
|
||||
the staging table into the table type, so SQL Server rejected every batch holding such a row
|
||||
(msg 8152, which names no column). That included every Company/Contact/ServiceRequest sync that
|
||||
`$expand`s the row. The result was about 1000 errors in five days, plus rows the short sync never
|
||||
re-fetched, because it only reads rows newer than the local `DateModified` watermark. Merge failures
|
||||
were silent until 8bcec40 began logging them, so earlier occurrences went unnoticed.
|
||||
|
||||
## Decision
|
||||
- Free-text mfr properties (`Note`, `Description`, `Comment`, …) are stored as `NVARCHAR(MAX)`.
|
||||
Bounded widths are kept only for values mfr structurally keeps short (ids, codes, names), and
|
||||
those are revisited when data proves otherwise.
|
||||
- A width change is made in the target table, its `mfr__tt__*` type **and** every table variable
|
||||
or type that copies the row with `SELECT *` (e.g. `@company` in `fds__getRequest_details`,
|
||||
`fds__prepInvoice`, `fds__prepStorno_recreate`), all in the same change. Otherwise the truncation
|
||||
just moves to the next copy.
|
||||
- Truncating at the mapping layer was rejected. The data service does **not** truncate
|
||||
silently. Instead, when a merge fails with 8152/2628, `FdsColumnWidths` compares the staged
|
||||
batch against the narrower of the table and type widths. The error log then names column, longest
|
||||
length, limit and an entity Id (`over-long columns: Note: 753 > 255 chars (1 row, e.g. Id=…)`).
|
||||
This lookup runs only on the failure path.
|
||||
|
||||
## Consequences
|
||||
- Changing a `mfr__tt__*` type means DROP + CREATE (types cannot be ALTERed). The procs that
|
||||
DECLARE the type must be dropped first and recreated. See
|
||||
`Fuchs_Database/Scripts/2026-09-30_mfr_truncation_fix.sql` for the pattern, including grants.
|
||||
- After fixing a rejected merge, queue the affected Ids in `fds__mfr_updaterequests`. The short
|
||||
sync will not revisit them, and the nightly full reset is disabled in `fds__getUpdateableTables`.
|
||||
- `NVARCHAR(MAX)` columns cannot be index key columns. None of the widened columns is indexed.
|
||||
|
||||
## Alternatives considered
|
||||
- **Truncate to the column width in `SetRowValue`**: this would keep syncing, but it loses user text
|
||||
silently. For keys such as `ExternalId` (DATEV customer number) it would corrupt data. Rejected.
|
||||
- **Widen to a larger fixed size (e.g. 1000, like `mfr__contacts.Note`)**: this just moves the
|
||||
cliff. mfr enforces no limit. Rejected for free text.
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
status: Accepted
|
||||
date: 2026-09-30
|
||||
applyTo:
|
||||
- "Fuchs_Database/dbo/Functions/fds__fn_IntermediateIsAllocatedToOther.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__setInvoiceFinal.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__prepInvoice.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__prepStorno_recreate.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__createStorno_*.sql"
|
||||
- "Fuchs_Database/Scripts/2026-09-30_backfill_intermediate_invoice_allocations.sql"
|
||||
supersededBy: ""
|
||||
---
|
||||
|
||||
# 0034 — An Abschlagsrechnung credit follows the live invoice
|
||||
|
||||
## Context
|
||||
|
||||
A Schlussrechnung credits the Abschlagsrechnungen (intermediate invoices, `InvoiceType = 'i'`)
|
||||
already paid for the same service requests via `IntInvoiceAllocation` lines. Finalising it records
|
||||
the claim in `fds__invoice_details.AllocatedTo_InvId` of each credited intermediate so that
|
||||
`fds__prepInvoice` does not offer the same Abschlagsrechnung again.
|
||||
|
||||
Until 2026-09-30 that claim was never actually written (the editor dropped `reference_InvId`, the
|
||||
MERGE aborted on the NULL). The moment it is written, the question "what happens to the claim when
|
||||
the crediting invoice is cancelled?" becomes real: a Storno (simple/copy) or a Storno + recreate
|
||||
leaves `AllocatedTo_InvId` pointing at the cancelled invoice, and the next invoice for the same
|
||||
service requests would silently bill the customer again for money already paid.
|
||||
|
||||
## Decision
|
||||
|
||||
- **A claim counts only while the crediting invoice is alive.** An `AllocatedTo_InvId` that points
|
||||
at a cancelled invoice (`IsCanceled = 1`), at a non-existent invoice, or at the invoice that the
|
||||
one being finalised replaces (`CanceledInvoice` line → `@Replaces`) is void: the Abschlagsrechnung
|
||||
is offered again by `fds__prepInvoice` / `fds__prepStorno_recreate` and may be taken over by the
|
||||
next finalisation. The single source of truth for "already credited elsewhere" is
|
||||
`[dbo].[fds__fn_IntermediateIsAllocatedToOther]`; `fds__setInvoiceFinal`'s `existing` CTE is its
|
||||
set-based twin and must apply the same rule.
|
||||
- **The link is not cleared on cancellation.** Cancellation has several paths (storno simple/copy,
|
||||
storno + recreate, the retired `fds__setInvoiceCancelled`); rather than releasing the link in each,
|
||||
the readers ignore dead links. The last claimant therefore stays visible for auditing.
|
||||
- **A Stornorechnung (`InvoiceType = 'c'`) never claims.** `fds__createStorno_copy` mirrors the
|
||||
cancelled invoice's `IntInvoiceAllocation` lines (with their references) so the storno document
|
||||
shows them, but finalising the storno must not move the allocation onto it.
|
||||
- **Storno + recreate carries the references.** `fds__prepStorno_recreate` emits its credit lines
|
||||
with the same type as `fds__prepInvoice` (`IntInvoiceAllocation`, not `PartialInvoice`) and the
|
||||
credited invoice in `reference_InvId`, and its `CanceledInvoice` line references the replaced
|
||||
invoice — otherwise the replacement neither takes over the credit nor records what it replaces.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The recreate flow now produces `Replaces_InvId` / `DateCancelled` on the replaced invoice via the
|
||||
regular `fds__setInvoiceFinal` bookkeeping (it did not before: no reference reached the proc).
|
||||
- A second recreate (cancel the replacement, recreate again) keeps the credit because the claim on
|
||||
the replaced invoice is treated as void during the replacement's finalisation.
|
||||
- The one-off backfill script records the *earliest* live claimant; it is consistent with this rule.
|
||||
- An intermediate that is itself cancelled is not offered (unchanged).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Release `AllocatedTo_InvId` when the crediting invoice is cancelled** — rejected: four write
|
||||
paths to keep in sync, and the audit trail of who last credited the Abschlagsrechnung is lost.
|
||||
- **Keep `PartialInvoice` for recreated credit lines and teach `fds__setInvoiceFinal` both types** —
|
||||
rejected: one line type per meaning; `PartialInvoice` remains the Abschlagsbetrag charge line of an
|
||||
intermediate invoice.
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
status: Accepted
|
||||
date: 2026-09-30
|
||||
applyTo:
|
||||
- "Fuchs_Database/dbo/Functions/fds__fn_invoiceDate.sql"
|
||||
- "Fuchs_Database/dbo/Functions/fds__r_getBalanceThisMonth.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__getDatevExports.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__getInvoices_list*.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__getFDSDocument.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__getReportDocument.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__getRequest_details.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__r_*.sql"
|
||||
- "Fuchs_Database/dbo/Stored Procedures/fds__*Reminder*.sql"
|
||||
- "Fuchs/code/InvoiceDates.cs"
|
||||
- "Fuchs/Services/ReminderService.cs"
|
||||
- "Fuchs/Services/*DraftEditService.cs"
|
||||
supersededBy: ""
|
||||
---
|
||||
|
||||
# 0035 — The invoice date is `DateFinalized` everywhere
|
||||
|
||||
## Context
|
||||
|
||||
ADR 0032 made `DateFinalized` the invoice date for the PDF, the e-invoice (BT-2) and the number
|
||||
sequence, and introduced backdating (which only moves `DateFinalized`). Everything else — invoice
|
||||
lists, revenue reports and charts, the DATEV export, document lookups, the request details — still
|
||||
dated and grouped invoices by `DateCreated`, the moment the *draft* was created. A backdated invoice
|
||||
therefore showed September on the document and October in every list, report and in DATEV, and any
|
||||
invoice drafted in one month and finalised in the next had the same split.
|
||||
|
||||
## Decision
|
||||
|
||||
- **One invoice date, everywhere:** the German calendar day of `DateFinalized`, falling back to
|
||||
`DateCreated` only for drafts. Both columns are stored in UTC.
|
||||
- **One implementation per side:** `[dbo].[fds__fn_invoiceDate](DateFinalized, DateCreated)` in SQL and
|
||||
`InvoiceDates.IssueDate` in C#. No proc converts or picks the date itself.
|
||||
- Applies to: the DATEV export (period filter, `Belegdatum`, `Belegfeld 2`), the invoice lists
|
||||
(`fds__getInvoices_list`, `list2`, `list_vario`: period filter, `Belegdatum`, `DateOfCreation`,
|
||||
ordering, overdue fallback), the revenue reports (`fds__r_*`: year/month filters and grouping,
|
||||
`fds__r_getBalanceThisMonth`), document lookups (`fds__getFDSDocument`, `fds__getReportDocument`),
|
||||
`fds__getRequest_details` (new column `InvoiceDate`, shown in the request's invoice list), and the
|
||||
invoice date shown in reminders (`fds__prepReminder`, `fds__createReminder`, `fds__getReminder`).
|
||||
- **Reminders use the same rule for their own date:** the reminder PDF's date is the reminder's
|
||||
`DateFinalized` (German day), draft: `DateCreated`.
|
||||
- **No transition safeguard.** The switch applies to all periods, past and future; already exported
|
||||
DATEV periods are not reconciled (confirmed by the business owner, 2026-09-30).
|
||||
- `DateCreated` keeps its meaning — the creation timestamp of the row — and is still returned where
|
||||
callers read it as such. Draft previews stamp it with `DateTime.UtcNow` to match the stored column.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Reports and exports for a past period can differ from what they showed before this change: an
|
||||
invoice drafted in month M and finalised in M+1 now counts in M+1.
|
||||
- Filters on the invoice date are not SARGable (they were not before either: `CAST(DateCreated as date)`).
|
||||
Acceptable at the current table size; an indexed computed column is the follow-up if it matters.
|
||||
- Any new list, report or export that shows or groups invoices by date must use
|
||||
`fds__fn_invoiceDate` — never `DateCreated` or a raw `DateFinalized`.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep `DateCreated` in lists/reports, `DateFinalized` only on documents** — rejected: the same
|
||||
invoice would carry two dates, and backdating would never reach the books.
|
||||
- **Transition safeguard for already exported DATEV periods** — rejected by the business owner.
|
||||
@@ -0,0 +1,85 @@
|
||||
---
|
||||
status: Accepted
|
||||
date: 2026-09-30
|
||||
applyTo:
|
||||
- "Fuchs_Database/**"
|
||||
supersededBy: ""
|
||||
---
|
||||
|
||||
# 0036 — The database project build is the T-SQL gate
|
||||
|
||||
## Context
|
||||
|
||||
A rewrite of `fds__getDatevExports` added a `JOIN @inv as i` next to `fds__invoices as iv`
|
||||
and left an unqualified `[file_guid]` in the select list. SQL Server rejects that statement
|
||||
with Msg 209 (ambiguous column name) — but only when it runs, and the working-tree SQL runs
|
||||
nowhere: the schema lives in the `Fuchs_Database` SSDT project, the dev database only gets
|
||||
what is explicitly published to it, and the xUnit suite cannot bind T-SQL. The defect reached
|
||||
code review as the only check.
|
||||
|
||||
The SSDT build would have caught it: it reported the line as `SQL71502` ("unresolved reference
|
||||
… or ambiguous"). Nobody saw that because
|
||||
|
||||
- the project did not build at all — the two `CREATE USER … FOR LOGIN` files referenced logins
|
||||
that were not in the model, which SSDT treats as error `SQL71501`;
|
||||
- the genuine warning sat among ~160 identical `SQL71502` warnings caused by references to
|
||||
`sys.*`, `INFORMATION_SCHEMA`, `sp_executesql`, `sp_getapplock` (no `master` reference) and
|
||||
to `ocms_fn_generatePassword`, an OCMS helper the project never contained;
|
||||
- a warning does not fail anything.
|
||||
|
||||
The same sweep found that `ocms_fn_generatePassword` (and its dependency `ocms_fn_randbetween`)
|
||||
is missing on the dev database as well, so every id-generating function (`fds__fn_invoice_id`,
|
||||
`fds__fn_reminder_id`, `fis_fn_id`, …) fails there with Msg 4121.
|
||||
|
||||
## Decision
|
||||
|
||||
1. `FuchsDatabase.sqlproj` references the `master` system dacpac
|
||||
(`$(DacPacRootPath)\…\SqlServer\170\SQLSchemas\master.dacpac`), so system objects bind.
|
||||
2. Everything the SQL references is in the project. The OCMS helpers `ocms_fn_generatePassword`
|
||||
and `ocms_fn_randbetween` are vendored copies of the OCMSsharp `ocms_mt_database` files.
|
||||
`Security/Logins.sql` scripts the two application logins with a placeholder,
|
||||
`MUST_CHANGE` password so the users bind; real passwords stay in Key Vault and SSDT never
|
||||
compares passwords, so existing logins are untouched on publish.
|
||||
3. `TreatTSqlWarningsAsErrors` is on. Suppressed: `71501` (dynamic SQL / cross-database
|
||||
references) and `71558` (identifier differs only by case — harmless under the `CI` model
|
||||
collation). Every other T-SQL warning, in particular `SQL71502`, fails the build.
|
||||
4. Building the database project is part of the workflow whenever `Fuchs_Database/**` changes
|
||||
(`msbuild Fuchs_Database/FuchsDatabase.sqlproj -p:Configuration=Debug`, from the Visual
|
||||
Studio MSBuild — `dotnet build` cannot build the legacy SSDT project format). A red database
|
||||
build is a failure like a red C# build.
|
||||
5. The C# side of the contract is a test: `Fuchs.Tests/SqlContractTests` scans every
|
||||
`[dbo].[…]` named in C# strings (`Fuchs`, `Fuchs_DataService`) and checks it against the
|
||||
SSDT files — the object exists, every `EXECUTE` binds (no unknown names, no surplus
|
||||
positional arguments, every parameter without a default supplied), every function call
|
||||
passes the declared argument count, and every `mfr__updt__*` proc declares the
|
||||
`@tblname, @referencetable, @tgtid` signature `FdsMfrClient` calls dynamically.
|
||||
|
||||
## Consequences
|
||||
|
||||
- An unqualified column that becomes ambiguous, a typo in a column name, or a call to a
|
||||
function that is not in the project is a build error before review, not a Msg 209/4121 at
|
||||
a customer.
|
||||
- `SELECT *` inside `EXISTS (…)` over joined tables trips `SQL71502` in SSDT even though SQL
|
||||
Server accepts it; write `SELECT 1` there (done for `fds__fn_IntermediateIsAllocatedToOther`
|
||||
and `fis_admin_testGroupMembership`).
|
||||
- New procedures that use `sys.*`/`sp_*` need no suppression any more. A new reference that is
|
||||
genuinely outside the model (another database, dynamic SQL) is suppressed per statement or
|
||||
by adding the object to the project — never by widening `SuppressTSqlWarnings` for `71502`.
|
||||
- A proc signature change that breaks a C# caller fails `dotnet test`, not the first request.
|
||||
Only literal SQL is covered: dynamically built names (`FuchsVisualization`'s report procs,
|
||||
resolved from `sys.parameters` at runtime) and result-set column names are not.
|
||||
- The build proves binding, not behaviour: publishing to the dev database (`SqlPackage
|
||||
/Action:Publish` with `ExcludeObjectTypes=Logins;Users;Permissions;RoleMembership`, or the
|
||||
schema compare) and exercising the code path remains the runtime check.
|
||||
- The dev database needs the two `ocms_*` helpers published before any id-generating path
|
||||
works there.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Run the changed procedures against the dev database before review.** Needs a publish of
|
||||
every working-tree change first, and does not cover code paths the reviewer does not think
|
||||
to exercise; the compile-time check is free and total.
|
||||
- **Suppress `SQL71502` and keep the build warning-free that way.** Removes exactly the signal
|
||||
that catches this class of bug.
|
||||
- **A C#-side test that parses the SQL for unqualified columns.** A parser-level heuristic
|
||||
without a schema cannot decide ambiguity; the SSDT model can.
|
||||
Reference in New Issue
Block a user