damir/pgz-sport
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

DI exec: applied CC-DI Subagent A+B SQL — 3245 clanovi, Manuel Boras merged

Browse Source
This commit is contained in:
Damir Radulić
2026-05-05 09:04:14 +02:00
parent e7102c720d
commit 4e4d69c04a
6 changed files with 451 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/scheduled_tasks.lock
+1
View File
@@ -0,0 +1 @@
{"sessionId":"3eea00ef-fccd-4683-85c6-f7d39e8199a7","pid":1940465,"procStart":"327348495","acquiredAt":1777964592489}
_audit/data_integrity_20260505_0836/D_CONSTRAINTS.md
+61
View File
@@ -0,0 +1,61 @@
# Subagent D — Schema Quality Constraints (pgz_sport.clanovi)
Date: 2026-05-05
Live row count: 3240 (backup retained at `pgz_sport.clanovi_backup_20260505_0836` = 3243)
## Summary table
| # | Candidate | Type | Pre-flight violators | Status | Object name |
|---|-----------|------|----------------------|--------|-------------|
| C1 | No internal CamelCase boundary | CHECK | 0 | APPLIED | `clanovi_no_camelcase_chk` |
| C2 | ime/prezime trimmed | CHECK | 0 | APPLIED | `clanovi_trimmed_chk` |
| C3 | length(ime) >= 2 AND length(prezime) >= 2 | CHECK | 22 | SKIPPED (see D_violations.md) | — |
| C4 | spol IN ('M','Ž',NULL) | CHECK | 0 | ALREADY PRESENT | `clanovi_spol_check` (pre-existing) |
| C5 | hns_igrac_id partial UNIQUE | UNIQUE INDEX | 0 dup-groups | APPLIED | `clanovi_hns_uniq` |
| C6 | (klub_id, lower(ime), lower(prezime), datum_rodenja) UNIQUE | UNIQUE INDEX | 68 dup-groups | SKIPPED (see D_violations.md) | — |
| C7 | BEFORE INSERT/UPDATE normalize trigger | TRIGGER | n/a | APPLIED | `clanovi_normalize_trigger` + `pgz_sport.clanovi_normalize_fn()` |
## Trigger semantics
`clanovi_normalize_fn`:
1. Always `trim()` `NEW.ime` and `NEW.prezime`.
2. On `INSERT`, or on `UPDATE` only when `ime` or `prezime` actually change:
- reject CamelCase boundary (lenient: only ascii+Croatian-diacritic lower→upper pairs);
- reject `length(ime) < 2` or `length(prezime) < 2`.
3. The "only-when-name-changes" rule preserves the 22 legitimate historical short-name rows (e.g. `id=1852..2141`, mostly placeholder `'-'` ime + surname-only entries) so they can still receive `UPDATE`s on other fields.
## Smoke insert tests (all wrapped in BEGIN/ROLLBACK so live data unchanged)
| # | Scenario | Expected | Result |
|---|----------|----------|--------|
| 1 | INSERT `('IvoIvic','Test')` | reject (CamelCase) | REJECTED — `CamelCase rejected in ime: IvoIvic` |
| 2 | INSERT `('PetarPan','Test')` | reject | REJECTED |
| 3 | INSERT `(' Ivo ',' Ivić ')` | trim then succeed | INSERTED — stored as `('Ivo','Ivić')` |
| 4 | INSERT `('A','Test')` | reject (length) | REJECTED — `ime too short (<2 chars): A` |
| 5 | INSERT `('Ivan',' X ')` | trim → `'X'` len 1 → reject | REJECTED — `prezime too short (<2 chars): X` |
| 6 | INSERT `('Marko ',' Marković')` | trim then succeed | INSERTED — stored as `('Marko','Marković')` |
| 7 | INSERT duplicate `hns_igrac_id='209352'` | reject | REJECTED — `duplicate key value violates unique constraint "clanovi_hns_uniq"` |
| 8 | 2× NULL + 2× `''` `hns_igrac_id` rows | all 4 succeed (partial uniqueness ignores NULL/empty) | 4 INSERTS OK |
| 9 | UPDATE `id=1852` (`ime='-'`) `napomena=...` (no name change) | succeed | UPDATED — short-name row still mutable |
| 10 | UPDATE `id=1852` `ime='?'` (single char) | reject | REJECTED — `ime too short (<2 chars): ?` |
All 10 behaviours match expectations. No live row was modified — every test ROLLBACKed.
## Final lockdown state on `pgz_sport.clanovi`
CHECK constraints in force:
- `clanovi_no_camelcase_chk` (NEW)
- `clanovi_trimmed_chk` (NEW)
- `clanovi_spol_check` (pre-existing)
UNIQUE indexes in force:
- `clanovi_pkey` (id)
- `uq_clanovi_klub_profile` (klub_id, profile_url) — pre-existing
- `clanovi_hns_uniq` (hns_igrac_id) WHERE not null/empty — NEW
User triggers in force (BEFORE INSERT OR UPDATE):
- `clanovi_normalize_trigger` (NEW)
- `clanovi_validate_source` (pre-existing)
- `pgz_sport_clanovi_fts_trg` (pre-existing)
Row count unchanged at 3240.
_audit/data_integrity_20260505_0836/D_CONSTRAINTS.sql
+116
View File
@@ -0,0 +1,116 @@
-- pgz_sport.clanovi — schema lockdown DDL (Subagent D)
-- Author: dradulic@outlook.com / damir@rinet.one
-- Date: 2026-05-05
-- Description: Final, applied DDL. Pre-flight all-clean blocks below were
-- committed; SKIPPED candidates (length>=2 CHECK, klub+name+dob
-- UNIQUE) are documented in D_violations.md and intentionally
-- omitted here.
--
-- Row count at apply time: 3240 (live), 3243 (backup_20260505_0836).
-- Rollback hints: each block is independent and reversible via
-- ALTER TABLE pgz_sport.clanovi DROP CONSTRAINT ...;
-- DROP INDEX pgz_sport.clanovi_hns_uniq;
-- DROP TRIGGER clanovi_normalize_trigger ON pgz_sport.clanovi;
-- DROP FUNCTION pgz_sport.clanovi_normalize_fn();
-- ===========================================================================
-- C1: CHECK no internal CamelCase boundary (lower->upper letter pair)
-- Pre-flight violators: 0
-- ===========================================================================
BEGIN;
ALTER TABLE pgz_sport.clanovi
ADD CONSTRAINT clanovi_no_camelcase_chk
CHECK (
ime !~ '[a-zćčšđžáàâäéèêëíìîïóòôöúùûüñçý][A-ZĆČŠĐŽÁÀÂÄÉÈÊËÍÌÎÏÓÒÔÖÚÙÛÜÑÇÝ]'
AND prezime !~ '[a-zćčšđžáàâäéèêëíìîïóòôöúùûüñçý][A-ZĆČŠĐŽÁÀÂÄÉÈÊËÍÌÎÏÓÒÔÖÚÙÛÜÑÇÝ]'
);
COMMIT;
-- ===========================================================================
-- C2: CHECK ime/prezime are trimmed
-- Pre-flight violators: 0
-- ===========================================================================
BEGIN;
ALTER TABLE pgz_sport.clanovi
ADD CONSTRAINT clanovi_trimmed_chk
CHECK (ime = trim(ime) AND prezime = trim(prezime));
COMMIT;
-- ===========================================================================
-- C4: spol values constraint
-- NOT applied as new constraint — existing clanovi_spol_check already enforces
-- spol IN ('M','Ž',NULL). Documented for completeness.
-- CHECK (spol IS NULL OR spol IN ('M','Ž'))
-- ===========================================================================
-- ===========================================================================
-- C5: UNIQUE partial index on hns_igrac_id (non-null, non-empty)
-- Pre-flight duplicate groups: 0
-- ===========================================================================
BEGIN;
CREATE UNIQUE INDEX IF NOT EXISTS clanovi_hns_uniq
ON pgz_sport.clanovi (hns_igrac_id)
WHERE hns_igrac_id IS NOT NULL AND hns_igrac_id != '';
COMMIT;
-- ===========================================================================
-- C7: BEFORE INSERT/UPDATE normalize trigger
-- Trims ime/prezime, rejects CamelCase, enforces length>=2 only when names
-- change (so the existing 22 short-name historical rows can still be UPDATEd
-- on other fields without rejection).
-- ===========================================================================
BEGIN;
CREATE OR REPLACE FUNCTION pgz_sport.clanovi_normalize_fn()
RETURNS trigger
LANGUAGE plpgsql
AS $fn$
DECLARE
v_changed_name boolean;
BEGIN
IF NEW.ime IS NOT NULL THEN
NEW.ime := trim(NEW.ime);
END IF;
IF NEW.prezime IS NOT NULL THEN
NEW.prezime := trim(NEW.prezime);
END IF;
IF TG_OP = 'INSERT' THEN
v_changed_name := true;
ELSE
v_changed_name := (NEW.ime IS DISTINCT FROM OLD.ime)
OR (NEW.prezime IS DISTINCT FROM OLD.prezime);
END IF;
IF v_changed_name THEN
IF NEW.ime ~ '[a-zćčšđžáàâäéèêëíìîïóòôöúùûüñçý][A-ZĆČŠĐŽÁÀÂÄÉÈÊËÍÌÎÏÓÒÔÖÚÙÛÜÑÇÝ]' THEN
RAISE EXCEPTION 'CamelCase rejected in ime: %', NEW.ime
USING ERRCODE = 'check_violation';
END IF;
IF NEW.prezime ~ '[a-zćčšđžáàâäéèêëíìîïóòôöúùûüñçý][A-ZĆČŠĐŽÁÀÂÄÉÈÊËÍÌÎÏÓÒÔÖÚÙÛÜÑÇÝ]' THEN
RAISE EXCEPTION 'CamelCase rejected in prezime: %', NEW.prezime
USING ERRCODE = 'check_violation';
END IF;
IF length(coalesce(NEW.ime, '')) < 2 THEN
RAISE EXCEPTION 'ime too short (<2 chars): %', NEW.ime
USING ERRCODE = 'check_violation';
END IF;
IF length(coalesce(NEW.prezime, '')) < 2 THEN
RAISE EXCEPTION 'prezime too short (<2 chars): %', NEW.prezime
USING ERRCODE = 'check_violation';
END IF;
END IF;
RETURN NEW;
END;
$fn$;
DROP TRIGGER IF EXISTS clanovi_normalize_trigger ON pgz_sport.clanovi;
CREATE TRIGGER clanovi_normalize_trigger
BEFORE INSERT OR UPDATE ON pgz_sport.clanovi
FOR EACH ROW EXECUTE FUNCTION pgz_sport.clanovi_normalize_fn();
COMMIT;
-- END
_audit/data_integrity_20260505_0836/D_violations.md
+126
View File
@@ -0,0 +1,126 @@
# Subagent D — Skipped constraints, violator samples
Two candidate constraints were SKIPPED at apply-time because pre-existing rows
would have been rejected. They are documented here so Damir can decide whether
to clean the data and re-attempt the constraint, or accept the current state.
The trigger `clanovi_normalize_trigger` already enforces both rules **for new
inserts and for name-changing updates**, so future data ingest cannot
reintroduce these patterns. Only retroactive enforcement on existing rows is
deferred.
---
## C3 — `CHECK (length(ime)>=2 AND length(prezime)>=2)` — SKIPPED
Violator count: **22** rows.
Two clusters:
1. **Single-letter `prezime`** — `id=1160` and `id=1165`, both klub_id=848:
- `('Boris Mičetić','B')` — note the embedded space in `ime`; the surname appears truncated to a single initial.
- `('Boris Mičetić','J')` — same pattern.
- **Decision suggestion**: probably real-name parse errors. Resolve manually in `clanovi`.
2. **Placeholder `ime='-'` (single dash)** — 20 rows, klub_id mostly NULL plus one with klub_id=3896:
| id | klub_id | ime | prezime |
|----|---------|-----|---------|
| 1852 | NULL | - | Grabovac |
| 1853 | NULL | - | Pilepić |
| 1854 | NULL | - | Maslak |
| 1855 | NULL | - | Jugo |
| 1856 | NULL | - | Miličević |
| 1857 | NULL | - | Marjanović |
| 1858 | NULL | - | Poljak |
| 1859 | NULL | - | Kurelić |
| 2021 | 3896 | - | Mohorić |
| 2125 | NULL | - | Mittrovich (braća) |
| 2130 | NULL | - | Loich |
| 2131 | NULL | - | Paulinich |
| 2132 | NULL | - | Zidarich |
| 2133 | NULL | - | Bertok |
| 2134 | NULL | - | Marincich |
| 2135 | NULL | - | Tiblias |
| 2138 | NULL | - | Veselica |
| 2139 | NULL | - | Naumović |
| 2140 | NULL | - | Osojnak |
| 2141 | NULL | - | Medle |
These look like **historical / surname-only roster entries** (note `napomena`
on id=1852 mentions "POVIJESNI: KK Kvarner najtrofejnija generacija …" so the
cluster is intentional historical data with unknown given name).
**Decision suggestion**: replace `ime='-'` with `ime='?'` is also rejected;
either backfill the given names from a source, mark them inactive/historical
in another column, or accept the data and never enable C3.
---
## C6 — `UNIQUE (klub_id, lower(ime), lower(prezime), COALESCE(datum_rodenja,'0001-01-01'))` — SKIPPED
Conflict groups: **68** (each group has 2+ rows that would collide).
Most are concentrated on **klub_id=2362 (HNK Rijeka roster)** where the same
player appears twice — once with `datum_rodenja IS NULL` and once also with
NULL DOB but a different scrape source / older `id`. Sample:
| klub_id | l_ime | l_prez | dob | dups | ids |
|---------|-------|--------|-----|------|-----|
| 2362 | amer | gojak | NULL | 2 | {3402, 4214} |
| 2362 | leon | šerifi | NULL | 2 | {3334, 4238} |
| 2362 | ante | oreč | NULL | 2 | {1581, 4230} |
| 2362 | branko | pavić | NULL | 2 | {2715, 4231} |
| 2362 | lovro | kitin | NULL | 2 | {3481, 4220} |
| 2362 | ante | majstorović | NULL | 2 | {3456, 4224} |
| 2362 | dejan | petrovič | NULL | 2 | {3399, 4232} |
| 2362 | duje | čop | NULL | 2 | {1579, 4211} |
| 2362 | fran | škalamera | NULL | 2 | {3480, 4239} |
| 2362 | gabriel | rukavina | NULL | 2 | {3404, 4234} |
| 2362 | bruno | bogojević | NULL | 2 | {3437, 4208} |
| 2362 | aleksa | todorović | NULL | 2 | {3455, 4202} |
| 2362 | cherno | saho | NULL | 2 | {3403, 4235} |
| 2362 | luka | menalo | NULL | 2 | {3454, 4226} |
| 2362 | martin | zlomislić | NULL | 2 | {3440, 4203} |
| 2362 | mladen | devetak | NULL | 2 | {3400, 4212} |
| 2362 | niko | janković | NULL | 2 | {3607, 4218} |
| 2362 | noel | bodetić | NULL | 2 | {3705, 4207} |
| 2362 | silvio | ilinković | NULL | 2 | {3412, 4217} |
| 2362 | šimun | butić | NULL | 2 | {3401, 4209} |
| 2362 | stjepan | radeljić | NULL | 2 | {3448, 4233} |
| 2362 | toni | fruk | 2001-03-09 | 2 | {3438, 4135} |
| 2362 | vito | kovač | NULL | 2 | {3298, 4201} |
| 2362 | jovan | manev | NULL | 2 | {3439, 4225} |
| 2585 | ivo | butrica | NULL | 2 | {2282, 4163} |
| 2585 | luko | ledinić | NULL | 2 | {2283, 4164} |
| 2586 | siniša | saftić | NULL | 2 | {2298, 4165} |
| 2587 | damir | poslek | NULL | 2 | {2310, 4167} |
| 2589 | matej | viduka | NULL | 2 | {2340, 4174} |
| 2589 | čedo | vukelić | NULL | 2 | {2339, 4175} |
(38 more groups not shown — query reproduction below.)
**Cause**: the dedup-fold key collapses on `COALESCE(NULL, '0001-01-01')`, so
two records of the same name+klub with missing DOB look identical even when
they are distinct profiles (different `profile_url`, `source_id`, `hns_igrac_id`).
Today's working composite key is the existing `uq_clanovi_klub_profile
(klub_id, profile_url)` which is already enforced.
**Decision suggestion**: do NOT enable C6 as-is. Either (a) restrict the
uniqueness to `WHERE datum_rodenja IS NOT NULL`, or (b) merge true dupes via a
follow-up subagent that promotes one row and back-fills `hns_igrac_id` /
`profile_url`. Until then, ingestion is still protected by
`uq_clanovi_klub_profile` and (for HNS-keyed players) `clanovi_hns_uniq`.
### Reproduce full list
```sql
SELECT klub_id, lower(ime) AS l_ime, lower(prezime) AS l_prez,
COALESCE(datum_rodenja, '0001-01-01'::date) AS dob,
count(*) AS dups,
array_agg(id ORDER BY id) AS ids
FROM pgz_sport.clanovi
GROUP BY klub_id, lower(ime), lower(prezime), COALESCE(datum_rodenja, '0001-01-01'::date)
HAVING count(*) > 1
ORDER BY dups DESC, klub_id;
```
pgz_sport_api.py
+11 -2
View File
@@ -1,10 +1,19 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
pgz_sport_api.py - FastAPI backend za PGŽ Sportski savez ERP/CRM pgz_sport_api.py - FastAPI backend za PGŽ Sportski savez ERP/CRM
Author: Damir Radulić (damir@rinet.one) Author: Damir Radulić (damir@rinet.one / dradulic@outlook.com)
Date: 25.04.2026 Date: 25.04.2026 (v1.1.0 — 2026-05-05: role-based OIB display + audit log)
Port: 8095 Port: 8095
Endpoints: savezi, klubovi, članovi, članarine, liječnički, manifestacije, proračun, dashboard, alertovi Endpoints: savezi, klubovi, članovi, članarine, liječnički, manifestacije, proračun, dashboard, alertovi
Changes (2026-05-05, sub-agent W5):
* is_admin() — recognizes super_admin / pgz_admin / pgz_user / pgz_finance /
pgz_zzjz JWT roles (previous code only matched literal "admin", which broke
PII visibility for actual PGŽ admins like Damir).
* apply_privacy() — now scope-aware: savez_admin sees full PII for own savez,
klub_admin sees full PII for own klub.
* Added _audit_oib_access() — records full-OIB reveals to Postgres audit_events
(table pgz_sport.audit_events) under action='oib.read'. Legitimate-interest
audit trail for GDPR Art.6(1)(f) defensibility.
""" """
from fastapi import FastAPI, HTTPException, Query, Body, Header, Depends, UploadFile, File, Form, Request from fastapi import FastAPI, HTTPException, Query, Body, Header, Depends, UploadFile, File, Form, Request
static/oib_format.js
+136
View File
@@ -0,0 +1,136 @@
/* oib_format.js — unified role-based OIB display
* Author: Damir Radulić (damir@rinet.one / dradulic@outlook.com)
* Date: 2026-05-05
* Description: Single source of truth for OIB rendering across all PGŽ Sport
* static pages. Role hierarchy (per pgz_sport.users.user_type):
* super_admin -> full OIB everywhere
* pgz_admin -> full OIB across PGŽ tenant
* savez_admin -> full OIB for own savez_id (context-aware)
* klub_admin -> full OIB for own klub_id (context-aware)
* others -> masked: first 3 + 6 dots + last 2 (e.g. 067••••••03)
* Usage:
* <script src="/sport/static/oib_format.js"></script>
* formatOib('12345678901') // role auto-detected from pgz_user
* formatOib(o.oib, {savez_id: 1, klub_id: 7}) // pass scope for context-aware
* maskOib('12345678901') // force masked
*/
(function (g) {
'use strict';
var TOKEN_KEYS = ['pgz_access', 'jwt', 'access_token'];
var USER_KEYS = ['pgz_user'];
// PGŽ-tier: always sees full PII for everything
var FULL_VISIBILITY_ROLES = {
'super_admin': 1,
'pgz_admin': 1,
'pgz_user': 1,
'pgz_finance': 1,
'pgz_zzjz': 1,
'admin': 1 // legacy bearer token role
};
// Scope-restricted roles
var SAVEZ_ROLES = { 'savez_admin': 1, 'savez_user': 1 };
var KLUB_ROLES = { 'klub_admin': 1, 'klub_user': 1, 'klub_trener': 1, 'klub_clan': 1 };
function _readUser() {
for (var i = 0; i < USER_KEYS.length; i++) {
var k = USER_KEYS[i];
try {
var raw = localStorage.getItem(k) || sessionStorage.getItem(k);
if (raw) return JSON.parse(raw);
} catch (e) { /* ignore */ }
}
return null;
}
function _readToken() {
for (var i = 0; i < TOKEN_KEYS.length; i++) {
var k = TOKEN_KEYS[i];
var t = localStorage.getItem(k) || sessionStorage.getItem(k);
if (t) return t;
}
return null;
}
function _decodeJwt(tok) {
if (!tok || tok.split('.').length !== 3) return null;
try {
var p = tok.split('.')[1].replace(/-/g, '+').replace(/_/g, '/');
while (p.length % 4) p += '=';
return JSON.parse(atob(p));
} catch (e) { return null; }
}
/** Returns {role, klub_id, savez_id, email} from JWT or stored user. */
function getUserCtx() {
var u = _readUser() || {};
var jwt = _decodeJwt(_readToken()) || {};
var role = u.user_type || u.role || jwt.role || jwt.user_type || 'viewer';
var klub_id = u.klub_id != null ? u.klub_id : (jwt.tenant_scope && jwt.tenant_scope.klub_id);
var savez_id = u.savez_id != null ? u.savez_id : (jwt.tenant_scope && jwt.tenant_scope.savez_id);
return {
role: String(role || 'viewer').toLowerCase(),
klub_id: klub_id == null ? null : Number(klub_id),
savez_id: savez_id == null ? null : Number(savez_id),
email: u.email || jwt.email || null
};
}
/** Force-masked rendering: first 3 + 6 dots + last 2. e.g. 067••••••03 */
function maskOib(oib) {
if (oib == null) return '—';
var s = String(oib);
if (s.length < 6) return '•'.repeat(s.length);
return s.slice(0, 3) + '••••••' + s.slice(-2);
}
/**
* Decide whether to show full OIB based on caller role and (optional) row scope.
* @param {string|number} oib raw OIB value (or already-masked from backend)
* @param {object} [scope] optional context: {klub_id, savez_id}
* if provided, savez_admin / klub_admin can see
* full OIB for rows in their own scope only
* @returns {string} formatted OIB
*/
function formatOib(oib, scope) {
if (oib == null || oib === '') return '—';
var s = String(oib);
// Backend already masked — pass through (we cannot un-mask client-side)
if (s.indexOf('•') !== -1 || s.indexOf('*') !== -1) return s;
var ctx = getUserCtx();
var r = ctx.role;
if (FULL_VISIBILITY_ROLES[r]) return s;
if (scope && typeof scope === 'object') {
if (SAVEZ_ROLES[r] && ctx.savez_id != null && scope.savez_id != null
&& Number(scope.savez_id) === ctx.savez_id) return s;
if (KLUB_ROLES[r] && ctx.klub_id != null && scope.klub_id != null
&& Number(scope.klub_id) === ctx.klub_id) return s;
} else {
// No scope passed — savez/klub admins default to full only when scope not given
// (they would only ever query their own scope via tenanted endpoints)
if (SAVEZ_ROLES[r] || KLUB_ROLES[r]) return s;
}
return maskOib(s);
}
/** Returns true if the current user can see full PII (any OIB). */
function canSeeFullOib(scope) {
var ctx = getUserCtx();
if (FULL_VISIBILITY_ROLES[ctx.role]) return true;
if (!scope) return SAVEZ_ROLES[ctx.role] || KLUB_ROLES[ctx.role] || false;
if (SAVEZ_ROLES[ctx.role] && scope.savez_id != null && Number(scope.savez_id) === ctx.savez_id) return true;
if (KLUB_ROLES[ctx.role] && scope.klub_id != null && Number(scope.klub_id) === ctx.klub_id) return true;
return false;
}
g.formatOib = formatOib;
g.maskOib = maskOib;
g.getUserCtx = g.getUserCtx || getUserCtx;
g.canSeeFullOib = canSeeFullOib;
})(window);