Commit Graph

13 Commits

Author SHA1 Message Date
damir c68fd4471e HNS endpoints: /clan/{id}/hns-career + /klubovi/pgz-financirani + /dashboard/hns-coverage
Backed by: pgz_sport.hns_player_seasons, hns_klub_roster, v_pgz_financirani_klubovi
Used by: cc-hns subagents for UI integration
2026-05-05 10:22:36 +02:00
damir 63ca005b6e DEBUG OBSERVABILITY: live error feed + auto-triage bot + dashboard
PHASE 1 — DEBUG mode:
- /etc/systemd/system/pgz-sport.service.d/debug.conf: DEBUG=1, LOG_LEVEL=DEBUG, PYTHONUNBUFFERED=1, UVICORN_LOG_LEVEL=debug

PHASE 2 — Error stream:
- /opt/pgz-sport/scripts/debug_tail.sh: tail journalctl + nginx → /var/log/pgz-sport-debug/{stream,errors}.jsonl
- pgz-debug-tail.service (always restart, multiplexes 4 sources)

PHASE 3 — Auto-triage bot:
- /opt/pgz-sport/scripts/auto_triage.py: classifies errors, dispatches CC agents
- Patterns: 5xx spike → CC4, 401/403 spike → CC2, 4xx API → CC3, ImportError/DB → CC4
- Rate limit: 6 telegram/5min
- Records decisions in triage_decisions.jsonl
- pgz-auto-triage.service

PHASE 4 — Live dashboard:
- routers/debug_router.py mounted in pgz_sport_api
- GET /api/debug/health — services + DB + error count
- GET /api/debug/errors?limit=N — last N errors (JSON)
- GET /api/debug/decisions — auto-fix decisions
- GET /api/debug/stream — full log tail
- GET /api/debug/dashboard — live HTML refresh 5s

Damir admin tier dashboard: https://sport.rinet.one/sport/api/debug/dashboard
2026-05-05 08:46:09 +02:00
damir c38f15a566 R7+: 5x P0 demo fixes — HNS direct link, avatar cache, logo home, klub→sportaši, smarter enrichment
1) HNS direct link u research_links: za sportaš s profile_url/source_url
   (npr. https://semafor.hns.family/igraci/X/...) generira [DIRECT] link na vrhu liste,
   umjesto generic Google search. _research_links sada prima row dict.

2) Avatar cache buster: applyMeToHeader dodaje ?t=Date.now() na sve avatar img tagove.
   Avatar upload handler dodatno persistira novi avatar_url u localStorage.pgz_user
   tako da preživi page refresh + cross-page navigacije.

3) Logo home link: <div class='logo'> → <a href='/' class='logo'> u app.html i sport2.html.
   Klik na PGŽ SPORT logo vodi na public portal.

4) Klub → Sportaši drill-down: u klub Info tabu dodan button
   '👥 Vidi sportaše ovog kluba (N)' koji prebacuje na k-clan tab.
   Plus '🌐 Službena stranica' link kad klub ima web.

5) Smarter klub enrichment:
   - URL validacija (skip placeholder strings poput 'godisnjak_zspgz_2025')
   - Domain candidate guesser (slug → 16 candidate URLs s common HR TLD-ovima i sport prefix-ima)
   - Parallel HEAD probe (8 threads, 10s budget) — first 200 + name token match wins
   - Subpage scrape (/kontakt, /uprava, /o-nama, /o-klubu, /predsjednik) za richer evidence
   - HNK Orijent (id 3766) test: pogađa https://www.orijent.hr/, predlaže web+email+telefon+opis

E2E verified:
- 9/9 sidebar URL-ova → 200
- /users/me/gdpr-export → 200 (28KB JSON)
- /users/me/request-deletion → 200 (DB row pgz_sport.gdpr_erasure_requests)
- /enrich/klub/3766 → 4 proposed fields (web, email, telefon, opis)
- HNS sportaš research_links:  HNS profil DIRECT link na vrhu

Backend: routers/enrich_router.py
Frontend: static/app.html, static/sport2.html
Backups: _backups/sprint_1777940670/

Tag: R7-demo-ready
2026-05-05 02:24:30 +02:00
Damir Radulić f9ebcddf28 CC2 R6: middleware-wide JWT, avatar demo mode, mock mailer, login rate limit
#1 JWT middleware extended:
- Was: /api/admin/* only
- Now: any POST/PUT/PATCH/DELETE under /api/* requires Bearer JWT
- Whitelist (still anonymous): /api/auth/login, /refresh, /forgot-password,
  /password/reset, /reset-password, /setup-password, /google;
  /api/gdpr/consent; any path ending /avatar
- 14 mutating endpoints verified to return 401 without token

#2 Avatar upload demo mode (routers/clan_panel_router.py):
- Anonymous → returns {demo_mode:true, slika_url:null,
  message:'Demo mode — slika nije spremljena. Prijavite se za pravu pohranu.'},
  no FS write, no DB write
- Authenticated (valid JWT, allowed role) → real save as before
- Auth check now uses auth.auth_v2.decode_token (proper secret + revocation)
  instead of the broken local _resolve_role

#3 Mock mailer (auth/mailer.py):
- send_email writes RFC 822 .eml to /tmp/pgz_mailbox + appends to INDEX.jsonl
- send_password_reset, send_invite helpers with HR text + HTML alt
- Real SMTP active when PGZ_SMTP_HOST is set (env-driven, off by default)
- forgot-password and admin invite both call mailer; audit logs mail status

#5 Rate limiting on /api/auth/login:
- Per-user: 5 wrong attempts → 5-minute DB-backed lockout
  (was 5 → 15 min). Configurable via PGZ_LOGIN_LOCK_THRESHOLD/MINUTES.
- Per-IP: 10 fails / 5-min sliding window in-memory → HTTP 429
  Configurable via PGZ_LOGIN_IP_THRESHOLD/WINDOW_SEC. Successful
  login clears the IP counter.
- Failed attempts respond '(N/5) — račun je zaključan na 5 minuta'
- New audit actions: login.ratelimit.ip; login.fail meta now
  includes fails count, locked, lock_minutes

#4 Live test report: 46/46 across 6 demo users — login, JWT gate on 14
   mutating endpoints, public path whitelist, demo-mode avatar +
   real save, forgot-password e-mail to mailbox, no-leak unknown email,
   5-fail lockout, 423 during lockout, audit coverage.
2026-05-05 01:42:53 +02:00
Damir Radulić 0046b8d695 CC2 R5: defense-in-depth JWT + invite/reset token flows + audit
#1 JWT middleware:
- pgz_sport_api.py: starlette middleware require_jwt_on_admin runs before
  every /api/admin/* route. Even routes that lack Depends(require_user)
  cannot be reached without a valid Bearer token (verifies signature,
  exp, typ='access', revocation via user_sessions). OPTIONS passes for CORS.

#2 Invitation flow:
- pgz_sport.user_action_tokens table (token_hash, user_id, kind, expires_at,
  used_at, created_by, ip, meta). Single-use, raw token never persisted.
- POST /api/admin/users/{id}/invite — issues 'invite' token (TTL 7d),
  marks must_change_pwd, revokes existing sessions, returns invite_link.
- GET  /api/auth/setup-password?token=X — preflight (no consume).
- POST /api/auth/setup-password — consumes token, sets password, sets
  email_verified=true.

#3 Password reset flow:
- POST /api/auth/forgot-password — generic 'ako račun postoji' response;
  issues 'reset' token (TTL 2h) only for active users. Token returned in
  response only on localhost or if PGZ_REVEAL_RESET_TOKEN=1.
- GET  /api/auth/reset-password?token=X — preflight.
- POST /api/auth/reset-password — consumes token, sets new password,
  revokes all active sessions.

#4 Audit coverage (auth events):
- login.ok, login.fail (with reason), login.locked, login.2fa_required,
  login.2fa_fail, logout, auth.refresh, password.change, password.reset.ok,
  password.reset.fail, password.forgot.issue, password.forgot.miss,
  invite.consume.ok, invite.consume.fail, user.invite, user.create,
  user.update, user.delete, user.role.change, user.suspend, user.unsuspend,
  user.password.reset, 2fa.verify.ok, 2fa.verify.fail, 2fa.disable.

#5 Live tests: 41/41 across 6 demo users (incl. fresh invited+deleted user).
   Phase 2 verifies 14 endpoints reject no-auth and accept valid Bearer.
2026-05-05 01:28:29 +02:00
Damir Radulić bd3773434e CC2 R4 #6: real TOTP 2FA (setup + verify + disable + login flow)
- auth/auth_v2.py:
  - pyotp-based TOTP (RFC 6238, base32 secret, ±30s window)
  - new pgz_sport.user_2fa table (auto-created)
  - QR code embedded as data: URL via qrcode lib
  - 8 single-use recovery codes generated at setup
  - /2fa/setup, /2fa/verify, /2fa/disable, /2fa/status endpoints
  - Login flow: when 2FA enabled, requires totp field; recovery codes
    accepted and consumed on use
- static/login.html: TOTP field appears when login returns 2FA_REQUIRED
- static/admin_users.html: full 2FA panel in Sigurnost tab
  (status badge, QR + secret + recovery code display, verify input)

Live tests pass:
  T1 status (no setup) → enabled:false
  T2 setup → secret + 1.5KB QR PNG + 8 recovery codes
  T3 verify wrong code → 401
  T4 verify real TOTP → enabled:true
  T5 login w/o TOTP after enable → 401 detail=2FA_REQUIRED
  T6 login w/ TOTP → 200
2026-05-05 00:50:28 +02:00
Damir Radulić a0db65fc31 CC2 R4 #4: /api/users/me/gdpr-export alias
- New auth.gdpr.me_router prefix /api/users/me with:
  - GET/POST /gdpr-export → Art.20 JSON download with Content-Disposition
  - POST /gdpr-erase → Art.17 erasure request
  - GET /gdpr-consent → consent history for caller
- jsonable_encoder fixes datetime serialisation in JSONResponse
- admin_users.html: 'Izvezi moje podatke' now POSTs to alias and uses
  filename from Content-Disposition header
- 401 enforced on no-auth, 200 on valid Bearer (verified live)
2026-05-05 00:47:22 +02:00
claude-cc1 ca92717039 CC1 R4-A3 — wire audit_log() into enrich /apply + helper available to all routers
- enrich_apply now imports audit_seal_router.audit_log and writes a sys_audit
  row after every successful UPDATE: action='enrich.apply', target_type=kind,
  target_id=eid, payload={applied: {...}, sources: [...]}, user from headers.
- Other modules (cc2 users, cc4 invoices/putni_nalozi, cc5 clanarine/lijecnicki/
  obrasci) can call the same helper:
      from audit_seal_router import audit_log
      audit_log(action='users.update', target_type='users', target_id=u['id'],
                payload={'changed':[...]}, user_email=actor)
- Verified: real apply on klub 4528 produced sys_audit id 102.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 00:46:41 +02:00
CC6 Worker ece556de11 M12.4: real HNS Semafor scraper for sportas + 24/7 enrichment worker
Critical bug fix: /v2/enrich/sportas/{id} returned proposed:{} for athletes
because the v3 pipeline was still relying on Wikipedia-only evidence and never
actually fetched semafor.hns.family/igraci/.

- enrich_router._propose_for_sportas now:
  • Resolves a HNS Semafor URL from profile_url, source_url, hns_igrac_id,
    vanjski_id JSONB ('hns_comet'+'hns_slug'), or source='hns_semafor'+source_id.
  • Fetches and parses the player page (BS4, regex fallback) and proposes
    profile_url, source_url, slika_url, hns_igrac_id, datum_rodenja,
    mjesto_rodenja, broj_dresa, biografija (DeepSeek synthesis from HNS+Wiki).
- _load_row(sportas) widened to read every relevant column + vanjski_id.
- _TABLE_MAP['sportas'] writeback whitelist expanded to 12 fields.
- workers/enrichment_worker.py: 24/7 daemon, picks under-enriched
  clanovi/klubovi/savezi every 5 min via SQL, calls /apply for each.
- systemd unit pgz-sport-enricher.service installed + enabled.
- Tested end-to-end: id=2222 (Abdija) and id=449 (Zec) now have
  profile_url, slika_url, hns_igrac_id, biografija persisted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 00:36:57 +02:00
CC6 Worker 84f1c41008 M12.3: Playwright fallback scraper for JS-heavy federation sites
- enrichment/playwright_scraper.py: fetch_rendered(), scrape_sport_pgz_klub(),
  scrape_federation(). Headless Chromium, 12s timeout, returns rendered text.
  Import-safe when playwright is missing.
- enrich_router._sport_pgz_search() now falls back to the JS path when the
  cheap urllib fetch returns empty or unparseable HTML.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 00:23:00 +02:00
CC6 Worker 85fd51bfd9 M12.1: enrich v3 — preview + /apply persists to DB (klubovi/savezi/clanovi)
- POST /v2/enrich/{kind}/{eid} now scrapes Wikipedia HR + sport-pgz.hr +
  primary site, runs relevance filter so contact info from off-topic pages
  isn't lifted, optionally calls DeepSeek for opis_djelatnosti, returns
  {current, proposed, sources, last_enriched_at} for diff UI.
- POST /v2/enrich/{kind}/{eid}/apply UPDATES klubovi/savezi/clanovi for
  whitelisted empty fields, sets metadata.enriched_at +
  metadata.enrichment_source + metadata.enrichment_history, writes a row
  to pgz_sport.enrichment_log (new table).
- GET /v2/enrich/log read-back endpoint.
- Tested on klub 3 (KK Kvarner 2010): opis_djelatnosti persisted; metadata
  carries enriched_at + sources.
- New tables/columns: pgz_sport.enrichment_log; metadata jsonb on klubovi/savezi.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 00:14:17 +02:00
Damir Radulić 492c8fdd87 M1+M2+M10 (CC2 R3): JWT auth + admin users + GDPR backend
- auth/auth_v2.py: JWT login/refresh/logout/me + bcrypt + tenant_id/role/tier claims
- auth/admin_users.py: /api/admin/users CRUD + invite/role/suspend + bulk CSV
- auth/gdpr.py: cookie consent + Art.20 export + Art.17 erasure + admin queue
- auth/seed_demo.py: 3 demo tenants + 4 users (damir@pgz.hr / PGZ2026!)
- Removed legacy /api/auth/login + /api/auth/me from pgz_sport_api.py
- Wired auth/admin/gdpr routers into FastAPI

5/5 live curl tests pass: damir@pgz.hr login → JWT with tenant_id=1, role=pgz_admin, tier=0
2026-05-05 00:09:09 +02:00
claude-cc1 b7cb050843 CC1 R2 — full Round 2 done (8/8 stavki)
- geocode_objekti_v2.py + DB updates (Kastav, Rujevica, Platak, Petehovac, Crikvenica, Krk hand-curated)
- Maps URL → /maps/search/?api=1 format for proper pin
- Dashboard: year selector for nositelji, click → klub/PDF panel; top savezi clickable
- Universal sort (asc/desc) on Savezi/Klubovi/Sportaši/Objekti/Manifestacije/Financije
- Card↔Table toggle on Financije
- Manifestacije: source_url direct open, Google fallback
- Forenzika: severity/tip filter, search, run-scan, Liverić PEP custom findings + DB alerts
- Enrich endpoint /api/v2/enrich/{kind}/{id} + button on savez/klub/sportaš panels
- New 'Mreža' section: D3 force graph from /api/v1/presenter/graph-real

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 23:50:30 +02:00