Logs: own folders, a daily cap, masked paths; workers finish running work on a deploy
CI / test (push) Successful in 1m34s
CI / build (push) Successful in 10m9s
CI / deploy (push) Skipped

Review fixes for the persistent logs:
- each service writes its own folder (the old-file cleanup matches by
  prefix, so 'worker' deleted the 'worker-ai' files);
- at most LOG_MAX_DAY_MB (50) a day, and at least 2 days kept;
- written by a background thread, not on the async workers;
- e-mail addresses and tokens in request paths are masked;
- a failing health check is logged again, a passing one is not;
- one-off commands (compare-models) log to the console only.

On a stop the worker takes nothing new and lets running work finish
(SHUTDOWN_DRAIN_SECS, worker 40 s, worker-ai 270 s, with matching stop
grace periods), so a deploy no longer cuts an analysis off, wastes its
model call and delays it by the 15-minute stuck-job requeue.

The CI cleanup caps the build cache at 10 GB: by age it had grown to
33 GB on the runner and filled its disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012vLHzh5qUhFtmJhzkcTbXL
This commit is contained in:
feco
2026-11-20 18:57:09 +00:00
co-authored by Claude Opus 5.5
parent ffb6060cfa
commit 16c40864bb
14 changed files with 315 additions and 56 deletions
+7
View File
@@ -134,6 +134,8 @@ services:
MAIL_FROM: ${MAIL_FROM:-Sorsfonalak <hello@sorsfonalak.hu>}
PUBLIC_BASE_URL: https://${DOMAIN}
LOG_DIR: /logs
# On a deploy the e-mails and invoices being sent finish first.
SHUTDOWN_DRAIN_SECS: 40
env_file:
- path: .env.secrets
required: false
@@ -143,6 +145,7 @@ services:
read_only: true
volumes:
- logs:/logs
stop_grace_period: 1m
logging: *logging
restart: unless-stopped
@@ -158,6 +161,9 @@ services:
PUBLIC_BASE_URL: https://${DOMAIN}
LOG_DIR: /logs
LOG_NAME: worker-ai
# On a deploy the analyses being written finish first (a detailed one
# takes a few minutes); what is not done by then goes back to the queue.
SHUTDOWN_DRAIN_SECS: 270
env_file:
- path: .env.secrets
required: false
@@ -167,6 +173,7 @@ services:
read_only: true
volumes:
- logs:/logs
stop_grace_period: 5m
logging: *logging
restart: unless-stopped