Logs: own folders, a daily cap, masked paths; workers finish running work on a deploy
CI / test (push) Successful in 1m34s
CI / build (push) Successful in 10m9s
CI / deploy (push) Skipped

Review fixes for the persistent logs:
- each service writes its own folder (the old-file cleanup matches by
  prefix, so 'worker' deleted the 'worker-ai' files);
- at most LOG_MAX_DAY_MB (50) a day, and at least 2 days kept;
- written by a background thread, not on the async workers;
- e-mail addresses and tokens in request paths are masked;
- a failing health check is logged again, a passing one is not;
- one-off commands (compare-models) log to the console only.

On a stop the worker takes nothing new and lets running work finish
(SHUTDOWN_DRAIN_SECS, worker 40 s, worker-ai 270 s, with matching stop
grace periods), so a deploy no longer cuts an analysis off, wastes its
model call and delays it by the 15-minute stuck-job requeue.

The CI cleanup caps the build cache at 10 GB: by age it had grown to
33 GB on the runner and filled its disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012vLHzh5qUhFtmJhzkcTbXL
This commit is contained in:
feco
2026-11-20 18:57:09 +00:00
co-authored by Claude Opus 5.5
parent ffb6060cfa
commit 16c40864bb
14 changed files with 315 additions and 56 deletions
+4 -9
View File
@@ -9,8 +9,7 @@ use axum::{Json, Router};
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use tower_http::cors::{AllowOrigin, Any, CorsLayer};
use tower_http::trace::{DefaultMakeSpan, DefaultOnResponse, TraceLayer};
use tracing::Level;
use tower_http::trace::TraceLayer;
use crate::AppState;
use crate::error::ApiError;
@@ -22,6 +21,7 @@ type AppResult<T> = Result<Json<T>, ApiError>;
pub fn router(state: Arc<AppState>) -> Router {
Router::new()
.route("/healthz", get(health))
.route("/v1/info", get(info))
.route("/v1/countries", get(countries))
.route("/v1/places", get(places))
@@ -30,13 +30,8 @@ pub fn router(state: Arc<AppState>) -> Router {
.route("/v1/synastry", post(synastry_chart))
.route("/v1/forecast", post(forecast_year))
.layer(cors())
.layer(
TraceLayer::new_for_http()
.make_span_with(DefaultMakeSpan::new().level(Level::INFO))
.on_response(DefaultOnResponse::new().level(Level::INFO)),
)
// After the layers: the Docker health check (every 30 s) stays out of the log.
.route("/healthz", get(health))
// The request log: masked paths, health checks only when they fail.
.layer(TraceLayer::new_for_http().make_span_with(service_kit::RequestLog).on_response(service_kit::RequestLog))
.with_state(state)
}