Add worker service with e-mail delivery and dev Mailpit
- worker: sends email_outbox mail over SMTP (lettre), woken by LISTEN/NOTIFY, SKIP LOCKED claiming, retries with backoff, hu/en templates (text + HTML), hourly cleanup of expired data - migration 0002: next_attempt_at, failed_at, notify trigger - docker-compose.override.yml: Mailpit for development only - Dockerfiles: lock shared cargo cache mounts for parallel builds Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012vLHzh5qUhFtmJhzkcTbXL
This commit is contained in:
@@ -0,0 +1,20 @@
|
||||
[package]
|
||||
name = "worker"
|
||||
description = "Sorsfonalak background jobs: e-mail delivery, cleanup (later: LLM analyses)"
|
||||
edition.workspace = true
|
||||
version.workspace = true
|
||||
license.workspace = true
|
||||
publish.workspace = true
|
||||
|
||||
[dependencies]
|
||||
db.workspace = true
|
||||
service-kit.workspace = true
|
||||
anyhow.workspace = true
|
||||
axum.workspace = true
|
||||
lettre.workspace = true
|
||||
serde_json.workspace = true
|
||||
sqlx.workspace = true
|
||||
time.workspace = true
|
||||
tokio.workspace = true
|
||||
tracing.workspace = true
|
||||
uuid.workspace = true
|
||||
@@ -0,0 +1,30 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
#
|
||||
# Build from the repository root:
|
||||
# docker build -f services/worker/Dockerfile -t sorsfonalak/worker .
|
||||
|
||||
ARG RUST_VERSION=1.99
|
||||
|
||||
# --- 1. static (musl) build ----------------------------------------------------
|
||||
FROM rust:${RUST_VERSION}-alpine AS build
|
||||
RUN apk add --no-cache musl-dev
|
||||
WORKDIR /app
|
||||
COPY Cargo.toml Cargo.lock ./
|
||||
COPY crates crates
|
||||
COPY services services
|
||||
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
|
||||
--mount=type=cache,target=/app/target,sharing=locked \
|
||||
cargo build --release --locked -p worker \
|
||||
&& cp target/release/worker /worker
|
||||
|
||||
# --- 2. runtime: the binary only (TLS roots are compiled in via webpki-roots) ---
|
||||
FROM scratch
|
||||
COPY --from=build /worker /worker
|
||||
ENV BIND_ADDR=0.0.0.0:8082 \
|
||||
LOG_FORMAT=json \
|
||||
RUST_LOG=info
|
||||
USER 65532:65532
|
||||
EXPOSE 8082
|
||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||
CMD ["/worker", "healthcheck"]
|
||||
ENTRYPOINT ["/worker"]
|
||||
@@ -0,0 +1,35 @@
|
||||
# worker
|
||||
|
||||
Háttérmunkák a közös PostgreSQL-adatbázis fölött. Nincs nyilvános API-ja, csak egy `/healthz` végpontja (8082) a Docker healthcheckhez. Induláskor lefuttatja a migrációkat, ugyanúgy, mint a backend.
|
||||
|
||||
## Munkák
|
||||
|
||||
**E-mail-küldés** (`email_outbox` → SMTP)
|
||||
- A backend a levelet az `email_outbox` táblába teszi. Egy adatbázis-trigger (`NOTIFY email_outbox`) azonnal felébreszti a workert, így a levél másodperceken belül kimegy. Ha az értesítés elveszne, a worker `POLL_INTERVAL_SECS` másodpercenként úgyis ránéz.
|
||||
- A sorokat `FOR UPDATE SKIP LOCKED` zárolással veszi fel, ezért több worker is futhat egyszerre, és nem küldik el kétszer ugyanazt. A kézbesítés „legalább egyszer” elvű: ha a worker éppen küldés közben áll le, a levél újra kimehet.
|
||||
- Átmeneti hibánál (hálózat, SMTP 4xx) újrapróbálja 1, 2, 4, … percenként, legfeljebb 6 óránként. `MAX_EMAIL_ATTEMPTS` próbálkozás után a levelet sikertelennek jelöli (`failed_at`, `last_error`).
|
||||
- Végleges hibánál nem próbálja újra: ilyen a hibás cím, az SMTP 5xx és az ismeretlen sablon.
|
||||
- Sablonok magyarul és angolul, a felhasználó nyelve szerint (`src/templates.rs`): `verify_email`, `reset_password`, `invite_accepted`. Mindegyik szöveges és HTML-változatban megy ki; a felhasználó nevét HTML-ben escape-eli, és a vezérlőkaraktereket eltávolítja belőle.
|
||||
- Ha nincs `SMTP_URL`, a worker nem küld semmit, a levelek az outboxban várnak.
|
||||
|
||||
**Takarítás** (induláskor, majd óránként)
|
||||
- lejárt munkamenetek és e-mail-tokenek;
|
||||
- elküldött e-mailek 7 nap után (tokenes linkek vannak bennük), sikertelenek 30 nap után;
|
||||
- el nem fogadott meghívók 30 nappal a lejáratuk után.
|
||||
|
||||
**Később:** az elemzések megírása (OpenRouter), kredit-visszatérítés hibánál.
|
||||
|
||||
## Konfiguráció
|
||||
|
||||
| Változó | Alapérték | Jelentés |
|
||||
|---|---|---|
|
||||
| `DATABASE_URL` | – (kötelező) | PostgreSQL kapcsolat |
|
||||
| `SMTP_URL` | – | `smtp://mailpit:1025` (fejlesztés), `smtp://user:pass@host:587?tls=required` (STARTTLS), `smtps://user:pass@host:465` (TLS) |
|
||||
| `MAIL_FROM` | `Sorsfonalak <noreply@sorsfonalak.local>` | feladó |
|
||||
| `POLL_INTERVAL_SECS` | `30` | tartalék lekérdezési időköz |
|
||||
| `MAX_EMAIL_ATTEMPTS` | `8` | ennyi próbálkozás után sikertelen a levél |
|
||||
| `BIND_ADDR`, `LOG_FORMAT`, `RUST_LOG` | `0.0.0.0:8082`, `text`, `info` | |
|
||||
|
||||
## Fejlesztés: Mailpit
|
||||
|
||||
A `docker-compose.override.yml` csak fejlesztéskor töltődik be. Elindítja a Mailpitet, és a workert rá irányítja. Minden levél ott landol, kifelé semmi nem megy ki. Webes felület: `http://<szerver>:8025`, API: `http://<szerver>:8025/api/v1/messages`.
|
||||
@@ -0,0 +1,47 @@
|
||||
//! Hourly removal of data that is no longer needed.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use sqlx::PgPool;
|
||||
use tokio::sync::watch;
|
||||
|
||||
const INTERVAL: Duration = Duration::from_secs(3600);
|
||||
|
||||
/// (what, statement) pairs; each returns the number of deleted rows.
|
||||
const STATEMENTS: [(&str, &str); 4] = [
|
||||
("sessions", "DELETE FROM sessions WHERE expires_at < now()"),
|
||||
("auth_tokens", "DELETE FROM auth_tokens WHERE expires_at < now()"),
|
||||
// Sent e-mails contain links with tokens; keep them a week for support.
|
||||
(
|
||||
"emails",
|
||||
"DELETE FROM email_outbox WHERE sent_at < now() - interval '7 days' \
|
||||
OR failed_at < now() - interval '30 days'",
|
||||
),
|
||||
("invites", "DELETE FROM synastry_invites WHERE accepted_at IS NULL AND expires_at < now() - interval '30 days'"),
|
||||
];
|
||||
|
||||
pub async fn run_once(db: &PgPool) -> Result<Vec<(&'static str, u64)>, sqlx::Error> {
|
||||
let mut deleted = Vec::new();
|
||||
for (what, sql) in STATEMENTS {
|
||||
deleted.push((what, sqlx::query(sql).execute(db).await?.rows_affected()));
|
||||
}
|
||||
Ok(deleted)
|
||||
}
|
||||
|
||||
pub async fn run(db: PgPool, stop: watch::Receiver<bool>) {
|
||||
loop {
|
||||
match run_once(&db).await {
|
||||
Ok(deleted) => {
|
||||
let total: u64 = deleted.iter().map(|(_, n)| n).sum();
|
||||
if total > 0 {
|
||||
tracing::info!(?deleted, "cleanup");
|
||||
}
|
||||
}
|
||||
Err(e) => tracing::error!(error = %e, "cleanup"),
|
||||
}
|
||||
tokio::select! {
|
||||
() = tokio::time::sleep(INTERVAL) => {}
|
||||
() = crate::stopped(stop.clone()) => break,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
//! Configuration from environment variables.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Context;
|
||||
use lettre::message::Mailbox;
|
||||
|
||||
pub struct Config {
|
||||
pub database_url: String,
|
||||
/// `smtp://host:port` (plain, e.g. Mailpit), `smtp://user:pass@host:587?tls=required`
|
||||
/// (STARTTLS) or `smtps://...` (TLS). Unset: e-mails stay in the outbox.
|
||||
pub smtp_url: Option<String>,
|
||||
/// Sender, e.g. `Sorsfonalak <noreply@sorsfonalak.hu>`.
|
||||
pub mail_from: Mailbox,
|
||||
/// Fallback polling interval; new e-mails are picked up at once via LISTEN/NOTIFY.
|
||||
pub poll_interval: Duration,
|
||||
/// After this many failed attempts an e-mail is marked as failed.
|
||||
pub max_email_attempts: i32,
|
||||
}
|
||||
|
||||
fn var(name: &str) -> Option<String> {
|
||||
std::env::var(name).ok().map(|v| v.trim().to_owned()).filter(|v| !v.is_empty())
|
||||
}
|
||||
|
||||
fn parsed<T: std::str::FromStr>(name: &str, default: T) -> anyhow::Result<T>
|
||||
where
|
||||
T::Err: std::fmt::Display,
|
||||
{
|
||||
match var(name) {
|
||||
Some(raw) => raw.parse().map_err(|e| anyhow::anyhow!("invalid {name} '{raw}': {e}")),
|
||||
None => Ok(default),
|
||||
}
|
||||
}
|
||||
|
||||
impl Config {
|
||||
pub fn from_env() -> anyhow::Result<Self> {
|
||||
let mail_from = var("MAIL_FROM").unwrap_or_else(|| "Sorsfonalak <noreply@sorsfonalak.local>".into());
|
||||
Ok(Self {
|
||||
database_url: var("DATABASE_URL").context("DATABASE_URL is required")?,
|
||||
smtp_url: var("SMTP_URL"),
|
||||
mail_from: mail_from.parse().with_context(|| format!("invalid MAIL_FROM '{mail_from}'"))?,
|
||||
poll_interval: Duration::from_secs(parsed("POLL_INTERVAL_SECS", 30)?),
|
||||
max_email_attempts: parsed("MAX_EMAIL_ATTEMPTS", 8)?,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
//! Delivers queued e-mails (`email_outbox`) over SMTP.
|
||||
//!
|
||||
//! New rows wake the job at once (`NOTIFY email_outbox`, see migration 0002);
|
||||
//! a slow poll picks up retries. Rows are claimed with `FOR UPDATE SKIP LOCKED`,
|
||||
//! so several workers can run side by side. Delivery is at-least-once.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use lettre::message::{Mailbox, MultiPart};
|
||||
use lettre::{AsyncSmtpTransport, AsyncTransport, Message, Tokio1Executor};
|
||||
use serde_json::Value;
|
||||
use sqlx::PgPool;
|
||||
use sqlx::postgres::PgListener;
|
||||
use tokio::sync::watch;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::config::Config;
|
||||
use crate::templates::{self, Rendered};
|
||||
|
||||
const BATCH: i64 = 10;
|
||||
const CHANNEL: &str = "email_outbox";
|
||||
|
||||
pub enum Mailer {
|
||||
Smtp(AsyncSmtpTransport<Tokio1Executor>),
|
||||
/// Collects messages instead of sending; fails for addresses listed in `fail`.
|
||||
#[cfg(test)]
|
||||
Memory {
|
||||
sent: std::sync::Mutex<Vec<(String, Rendered)>>,
|
||||
fail: Vec<(String, bool)>,
|
||||
},
|
||||
}
|
||||
|
||||
/// Why sending failed; permanent errors (bad address, SMTP 5xx) are not retried.
|
||||
#[derive(Debug)]
|
||||
pub struct SendError {
|
||||
pub permanent: bool,
|
||||
pub message: String,
|
||||
}
|
||||
|
||||
impl SendError {
|
||||
fn permanent(message: impl ToString) -> Self {
|
||||
Self { permanent: true, message: message.to_string() }
|
||||
}
|
||||
}
|
||||
|
||||
/// The host part of an SMTP URL, for logs (no credentials).
|
||||
pub fn smtp_host(url: &str) -> String {
|
||||
let rest = url.split_once("://").map_or(url, |(_, r)| r);
|
||||
let rest = rest.rsplit_once('@').map_or(rest, |(_, r)| r);
|
||||
rest.split(['/', '?']).next().unwrap_or_default().to_owned()
|
||||
}
|
||||
|
||||
impl Mailer {
|
||||
pub fn smtp(url: &str) -> anyhow::Result<Self> {
|
||||
let transport = AsyncSmtpTransport::<Tokio1Executor>::from_url(url)
|
||||
.map_err(|e| anyhow::anyhow!("invalid SMTP_URL for {}: {e}", smtp_host(url)))?
|
||||
.timeout(Some(Duration::from_secs(30)))
|
||||
.build();
|
||||
Ok(Mailer::Smtp(transport))
|
||||
}
|
||||
|
||||
async fn send(&self, from: &Mailbox, to: &str, mail: &Rendered) -> Result<(), SendError> {
|
||||
let to_mailbox: Mailbox = to.parse().map_err(SendError::permanent)?;
|
||||
let message = Message::builder()
|
||||
.from(from.clone())
|
||||
.to(to_mailbox)
|
||||
.subject(&mail.subject)
|
||||
.multipart(MultiPart::alternative_plain_html(mail.text.clone(), mail.html.clone()))
|
||||
.map_err(SendError::permanent)?;
|
||||
match self {
|
||||
Mailer::Smtp(transport) => transport
|
||||
.send(message)
|
||||
.await
|
||||
.map(drop)
|
||||
.map_err(|e| SendError { permanent: e.is_permanent(), message: e.to_string() }),
|
||||
#[cfg(test)]
|
||||
Mailer::Memory { sent, fail } => {
|
||||
if let Some((_, permanent)) = fail.iter().find(|(addr, _)| addr == to) {
|
||||
return Err(SendError { permanent: *permanent, message: "test failure".into() });
|
||||
}
|
||||
sent.lock().unwrap().push((to.to_owned(), mail.clone()));
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct Delivery {
|
||||
pub db: PgPool,
|
||||
pub mailer: Mailer,
|
||||
pub from: Mailbox,
|
||||
pub max_attempts: i32,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct OutboxRow {
|
||||
id: Uuid,
|
||||
to_email: String,
|
||||
template: String,
|
||||
locale: String,
|
||||
payload: Value,
|
||||
attempts: i32,
|
||||
}
|
||||
|
||||
/// Wait before the next attempt: 1, 2, 4, ... minutes, at most 6 hours.
|
||||
pub fn backoff(attempts: i32) -> Duration {
|
||||
let minutes = 1u64 << attempts.clamp(1, 10).saturating_sub(1);
|
||||
Duration::from_secs((minutes * 60).min(6 * 3600))
|
||||
}
|
||||
|
||||
impl Delivery {
|
||||
pub fn new(db: PgPool, mailer: Mailer, config: &Config) -> Self {
|
||||
Self { db, mailer, from: config.mail_from.clone(), max_attempts: config.max_email_attempts }
|
||||
}
|
||||
|
||||
/// Sends up to one batch of due e-mails; returns how many rows were handled.
|
||||
pub async fn deliver_due(&self) -> anyhow::Result<usize> {
|
||||
let mut tx = self.db.begin().await?;
|
||||
let rows: Vec<OutboxRow> = sqlx::query_as(
|
||||
"SELECT id, to_email, template, locale, payload, attempts FROM email_outbox \
|
||||
WHERE sent_at IS NULL AND failed_at IS NULL AND next_attempt_at <= now() \
|
||||
ORDER BY next_attempt_at LIMIT $1 FOR UPDATE SKIP LOCKED",
|
||||
)
|
||||
.bind(BATCH)
|
||||
.fetch_all(&mut *tx)
|
||||
.await?;
|
||||
|
||||
for row in &rows {
|
||||
let attempts = row.attempts + 1;
|
||||
let result = match templates::render(&row.template, &row.locale, &row.payload) {
|
||||
Ok(mail) => self.mailer.send(&self.from, &row.to_email, &mail).await,
|
||||
Err(e) => Err(SendError::permanent(e)),
|
||||
};
|
||||
match result {
|
||||
Ok(()) => {
|
||||
sqlx::query(
|
||||
"UPDATE email_outbox SET sent_at = now(), attempts = $2, last_error = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(row.id)
|
||||
.bind(attempts)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
tracing::info!(id = %row.id, template = %row.template, "e-mail sent");
|
||||
}
|
||||
Err(e) if e.permanent || attempts >= self.max_attempts => {
|
||||
sqlx::query(
|
||||
"UPDATE email_outbox SET failed_at = now(), attempts = $2, last_error = $3 WHERE id = $1",
|
||||
)
|
||||
.bind(row.id)
|
||||
.bind(attempts)
|
||||
.bind(&e.message)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
tracing::error!(id = %row.id, template = %row.template, attempts, error = %e.message, "e-mail failed");
|
||||
}
|
||||
Err(e) => {
|
||||
let wait = backoff(attempts);
|
||||
sqlx::query(
|
||||
"UPDATE email_outbox SET attempts = $2, last_error = $3, \
|
||||
next_attempt_at = now() + make_interval(secs => $4) WHERE id = $1",
|
||||
)
|
||||
.bind(row.id)
|
||||
.bind(attempts)
|
||||
.bind(&e.message)
|
||||
.bind(wait.as_secs_f64())
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
tracing::warn!(id = %row.id, attempts, retry_in_secs = wait.as_secs(), error = %e.message, "e-mail will be retried");
|
||||
}
|
||||
}
|
||||
}
|
||||
tx.commit().await?;
|
||||
Ok(rows.len())
|
||||
}
|
||||
}
|
||||
|
||||
async fn listen(db: &PgPool) -> Option<PgListener> {
|
||||
let result = async {
|
||||
let mut listener = PgListener::connect_with(db).await?;
|
||||
listener.listen(CHANNEL).await?;
|
||||
Ok::<_, sqlx::Error>(listener)
|
||||
}
|
||||
.await;
|
||||
result.map_err(|e| tracing::warn!(error = %e, "LISTEN failed; polling only")).ok()
|
||||
}
|
||||
|
||||
pub async fn run(delivery: Delivery, poll: Duration, stop: watch::Receiver<bool>) {
|
||||
let mut listener = listen(&delivery.db).await;
|
||||
loop {
|
||||
if *stop.borrow() {
|
||||
break;
|
||||
}
|
||||
match delivery.deliver_due().await {
|
||||
Ok(n) if n as i64 == BATCH => continue,
|
||||
Ok(_) => {}
|
||||
Err(e) => tracing::error!(error = %e, "e-mail delivery"),
|
||||
}
|
||||
let notified = async {
|
||||
match listener.as_mut() {
|
||||
// Reconnects by itself; an error means the connection was lost for now.
|
||||
Some(l) => {
|
||||
if let Err(e) = l.recv().await {
|
||||
tracing::warn!(error = %e, "LISTEN connection lost");
|
||||
tokio::time::sleep(Duration::from_secs(5)).await;
|
||||
}
|
||||
}
|
||||
None => std::future::pending().await,
|
||||
}
|
||||
};
|
||||
tokio::select! {
|
||||
() = notified => {}
|
||||
() = tokio::time::sleep(poll) => {}
|
||||
() = crate::stopped(stop.clone()) => break,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
//! Sorsfonalak worker: background jobs that read and write the shared
|
||||
//! PostgreSQL database.
|
||||
//!
|
||||
//! * e-mail delivery from `email_outbox` over SMTP (Mailpit in development)
|
||||
//! * hourly cleanup of expired sessions, tokens, invites and old e-mails
|
||||
//! * later: writing the text analyses (LLM)
|
||||
//!
|
||||
//! Configuration: see [`config::Config`]; plus `BIND_ADDR` (default
|
||||
//! `0.0.0.0:8082`, only `/healthz`), `LOG_FORMAT` and `RUST_LOG`.
|
||||
//! `worker healthcheck` performs a local health check (for Docker).
|
||||
|
||||
mod cleanup;
|
||||
mod config;
|
||||
mod email;
|
||||
mod templates;
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Context;
|
||||
use axum::Router;
|
||||
use axum::extract::State;
|
||||
use axum::http::StatusCode;
|
||||
use axum::routing::get;
|
||||
use sqlx::PgPool;
|
||||
use tokio::sync::watch;
|
||||
|
||||
use crate::config::Config;
|
||||
|
||||
const DEFAULT_BIND_ADDR: &str = "0.0.0.0:8082";
|
||||
|
||||
/// Resolves once shutdown has been requested.
|
||||
pub async fn stopped(mut stop: watch::Receiver<bool>) {
|
||||
let _ = stop.wait_for(|s| *s).await;
|
||||
}
|
||||
|
||||
async fn healthz(State(db): State<PgPool>) -> StatusCode {
|
||||
match sqlx::query("SELECT 1").execute(&db).await {
|
||||
Ok(_) => StatusCode::OK,
|
||||
Err(_) => StatusCode::SERVICE_UNAVAILABLE,
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
let addr = service_kit::bind_addr(DEFAULT_BIND_ADDR)?;
|
||||
if std::env::args().nth(1).as_deref() == Some("healthcheck") {
|
||||
return service_kit::healthcheck(addr);
|
||||
}
|
||||
service_kit::init_logging();
|
||||
let config = Config::from_env()?;
|
||||
let pool = db::connect(&config.database_url).await.context("connecting to the database")?;
|
||||
db::MIGRATOR.run(&pool).await.context("running database migrations")?;
|
||||
|
||||
let (stop_tx, stop) = watch::channel(false);
|
||||
let mut jobs = tokio::task::JoinSet::new();
|
||||
match &config.smtp_url {
|
||||
Some(url) => {
|
||||
let delivery = email::Delivery::new(pool.clone(), email::Mailer::smtp(url)?, &config);
|
||||
tracing::info!(host = %email::smtp_host(url), "e-mail delivery enabled");
|
||||
jobs.spawn(email::run(delivery, config.poll_interval, stop.clone()));
|
||||
}
|
||||
None => tracing::warn!("SMTP_URL is not set: e-mails stay in the outbox"),
|
||||
}
|
||||
jobs.spawn(cleanup::run(pool.clone(), stop.clone()));
|
||||
|
||||
let app = Router::new().route("/healthz", get(healthz)).with_state(pool);
|
||||
let listener = tokio::net::TcpListener::bind(addr).await.with_context(|| format!("binding {addr}"))?;
|
||||
tracing::info!(%addr, "worker running");
|
||||
let server = axum::serve(listener, app).with_graceful_shutdown(stopped(stop.clone()));
|
||||
tokio::select! {
|
||||
r = server => r?,
|
||||
() = service_kit::shutdown_signal() => {}
|
||||
}
|
||||
let _ = stop_tx.send(true);
|
||||
// Let jobs finish the batch they are working on.
|
||||
let finished = tokio::time::timeout(Duration::from_secs(30), async { while jobs.join_next().await.is_some() {} });
|
||||
if finished.await.is_err() {
|
||||
tracing::warn!("jobs did not stop in time");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
@@ -0,0 +1,197 @@
|
||||
//! E-mail texts (Hungarian and English), as plain text and HTML.
|
||||
//!
|
||||
//! The payloads are written by the backend (`queue_email`); the validity
|
||||
//! periods in the texts match the token lifetimes there.
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct Rendered {
|
||||
pub subject: String,
|
||||
pub text: String,
|
||||
pub html: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum TemplateError {
|
||||
UnknownTemplate(String),
|
||||
MissingField(&'static str),
|
||||
}
|
||||
|
||||
impl std::fmt::Display for TemplateError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
TemplateError::UnknownTemplate(t) => write!(f, "unknown e-mail template '{t}'"),
|
||||
TemplateError::MissingField(field) => write!(f, "e-mail payload has no '{field}'"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, PartialEq)]
|
||||
enum Lang {
|
||||
Hu,
|
||||
En,
|
||||
}
|
||||
|
||||
/// One e-mail: greeting, paragraphs, a button and a closing note.
|
||||
struct Content {
|
||||
subject: String,
|
||||
greeting: String,
|
||||
intro: String,
|
||||
button: &'static str,
|
||||
url: String,
|
||||
note: &'static str,
|
||||
}
|
||||
|
||||
pub fn render(template: &str, locale: &str, payload: &Value) -> Result<Rendered, TemplateError> {
|
||||
let lang = if locale.eq_ignore_ascii_case("hu") { Lang::Hu } else { Lang::En };
|
||||
let url = payload["url"].as_str().filter(|u| !u.is_empty()).ok_or(TemplateError::MissingField("url"))?.to_owned();
|
||||
let greeting = |name: Option<String>| match (lang, name) {
|
||||
(Lang::Hu, Some(n)) => format!("Szia {n}!"),
|
||||
(Lang::Hu, None) => "Szia!".to_owned(),
|
||||
(Lang::En, Some(n)) => format!("Hi {n},"),
|
||||
(Lang::En, None) => "Hi,".to_owned(),
|
||||
};
|
||||
let content = match (template, lang) {
|
||||
("verify_email", Lang::Hu) => Content {
|
||||
subject: "Erősítsd meg az e-mail-címed".into(),
|
||||
greeting: greeting(name(payload, "display_name")),
|
||||
intro: "Köszönjük, hogy regisztráltál a Sorsfonalakon! Az alábbi gombbal erősítsd meg az e-mail-címed."
|
||||
.into(),
|
||||
button: "E-mail-cím megerősítése",
|
||||
url,
|
||||
note: "A link 3 napig érvényes. Ha nem te regisztráltál, nyugodtan hagyd figyelmen kívül ezt a levelet.",
|
||||
},
|
||||
("verify_email", Lang::En) => Content {
|
||||
subject: "Confirm your e-mail address".into(),
|
||||
greeting: greeting(name(payload, "display_name")),
|
||||
intro: "Thanks for signing up for Sorsfonalak! Please confirm your e-mail address with the button below."
|
||||
.into(),
|
||||
button: "Confirm e-mail address",
|
||||
url,
|
||||
note: "The link is valid for 3 days. If you did not sign up, you can safely ignore this e-mail.",
|
||||
},
|
||||
("reset_password", Lang::Hu) => Content {
|
||||
subject: "Új jelszó megadása".into(),
|
||||
greeting: greeting(name(payload, "display_name")),
|
||||
intro: "Jelszó-visszaállítást kértél a Sorsfonalak-fiókodhoz. Az alábbi gombbal adhatsz meg új jelszót."
|
||||
.into(),
|
||||
button: "Új jelszó megadása",
|
||||
url,
|
||||
note: "A link 1 óráig érvényes, és csak egyszer használható. Ha nem te kérted, hagyd figyelmen kívül ezt a levelet: a jelszavad nem változik.",
|
||||
},
|
||||
("reset_password", Lang::En) => Content {
|
||||
subject: "Set a new password".into(),
|
||||
greeting: greeting(name(payload, "display_name")),
|
||||
intro:
|
||||
"You asked to reset the password of your Sorsfonalak account. Use the button below to set a new one."
|
||||
.into(),
|
||||
button: "Set a new password",
|
||||
url,
|
||||
note: "The link is valid for 1 hour and can be used once. If you did not ask for this, ignore this e-mail: your password stays the same.",
|
||||
},
|
||||
("invite_accepted", Lang::Hu) => {
|
||||
let who = name(payload, "invitee_name");
|
||||
Content {
|
||||
subject: match &who {
|
||||
Some(n) => format!("{n} elfogadta a meghívásod"),
|
||||
None => "Elfogadták a meghívásod".into(),
|
||||
},
|
||||
greeting: greeting(None),
|
||||
intro: format!(
|
||||
"{} elfogadta a „Mennyire illünk össze?” meghívásod, és elkészült a szinasztriátok.",
|
||||
who.as_deref().unwrap_or("A meghívott")
|
||||
),
|
||||
button: "Szinasztria megnyitása",
|
||||
url,
|
||||
note: "A szinasztriát mindketten látjátok, de egymás születési adatait nem.",
|
||||
}
|
||||
}
|
||||
("invite_accepted", Lang::En) => {
|
||||
let who = name(payload, "invitee_name");
|
||||
Content {
|
||||
subject: match &who {
|
||||
Some(n) => format!("{n} accepted your invitation"),
|
||||
None => "Your invitation was accepted".into(),
|
||||
},
|
||||
greeting: greeting(None),
|
||||
intro: format!(
|
||||
"{} accepted your “How compatible are we?” invitation, and your synastry is ready.",
|
||||
who.as_deref().unwrap_or("Your invitee")
|
||||
),
|
||||
button: "Open synastry",
|
||||
url,
|
||||
note: "You both see the synastry, but not each other's birth data.",
|
||||
}
|
||||
}
|
||||
_ => return Err(TemplateError::UnknownTemplate(template.to_owned())),
|
||||
};
|
||||
Ok(content.render(lang))
|
||||
}
|
||||
|
||||
/// A user-supplied name, safe for a subject line.
|
||||
fn name(payload: &Value, field: &str) -> Option<String> {
|
||||
let name: String = payload[field].as_str()?.chars().filter(|c| !c.is_control()).take(60).collect();
|
||||
let name = name.trim();
|
||||
(!name.is_empty()).then(|| name.to_owned())
|
||||
}
|
||||
|
||||
fn escape(s: &str) -> String {
|
||||
let mut out = String::with_capacity(s.len());
|
||||
for c in s.chars() {
|
||||
match c {
|
||||
'&' => out.push_str("&"),
|
||||
'<' => out.push_str("<"),
|
||||
'>' => out.push_str(">"),
|
||||
'"' => out.push_str("""),
|
||||
'\'' => out.push_str("'"),
|
||||
_ => out.push(c),
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
impl Content {
|
||||
fn render(self, lang: Lang) -> Rendered {
|
||||
let footer = match lang {
|
||||
Lang::Hu => "Ezt a levelet a Sorsfonalak küldte.",
|
||||
Lang::En => "This e-mail was sent by Sorsfonalak.",
|
||||
};
|
||||
let text = format!(
|
||||
"{}\n\n{}\n\n{}:\n{}\n\n{}\n\n-- \nSorsfonalak\n",
|
||||
self.greeting, self.intro, self.button, self.url, self.note
|
||||
);
|
||||
let html = format!(
|
||||
r#"<!doctype html>
|
||||
<html lang="{lang}">
|
||||
<head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"><title>{subject}</title></head>
|
||||
<body style="margin:0;padding:0;background:#f4f1ea;font-family:Georgia,'Times New Roman',serif;color:#2a2620">
|
||||
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background:#f4f1ea;padding:32px 16px">
|
||||
<tr><td align="center">
|
||||
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="max-width:520px;background:#ffffff;border-radius:8px;padding:32px">
|
||||
<tr><td style="font-size:20px;letter-spacing:2px;color:#9a7b3c;padding-bottom:24px">SORSFONALAK</td></tr>
|
||||
<tr><td style="font-size:16px;line-height:1.6;padding-bottom:12px">{greeting}</td></tr>
|
||||
<tr><td style="font-size:16px;line-height:1.6;padding-bottom:24px">{intro}</td></tr>
|
||||
<tr><td style="padding-bottom:24px"><a href="{url}" style="display:inline-block;background:#9a7b3c;color:#ffffff;text-decoration:none;padding:12px 24px;border-radius:6px;font-size:16px">{button}</a></td></tr>
|
||||
<tr><td style="font-size:14px;line-height:1.6;color:#6b645a;padding-bottom:16px">{note}</td></tr>
|
||||
<tr><td style="font-size:12px;line-height:1.5;color:#9b948a;word-break:break-all">{url_text}</td></tr>
|
||||
</table>
|
||||
<p style="font-size:12px;color:#9b948a;max-width:520px">{footer}</p>
|
||||
</td></tr>
|
||||
</table>
|
||||
</body>
|
||||
</html>
|
||||
"#,
|
||||
lang = if lang == Lang::Hu { "hu" } else { "en" },
|
||||
subject = escape(&self.subject),
|
||||
greeting = escape(&self.greeting),
|
||||
intro = escape(&self.intro),
|
||||
url = escape(&self.url),
|
||||
button = escape(self.button),
|
||||
note = escape(self.note),
|
||||
url_text = escape(&self.url),
|
||||
footer = escape(footer),
|
||||
);
|
||||
Rendered { subject: self.subject, text, html }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,182 @@
|
||||
use serde_json::json;
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::email::{Delivery, Mailer, backoff};
|
||||
use crate::templates::{TemplateError, render};
|
||||
|
||||
#[test]
|
||||
fn templates_in_both_languages() {
|
||||
let payload = json!({"url": "https://example.com/verify-email?token=abc", "display_name": "Anna"});
|
||||
let hu = render("verify_email", "hu", &payload).unwrap();
|
||||
assert_eq!(hu.subject, "Erősítsd meg az e-mail-címed");
|
||||
assert!(hu.text.starts_with("Szia Anna!") && hu.text.contains("token=abc"));
|
||||
assert!(
|
||||
hu.html.contains(r#"<html lang="hu">"#)
|
||||
&& hu.html.contains(r#"href="https://example.com/verify-email?token=abc""#)
|
||||
);
|
||||
|
||||
let en = render("reset_password", "en", &payload).unwrap();
|
||||
assert_eq!(en.subject, "Set a new password");
|
||||
assert!(en.text.starts_with("Hi Anna,"));
|
||||
// Unknown locales fall back to English.
|
||||
assert_eq!(render("reset_password", "de", &payload).unwrap(), en);
|
||||
|
||||
let accepted = render("invite_accepted", "hu", &json!({"url": "u", "invitee_name": "Béla"})).unwrap();
|
||||
assert_eq!(accepted.subject, "Béla elfogadta a meghívásod");
|
||||
let anonymous = render("invite_accepted", "en", &json!({"url": "u", "invitee_name": null})).unwrap();
|
||||
assert_eq!(anonymous.subject, "Your invitation was accepted");
|
||||
assert!(anonymous.text.contains("Your invitee accepted"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn template_input_is_escaped_and_checked() {
|
||||
let mail = render("verify_email", "hu", &json!({"url": "u", "display_name": "<b>X</b>\r\nBcc: y"})).unwrap();
|
||||
assert!(mail.html.contains("Szia <b>X</b>Bcc: y!"));
|
||||
assert!(!mail.html.contains("<b>X"));
|
||||
assert_eq!(render("verify_email", "hu", &json!({})), Err(TemplateError::MissingField("url")));
|
||||
assert_eq!(render("nope", "hu", &json!({"url": "u"})), Err(TemplateError::UnknownTemplate("nope".into())));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn backoff_grows_and_is_capped() {
|
||||
assert_eq!(backoff(1).as_secs(), 60);
|
||||
assert_eq!(backoff(2).as_secs(), 120);
|
||||
assert_eq!(backoff(4).as_secs(), 480);
|
||||
assert_eq!(backoff(20).as_secs(), 6 * 3600);
|
||||
}
|
||||
|
||||
fn delivery(db: PgPool, fail: Vec<(&str, bool)>) -> Delivery {
|
||||
Delivery {
|
||||
db,
|
||||
mailer: Mailer::Memory {
|
||||
sent: Default::default(),
|
||||
fail: fail.into_iter().map(|(a, p)| (a.to_owned(), p)).collect(),
|
||||
},
|
||||
from: "Sorsfonalak <noreply@example.com>".parse().unwrap(),
|
||||
max_attempts: 3,
|
||||
}
|
||||
}
|
||||
|
||||
fn sent(d: &Delivery) -> Vec<(String, String)> {
|
||||
let Mailer::Memory { sent, .. } = &d.mailer else { unreachable!() };
|
||||
sent.lock().unwrap().iter().map(|(to, m)| (to.clone(), m.subject.clone())).collect()
|
||||
}
|
||||
|
||||
async fn queue(db: &PgPool, to: &str, template: &str) -> Uuid {
|
||||
let id = Uuid::now_v7();
|
||||
sqlx::query("INSERT INTO email_outbox (id, to_email, template, locale, payload) VALUES ($1, $2, $3, 'hu', $4)")
|
||||
.bind(id)
|
||||
.bind(to)
|
||||
.bind(template)
|
||||
.bind(json!({"url": "https://example.com/x"}))
|
||||
.execute(db)
|
||||
.await
|
||||
.unwrap();
|
||||
id
|
||||
}
|
||||
|
||||
type State = (i32, bool, bool, Option<String>, bool);
|
||||
|
||||
async fn state(db: &PgPool, id: Uuid) -> State {
|
||||
sqlx::query_as(
|
||||
"SELECT attempts, sent_at IS NOT NULL, failed_at IS NOT NULL, last_error, next_attempt_at > now() \
|
||||
FROM email_outbox WHERE id = $1",
|
||||
)
|
||||
.bind(id)
|
||||
.fetch_one(db)
|
||||
.await
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[sqlx::test(migrator = "db::MIGRATOR")]
|
||||
async fn delivers_retries_and_gives_up(db: PgPool) {
|
||||
let ok = queue(&db, "anna@example.com", "verify_email").await;
|
||||
let flaky = queue(&db, "flaky@example.com", "reset_password").await;
|
||||
let rejected = queue(&db, "rejected@example.com", "verify_email").await;
|
||||
let broken = queue(&db, "anna@example.com", "no_such_template").await;
|
||||
let d = delivery(db.clone(), vec![("flaky@example.com", false), ("rejected@example.com", true)]);
|
||||
|
||||
assert_eq!(d.deliver_due().await.unwrap(), 4);
|
||||
assert_eq!(sent(&d), vec![("anna@example.com".into(), "Erősítsd meg az e-mail-címed".into())]);
|
||||
assert_eq!(state(&db, ok).await, (1, true, false, None, false));
|
||||
// Temporary failure: retried later, not at once.
|
||||
assert_eq!(state(&db, flaky).await, (1, false, false, Some("test failure".into()), true));
|
||||
// Permanent failures are not retried.
|
||||
assert_eq!(state(&db, rejected).await, (1, false, true, Some("test failure".into()), false));
|
||||
let (_, _, failed, error, _) = state(&db, broken).await;
|
||||
assert!(failed && error.unwrap().contains("unknown e-mail template"));
|
||||
assert_eq!(d.deliver_due().await.unwrap(), 0);
|
||||
|
||||
// Retries until max_attempts (3), then gives up.
|
||||
for expected in [2, 3] {
|
||||
sqlx::query("UPDATE email_outbox SET next_attempt_at = now() WHERE id = $1")
|
||||
.bind(flaky)
|
||||
.execute(&db)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(d.deliver_due().await.unwrap(), 1);
|
||||
assert_eq!(state(&db, flaky).await.0, expected);
|
||||
}
|
||||
let (_, sent_ok, failed, _, _) = state(&db, flaky).await;
|
||||
assert!(!sent_ok && failed);
|
||||
assert_eq!(sent(&d).len(), 1);
|
||||
}
|
||||
|
||||
#[sqlx::test(migrator = "db::MIGRATOR")]
|
||||
async fn concurrent_workers_do_not_send_twice(db: PgPool) {
|
||||
for i in 0..15 {
|
||||
queue(&db, &format!("u{i}@example.com"), "verify_email").await;
|
||||
}
|
||||
let (a, b) = (delivery(db.clone(), vec![]), delivery(db.clone(), vec![]));
|
||||
let (na, nb) = tokio::join!(async { a.deliver_due().await.unwrap() }, async { b.deliver_due().await.unwrap() });
|
||||
let (ra, rb) = (a.deliver_due().await.unwrap(), b.deliver_due().await.unwrap());
|
||||
assert_eq!(na + nb + ra + rb, 15);
|
||||
let mut all: Vec<String> = sent(&a).into_iter().chain(sent(&b)).map(|(to, _)| to).collect();
|
||||
all.sort();
|
||||
all.dedup();
|
||||
assert_eq!(all.len(), 15);
|
||||
}
|
||||
|
||||
#[sqlx::test(migrator = "db::MIGRATOR")]
|
||||
async fn insert_notifies_the_worker(db: PgPool) {
|
||||
let mut listener = sqlx::postgres::PgListener::connect_with(&db).await.unwrap();
|
||||
listener.listen("email_outbox").await.unwrap();
|
||||
queue(&db, "anna@example.com", "verify_email").await;
|
||||
let n = tokio::time::timeout(std::time::Duration::from_secs(5), listener.recv()).await.unwrap().unwrap();
|
||||
assert_eq!(n.channel(), "email_outbox");
|
||||
}
|
||||
|
||||
#[sqlx::test(migrator = "db::MIGRATOR")]
|
||||
async fn cleanup_removes_only_expired_data(db: PgPool) {
|
||||
let user = Uuid::now_v7();
|
||||
sqlx::query("INSERT INTO users (id, email) VALUES ($1, 'anna@example.com')").bind(user).execute(&db).await.unwrap();
|
||||
for (hash, expires) in
|
||||
[(b"live".as_slice(), "now() + interval '1 day'"), (b"dead".as_slice(), "now() - interval '1 second'")]
|
||||
{
|
||||
sqlx::query(&format!(
|
||||
"INSERT INTO sessions (id, user_id, token_hash, expires_at) VALUES ($1, $2, $3, {expires})"
|
||||
))
|
||||
.bind(Uuid::now_v7())
|
||||
.bind(user)
|
||||
.bind(hash)
|
||||
.execute(&db)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
let old = queue(&db, "anna@example.com", "verify_email").await;
|
||||
let recent = queue(&db, "anna@example.com", "verify_email").await;
|
||||
let unsent = queue(&db, "anna@example.com", "verify_email").await;
|
||||
sqlx::query("UPDATE email_outbox SET sent_at = now() - interval '8 days' WHERE id = $1")
|
||||
.bind(old)
|
||||
.execute(&db)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query("UPDATE email_outbox SET sent_at = now() WHERE id = $1").bind(recent).execute(&db).await.unwrap();
|
||||
|
||||
let deleted = crate::cleanup::run_once(&db).await.unwrap();
|
||||
assert_eq!(deleted, vec![("sessions", 1), ("auth_tokens", 0), ("emails", 1), ("invites", 0)]);
|
||||
let (left,): (Vec<Uuid>,) =
|
||||
sqlx::query_as("SELECT array_agg(id ORDER BY id) FROM email_outbox").fetch_one(&db).await.unwrap();
|
||||
assert_eq!(left, vec![recent, unsent]);
|
||||
}
|
||||
Reference in New Issue
Block a user