Skip to content

Runtime hardening

This document describes the hardening defences Merovingian applies at build time, at startup, and while serving traffic. It covers defences that are cross_platform_ and defences that are platform_specific (Linux, FreeBSD, NetBSD, OpenBSD, and the portable/service-manager profile).

For open hardening work and production-gating status, see todos/capability-gaps.md.

Cross-platform defences

These defences are present on every supported platform, either inside the binary or in the service-manager configuration that ships with the packages.

Build-time toolchain hardening

meson.build adds the following compile-and-link hardening whenever -Dhardening=true is set (the default for packages):

Defence Where it lives Notes
Stack protector hardening_compile_flags (-fstack-protector-strong) Compiler inserts stack canaries.
Stack clash protection hardening_compile_flags (-fstack-clash-protection) Guards against stack clash attacks.
Control-flow protection hardening_compile_flags (-fcf-protection=full) CET/IBT on x86_64.
FORTIFY_SOURCE hardening_compile_flags (-D_FORTIFY_SOURCE=3) when optimization != '0' Checked libc wrappers.
Hidden visibility hardening_compile_flags (-fvisibility=hidden) Limits ELF symbol exposure.
Trivial auto-var init hardening_compile_flags (-ftrivial-auto-var-init=zero) Uninitialised locals are zeroed.
Position-independent executable hardening_compile_flags (-fPIE) and link args (-pie) Enables ASLR.
No-exec stack hardening_link_flags (-Wl,-z,noexecstack) ELF GNU_STACK note is non-executable.
RELRO + BIND_NOW hardening_link_flags (-Wl,-z,relro -Wl,-z,now) on GNU/Linux Full RELRO; dynamic relocations resolved at load time.
Static PIE fallback scripts/build-static-linux.sh (-static-pie) Fully static, position-independent musl build for Linux.

The startup hardening self-check probes the same flags at runtime:

  • compiler hardening checks for __SSP__/__SSP_STRONG__/__SSP_ALL__, _FORTIFY_SOURCE > 0, and __PIE__/__pie__.
  • linker hardening, PIE, and RELRO parse /proc/self/exe on Linux to confirm PT_GNU_RELRO, DT_BIND_NOW, and PT_GNU_STACK without PF_X. Static or non-ELF builds report unknown rather than disabled.

C++ memory and type safety

The project uses C++26 with strict rules that reduce memory-safety bugs:

  • RAII everywhere; no raw new/delete, malloc/free.
  • Smart pointers for dynamic ownership; references preferred over pointers.
  • core::FileDescriptor is a move-only RAII wrapper that closes its fd on destruction and provides set_cloexec().
  • core::SecretBuffer owns isolated sodium_malloc pages, explicitly checks sodium_mlock success, and wipes/unlocks through sodium_free on destruction. Moves transfer ownership and leave the source empty. Separate allocations do not share data pages, so destroying one cannot unlock another live secret. It is move-only and non-copyable. Derived 32-byte keys use SecureKeyBytes; every copy must obtain its own locked allocation or fail. Linux kernel mapping checks and forced locked-memory exhaustion cover the storage and both caches. No key-derivation or new-token-issuance path silently accepts an unlocked key. src/core links libsodium; historical derivations are retained (ADR-0128).

Cryptographic boundary

All cryptography is delegated to libsodium. The project does not implement its own primitives.

  • sodium_init() is wrapped in per-module static sodium_is_ready() helpers so it is called once and failures are checked (src/events/event_signer.cpp, src/homeserver/auth_service.cpp, src/crypto/secret_box.cpp, etc.).
  • Passwords and the registration token are hashed with Argon2id (crypto_pwhash_str / crypto_pwhash_str_verify).
  • Access-token HMAC and signing-secret encryption keys are derived from the operator's master key with domain-separated libsodium generic hashes (crypto_generichash).
  • The Ed25519 server signing secret is stored encrypted at rest via crypto::secret_box_encrypt (XSalsa20-Poly1305 with a random nonce).
  • Constant-time comparison for fixed-size values uses sodium_memcmp. Variable length secrets are compared by hashing both inputs with a domain-separated crypto_generichash context and then comparing the fixed-size digests with sodium_memcmp, so the comparison does not leak the secret length.
  • Short-lived plaintext secret buffers attempt sodium_mlock and are wiped on release. Master-key loading and SecureKeyBytes require a successful lock; general SecretBuffer allocations record failure without rejecting it. Not every signing-secret copy checks that result. See crypto-boundary.md for the precise guarantees.
  • The seccomp-bpf allowlist permits mlock, munlock, mlockall, munlockall, and getrandom so libsodium can lock pages and fetch entropy.
  • Main alone admits fchmod(fd, 0600) for descriptor-bound SQLite permission tightening; any other mode is denied. Federation and thumbnail profiles still deny all fchmod, and path-based chmod remains denied. The grant is mode-specific, not file-specific; database admission separately validates descriptor ownership/type. See ADR-0129.
  • Main also admits only umask(0077): supervised executables inherit main's seccomp across exec and must establish their private startup mask before their own sandbox installation. Other masks remain denied in main, and both workers deny every umask change once their stricter profiles are installed. Direct launches still run the same private-mask startup call. See ADR-0130.

Configuration and secret file permissions

Before loading configuration or secret files, the bootstrap path validates POSIX metadata:

  • src/platform/file_metadata.cpp reads file kind and mode with lstat.
  • Configuration and TLS certificate files must be regular files without group or other write or any execute bit (is_secure_config_file).
  • Secret files (master key, TLS private key, registration token) must be regular owner-read-only, non-executable files with no group/other access (is_secure_secret_file). Until 0.12.5 the predicate did not actually check owner_write, so 0600 was accepted despite this line; it now enforces what it documents, and operators upgrading from an earlier release need a one-time chmod 0400 on each secret file (see docs/user-manual.md).
  • security.secrets.master_key_file was added to the startup metadata check in 0.12.13. Before then the file was opened with a plain std::ifstream without an ownership, symlink, or file-kind check, so a group/world-readable or swapped-in master key was silently accepted.

  • The --log-file path (ISO-6, ADR-0146) is opened O_NOFOLLOW|O_CLOEXEC and used only when it is a regular file the process owns; it is then made 0600 and truncated. A symlink, FIFO, device or another user's file is refused and the logger writes no file output.

  • Core-dump policy. setrlimit(RLIMIT_CORE, 0) and, on Linux, prctl(PR_SET_DUMPABLE, 0) must both succeed. The prctl result was discarded until 0.12.5: RLIMIT_CORE=0 stops the kernel writing a core file for an ordinary crash, but a core_pattern piping to a handler is not bound by the process rlimit, so the process image could still be captured while the policy reported success. setrlimit is POSIX and was only ever applied on the Linux path; BSD servers now clamp it too.

Platform coverage of the hardening controls

The startup self-check runs the same probe on every platform where the control exists, and reports "not applicable on this platform" only where the mechanism itself does not exist. Before 0.12.5 the POSIX probes were behind an __linux__ guard, so a BSD server reported unknown for controls it had applied and is_ready() refused to start it — on platforms that are documented Tier 1.

Control Linux FreeBSD OpenBSD NetBSD
Privilege drop / filesystem restrictions (geteuid) probed probed probed probed
Core-dump policy (RLIMIT_CORE) probed, plus PR_GET_DUMPABLE probed probed probed
no_new_privs probed n/a n/a n/a
Capability bounding probed n/a n/a n/a
seccomp-bpf probed n/a n/a n/a
pledge / unveil n/a n/a probed n/a
Capsicum n/a probed n/a n/a

Thumbnail worker sandbox

The out-of-process thumbnail worker decodes untrusted PNG and JPEG bytes, so it enters a sandbox before reading any input. By that point stdin and stdout are already open and it needs nothing else for the rest of its life.

Platform Primitive
Linux seccomp-bpf syscall allowlist (platform::apply_decoder_seccomp_filter())
OpenBSD pledge("stdio")
FreeBSD Capsicum capability mode (cap_enter)
NetBSD rlimits only — no equivalent in-process primitive

NetBSD is the remaining gap. Until one exists, run the worker under whatever confinement the service manager provides; the rlimits (address space, CPU, file size, descriptor count) still apply.

The library report the server requests at startup (0.12.45) and --version are answered inside the same sandbox: the worker applies its hardening first and only then calls libpng or describes either library, and it writes the answer with write(2) as it does a thumbnail, never through iostreams (whose first write makes the C library fstat stdout).

The Linux profile is its own allowlist, not the general server filter or the federation worker's filter (fixed 0.12.7). Before this fix, the decoder's harden() called platform::apply_seccomp_filter() — the general server allowlist, which permits socket, connect, sendto, openat, unlinkat, execve, and clone — so on the primary production platform the decoder was effectively unconfined, while its own source comment claimed a decoder exploit had "nothing to reach for". apply_worker_seccomp_filter(), added for the federation worker, is not a substitute either: it removes only execve/execveat and still permits sockets, openat, and clone, because the federation worker genuinely needs network and thread creation. apply_decoder_seccomp_filter() is a third, dedicated profile whose allowlist is the seccomp-bpf equivalent of pledge("stdio") — I/O on already-open descriptors, memory management, the per-thread syscalls glibc issues from inside malloc, signal return, clock reads, and exit — denying every socket call, every path-based filesystem call, execve/execveat, and clone/clone3/fork/vfork. All three platforms now confine the decoder equivalently. tests/unit/test_seccomp_hardening.cpp asserts the decoder profile is strictly tighter than the worker profile on sockets, openat, and clone, so a later "simplification" onto the shared filter fails the suite rather than silently reopening this gap. The rule this sets for future code: sandbox profiles are named for the threat they contain, not for the kind of process that installs them — a new isolated child process gets its own profile unless its syscall needs are genuinely identical to an existing one. See ADR-0053. * These checks run in src/main.cpp before any listener is created.

Signal handling and graceful shutdown

src/net/shutdown_signal.cpp installs a self-pipe and SIGINT/SIGTERM handlers. The handler does only signal-safe work: it writes one byte to the pipe and sets an atomic flag. The main thread unblocks poll() and initiates clean shutdown. SIGPIPE is ignored process-wide so a peer that vanishes mid-write cannot terminate the process (HTTP-5, security-audit-report-2026-09-29.md). Before the HTTP-5 fix this was only true after the first thumbnail was generated, because the sole SIG_IGN sat in src/media/thumbnailer.cpp. OpenSSL's socket BIO writes with plain write() and no MSG_NOSIGNAL, so a TLS client that completed the handshake, sent a partial request head and reset the connection made the server's 408 write raise SIGPIPE, and the default disposition killed the homeserver. The plain-HTTP and IPC paths use MSG_NOSIGNAL and were never affected. Every executable (merovingian-server, merovingian-fed-worker, merovingian-thumbnail-worker, merovingian-db-migrate) now calls platform::ignore_sigpipe() (sigaction(SIGPIPE, SIG_IGN), src/platform/signal_hardening.cpp) first thing in main(), before any thread starts and before any seccomp filter exists. A failure is fatal: the server and db-migrate exit with bootstrap::ExitCode::runtime_start_error (80), the two workers exit 1. The lazy ::signal(SIGPIPE, SIG_IGN) in the thumbnailer is kept as a second line of defence for the unit and integration tests, which call the thumbnailer without going through any main() of ours. tests/unit/test_signal_hardening.cpp forks a child that resets SIGPIPE to SIG_DFL (so it never inherits a harness SIG_IGN), and proves both that the child is killed without the call and that it survives with it.

Out-of-process thumbnail worker sandbox

The main server never decodes untrusted image bytes in-process. It spawns merovingian-thumbnail-worker via fork()/execve() (src/media/thumbnailer.cpp and src/media/thumbnail_worker_main.cpp):

  • Parent and child communicate through O_CLOEXEC pipes (pipe2() on Linux, pipe() + fcntl(F_SETFD, FD_CLOEXEC) elsewhere).
  • The child dup2()s the pipe ends onto stdio, then calls core::close_all_file_descriptors_except() with a std::span<int const> to close every inherited fd other than stdio. The span overload is allocation-free and async-signal-safe so it can run immediately after fork() in a multi-threaded parent. The sweep:
  • uses /proc/self/fd on Linux;
  • skips /dev/fd directory walks on FreeBSD, NetBSD, and OpenBSD (those directories contain an entry for every possible fd, not only open ones);
  • falls back to a capped fcntl(F_GETFD) scan of at most 1024 descriptors.
  • The child executes with only PATH=/usr/bin:/bin, including the FreeBSD descriptor-based fexecve() path. Parent environment entries are not inherited. Before exec, fd 2 is redirected to an immutable, RAII-owned /dev/null sink opened before filesystem sandboxing. Its original descriptor is CLOEXEC and kept above stdio even when the service starts with stderr closed. A missing sink or failed stderr duplication prevents execution with inherited diagnostics.
  • Before execve() the child sets prctl(PR_SET_NO_NEW_PRIVS, 1, ...) on Linux so a compromised worker cannot escalate through setuid/setcap helpers.
  • Inside the worker, main() calls media::apply_decoder_hardening() (include/merovingian/media/decoder_hardening.hpp, src/media/decoder_hardening.cpp) before reading a single byte of stdin, and it is fail-closed (0.12.13 audit fix): the sequence clamps resources, then installs the platform sandbox, and if any applicable control fails the worker prints the failed control's name to stderr and exits 1 without ever reading its input or invoking libpng/libjpeg-turbo. Before 0.12.13, harden() discarded every hardening call's result with std::ignore, so a sandbox that failed to install still let the worker decode attacker-controlled bytes completely unconfined, silently. The parent (src/media/thumbnailer.cpp) already treats a worker that exits before producing output as a normal failed-thumbnail response (malformed/ empty worker response → HTTP 502), so this fails closed without any parent-side change.
  • RLIMIT_CPU = 15 s in production release builds, 60 s in non-release builds, 120 s under sanitizers (ASan/UBSan/TSan are slow on CI QEMU);
  • RLIMIT_FSIZE = 64 MiB;
  • RLIMIT_CORE = 0;
  • RLIMIT_NOFILE = 16;
  • RLIMIT_AS = 768 MiB in production builds (skipped under sanitizers); OpenBSD has no RLIMIT_AS, so there the same cap is RLIMIT_DATA, which on OpenBSD also bounds anonymous mmap (how its malloc allocates);
  • platform::apply_decoder_seccomp_filter() is installed in production builds (skipped under sanitizers) — the decoder's own allowlist, not the general server filter and not apply_worker_seccomp_filter(); see "Thumbnail worker sandbox" above and ADR-0053.
  • Each control's syscall is behind an injectable function table (DecoderHardeningOps) so tests/unit/test_media_decoder_hardening.cpp can assert the fail-closed sequencing and the failed-control name for every control without actually installing a broken sandbox in the test process.
  • The parent independently validates the complete PNG signature, the first chunk's 13-byte IHDR framing and nonzero, legal dimensions, and their equality to the worker's reported dimensions. A malformed or mismatched successful response produces 502 without media bytes, content type or dimensions. This is fixed-header validation, not parent-side decoding or CRC/image-data validation.
  • The worker rejects decoded images when either axis exceeds 4096 or their area exceeds the effective pixel budget. Zero requests still use its 4,096,000-pixel ceiling; positive requests cannot raise that ceiling. The final thumbnail and actual crop-fill intermediate share that pixel budget (MED-8). Before resampling, the worker checks the rounded fill axes in uint64_t and uses division to reject excess area before narrowing or allocating. This does not add a decoded-axis limit to intermediate images or an aggregate-memory cap. After hardening, a prebuilt empty internal_error frame lets the main exception boundary contain unexpected C++ failures without another string allocation. Startup exceptions before that frame exists fail closed without reading input. Signals and closed-pipe delivery are not guaranteed by the exception boundary. See proposed ADR-0143.

Out-of-process federation worker IPC security

When federation is enabled, merovingian-server spawns merovingian-fed-worker and communicates through an AF_UNIX SOCK_STREAM socket pair created with SOCK_CLOEXEC before the child is spawned via posix_spawn. The worker inherits the client fd at file descriptor 3 (--ipc-fd), the read end of a one-shot pipe carrying the IPC auth key at file descriptor 4 (--ipc-key-fd, see below), and — when database.backend=postgresql and a separate worker database role applies (ADR-0062 part 2, see below) — the read end of a third one-shot pipe carrying that role's connection URI at file descriptor 5 (--db-uri-fd). All three are placed by posix_spawn_file_actions_adddup2, which clears FD_CLOEXEC only on the child's copies. Every other fd is FD_CLOEXEC and does not survive the exec. None of the three sources is allowed to sit on fd 3, 4 or 5 in main: when one does (for example the socket pair returns fds 0 and 3 because main runs with stdin closed), make_worker_ipc_socketpair and make_worker_secret_pipe first move it past fd 5 with F_DUPFD_CLOEXEC. A source already on its own target would make the adddup2 a same-fd dup2, which some libcs treat as a no-op that leaves FD_CLOEXEC set, and the worker would start without that fd; a source on another target would be overwritten before its own dup2 ran.

The channel is hardened against a local attacker who gains access to a separate process on the same host:

  • Authenticated ephemeral encryption (#318, key handoff redesigned 0.12.13 finding N1 / ADR-0062): every session uses a fresh crypto_kx_keypair pair and crypto_secretstream_xchacha20poly1305 AEAD, and the key exchange itself is authenticated — both processes MAC each other's ephemeral public keys (and role) with crypto_auth, keyed by the same 32-byte IPC auth key, before deriving session keys. Through 0.12.12 both processes independently derived that key from the operator master-key file (domain-separated label merovingian:ipc-channel-auth:1), which meant the worker opened the master key file itself. As of 0.12.13 only main ever opens that file: WorkerPool derives the auth key once (crypto::derive_ipc_auth_key) and WorkerSupervisor::spawn_and_connect writes exactly those 32 bytes into a pipe2(O_CLOEXEC) pipe and closes its own write end (make_worker_key_pipe). The read end stays FD_CLOEXEC in main — clearing it in the multithreaded parent would let a concurrent spawn inherit the key — and an adddup2 file action places it at fd 4 in the child only (--ipc-key-fd 4). The worker (federation_worker::read_ipc_auth_key) reads exactly that many bytes, requires EOF immediately after (rejecting a short or long write as fail-closed rather than silently truncating or padding), and closes the fd. A local process that reaches the inherited ipc-fd without a valid key on the key-fd cannot complete the handshake or inject AEAD frames. The session keys are never stored or logged; a captured IPC stream is useless after the session ends.
  • The worker's own config copy cannot reopen the master key file either: immediately after the key-fd read succeeds, WorkerEventLoop::run() calls federation_worker::clear_master_key_file(), which empties security.secrets.master_key_file on the worker's in-memory config::Config before homeserver::start_runtime() is ever called — so no runtime code path reachable inside the worker, today or added later, has a path string to pass to crypto::load_master_key_material. See docs/threat-model.md, "Operator master key reachable from the federation worker" — part 3 (Linux Landlock, below) closes the filesystem-level residual gap this entry used to record.
  • A separate, least-privilege database login, delivered the same way (ADR-0062 part 2): when database.backend=postgresql, WorkerPool::WorkerPool reads federation.worker.database_uri_file once (same owner-only-file validation as database.uri_file) and hands each WorkerSupervisor its own core::SecretBuffer copy of the bytes. spawn_and_connect creates a third pipe2(O_CLOEXEC) pipe (homeserver::make_worker_db_uri_pipe, a generalization of make_worker_key_pipe that keeps the read end off all three fixed child fd numbers), writes the URI, and adds a third adddup2 onto fd 5 (homeserver::kWorkerDbUriFd, --db-uri-fd 5) — again only when a separate URI actually applies; a SQLite backend or the allow_shared_database_credentials=true opt-out means neither the flag nor the pipe exist at all. The worker (federation_worker:: read_worker_database_uri) reads until EOF (bounded to 4096 bytes, unlike the fixed-length auth key), rejects an empty or oversized result, and federation_worker::apply_worker_database_uri sets the delivered URI as the worker's own connection override while clearing database.uri_file, runtime_role, and migration_role on its config copy — so the worker never has a path to main's credentials file and never attempts a SET ROLE onto a role it was never granted membership of.
  • Linux Landlock filesystem restriction (ADR-0062 part 3): federation_worker::main() calls platform::apply_worker_landlock() immediately before the worker seccomp filter below (Landlock's three syscalls are not on that filter's allowlist — applying Landlock first avoids adding a startup-only capability to it permanently) and before the event loop opens the database or handles any inbound request. It queries the kernel's Landlock ABI version, builds a rights mask downgraded to whatever that ABI supports, and grants exactly the paths platform::build_worker_landlock_rules() derives from the worker's own config copy: the SQLite database directory (read-write, required, when database.backend=sqlite) plus a fixed set of best-effort, read-only or read-execute OS-integration paths (CA trust store candidates, resolver configuration, NSS/dynamic-linker library directories, timezone data). The master key file, both database URI files, and TLS private keys are never in the rule set, and the worker refuses to start if any rule covers one of them — a rule on a directory grants everything beneath it, so a secret kept beside the SQLite database would otherwise be exposed. The CA grants cover only the certificate stores, never all of /etc/ssl or /etc/pki, which hold the conventional private-key directories. Because a seccomp filter survives execve, the worker inherits the main server's filter, which therefore allows the three Landlock syscalls. When the running kernel has no Landlock support (older than Linux 5.13, or disabled at boot), the worker refuses to start unless federation.worker.allow_without_landlock=true is set, which logs CRITICAL on every start — mirroring the fail-closed seccomp policy below; any other Landlock failure (ruleset creation, a required rule, or landlock_restrict_self itself) is always fatal regardless of that opt-out. See ADR-0062, part 3, for the allowlist derivation.
  • No peer credentials in transit (#323): the main process verifies the inbound X-Matrix signature itself and forwards only the verified peer identity (origin/key_id/sig_verified); the raw peer access_token and Authorization/X-Matrix headers are stripped from the fed_request frame and never cross IPC, so a compromised worker cannot harvest or replay peer homeserver credentials. The Ed25519 signing key is never forwarded either, and the worker cannot ask main to sign: there is no sign_request frame and the worker's provider refuses every request (#317, ADR-0078), so a compromised worker has neither the key nor a signing oracle. (The outbound Authorization header that does cross IPC is our own request-bound X-Matrix signature, not a reusable peer credential.)
  • No filesystem socket path: the transport is an AF_UNIX socket pair with no pathname in the filesystem namespace, so there is no socket file to impersonate or intercept via filesystem access.
  • SOCK_CLOEXEC: the socket pair is created SOCK_CLOEXEC so it is not inherited by any further child processes.
  • Minimal worker environment (#330): the worker is spawned with an explicit minimal environment (PATH only) rather than inheriting the parent environment, so no parent secret leaked via environment reaches the lower-privilege worker child.
  • Bounded IPC frames (#325): the per-frame cap is 24 MiB (merovingian::ipc::kIpcMaxFrameBytes) by default, lowered from a prior 50 MiB cap to bound memory-exhaustion DoS across concurrent workers; oversize frames are logged and propagated as send/request failures rather than silently dropped. Response bodies (fed_response/outbound_http_response) are base64-encoded before framing — a fixed 4/3 expansion — instead of raw JSON-string escaping, whose expansion depends on the response content and could otherwise push a response already at the response cap over the frame cap after escaping (issue #342). WorkerPool and the worker process each independently derive the channel's actual max_frame_bytes from security.federation.join_response_max_size via ipc::frame_bytes_for_response_cap (main process: WorkerPool::WorkerPool; worker: WorkerEventLoop::run) — both sides parse the same --config file, so they agree without negotiating the value over the channel itself. The 24 MiB constant is only the floor used when that config value is at its 16 MiB-equivalent minimum; raising join_response_max_size raises the frame cap in lockstep, and both processes must restart to pick up a change (see docs/user-manual.md).
  • Per-channel in-flight IPC cap (0.12.13 finding H2 / ADR-0065): each worker channel is independently capped by federation.worker.ipc_max_in_flight_requests (default 256). Main counts the requests it has accepted from that channel but not yet finished responding to; a request that arrives at the cap is rejected immediately with an explicit main_overloaded reply. The worker turns that reply into a retryable 503 M_UNKNOWN for the remote server, preserving Matrix retry semantics, instead of silently dropping the request or queueing it on an unbounded handler pool. The cap is per channel, not global, so one slow or abusive worker cannot starve others and a crashed channel's count is released with the channel.
  • Bounded IPC dispatch queue (CRY-2, ADR-0111): the in-flight cap is applied when a request is dequeued, so the reader thread's queue in front of it is bounded separately, by federation.worker.ipc_max_dispatch_queue_count (default 1024 frames) and federation.worker.ipc_max_dispatch_queue_bytes (default 128 MiB). A worker that floods past either marks its channel unhealthy and the reader stops, so the supervisor replaces the worker instead of main buffering without limit. Every IPC socket has a 30 s send timeout, so a peer that stops reading cannot pin a writer.
  • The worker cannot signal or re-limit other processes (ISO-2, ADR-0112): the worker seccomp filter has no kill, tkill, setrlimit or prlimit64, and allows tgkill only when its thread-group argument is the worker itself. On Landlock ABI 6 and later the ruleset also scopes signals to the worker's domain. The worker still shares main's uid, so these filters are what stop kill(getppid(), SIGSTOP). [iso2] in tests/unit/test_worker_hardening_threads.cpp runs the real filter in a forked child.
  • The worker opens only the socket families it uses (ISO-8, ADR-0147): socket() admits AF_UNIX, AF_INET and AF_INET6; AF_NETLINK returns EAFNOSUPPORT so glibc's getaddrinfo address-sorting probe falls back instead of killing the worker; any other family (AF_ALG, AF_PACKET, …) kills it. socketpair() admits AF_UNIX only. Landlock does not govern connect() to a pathname AF_UNIX socket, so give the worker's PostgreSQL role password or certificate authentication, not peer. [iso8] in tests/unit/test_seccomp_hardening.cpp runs the real filter in forked children.
  • No thread exists before hardening; seccomp is installed with TSYNC (ISO-1, ADR-0082): seccomp filters and Landlock rulesets attach to the calling thread only, so a thread that already exists when they are installed keeps whatever it had. Before this fix the logger's constructor started two writer threads, the worker's first LOG_INFO constructed the logger before Landlock and the worker filter were applied, and those two threads were left with only the filter inherited from main, which allows execve, open, socket and connect. A compromised worker thread could aim a signal handler at one of them (rt_sigaction and tgkill are allowed) and run unconfined, reading the master key or execing a shell. Three controls now close this:
  • Ordering. observability::SingleLog starts no thread in its constructor. Until start_writers() is called it writes each line synchronously under a mutex (stdout, and the log file when one is open), so an early message is neither lost nor able to deadlock. The worker and the server call start_writers() only after their hardening is applied. The server's request pools and its connection dispatcher thread (ADR-0077) are likewise created in serve_until_shutdown, after the hardening self-check has passed. Because Landlock has no thread-sync flag on older kernels, this ordering is the only thing that confines a thread against Landlock.
  • SECCOMP_FILTER_FLAG_TSYNC. Every seccomp filter (main, worker and decoder profiles) is installed with the seccomp(2) syscall and SECCOMP_FILTER_FLAG_TSYNC, so any thread that already exists is confined in the same call. A failure, including TSYNC reporting a thread it could not synchronise, is fatal; there is no fallback to a per-thread install.
  • Self-check reads every task. platform::probe_seccomp_status() reads /proc/self/task/<tid>/status for every task and reports Seccomp: 2 and NoNewPrivs: 1 only if all of them have both. Reading /proc/self/status alone reports the thread-group leader and cannot see an unconfined sibling. A task that cannot be read or parsed counts as unconfined. tests/unit/test_worker_hardening_threads.cpp covers both properties in a forked child against the real kernel: a thread created before hardening is killed on execve and every task reports the confined state, and a logger that logged before hardening has created no thread, so the writers it starts afterwards inherit Landlock and seccomp and cannot open the master key path. A kernel without seccomp or Landlock makes those scenarios SKIP with a message rather than pass.
  • Worker-specific seccomp + runtime hardening (#319): the worker applies PR_SET_NO_NEW_PRIVS, drops capabilities, sets resource limits, and installs a stricter seccomp-bpf filter (on top of the inherited server filter) that denies execve/execveat/spawn-oriented clone — the worker never execs or spawns, so those syscalls are unreachable in normal operation and kill the worker on attempted abuse. This sequence (apply_worker_hardening() in src/platform/runtime_hardening.cpp) is Linux-only, gated by federation.worker.apply_hardening (default true). Fail-closed on other platforms (0.12.1): on FreeBSD/OpenBSD/NetBSD there is no in-process worker sandbox equivalent to seccomp, and until 0.12.1 apply_worker_hardening() silently reported success there regardless — the worker logged "runtime hardening applied" and ran fully unsandboxed. It now returns a fail-closed rejection (worker_hardening_unavailable_decision()) naming the unavailable controls, and the worker refuses to start; the WorkerSupervisor treats this as a crash-looping child and federation degrades to 503 rather than ever accepting unsandboxed federation traffic. An operator who accepts the risk can still run the worker there by setting federation.worker.apply_hardening=false explicitly.
  • ASan exit-time leak check disabled in the worker (#319): the worker filter denies ptrace — an escalation primitive a compromised worker must never possess. AddressSanitizer's exit-time LeakSanitizer suspends all threads with ptrace (StopTheWorld) before scanning; with ptrace denied the tracer is killed and the scan spins forever, so the worker would never exit and the supervisor's waitpid would hang. The worker therefore defines __asan_default_options() returning detect_leaks=0 (guarded to ASan builds, C linkage at global scope). This affects only sanitised test builds — production builds compile the guard out — and only the worker; the main process retains full ASan/LSan coverage, and the worker's correctness is covered by its own unit and integration tests.
  • WorkerSupervisor isolation: the supervisor thread monitors the child pid with waitpid. If the worker exits unexpectedly the supervisor closes the channel and respawns — the main process never acts on a half-open or unauthenticated channel.
  • PDU write authority: the worker does not write accepted PDUs to the database directly. It sends a pdu_ingest call back to the main process over the IPC channel; only the main process writes to the persistent store and advances the authoritative stream_ordering counter.

Platform-specific defences

Linux

Linux receives the richest set of in-process controls.

Defence Implementation Notes
seccomp-bpf syscall allowlist src/platform/seccomp_hardening.cpp Installed in main.cpp before listeners bind and inside the thumbnail worker, always with SECCOMP_FILTER_FLAG_TSYNC so every existing thread is covered (ADR-0082).
Architecture guard src/platform/seccomp_hardening.cpp Filter starts with an AUDIT_ARCH_X86_64 or AUDIT_ARCH_AARCH64 guard and fails closed on unsupported architectures.
Fail-closed default src/platform/seccomp_hardening.cpp Unlisted syscalls return SECCOMP_RET_KILL_PROCESS.
No new privileges prctl(PR_SET_NO_NEW_PRIVS, 1, ...) Applied by apply_seccomp_filter() and apply_runtime_hardening_controls().
Capability bounding set drop apply_linux_capability_bounding_set() Calls prctl(PR_CAPBSET_DROP, cap, ...) for every capability.
Core dump policy apply_linux_core_dump_policy() setrlimit(RLIMIT_CORE, {0, 0}) and prctl(PR_SET_DUMPABLE, 0).
Self-check probes src/platform/hardening_self_check.cpp Confirms Seccomp: 2 and NoNewPrivs: 1 for every task under /proc/self/task, PR_GET_NO_NEW_PRIVS, and RLIMIT_CORE == 0.
Ignored SIGPIPE src/platform/signal_hardening.cpp platform::ignore_sigpipe() is the first call in every executable's main().
systemd sandboxing packaging/systemd/merovingian.service PrivateTmp=true, ProtectSystem=strict, ProtectHome=true, NoNewPrivileges=true, CapabilityBoundingSet=, SystemCallArchitectures=native, MemoryDenyWriteExecute=true, UMask=0077, RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6, RestrictNamespaces=true, ProtectKernelTunables=/Modules=/Logs=, ProtectControlGroups=, ProtectClock=, ProtectHostname= (ISO-8, ADR-0147), etc.

The seccomp-bpf filter is deliberately narrow: it allows only the syscalls the runtime actually needs. The filter is installed in main.cpp before start_client_server is called, so the database layer (SQLite) runs under the filter from its first access onwards.

Filesystem syscalls in the allowlist — the filter permits the following filesystem-mutating calls because SQLite requires them for correct operation:

Syscall Reason
ftruncate WAL file truncation during checkpoints and journal rollback.
unlink, unlinkat Journal file deletion on commit in DELETE journal mode.
rename, renameat, renameat2 Atomic commit in some SQLite configurations; std::filesystem::rename.
fstatfs, statfs Device sector-size probe during WAL-mode open.
fallocate posix_fallocate(3) path: SQLite pre-allocates database and WAL file space; glibc maps this to fallocate(2) on filesystems that support it.

The path-based truncate and permission-mutation calls (chmod, fchmod, fchmodat, umask, mkdir) remain blocked. The service-manager sandbox (ProtectSystem=strict, writable paths limited to /var/lib/merovingian and /run/merovingian) bounds what unlink/rename/ftruncate/fallocate can actually reach.

glibc per-CPU and synchronisation syscalls — allowed because modern glibc (2.31+–2.35+) calls these from malloc, thread initialisation, process spawning, and after fork():

Syscall Nr (x86-64/aarch64) Reason
clone3 435 glibc 2.34+ uses clone3 instead of clone for pthread_create and posix_spawn. Not defining __NR_clone3 in the build headers silently omits this entry and causes SIGSYS at the first thread or process creation after seccomp is applied.
close_range 436 glibc 2.34+ posix_spawn uses close_range in the child to close inherited file descriptors before exec. The child inherits this filter, so the entry must be present even though only the child calls it.
faccessat2 439 glibc 2.33+ uses faccessat2 (Linux 5.8+) for faccessat() calls with AT_SYMLINK_NOFOLLOW or AT_EACCESS flags. getaddrinfo and NSS module probing trigger this on Fedora 36+ hosts.
rseq — glibc 2.35+ registers a per-thread restartable-sequence area after fork(). glibc 2.36+ also uses rseq inside the malloc per-CPU cache.
membarrier — glibc 2.31+ issues MEMBARRIER_CMD_PRIVATE_EXPEDITED from the malloc fast path on multi-processor systems.
getcpu — Returns the running CPU/NUMA node; used by glibc's per-CPU TLS and malloc implementation.

Runtime-instrumentation syscalls — allowed because sanitiser-instrumented binaries use them after the filter is installed. The federation worker inherits the server's seccomp filter across execve, so any syscall the sanitizer runtime needs in the child must also be present:

Syscall Nr (x86-64/aarch64) Reason
personality 135 ThreadSanitizer calls personality(ADDR_NO_RANDOMIZE) in the worker after execve to disable ASLR for deterministic shadow-memory layout. Blocking it kills the child with SIGSYS before the IPC handshake completes.

personality is gated behind __has_feature(thread_sanitizer) / __SANITIZE_THREAD__ in both the main and worker allowlists (#428): it is compiled into the filter only for TSan builds. A production build must never carry it — personality(ADDR_NO_RANDOMIZE) lets a caller with arbitrary code execution disable ASLR, weakening a core exploit mitigation the platform self-check otherwise enforces.

Build-header caveat — clone3, close_range, and faccessat2 were added in Linux 5.3, 5.9, and 5.8 respectively. Kernel headers shipped with older build environments (e.g. Ubuntu 20.04 linux-libc-dev at 5.4) may not define the corresponding __NR_* macros; a naive #ifdef __NR_clone3 guard would silently drop the entry. The filter uses an #elif defined(__x86_64__) || defined(__aarch64__) fallback with the raw numeric constant for each of these three syscalls, paralleling the pattern already used for SECCOMP_RET_KILL_PROCESS.

FreeBSD / NetBSD / OpenBSD (BSD)

Merovingian builds and runs on all three BSDs. OpenBSD and FreeBSD have full in-process sandbox support; NetBSD is served by the portable service-manager profile.

OpenBSD — pledge(2) + unveil(2)

apply_runtime_hardening_controls() (called from start_client_server()) first plans every unveil(2) call with openbsd_unveil_rules(). If any listed path is missing and so is the directory it would be created in, unveil would fail with ENOENT. In that case no call is made and the control reports failure. The first unveil hides the rest of the filesystem and cannot be undone, so a failure partway used to leave the process seeing only the paths before it (ADR-0162). The calls are:

  1. unveil(path, "rx") for every path in filesystem.read_only_paths (e.g. /usr, /etc/merovingian).
  2. unveil(path, "rwc") for every path in filesystem.writable_paths (e.g. /var/lib/merovingian, /var/run/merovingian, /tmp). /tmp is included because the test harness and runtime scratch utilities create temporary files via std::filesystem::temp_directory_path(), which resolves to /tmp on OpenBSD.
  3. unveil("/etc/ssl", "r") — LibreSSL CA bundle for outbound TLS.
  4. unveil(NULL, NULL) — locks the vnode allowlist.
  5. pledge("stdio rpath wpath cpath flock inet unix dns proc exec", NULL) — the proc exec promises allow fork()/exec() for the thumbnail worker; unix allows the AF_UNIX IPC channel to the federation worker.

openbsd_pledge_is_active() returns true after step 5. The final startup hardening self-check runs in run_server() after start_client_server() has applied the BSD profile, so the snapshot reflects the post-apply state.

FreeBSD — Capsicum cap_enter(2)

FreeBSD Capsicum capability mode forbids opening files by path after cap_enter(). The call therefore happens after all resources are open:

  1. apply_runtime_hardening_controls() validates the BSD profile and returns accepted without calling cap_enter().
  2. TCP listeners bind, TLS certificates load, and the federation worker is spawned via posix_spawn() (path-based — must run before capability mode).
  3. run_server() in main.cpp pre-opens the thumbnail worker binary with open(path, O_RDONLY | O_EXEC | O_CLOEXEC) and stores the fd in RuntimeMediaConfig::thumbnail_worker_fd.
  4. apply_freebsd_capsicum_capability_mode() calls cap_enter().
  5. The child process (thumbnail worker) uses fexecve(fd, argv, environ) instead of execv(path, argv) because opening the binary by path is now forbidden.

freebsd_capsicum_is_active() probes cap_getmode(2). The final startup hardening self-check runs in run_server() after cap_enter() has been called, so the snapshot reflects the confined state.

NetBSD and other BSDs

No in-process sandbox primitives are implemented. The BSD hardening profile is validated as documented, but apply_runtime_hardening_controls() returns a mode-gated rejection. Service-manager privilege drop and filesystem restrictions apply via rc.d scripts.

  • The portable setrlimit gates are validated by the BSD hardening profile, and the thumbnail worker applies RLIMIT_CPU, RLIMIT_FSIZE, RLIMIT_CORE, RLIMIT_NOFILE, and RLIMIT_AS (RLIMIT_DATA on OpenBSD, which has no RLIMIT_AS).
  • The thumbnail worker fd sweep uses the capped fcntl(F_GETFD) fallback on every BSD instead of walking /dev/fd.
  • Service-manager scripts apply the privilege drop and filesystem restrictions:
  • OpenRC: packaging/openrc/merovingian (command_user=merovingian:merovingian, directory=/var/lib/merovingian).
  • FreeBSD/NetBSD/OpenBSD rc.d: packaging/rc.d/merovingian and packaging/openbsd/rc.d/merovingian (merovingian_user, runtime and state directories owned by the service user).

Portable / other Unix

On platforms that are neither Linux nor a recognised BSD, Merovingian uses the portable hardening profile (default_portable_hardening_profile()). The profile:

  • requires that the hardening plan documents privilege drop, filesystem restrictions, resource limits, memory locking, random source, and signal handling;
  • applies no kernel-specific syscalls itself;
  • relies on the service manager to drop privileges and confine the filesystem.

Startup hardening self-check

src/platform/hardening_self_check.cpp is invoked from src/main.cpp after all platform hardening controls have been applied (seccomp-bpf, Linux capability bounding / no_new_privs / core-dump policy, OpenBSD pledge/unveil, FreeBSD Capsicum). Each check reports one of:

  • enabled — the defence is active;
  • disabled — the defence was explicitly turned off (hard failure);
  • unknown — the probe could not confirm the defence (e.g. static binary, unsupported kernel).

run_server() refuses to start serving traffic unless every check reports enabled. HardeningSelfCheck::is_ready() returns true only when there are no blockers, and production_blockers() lists every check that is not enabled.

Packaging and deployment hardening

The packaging scripts request the same linker hardening that meson.build advertises:

  • scripts/build-deb.sh and scripts/build-rpm.sh pass -pie -Wl,-z,relro -Wl,-z,now.
  • scripts/build-static-linux.sh passes -static-pie -Wl,-z,relro -Wl,-z,now and verifies the resulting binary has no dynamic interpreter.
  • The RPM specs (packaging/rpm/merovingian.spec, packaging/rhel/merovingian.spec, packaging/opensuse/merovingian.spec) pass the same flags and strip conflicting _FORTIFY_SOURCE definitions from the distribution's %optflags.

Every packaging path also asks for an optimised build (0.12.26, audit OPS-5). Meson builds debug (-O0) when no buildtype is given, _FORTIFY_SOURCE is only defined when optimising, and the startup self-check refuses to start a binary whose FORTIFY_SOURCE check is not enabled. So:

  • scripts/build-deb.sh, scripts/build-freebsd-pkg.sh, scripts/build-netbsd-pkg.sh, scripts/build-openbsd-pkg.sh, scripts/build-static-linux.sh and the Dockerfile pass --buildtype=release. The RPM specs use the distribution %meson macro, whose %optflags supply -O2. meson.build keeps Meson's debug default for development and test builds (ADR-0148), and warns at configure time when hardening is on at optimization=0.
  • tests/tooling/test_packages_workflow.py fails if any of those scripts, or the Dockerfile, configures Meson without --buildtype=release.

The Dockerfile:

  • pins both stages' debian:trixie-slim base by the digest of its multi-architecture index, so a re-pushed tag cannot change the image;
  • builds with libpq, OpenSSL, libpng and libturbojpeg, so the thumbnail worker is built rather than silently skipped;
  • ships merovingian-server, merovingian-db-migrate, and both workers (/usr/libexec/merovingian/merovingian-fed-worker, the path the server spawns by default, and merovingian-thumbnail-worker), with their runtime libraries, and fails the image build if ldd finds an unresolved library in any of them;
  • runs as the unprivileged merovingian user. A non-root process cannot drop its own capability bounding set, so the container must be started with --cap-drop=ALL (and should be started with --security-opt=no-new-privileges), or the capability bounding check stays unknown and the server refuses to start. The systemd unit gets the same result from CapabilityBoundingSet=.

scripts/check-startup-hardening-gate.sh proves a packaged binary actually passes the gate. It boots the server against a throwaway SQLite config bound to loopback with a fresh owner-only master key, waits for the final self-check, and passes only when every check reports enabled, the number of enabled checks equals the number the server says it ran, readiness is ready=true blockers=0, the federation worker is spawned and still running a few seconds later (its own seccomp and Landlock policy applies after it starts, and a worker killed by it would otherwise go unnoticed while the supervisor restarts it), and the server then exits 0 on SIGTERM. Unless --federation-worker is given it leaves the worker path to the binary's built-in libexec default, so a pass also proves the package installed the worker where the server looks for it. Its decisions are covered by tests/tooling/test_startup_hardening_gate.py against a fake server.

CI gates

Hardening is exercised by automated tests:

  • tests/unit/test_seccomp_hardening.cpp asserts the fail-closed default action, the architecture guard, that SQLite journal syscalls are allowed, that privilege-mutation syscalls (chmod, fchmod, fchmodat, umask, mkdir, truncate) remain denied, and that clone3 (435), close_range (436), and faccessat2 (439) are always present in the compiled filter regardless of build-time __NR_* macro availability.
  • tests/unit/test_file_descriptor.cpp exercises FileDescriptor::set_cloexec() and close_all_file_descriptors_except().
  • tests/unit/test_media_thumbnailer.cpp covers the CLOEXEC pipe path and the sandboxed worker round-trip.
  • tests/unit/test_runtime_hardening.cpp validates profile accept/reject logic, including (0.12.1) worker_hardening_unavailable_decision() — the pure, syscall-free decision builder apply_worker_hardening() falls back to on non-Linux platforms — proving it is fail-closed rather than a silent accept(), plus a non-Linux-only scenario exercising apply_worker_hardening() itself end to end.
  • tests/integration/test_server_startup_hardening_flow.cpp spawns the real merovingian-server binary and verifies it either starts under full hardening or refuses to start when a control cannot be enabled. As of 0.12.1 the "starts under full hardening" branch also issues a real HTTP GET over a real TCP socket against the spawned process's client listener and asserts a genuine 200 response, plus a post-shutdown connect-refused check — proof the process actually serves traffic, not merely that it stays alive. The test skips in environments that cannot satisfy every control (for example, a debug Meson build omits _FORTIFY_SOURCE, or a non-root Linux process lacks CAP_SETPCAP to drop the capability bounding set). To exercise the live-serving path, build with --buildtype debugoptimized or release (or --profile hardened, which implies release + -Dhardening=true) and run the test as a non-root user that retains CAP_SETPCAP (ambient capability, sudo setcap cap_setpcap+ep on the binary, or a suitable container/security profile). CI gate added in 0.12.1: the ubuntu-hardened-listener-coverage job in .github/workflows/ci.yml builds the hardened profile and grants merovingian-server CAP_SETPCAP via sudo setcap (GitHub-hosted runners provide passwordless sudo) specifically so this scenario reaches its live-serving assertions in CI instead of WARN-skipping, which every other CI job still does (root containers fail the privilege-drop check instead).
  • scripts/check-elf-hardening.sh (0.12.1) statically inspects a built binary's ELF headers (PIE, PT_GNU_RELRO, DT_BIND_NOW, non-executable PT_GNU_STACK) and fails closed if any is missing — the CI-time counterpart to the runtime ELF probe above, needing no live process so it runs the same whether the job is root, non-root, or containerized. Run against every shipped binary in the ubuntu-hardened-listener-coverage CI job and per-platform in .github/workflows/release.yml via scripts/collect-release-evidence.sh.
  • scripts/check-startup-hardening-gate.sh (0.12.26, audit OPS-5) runs the real startup gate against what is shipped. In .github/workflows/packages.yml the deb and debian-pkg jobs install the built .deb and run it as the merovingian user under setpriv --bounding-set=-all, and the docker-image job builds the Dockerfile and runs it inside the image with --cap-drop=ALL --security-opt=no-new-privileges. The BSD package jobs do not run it yet.

Because seccomp/pledge/unveil are permanent in-process, the test suite sets MEROVINGIAN_TEST_DISABLE_HARDENING=1 when invoking meson test. The build scripts export this variable automatically; direct invocations of the test binary (such as the PostgreSQL integration workflow) must set it manually. Production server binaries never see this variable and always apply the platform profile.

What is intentionally deferred

The following controls are implemented as service-manager policy rather than in-process syscalls:

  • In-process privilege drop (setresgid/setresuid) — the server must never run as root; the service manager supplies a dedicated user.
  • In-process Linux filesystem confinement (Landlock) for the main merovingian-server process — service-manager sandboxing (systemd/OpenRC/FreeBSD rc.d) enforces filesystem restrictions; the Merovingian hardening profile documents these requirements. The federation worker is the exception: as of ADR-0062 part 3 (0.12.13) it applies its own in-process Landlock ruleset (see "Out-of-process federation worker IPC security" above) — the worker is the more exposed, more frequently restarted process, so it gets an in-process control the main process still relies on the service manager for.
  • A platform-specific in-process sandbox for the federation worker (merovingian-fed-worker) on FreeBSD/OpenBSD/NetBSD — unlike the main server, the worker has no pledge/unveil- or Capsicum-equivalent hardening sequence there. As of 0.12.1 this is a documented, fail-closed gap rather than a silent one: federation.worker.apply_hardening=true (the production default) makes the worker refuse to start on those platforms instead of running unsandboxed. An operator can opt into running an unsandboxed worker there with federation.worker.apply_hardening=false, relying solely on service-manager confinement.

OpenBSD pledge/unveil and FreeBSD cap_enter() are fully wired and exercised in CI. The startup readiness gate will not pass until every hardening check reports enabled. See todos/capability-gaps.md for the current list.