Compare commits

...

13 Commits

Author SHA1 Message Date
j4n
07938544a1 docker: trim compose override example 2026-02-20 17:02:34 +01:00
j4n
3cc74a4c9a docker: get rid of CHATMAIL_* in compose 2026-02-20 16:56:05 +01:00
j4n
77676a4e87 docker: streamline overrides, rename datadirs, external TLS 2026-02-20 16:38:35 +01:00
j4n
dc2a6fda05 docker: migrate to new external tls logic
- remove all traces of CHATMAIL_NOACME; purge certwatch service
- introduce TLS_EXTERNAL_CERT_AND_KEY as per new logic
2026-02-20 10:00:44 +01:00
j4n
d9dce2ccee Merge remote-tracking branch 'origin/hpk/tls-external' into j4n/docker-traefik 2026-02-19 21:04:21 +01:00
j4n
fcfc2cca1a fix(docker): remove CHATMAIL_INI from env 2026-02-19 20:41:18 +01:00
j4n
beb4041e3f fix(docker): Add TZ to env 2026-02-19 20:36:51 +01:00
holger krekel
da3d726fb1 feat: support externally managed TLS via tls_external_cert_and_key option
Adds a new tls_external_cert_and_key config option for chatmail servers
that manage their own TLS certificates (e.g. via an external ACME client
or a load balancer).

A systemd path unit (tls-cert-reload.path) watches the certificate file
via inotify and automatically reloads dovecot and nginx when it changes.
Postfix reads certs per TLS handshake so needs no reload.

Also extracts openssl_selfsigned_args() so cert generation parameters
are shared between SelfSignedTlsDeployer and the e2e test.
2026-02-19 19:49:53 +01:00
j4n
854b7ef368 typo 2026-02-19 16:03:41 +01:00
j4n
7e30bafd57 docker: clear up docker compose v1/v2 differences (doc/compose.yaml) 2026-02-19 16:03:41 +01:00
j4n
3ef59c3def feat: add Docker and Compose support
Add Docker-based deployment: Dockerfile based on systemd image,
docker-compose.yaml, build script, entrypoint, external certificate
monitoring, CI workflow, and documentation.

This builds on the chatmaild/cmdeploy preparation in the previous
commit (j4n/docker-prep-chatmail) which added the env-var-driven
feature flags (CHATMAIL_NOSYSCTL, CHATMAIL_NOPORTCHECK, CHATMAIL_NOACME)
and @local deployment support needed by the container.

This is commit 2 of 3 to merge squashed changes on j4n/docker and docker
branches, original commits were beef0ec..606f36e

Architecture overview (mostly by original author Keonik1):
- Debian-systemd image wrapping the existing cmdeploy install
- Host networking to not manually expose the many ports needed
- Config via MAIL_DOMAIN env var or (new) mounted chatmail.ini
- New: cmdeploy stages: install at build, configure+activate at startup
- New: Monitoring service for external certs via systemd timer (chatmail-certmon)
- New: Image version tracking for automatic upgrade detection (cm + config hash)
- New: docker-compose.override.yaml pattern for user customizations
- New: GitHub Actions CI for ghcr.io image builds

Traefik reverse-proxy support is prepared but the specific files are
excluded from this PR and will be submitted separately.

TODO:
- [ ] Pull out CHATMAIL_NOACME as PR #855 introduced a proper mechanism
- [ ] Check if underlying image could be based on regular debian-slim
  images with a step to enable systemd, similar to
  https://github.com/alexdzyoba/docker-debian-systemd

Files added:
  .dockerignore
  .github/workflows/docker-build.yaml
  docker-compose.yaml
  docker-compose.override.yaml.example
  docker/build.sh
  docker/chatmail_relay.dockerfile
  docker/files/chatmail-certmon.{service,sh,timer}
  docker/files/entrypoint.sh
  docker/files/setup_chatmail.service
  docker/files/setup_chatmail_docker.sh
  env.example
  doc/source/docker.rst

Files modified:
  .gitignore
  doc/source/getting_started.rst
  doc/source/index.rst

Co-authored-by: Keonik1 <keonik.dev@gmail.com>
Co-authored-by: missytake <missytake@systemli.org>
2026-02-19 16:03:41 +01:00
j4n
a7b3893fee cmdeploy: prepare chatmaild/cmdeploy changes for Docker support
- chatmaild:
  - basedeploy.py: Add has_systemd() guard. During Docker image builds
    there's no running systemd, so deployers that query SystemdEnabled
    facts would crash; this change might also be helpful for non-systemd
    platforms.
- cmdeploy:
  - cmdeploy.py:
    - when deploying to @docker, auto-set CHATMAIL_NOPORTCHECK and
      CHATMAIL_NOSYSCTL since neither makes sense inside a container
    - --config default now reads CHATMAIL_INI env var, so Docker
      entrypoints can point to a mounted ini without CLI flags.
  - deployers.py:
    - skip port check / CHATMAIL_NOPORTCHECK
    - skip echobot systemd cleanup w/ has_systemd
  - dovecot/deployer.py:
    - Guard sysctl writes behind CHATMAIL_NOSYSCTL
    - invert dovecot install check so it works without systemd
  - sshexec.py: Add __call__ to LocalExec so cmdeploy status works with
    @local target. Without it, cmdeploy status tried to call the
    executor directly and got TypeError.

Consolidated from j4n/docker branch commits (selection):
- 8953fde feat(cmdeploy): read CHATMAIL_INI env var for default --config path
- 81d7782 fix(cmdeploy): add __call__ to LocalExec so status works with @local
- 8bba78e docker: disable port check if docker is running. fix #694
- 865b514 docker: replace config flags with env vars, drop docker param (instead of f26cb08)

Files: cmdeploy/src/cmdeploy/{basedeploy,cmdeploy,deployers,sshexec,dovecot/deployer}.py

Co-authored-by: Keonik1 <keonik.dev@gmail.com>
Co-authored-by: missytake <missytake@systemli.org>
2026-02-19 16:03:41 +01:00
j4n
58fa5e5c98 cmdeploy: prepare chatmaild/cmdeploy changes for Docker support
- chatmaild:
  - basedeploy.py: Add has_systemd() guard. During Docker image builds
    there's no running systemd, so deployers that query SystemdEnabled
    facts would crash; this change might also be helpful for non-systemd
    platforms.
- cmdeploy:
  - cmdeploy.py:
    - when deploying to @docker, auto-set CHATMAIL_NOPORTCHECK and
      CHATMAIL_NOSYSCTL since neither makes sense inside a container
    - --config default now reads CHATMAIL_INI env var, so Docker
      entrypoints can point to a mounted ini without CLI flags.
  - deployers.py:
    - skip port check / CHATMAIL_NOPORTCHECK
    - skip echobot systemd cleanup w/ has_systemd
  - dovecot/deployer.py:
    - Guard sysctl writes behind CHATMAIL_NOSYSCTL
    - invert dovecot install check so it works without systemd
  - sshexec.py: Add __call__ to LocalExec so cmdeploy status works with
    @local target. Without it, cmdeploy status tried to call the
    executor directly and got TypeError.

Consolidated from j4n/docker branch commits (selection):
- 8953fde feat(cmdeploy): read CHATMAIL_INI env var for default --config path
- 81d7782 fix(cmdeploy): add __call__ to LocalExec so status works with @local
- 8bba78e docker: disable port check if docker is running. fix #694
- 865b514 docker: replace config flags with env vars, drop docker param (instead of f26cb08)

Files: cmdeploy/src/cmdeploy/{basedeploy,cmdeploy,deployers,sshexec,dovecot/deployer}.py

Co-authored-by: Keonik1 <keonik.dev@gmail.com>
Co-authored-by: missytake <missytake@systemli.org>
2026-02-19 16:03:39 +01:00
31 changed files with 1439 additions and 73 deletions

18
.dockerignore Normal file
View File

@@ -0,0 +1,18 @@
data/
venv/
__pycache__
*.pyc
*.orig
*.ini
.pytest_cache
.env
# Slim build context — .git/ alone can be 100s of MB
.git
.github/
docs/
tests/
# Exclude markdown files but keep www/src/*.md (used by WebsiteDeployer)
*.md
!www/**/*.md

76
.github/workflows/docker-build.yaml vendored Normal file
View File

@@ -0,0 +1,76 @@
name: Docker Build
on:
pull_request:
paths:
- 'docker/**'
- 'docker-compose.yaml'
- '.dockerignore'
- 'chatmaild/**'
- 'cmdeploy/**'
- '.github/workflows/docker-build.yaml'
push:
branches:
- main
- j4n/docker
paths:
- 'docker/**'
- 'docker-compose.yaml'
- '.dockerignore'
- 'chatmaild/**'
- 'cmdeploy/**'
- '.github/workflows/docker-build.yaml'
tags:
- 'v*'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build:
name: Build Docker image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GHCR
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
# Tagged releases: v1.2.3 → :1.2.3, :1.2, :latest
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
# Branch pushes: j4n/docker → :j4n-docker
type=ref,event=branch
# Always: :sha-<hash>
type=sha
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: docker/chatmail_relay.dockerfile
push: ${{ github.event_name == 'push' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
GIT_HASH=${{ github.sha }}

View File

@@ -0,0 +1,37 @@
name: test tls_external_cert_and_key on staging2.testrun.org
on:
workflow_run:
workflows:
- "deploy on staging2.testrun.org, and run tests"
types:
- completed
jobs:
test-tls-external:
name: test tls_external_cert_and_key
runs-on: ubuntu-latest
timeout-minutes: 30
concurrency: staging2.testrun.org
environment:
name: staging2.testrun.org
steps:
- uses: actions/checkout@v4
- name: prepare SSH
run: |
mkdir -p ~/.ssh
echo "${{ secrets.STAGING_SSH_KEY }}" >> ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
ssh-keyscan staging2.testrun.org >> ~/.ssh/known_hosts 2>/dev/null
- run: scripts/initenv.sh
- name: append venv/bin to PATH
run: echo venv/bin >>$GITHUB_PATH
- name: run tls_external e2e test
run: |
python -m cmdeploy.tests.setup_tls_external \
staging2.testrun.org

6
.gitignore vendored
View File

@@ -164,3 +164,9 @@ cython_debug/
#.idea/
chatmail.zone
# docker
/data/
/custom/
docker-compose.override.yaml
.env

View File

@@ -60,10 +60,24 @@ class Config:
self.privacy_pdo = params.get("privacy_pdo")
self.privacy_supervisor = params.get("privacy_supervisor")
# TLS certificate management: derived from the domain name.
# Domains starting with "_" use self-signed certificates
# All other domains use ACME.
if self.mail_domain.startswith("_"):
# TLS certificate management.
# If tls_external_cert_and_key is set, use externally managed certs.
# Otherwise derived from the domain name:
# - Domains starting with "_" use self-signed certificates
# - All other domains use ACME.
external = params.get("tls_external_cert_and_key", "").strip()
if external:
parts = external.split()
if len(parts) != 2:
raise ValueError(
"tls_external_cert_and_key must have two space-separated"
" paths: CERT_PATH KEY_PATH"
)
self.tls_cert_mode = "external"
self.tls_cert_path = parts[0]
self.tls_key_path = parts[1]
elif self.mail_domain.startswith("_"):
self.tls_cert_mode = "self"
self.tls_cert_path = "/etc/ssl/certs/mailserver.pem"
self.tls_key_path = "/etc/ssl/private/mailserver.key"

View File

@@ -48,6 +48,13 @@ passthrough_senders =
# (space-separated, item may start with "@" to whitelist whole recipient domains)
passthrough_recipients =
# Use externally managed TLS certificates instead of built-in acmetool.
# Paths refer to files on the deployment server (not the build machine).
# Both files must already exist before running cmdeploy.
# Certificate renewal is your responsibility; changed files are
# picked up automatically by all relay services.
# tls_external_cert_and_key = /path/to/fullchain.pem /path/to/privkey.pem
# path to www directory - documented here: https://chatmail.at/doc/relay/getting_started.html#custom-web-pages
#www_folder = www

View File

@@ -87,3 +87,36 @@ def test_config_tls_self(make_config):
assert config.tls_cert_mode == "self"
assert config.tls_cert_path == "/etc/ssl/certs/mailserver.pem"
assert config.tls_key_path == "/etc/ssl/private/mailserver.key"
def test_config_tls_external(make_config):
config = make_config(
"chat.example.org",
{
"tls_external_cert_and_key": "/custom/fullchain.pem /custom/privkey.pem",
},
)
assert config.tls_cert_mode == "external"
assert config.tls_cert_path == "/custom/fullchain.pem"
assert config.tls_key_path == "/custom/privkey.pem"
def test_config_tls_external_overrides_underscore(make_config):
config = make_config(
"_test.example.org",
{
"tls_external_cert_and_key": "/certs/fullchain.pem /certs/privkey.pem",
},
)
assert config.tls_cert_mode == "external"
assert config.tls_cert_path == "/certs/fullchain.pem"
def test_config_tls_external_bad_format(make_config):
with pytest.raises(ValueError, match="two space-separated"):
make_config(
"chat.example.org",
{
"tls_external_cert_and_key": "/only/one/path.pem",
},
)

View File

@@ -5,6 +5,11 @@ import os
from pyinfra.operations import files, server, systemd
def has_systemd():
"""Returns False during Docker image builds or any other non-systemd environment."""
return os.path.isdir("/run/systemd/system")
def get_resource(arg, pkg=__package__):
return importlib.resources.files(pkg).joinpath(arg)

View File

@@ -110,6 +110,9 @@ def run_cmd(args, out):
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
if ssh_host in ["localhost", "@docker"]:
if ssh_host == "@docker":
env["CHATMAIL_NOPORTCHECK"] = "True"
env["CHATMAIL_NOSYSCTL"] = "True"
cmd = f"{pyinf} @local {deploy_path} -y"
if version.parse(pyinfra.__version__) < version.parse("3"):
@@ -332,7 +335,7 @@ def add_config_option(parser):
"--config",
dest="inipath",
action="store",
default=Path("chatmail.ini"),
default=Path(os.environ.get("CHATMAIL_INI", "chatmail.ini")),
type=Path,
help="path to the chatmail.ini file",
)

View File

@@ -2,6 +2,7 @@
Chat Mail pyinfra deploy.
"""
import os
import shutil
import subprocess
import sys
@@ -19,13 +20,14 @@ from pyinfra.operations import apt, files, pip, server, systemd
from cmdeploy.cmdeploy import Out
from .acmetool import AcmetoolDeployer
from .selfsigned.deployer import SelfSignedTlsDeployer
from .external.deployer import ExternalTlsDeployer
from .basedeploy import (
Deployer,
Deployment,
activate_remote_units,
configure_remote_units,
get_resource,
has_systemd,
)
from .dovecot.deployer import DovecotDeployer
from .filtermail.deployer import FiltermailDeployer
@@ -33,6 +35,7 @@ from .mtail.deployer import MtailDeployer
from .nginx.deployer import NginxDeployer
from .opendkim.deployer import OpendkimDeployer
from .postfix.deployer import PostfixDeployer
from .selfsigned.deployer import SelfSignedTlsDeployer
from .www import build_webpages, find_merge_conflict, get_paths
@@ -66,6 +69,8 @@ def _build_chatmaild(dist_dir) -> None:
def remove_legacy_artifacts():
if not has_systemd():
return
# disable legacy doveauth-dictproxy.service
if host.get_fact(SystemdEnabled).get("doveauth-dictproxy.service"):
systemd.service(
@@ -300,7 +305,7 @@ class LegacyRemoveDeployer(Deployer):
present=False,
)
# remove echobot if it is still running
if host.get_fact(SystemdEnabled).get("echobot.service"):
if has_systemd() and host.get_fact(SystemdEnabled).get("echobot.service"):
systemd.service(
name="Disable echobot.service",
service="echobot.service",
@@ -536,6 +541,20 @@ class GithashDeployer(Deployer):
)
def get_tls_deployer(config, mail_domain):
"""Select the appropriate TLS deployer based on config."""
tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"]
if config.tls_cert_mode == "acme":
return AcmetoolDeployer(config.acme_email, tls_domains)
elif config.tls_cert_mode == "self":
return SelfSignedTlsDeployer(mail_domain)
elif config.tls_cert_mode == "external":
return ExternalTlsDeployer(config.tls_cert_path, config.tls_key_path)
else:
raise ValueError(f"Unknown tls_cert_mode: {config.tls_cert_mode}")
def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -> None:
"""Deploy a chat-mail instance.
@@ -567,44 +586,40 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
Out().red(f"Deploy failed: mtail_address {config.mtail_address} is not available (VPN up?).\n")
exit(1)
port_services = [
(["master", "smtpd"], 25),
("unbound", 53),
]
if config.tls_cert_mode == "acme":
port_services.append(("acmetool", 80))
port_services += [
(["imap-login", "dovecot"], 143),
("nginx", 443),
(["master", "smtpd"], 465),
(["master", "smtpd"], 587),
(["imap-login", "dovecot"], 993),
("iroh-relay", 3340),
("mtail", 3903),
("stats", 3904),
("nginx", 8443),
(["master", "smtpd"], config.postfix_reinject_port),
(["master", "smtpd"], config.postfix_reinject_port_incoming),
("filtermail", config.filtermail_smtp_port),
("filtermail", config.filtermail_smtp_port_incoming),
]
for service, port in port_services:
print(f"Checking if port {port} is available for {service}...")
running_service = host.get_fact(Port, port=port)
services = [service] if isinstance(service, str) else service
if running_service:
if running_service not in services:
Out().red(
f"Deploy failed: port {port} is occupied by: {running_service}"
)
exit(1)
if not os.environ.get("CHATMAIL_NOPORTCHECK"):
port_services = [
(["master", "smtpd"], 25),
("unbound", 53),
]
if config.tls_cert_mode == "acme":
port_services.append(("acmetool", 80))
port_services += [
(["imap-login", "dovecot"], 143),
("nginx", 443),
(["master", "smtpd"], 465),
(["master", "smtpd"], 587),
(["imap-login", "dovecot"], 993),
("iroh-relay", 3340),
("mtail", 3903),
("stats", 3904),
("nginx", 8443),
(["master", "smtpd"], config.postfix_reinject_port),
(["master", "smtpd"], config.postfix_reinject_port_incoming),
("filtermail", config.filtermail_smtp_port),
("filtermail", config.filtermail_smtp_port_incoming),
]
for service, port in port_services:
print(f"Checking if port {port} is available for {service}...")
running_service = host.get_fact(Port, port=port)
services = [service] if isinstance(service, str) else service
if running_service:
if running_service not in services:
Out().red(
f"Deploy failed: port {port} is occupied by: {running_service}"
)
exit(1)
tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"]
if config.tls_cert_mode == "acme":
tls_deployer = AcmetoolDeployer(config.acme_email, tls_domains)
else:
tls_deployer = SelfSignedTlsDeployer(mail_domain)
tls_deployer = get_tls_deployer(config, mail_domain)
all_deployers = [
ChatmailDeployer(mail_domain),

View File

@@ -1,3 +1,5 @@
import os
from chatmaild.config import Config
from pyinfra import host
from pyinfra.facts.server import Arch, Sysctl
@@ -9,6 +11,7 @@ from cmdeploy.basedeploy import (
activate_remote_units,
configure_remote_units,
get_resource,
has_systemd,
)
@@ -22,10 +25,11 @@ class DovecotDeployer(Deployer):
def install(self):
arch = host.get_fact(Arch)
if not host.get_fact(SystemdEnabled).get("dovecot.service"):
_install_dovecot_package("core", arch)
_install_dovecot_package("imapd", arch)
_install_dovecot_package("lmtpd", arch)
if has_systemd() and "dovecot.service" in host.get_fact(SystemdEnabled):
return # already installed and running
_install_dovecot_package("core", arch)
_install_dovecot_package("imapd", arch)
_install_dovecot_package("lmtpd", arch)
def configure(self):
configure_remote_units(self.config.mail_domain, self.units)
@@ -116,18 +120,19 @@ def _configure_dovecot(config: Config, debug: bool = False) -> (bool, bool):
# as per https://doc.dovecot.org/2.3/configuration_manual/os/
# it is recommended to set the following inotify limits
for name in ("max_user_instances", "max_user_watches"):
key = f"fs.inotify.{name}"
if host.get_fact(Sysctl)[key] > 65535:
# Skip updating limits if already sufficient
# (enables running in incus containers where sysctl readonly)
continue
server.sysctl(
name=f"Change {key}",
key=key,
value=65535,
persist=True,
)
if not os.environ.get("CHATMAIL_NOSYSCTL"):
for name in ("max_user_instances", "max_user_watches"):
key = f"fs.inotify.{name}"
if host.get_fact(Sysctl)[key] > 65535:
# Skip updating limits if already sufficient
# (enables running in incus containers where sysctl readonly)
continue
server.sysctl(
name=f"Change {key}",
key=key,
value=65535,
persist=True,
)
timezone_env = files.line(
name="Set TZ environment variable",

View File

@@ -0,0 +1,69 @@
from pyinfra.operations import files, server, systemd
from cmdeploy.basedeploy import Deployer, get_resource
class ExternalTlsDeployer(Deployer):
"""Expects TLS certificates to be managed on the server.
Validates that the configured certificate and key files
exist on the remote host. Installs a systemd path unit
that watches the certificate file and automatically
restarts/reloads affected services when it changes.
"""
def __init__(self, cert_path, key_path):
self.cert_path = cert_path
self.key_path = key_path
def configure(self):
server.shell(
name="Verify external TLS certificate and key exist",
commands=[
f"test -f {self.cert_path} && test -f {self.key_path}",
],
)
# Deploy the .path unit (templated with the cert path).
source = get_resource("tls-cert-reload.path.f", pkg=__package__)
content = source.read_text().format(cert_path=self.cert_path).encode()
import io
path_unit = files.put(
name="Upload tls-cert-reload.path",
src=io.BytesIO(content),
dest="/etc/systemd/system/tls-cert-reload.path",
user="root",
group="root",
mode="644",
)
service_unit = files.put(
name="Upload tls-cert-reload.service",
src=get_resource("tls-cert-reload.service", pkg=__package__),
dest="/etc/systemd/system/tls-cert-reload.service",
user="root",
group="root",
mode="644",
)
if path_unit.changed or service_unit.changed:
self.need_restart = True
def activate(self):
systemd.service(
name="Enable tls-cert-reload path watcher",
service="tls-cert-reload.path",
running=True,
enabled=True,
restarted=self.need_restart,
daemon_reload=self.need_restart,
)
# Always trigger a reload so services pick up the current cert.
# The path unit handles future changes via inotify.
server.shell(
name="Reload TLS services for current certificate",
commands=["systemctl start tls-cert-reload.service"],
)

View File

@@ -0,0 +1,11 @@
# Watch the TLS certificate file for changes.
# When the cert is updated (e.g. renewed by an external process),
# this triggers tls-cert-reload.service to restart the affected services.
[Unit]
Description=Watch TLS certificate for changes
[Path]
PathChanged={cert_path}
[Install]
WantedBy=multi-user.target

View File

@@ -0,0 +1,15 @@
# Reload services that cache the TLS certificate.
#
# dovecot: caches the cert at startup; reload re-reads SSL certs
# without dropping existing connections.
# nginx: caches the cert at startup; reload gracefully picks up
# the new cert for new connections.
# postfix: reads the cert fresh on each TLS handshake,
# does NOT need a reload/restart.
[Unit]
Description=Reload TLS services after certificate change
[Service]
Type=oneshot
ExecStart=/bin/systemctl reload dovecot
ExecStart=/bin/systemctl reload nginx

View File

@@ -84,7 +84,7 @@ http {
}
location /new {
{% if config.tls_cert_mode == "acme" %}
{% if config.tls_cert_mode != "self" %}
if ($request_method = GET) {
# Redirect to Delta Chat,
# which will in turn do a POST request.
@@ -106,7 +106,7 @@ http {
#
# Redirects are only for browsers.
location /cgi-bin/newemail.py {
{% if config.tls_cert_mode == "acme" %}
{% if config.tls_cert_mode != "self" %}
if ($request_method = GET) {
return 301 dcaccount:https://{{ config.mail_domain }}/new;
}

View File

@@ -1,8 +1,29 @@
from pyinfra.operations import apt, files, server
import shlex
from pyinfra.operations import apt, server
from cmdeploy.basedeploy import Deployer
def openssl_selfsigned_args(domain, cert_path, key_path, days=36500):
"""Return the openssl argument list for a self-signed certificate.
The certificate uses an EC P-256 key with SAN entries for *domain*,
``www.<domain>`` and ``mta-sts.<domain>``.
"""
return [
"openssl", "req", "-x509",
"-newkey", "ec", "-pkeyopt", "ec_paramgen_curve:P-256",
"-noenc", "-days", str(days),
"-keyout", str(key_path),
"-out", str(cert_path),
"-subj", f"/CN={domain}",
"-addext", "extendedKeyUsage=serverAuth,clientAuth",
"-addext",
f"subjectAltName=DNS:{domain},DNS:www.{domain},DNS:mta-sts.{domain}",
]
class SelfSignedTlsDeployer(Deployer):
"""Generates a self-signed TLS certificate for all chatmail endpoints."""
@@ -18,18 +39,13 @@ class SelfSignedTlsDeployer(Deployer):
)
def configure(self):
args = openssl_selfsigned_args(
self.mail_domain, self.cert_path, self.key_path,
)
cmd = shlex.join(args)
server.shell(
name="Generate self-signed TLS certificate if not present",
commands=[
f"[ -f {self.cert_path} ] || openssl req -x509"
f" -newkey ec -pkeyopt ec_paramgen_curve:P-256"
f" -noenc -days 36500"
f" -keyout {self.key_path}"
f" -out {self.cert_path}"
f' -subj "/CN={self.mail_domain}"'
f' -addext "extendedKeyUsage=serverAuth,clientAuth"'
f' -addext "subjectAltName=DNS:{self.mail_domain},DNS:www.{self.mail_domain},DNS:mta-sts.{self.mail_domain}"',
],
commands=[f"[ -f {self.cert_path} ] || {cmd}"],
)
def activate(self):

View File

@@ -89,6 +89,11 @@ class LocalExec:
self.verbose = verbose
self.docker = docker
def __call__(self, call, kwargs=None, log_callback=None):
if kwargs is None:
kwargs = {}
return call(**kwargs)
def logged(self, call, kwargs: dict):
where = "locally"
if self.docker:

View File

@@ -0,0 +1,362 @@
"""Setup and verify external TLS certificates for a chatmail server.
Generates a self-signed TLS certificate, uploads it to the chatmail
server via SCP, runs ``cmdeploy run``, and then probes all TLS-enabled
ports (nginx, postfix, dovecot) to verify the certificate is actually
served. After probing, checks remote service logs for errors.
Prerequisites
~~~~~~~~~~~~~
- SSH root access to the target server (same as ``cmdeploy run``)
- ``cmdeploy`` in PATH (activate the venv first)
How to run
~~~~~~~~~~
From the repository root::
# Full run: generate cert, deploy, probe ports, check services
python -m cmdeploy.tests.setup_tls_external DOMAIN
# Re-probe only (after a previous deploy)
python -m cmdeploy.tests.setup_tls_external DOMAIN \\
--skip-deploy --skip-certgen
# Override SSH host (e.g. when domain doesn't resolve to the server)
python -m cmdeploy.tests.setup_tls_external DOMAIN \\
--ssh-host staging-ipv4.testrun.org
Arguments
~~~~~~~~~
DOMAIN mail domain for the chatmail server (SSH root login must work)
Options
~~~~~~~
--skip-deploy skip ``cmdeploy run``, only probe ports
--skip-certgen skip cert generation/upload, use certs already on server
--ssh-host HOST SSH host override (defaults to DOMAIN)
"""
import argparse
import shutil
import smtplib
import socket
import ssl
import subprocess
import sys
import tempfile
import time
from pathlib import Path
# Cert paths on the remote server
REMOTE_CERT = "/etc/ssl/certs/tmp_fullchain.pem"
REMOTE_KEY = "/etc/ssl/private/tmp_privkey.pem"
# ---------------------------------------------------------------------------
# Config generation
# ---------------------------------------------------------------------------
def generate_config(domain: str, config_dir: Path) -> Path:
"""Generate a chatmail.ini with tls_external_cert_and_key for *domain*."""
from chatmaild.config import write_initial_config
ini_path = config_dir / "chatmail.ini"
write_initial_config(
ini_path,
domain,
overrides={
"tls_external_cert_and_key": f"{REMOTE_CERT} {REMOTE_KEY}",
},
)
print(f"[+] Generated chatmail.ini for {domain} in {config_dir}")
return ini_path
# ---------------------------------------------------------------------------
# Certificate generation
# ---------------------------------------------------------------------------
def generate_cert(domain: str, cert_dir: Path) -> tuple:
"""Generate a self-signed TLS cert+key for *domain* with proper SANs."""
from cmdeploy.selfsigned.deployer import openssl_selfsigned_args
cert_path = cert_dir / "fullchain.pem"
key_path = cert_dir / "privkey.pem"
subprocess.check_call(openssl_selfsigned_args(domain, cert_path, key_path, days=30))
print(f"[+] Generated cert for {domain} in {cert_dir}")
return cert_path, key_path
# ---------------------------------------------------------------------------
# Upload certs to remote server
# ---------------------------------------------------------------------------
def upload_certs(
ssh_host: str,
cert_path: Path,
key_path: Path,
) -> None:
"""SCP cert and key to the remote server."""
subprocess.check_call([
"scp", str(cert_path), f"root@{ssh_host}:{REMOTE_CERT}",
])
subprocess.check_call([
"scp", str(key_path), f"root@{ssh_host}:{REMOTE_KEY}",
])
# Ensure cert is world-readable and key is readable by ssl-cert group
# (dovecot/postfix/nginx need to read these files)
subprocess.check_call([
"ssh", f"root@{ssh_host}",
f"chmod 644 {REMOTE_CERT} && chmod 640 {REMOTE_KEY}"
f" && chgrp ssl-cert {REMOTE_KEY}",
])
print(f"[+] Uploaded cert/key to {ssh_host}")
# ---------------------------------------------------------------------------
# Deploy
# ---------------------------------------------------------------------------
def run_deploy(ini_path: str) -> None:
"""Run ``cmdeploy run --skip-dns-check --config <ini>``."""
cmd = ["cmdeploy", "run", "--config", str(ini_path), "--skip-dns-check"]
print(f"[+] Running: {' '.join(cmd)}")
subprocess.check_call(cmd)
print("[+] Deploy completed successfully")
# ---------------------------------------------------------------------------
# TLS port probing
# ---------------------------------------------------------------------------
def get_peer_cert_binary(host: str, port: int) -> bytes:
"""Connect to host:port with TLS and return the DER-encoded peer cert."""
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
with socket.create_connection((host, port), timeout=15) as sock:
with ctx.wrap_socket(sock, server_hostname=host) as ssock:
return ssock.getpeercert(binary_form=True)
def get_smtp_starttls_cert_binary(host: str, port: int = 587) -> bytes:
"""Connect via SMTP STARTTLS and return the DER cert."""
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
with smtplib.SMTP(host, port, timeout=15) as smtp:
smtp.starttls(context=ctx)
return smtp.sock.getpeercert(binary_form=True)
def check_cert_matches(
label: str, served_der: bytes, expected_der: bytes,
) -> bool:
"""Compare served DER cert against the expected cert."""
if served_der == expected_der:
print(f" [OK] {label}: certificate matches")
return True
else:
print(f" [FAIL] {label}: certificate does NOT match")
return False
def load_cert_der(cert_pem_path: Path) -> bytes:
"""Load a PEM cert file and return its DER encoding."""
pem_text = cert_pem_path.read_text()
start = pem_text.index("-----BEGIN CERTIFICATE-----")
end = pem_text.index("-----END CERTIFICATE-----") + len(
"-----END CERTIFICATE-----"
)
return ssl.PEM_cert_to_DER_cert(pem_text[start:end])
def probe_all_ports(host: str, expected_cert_der: bytes) -> bool:
"""Probe TLS ports and verify the served certificate matches.
Checks ports 993 (IMAP), 465 (SMTPS), 587 (STARTTLS), and 443
(nginx stream). Port 8443 is skipped as nginx binds it to
localhost behind the stream proxy on 443.
"""
print(f"\n[+] Probing TLS ports on {host}...")
all_ok = True
for label, port in [
("IMAP/TLS (993)", 993),
("SMTP/TLS (465)", 465),
]:
try:
served = get_peer_cert_binary(host, port)
if not check_cert_matches(label, served, expected_cert_der):
all_ok = False
except Exception as e:
print(f" [FAIL] {label}: connection failed: {e}")
all_ok = False
# STARTTLS on port 587
try:
served = get_smtp_starttls_cert_binary(host, 587)
if not check_cert_matches("SMTP/STARTTLS (587)", served, expected_cert_der):
all_ok = False
except Exception as e:
print(f" [FAIL] SMTP/STARTTLS (587): connection failed: {e}")
all_ok = False
# Port 443 (nginx stream proxy with ALPN routing)
try:
served = get_peer_cert_binary(host, 443)
if not check_cert_matches("nginx/443 (stream)", served, expected_cert_der):
all_ok = False
except Exception as e:
print(f" [FAIL] nginx/443 (stream): connection failed: {e}")
all_ok = False
return all_ok
# ---------------------------------------------------------------------------
# Post-deploy service health checks
# ---------------------------------------------------------------------------
SERVICES = ["dovecot", "postfix", "nginx"]
def check_remote_services(ssh_host: str, since: str = "") -> bool:
"""SSH to the server and check for service failures or errors.
*since* is a ``journalctl --since`` timestamp (e.g. ``"5 min ago"``).
If empty, checks the entire boot journal.
"""
print(f"\n[+] Checking remote service health on {ssh_host}...")
all_ok = True
for svc in SERVICES:
try:
result = subprocess.run(
["ssh", f"root@{ssh_host}",
f"systemctl is-active {svc}.service"],
capture_output=True, text=True, timeout=15, check=False,
)
status = result.stdout.strip()
if status == "active":
print(f" [OK] {svc}: active")
else:
print(f" [FAIL] {svc}: {status}")
all_ok = False
except Exception as e:
print(f" [FAIL] {svc}: check failed: {e}")
all_ok = False
since_arg = f'--since="{since}"' if since else ""
print(f"\n[+] Checking journal for errors on {ssh_host}...")
for svc in SERVICES:
try:
result = subprocess.run(
["ssh", f"root@{ssh_host}",
f"journalctl -u {svc}.service {since_arg}"
f" --no-pager -p err -q"],
capture_output=True, text=True, timeout=15, check=False,
)
errors = result.stdout.strip()
if errors:
print(f" [WARN] {svc} errors in journal:")
for line in errors.splitlines()[:10]:
print(f" {line}")
all_ok = False
else:
print(f" [OK] {svc}: no errors in journal")
except Exception as e:
print(f" [FAIL] {svc}: journal check failed: {e}")
all_ok = False
return all_ok
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def main():
parser = argparse.ArgumentParser(
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"domain",
help="mail domain (SSH root login must work to this host)",
)
parser.add_argument(
"--skip-deploy",
action="store_true",
help="skip cmdeploy run, only probe ports",
)
parser.add_argument(
"--skip-certgen",
action="store_true",
help="skip cert generation and upload (use existing)",
)
parser.add_argument(
"--ssh-host",
help="SSH host override (defaults to DOMAIN)",
)
args = parser.parse_args()
domain = args.domain
ssh_host = args.ssh_host or domain
print(f"[+] Domain: {domain}")
print(f"[+] SSH host: {ssh_host}")
print(f"[+] Remote cert: {REMOTE_CERT}")
print(f"[+] Remote key: {REMOTE_KEY}")
work_dir = Path(tempfile.mkdtemp(prefix="tls-external-test-"))
try:
# Generate chatmail.ini
ini_path = generate_config(domain, work_dir)
if not args.skip_certgen:
local_cert, local_key = generate_cert(domain, work_dir)
upload_certs(ssh_host, local_cert, local_key)
else:
local_cert = work_dir / "fullchain.pem"
subprocess.check_call([
"scp", f"root@{ssh_host}:{REMOTE_CERT}", str(local_cert),
])
# Record timestamp before deploy for journal filtering
deploy_start = time.strftime("%Y-%m-%d %H:%M:%S")
if not args.skip_deploy:
run_deploy(ini_path)
# Probe TLS ports
expected_der = load_cert_der(local_cert)
ports_ok = probe_all_ports(domain, expected_der)
# Check service health (only errors since deploy started)
services_ok = check_remote_services(ssh_host, since=deploy_start)
if ports_ok and services_ok:
print(
"\n[SUCCESS] All TLS port probes passed and services are healthy"
)
return 0
else:
if not ports_ok:
print("\n[FAILURE] Some TLS port probes failed", file=sys.stderr)
if not services_ok:
print(
"\n[FAILURE] Some services have errors", file=sys.stderr
)
return 1
finally:
shutil.rmtree(work_dir, ignore_errors=True)
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,78 @@
"""Functional tests for tls_external_cert_and_key option."""
import json
import chatmaild.newemail
import pytest
from chatmaild.config import read_config, write_initial_config
def make_external_config(tmp_path, cert_key=None):
inipath = tmp_path / "chatmail.ini"
overrides = {}
if cert_key is not None:
overrides["tls_external_cert_and_key"] = cert_key
write_initial_config(inipath, "chat.example.org", overrides=overrides)
return inipath
def test_external_tls_config_reads_paths(tmp_path):
inipath = make_external_config(
tmp_path,
cert_key=(
"/etc/letsencrypt/live/chat.example.org/fullchain.pem"
" /etc/letsencrypt/live/chat.example.org/privkey.pem"
),
)
config = read_config(inipath)
assert config.tls_cert_mode == "external"
assert (
config.tls_cert_path == "/etc/letsencrypt/live/chat.example.org/fullchain.pem"
)
assert config.tls_key_path == "/etc/letsencrypt/live/chat.example.org/privkey.pem"
def test_external_tls_missing_option_uses_acme(tmp_path):
config = read_config(make_external_config(tmp_path))
assert config.tls_cert_mode == "acme"
def test_external_tls_bad_format_raises(tmp_path):
inipath = make_external_config(tmp_path, cert_key="/only/one/path.pem")
with pytest.raises(ValueError, match="two space-separated"):
read_config(inipath)
def test_external_tls_three_paths_raises(tmp_path):
inipath = make_external_config(tmp_path, cert_key="/a /b /c")
with pytest.raises(ValueError, match="two space-separated"):
read_config(inipath)
def test_external_tls_no_dclogin_url(tmp_path, capsys, monkeypatch):
inipath = make_external_config(
tmp_path, cert_key="/certs/fullchain.pem /certs/privkey.pem"
)
monkeypatch.setattr(chatmaild.newemail, "CONFIG_PATH", str(inipath))
chatmaild.newemail.print_new_account()
out, _ = capsys.readouterr()
lines = out.split("\n")
dic = json.loads(lines[2])
assert "dclogin_url" not in dic
def test_external_tls_selects_correct_deployer(tmp_path):
from cmdeploy.deployers import get_tls_deployer
from cmdeploy.external.deployer import ExternalTlsDeployer
from cmdeploy.selfsigned.deployer import SelfSignedTlsDeployer
inipath = make_external_config(
tmp_path, cert_key="/certs/fullchain.pem /certs/privkey.pem"
)
config = read_config(inipath)
deployer = get_tls_deployer(config, "chat.example.org")
assert isinstance(deployer, ExternalTlsDeployer)
assert not isinstance(deployer, SelfSignedTlsDeployer)
assert deployer.cert_path == "/certs/fullchain.pem"
assert deployer.key_path == "/certs/privkey.pem"

260
doc/source/docker.rst Normal file
View File

@@ -0,0 +1,260 @@
Docker installation
===================
This section provides instructions for installing a chatmail relay
using Docker Compose.
.. note::
- Docker support is experimental and not yet covered by automated tests, please report bugs.
- This preliminary image simply wraps the cmdeploy process detailed in the :doc:`getting_started` instructions in a full Debian-systemd image with r/w access to `/sys/fs`
- Currently, the image has only been tested and built on amd64, though arm64 should theoretically work as well.
Setup Preparation
-----------------
We use ``chat.example.org`` as the chatmail domain in the following
steps. Please substitute it with your own domain.
1. Install docker and docker compose v2 (check with `docker compose version`), install, e.g., through
- Debian 12 through the `official install instructions <https://docs.docker.com/engine/install/debian/#install-using-the-repository>`_
- Debian 13+ with `apt install docker docker-compose`
If you must use v1 (EOL since 2023), use `docker-compose` in the following and modify the `docker-compose.yaml` to use `privileged: true` instead of `cgroup: host`, though that will run give the container all priviledges.
2. Setup the initial DNS records.
The following is an example in the familiar BIND zone file format with
a TTL of 1 hour (3600 seconds).
Please substitute your domain and IP addresses.
::
chat.example.org. 3600 IN A 198.51.100.5
chat.example.org. 3600 IN AAAA 2001:db8::5
www.chat.example.org. 3600 IN CNAME chat.example.org.
mta-sts.chat.example.org. 3600 IN CNAME chat.example.org.
3. Configure kernel parameters on the host, as these can not be set from the container::
echo "fs.inotify.max_user_instances=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
echo "fs.inotify.max_user_watches=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
sudo sysctl --system
Docker Compose Setup
--------------------
Pre-built images are available from GitHub Container Registry. The
``main`` branch and tagged releases are pushed automatically by CI::
docker pull ghcr.io/chatmail/relay:main # latest main branch
docker pull ghcr.io/chatmail/relay:1.2.3 # tagged release
Create service directory
^^^^^^^^^^^^^^^^^^^^^^^^
Either:
- Create a service directory, e.g., `/srv/chatmail-relay`::
mkdir -p /srv/chatmail-relay && cd /srv/chatmail-relay
wget https://raw.githubusercontent.com/chatmail/relay/refs/heads/main/docker-compose.yaml
wget https://raw.githubusercontent.com/chatmail/relay/refs/heads/main/docker-compose.override.yaml.example -O docker-compose.override.yaml
- or clone the chatmail repo ::
git clone https://github.com/chatmail/relay
cd relay
Customize and start
^^^^^^^^^^^^^^^^^^^
1. Set the fully qualified domain name of the relay::
echo 'MAIL_DOMAIN=chat.example.org' > .env
The container generates a ``chatmail.ini`` with defaults from
``MAIL_DOMAIN`` on first start. To customize chatmail settings, mount
your own ``chatmail.ini`` instead (see `Custom chatmail.ini`_ below).
2. All local customizations (data paths, extra volumes, config mounts) go in
``docker-compose.override.yaml``, which Compose merges automatically with
the base file. By default, all data is stored in docker volumes, you will
likely want to at least create and configure the mail storage location, but
you might also want to configure external TLS certificates there.
3. Start the container::
docker compose up -d
docker compose logs -f chatmail # view logs, Ctrl+C to exit
4. After installation is complete, open ``https://chat.example.org`` in
your browser.
Finish install and test
-----------------------
You can test the installation with::
pip install cmping chat.example.org # or
uvx cmping chat.example.org # if you use https://docs.astral.sh/uv/
You should check and extend your DNS records for better interoperability::
# Show required DNS records
docker exec chatmail /opt/cmdeploy/bin/cmdeploy dns --ssh-host @local
You can check server status with::
docker exec chatmail /opt/cmdeploy/bin/cmdeploy status --ssh-host @local
You can run some benchmarks (can also run from any machine with cmdeploy installed)
docker exec chatmail /opt/cmdeploy/bin/cmdeploy bench chat.example.org
You can run the test suite with
docker exec chatmail /opt/cmdeploy/bin/cmdeploy test chat.example.org --ssh-host localhost
Customization
-------------
Website
^^^^^^^^^^^^^^
You can customize the chatmail landing page by mounting a directory with
your own website source files.
1. Create a directory with your custom website source::
mkdir -p ./custom/www/src
nano ./custom/www/src/index.md
2. Add the volume mount in ``docker-compose.override.yaml``::
services:
chatmail:
volumes:
- ./custom/www:/opt/chatmail-www
3. Restart the service::
docker compose down
docker compose up -d
Custom chatmail.ini
^^^^^^^^^^^^^^^^^^^
If you want to go beyond simply setting the ``MAIL_DOMAIN`` in ``.env``, you
can use a regular `chatmail.ini` to give you full control.
1. Extract the generated config from a running container::
docker cp chatmail:/etc/chatmail/chatmail.ini ./chatmail.ini
2. Edit ``chatmail.ini`` as needed.
3. Add the volume mount in ``docker-compose.override.yaml`` ::
services:
chatmail:
volumes:
- ./chatmail.ini:/etc/chatmail/chatmail.ini
4. Restart the container, the container skips generating a new one: ::
docker compose down && docker compose up -d
External TLS certificates
^^^^^^^^^^^^^^^^^^^^^^^^^
If TLS certificates are managed outside the container (e.g. by certbot,
acmetool, or Traefik on the host), mount them into the container and set
``TLS_EXTERNAL_CERT_AND_KEY`` in ``docker-compose.override.yaml``.
Changed certificates are picked up automatically via inotify.
See the examples in the example override and :ref:`external-tls` in the getting started guide for details.
Migrating from a bare-metal install
------------------------------------
If you have an existing bare-metal chatmail installation and want to
switch to Docker:
1. Stop all existing services::
systemctl stop postfix dovecot doveauth nginx opendkim unbound \
acmetool-redirector filtermail filtermail-incoming chatmail-turn \
iroh-relay chatmail-metadata lastlogin mtail
systemctl disable postfix dovecot doveauth nginx opendkim unbound \
acmetool-redirector filtermail filtermail-incoming chatmail-turn \
iroh-relay chatmail-metadata lastlogin mtail
2. Copy your existing ``chatmail.ini`` and mount it into the container
(see `Custom chatmail.ini`_ above)::
cp /usr/local/lib/chatmaild/chatmail.ini ./chatmail.ini
3. Copy persistent data into the ``./data/`` subdirectories (for example, as configured in `Customize and start`_) ::
mkdir -p data/dkim data/certs data/mail
# DKIM keys
cp -a /etc/dkimkeys/* data/dkim/
# TLS certificates
rsync -a /var/lib/acme/ data/certs/
Note that ownership of dkim and acme is adjusted on container start.
For the mail directory::
rsync -a /home/vmail/ data/mail/
Alternatively, mount ``/home/vmail`` directly by changing the volume
in ``docker-compose-override.yaml``::
- /home/vmail:/home/vmail
The three ``./data/`` subdirectories cover all persistent state.
Everything else is regenerated by the ``configure`` and ``activate``
stages on container start.
Building the image
------------------
Clone the repository and build the Docker image::
git clone https://github.com/chatmail/relay
cd relay
docker compose build chatmail
The build bakes all binaries, Python packages, and the install stage
into the image. After building, only ``docker-compose.yaml`` and a ``.env`` with
``MAIL_DOMAIN`` are needed to run the container.
You can transfer a locally built image to your server directly (pigz is parallel `gzip` which can be used instead as well) ::
docker save chatmail-relay:latest | pigz | ssh chat.example.org 'pigz -d | docker load'
Forcing a full reinstall
------------------------
On container start, only the ``configure`` and ``activate`` stages run by default.
To force a full reinstall (e.g. after updating the source), either
rebuild the image::
docker compose build chatmail
docker compose up -d
Or override the stages at runtime without rebuilding::
CMDEPLOY_STAGES="install,configure,activate" docker compose up -d

View File

@@ -98,6 +98,12 @@ steps. Please substitute it with your own domain.
configure at your DNS provider (it can take some time until they are
public).
Docker installation
-------------------
There is experimental support for running chatmail via Docker Compose.
See :doc:`docker` for full setup instructions.
Other helpful commands
----------------------
@@ -198,6 +204,40 @@ and all other relays will accept connections from it
without requiring certificate verification.
This is useful for experimental setups and testing.
.. _external-tls:
Running a relay with externally managed certificates
-----------------------------------------------------
If you already have a TLS certificate manager
(e.g. Traefik, certbot, or another ACME client)
running on the deployment server,
you can configure the relay to use those certificates
instead of the built-in ``acmetool``.
Set the following in ``chatmail.ini``::
tls_external_cert_and_key = /path/to/fullchain.pem /path/to/privkey.pem
The paths must point to certificate and key files
on the deployment server.
During ``cmdeploy run``, these paths are written into
the Postfix, Dovecot, and Nginx configurations.
No certificate files are transferred from the build machine —
they must already exist on the server,
managed by your external certificate tool.
The deploy will verify that both files exist on the server.
``acmetool`` is **not** installed or run in this mode.
.. note::
You are responsible for certificate renewal.
When the certificate file changes on disk,
all relay services pick up the new certificate automatically
(via a systemd path watcher installed during deploy).
Migrating to a new build machine
----------------------------------

View File

@@ -13,6 +13,7 @@ Contributions and feedback welcome through the https://github.com/chatmail/relay
:maxdepth: 5
getting_started
docker
proxy
migrate
overview

View File

@@ -308,6 +308,11 @@ When providing a TLS certificate to your chatmail relay server, make
sure to provide the full certificate chain and not just the last
certificate.
If you use an external certificate manager (e.g. Traefik or certbot),
set ``tls_external_cert_and_key`` in ``chatmail.ini``
to provide the certificate and key paths.
See :ref:`external-tls` for details.
If you are running an Exim server and dont see incoming connections
from a chatmail relay server in the logs, make sure ``smtp_no_mail`` log
item is enabled in the config with ``log_selector = +smtp_no_mail``. By

View File

@@ -0,0 +1,40 @@
# Local overrides — copy to docker-compose.override.yaml in the repo root.
# Compose automatically merges this with docker-compose.yaml.
#
# cp docker-compose.override.yaml.example docker-compose.override.yaml
#
# Volumes are APPENDED to the base file's volumes list.
# Environment and other scalar keys are MERGED by key.
services:
chatmail:
volumes:
## Data paths — bind-mount to host directories for easy access/backup.
# - ./data/dkim:/etc/dkimkeys
# - ./data/certs:/var/lib/acme
# - ./data/mail:/home/vmail
## Or mount from an existing bare-metal install.
# - /home/vmail:/home/vmail
## Mount your own chatmail.ini (skips auto-generation):
# - ./chatmail.ini:/etc/chatmail/chatmail.ini
## Custom website:
# - ./custom/www:/opt/chatmail-www
## Debug — mount scripts from the repo for live editing:
# - ./docker/files/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
# - ./docker/files/entrypoint.sh:/entrypoint.sh
# environment:
## Mount certs (above) and set TLS_EXTERNAL_CERT_AND_KEY to in-container paths.
## Changed certs are picked up automatically (inotify via tls-cert-reload.path).
##
## Host acmetool (bare-metal migration): create mount above, and
## rsync -a /var/lib/acme/live data/certs
# TLS_EXTERNAL_CERT_AND_KEY: "/var/lib/acme/live/${MAIL_DOMAIN}/fullchain /var/lib/acme/live/${MAIL_DOMAIN}/privkey"
##
## (Untested) Traefik certs-dumper (see docker/docker-compose-traefik.yaml) - also add volume:
## - traefik-certs:/certs:ro
# TLS_EXTERNAL_CERT_AND_KEY: "/certs/${MAIL_DOMAIN}/certificate.crt /certs/${MAIL_DOMAIN}/privatekey.key"

47
docker-compose.yaml Normal file
View File

@@ -0,0 +1,47 @@
# Base compose file — do not edit. Put customizations (data paths, extra
# volumes, env overrides) in docker-compose.override.yaml instead.
# See docker/docker-compose.override.yaml.example for a starting point.
#
# Security note: this container uses network_mode:host (chatmail needs many
# ports: 25, 53, 80, 143, 443, 465, 587, 993, 3340, 8443) and cgroup:host
# (required for systemd). Together these give the container near-host-level
# access. This is acceptable for a dedicated mail server, but be aware that
# the container can bind any port and see all host network traffic.
services:
chatmail:
build:
context: ./
dockerfile: docker/chatmail_relay.dockerfile
args:
GIT_HASH: ${GIT_HASH:-unknown}
image: chatmail-relay:latest
restart: unless-stopped
container_name: chatmail
# Required for systemd — use only one of the following:
cgroup: host # compose v2
# privileged: true # compose v1 (less restricted)
tty: true # required for logs
tmpfs: # required for systemd
- /tmp
- /run
- /run/lock
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
environment:
MAIL_DOMAIN: $MAIL_DOMAIN
network_mode: "host"
volumes:
## system (required)
- /sys/fs/cgroup:/sys/fs/cgroup:rw
## data (defaults — override in docker-compose.override.yaml)
- mail:/home/vmail
- dkim:/etc/dkimkeys
- certs:/var/lib/acme
volumes:
mail:
dkim:
certs:

9
docker/build.sh Executable file
View File

@@ -0,0 +1,9 @@
#!/bin/sh
# Build the chatmail Docker image with the current git hash baked in.
# Usage: ./docker/build.sh [extra docker-compose build args...]
#
# .git/ is excluded from the build context (.dockerignore) so the hash
# must be passed as a build arg from the host.
export GIT_HASH=$(git rev-parse --short HEAD)
exec docker compose build "$@"

View File

@@ -0,0 +1,99 @@
FROM jrei/systemd-debian:12 AS base
ENV LANG=en_US.UTF-8
RUN echo 'APT::Install-Recommends "0";' > /etc/apt/apt.conf.d/01norecommend && \
echo 'APT::Install-Suggests "0";' >> /etc/apt/apt.conf.d/01norecommend && \
apt-get update && \
apt-get install -y \
ca-certificates && \
DEBIAN_FRONTEND=noninteractive \
TZ=UTC \
apt-get install -y tzdata && \
apt-get install -y locales && \
sed -i -e "s/# $LANG.*/$LANG UTF-8/" /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=$LANG \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && \
apt-get install -y \
git \
python3 \
python3-venv \
python3-virtualenv \
gcc \
python3-dev \
opendkim \
opendkim-tools \
curl \
rsync \
unbound \
unbound-anchor \
dnsutils \
postfix \
acl \
nginx \
libnginx-mod-stream \
fcgiwrap \
cron \
&& rm -rf /var/lib/apt/lists/*
# --- Build-time: install cmdeploy venv and run install stage ---
# Editable install so importlib.resources reads directly from the source tree.
# On container start only "configure,activate" stages run.
COPY . /opt/chatmail/
WORKDIR /opt/chatmail
RUN printf '[params]\nmail_domain = build.local\n' > /tmp/chatmail.ini
# Dummy git repo init: .git/ is excluded from the build context (.dockerignore)
# but setuptools calls `git ls-files` when building the sdist.
RUN git init -q && \
python3 -m venv /opt/cmdeploy && \
/opt/cmdeploy/bin/pip install --no-cache-dir \
-e chatmaild/ -e cmdeploy/
RUN CMDEPLOY_STAGES=install \
CHATMAIL_INI=/tmp/chatmail.ini \
CHATMAIL_NOSYSCTL=True \
CHATMAIL_NOPORTCHECK=True \
/opt/cmdeploy/bin/pyinfra @local \
/opt/chatmail/cmdeploy/src/cmdeploy/run.py -y
RUN cp -a www/ /opt/chatmail-www/
RUN rm -f /tmp/chatmail.ini
# Record image version (used in deploy fingerprint at runtime).
# GIT_HASH is passed as a build arg (from docker-compose or CI) so that
# .git/ can be excluded from the build context via .dockerignore.
ARG GIT_HASH=unknown
RUN echo "$GIT_HASH" > /etc/chatmail-image-version && \
echo "$GIT_HASH" > /etc/chatmail-version
# --- End build-time install ---
ENV TZ=:/etc/localtime
ENV PATH="/opt/cmdeploy/bin:${PATH}"
RUN ln -s /etc/chatmail/chatmail.ini /opt/chatmail/chatmail.ini
ARG SETUP_CHATMAIL_SERVICE_PATH=/lib/systemd/system/setup_chatmail.service
COPY ./docker/files/setup_chatmail.service "$SETUP_CHATMAIL_SERVICE_PATH"
RUN ln -sf "$SETUP_CHATMAIL_SERVICE_PATH" "/etc/systemd/system/multi-user.target.wants/setup_chatmail.service"
# Remove default nginx site config at build time (not in entrypoint)
RUN rm -f /etc/nginx/sites-enabled/default
COPY --chmod=555 ./docker/files/setup_chatmail_docker.sh /setup_chatmail_docker.sh
COPY --chmod=555 ./docker/files/entrypoint.sh /entrypoint.sh
HEALTHCHECK --interval=60s --timeout=10s --retries=3 \
CMD systemctl is-active dovecot postfix nginx unbound opendkim filtermail doveauth chatmail-metadata || exit 1
STOPSIGNAL SIGRTMIN+3
ENTRYPOINT ["/entrypoint.sh"]
CMD [ "--default-standard-output=journal+console", \
"--default-standard-error=journal+console" ]

12
docker/files/entrypoint.sh Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
set -eo pipefail
SETUP_CHATMAIL_SERVICE_PATH="${SETUP_CHATMAIL_SERVICE_PATH:-/lib/systemd/system/setup_chatmail.service}"
# Whitelist only the env vars needed by setup_chatmail_docker.sh.
# Forwarding all env vars (via printenv) would leak Docker internals,
# orchestrator secrets, and other unrelated variables into systemd.
env_vars="MAIL_DOMAIN CMDEPLOY_STAGES CHATMAIL_INI TLS_EXTERNAL_CERT_AND_KEY PATH"
sed -i "s|<envs_list>|$env_vars|g" "$SETUP_CHATMAIL_SERVICE_PATH"
exec /lib/systemd/systemd "$@"

View File

@@ -0,0 +1,14 @@
[Unit]
Description=Run container setup commands
After=multi-user.target
ConditionPathExists=/setup_chatmail_docker.sh
[Service]
Type=oneshot
ExecStart=/bin/bash /setup_chatmail_docker.sh
RemainAfterExit=true
WorkingDirectory=/opt/chatmail
PassEnvironment=<envs_list>
[Install]
WantedBy=multi-user.target

View File

@@ -0,0 +1,63 @@
#!/bin/bash
set -euo pipefail
export CHATMAIL_INI="${CHATMAIL_INI:-/etc/chatmail/chatmail.ini}"
export CHATMAIL_NOSYSCTL=True
export CHATMAIL_NOPORTCHECK=True
CMDEPLOY=/opt/cmdeploy/bin/cmdeploy
if [ -z "$MAIL_DOMAIN" ]; then
echo "ERROR: Environment variable 'MAIL_DOMAIN' must be set!" >&2
exit 1
fi
### MAIN
if [ ! -f /etc/dkimkeys/opendkim.private ]; then
/usr/sbin/opendkim-genkey -D /etc/dkimkeys -d "$MAIL_DOMAIN" -s opendkim
fi
# Fix ownership for bind-mounted keys (host opendkim UID may differ from container)
chown -R opendkim:opendkim /etc/dkimkeys
# Journald: forward to console for docker logs
grep -q '^ForwardToConsole=yes' /etc/systemd/journald.conf \
|| echo "ForwardToConsole=yes" >> /etc/systemd/journald.conf
systemctl restart systemd-journald
# Create chatmail.ini (skips if file already exists, e.g. volume-mounted)
mkdir -p "$(dirname "$CHATMAIL_INI")"
if [ ! -f "$CHATMAIL_INI" ]; then
$CMDEPLOY init --config "$CHATMAIL_INI" "$MAIL_DOMAIN"
fi
# Inject external TLS paths from env var (unless user mounted their own ini)
if [ -n "${TLS_EXTERNAL_CERT_AND_KEY:-}" ]; then
if ! grep -q '^tls_external_cert_and_key' "$CHATMAIL_INI"; then
echo "tls_external_cert_and_key = $TLS_EXTERNAL_CERT_AND_KEY" >> "$CHATMAIL_INI"
fi
fi
# --- Deploy fingerprint: skip cmdeploy run if nothing changed ---
# On restart with identical image+config, systemd already brings up all
# enabled services — the full cmdeploy run is redundant (~30s saved).
# The install stage runs at image build time (Dockerfile), so only
# configure+activate are needed here.
IMAGE_VERSION_FILE="/etc/chatmail-image-version"
FINGERPRINT_FILE="/etc/chatmail/.deploy-fingerprint"
image_ver="none"
[ -f "$IMAGE_VERSION_FILE" ] && image_ver=$(cat "$IMAGE_VERSION_FILE")
config_hash=$(sha256sum "$CHATMAIL_INI" | cut -c1-16)
current_fp="${image_ver}:${config_hash}"
# CMDEPLOY_STAGES non-empty in env = operator override → always run.
# Otherwise, if fingerprint matches the last successful deploy, skip.
if [ -z "${CMDEPLOY_STAGES:-}" ] \
&& [ -f "$FINGERPRINT_FILE" ] \
&& [ "$(cat "$FINGERPRINT_FILE")" = "$current_fp" ]; then
echo "[INFO] No changes detected ($current_fp), skipping deploy."
else
export CMDEPLOY_STAGES="${CMDEPLOY_STAGES:-configure,activate}"
$CMDEPLOY run --config "$CHATMAIL_INI" --ssh-host @local
echo "$current_fp" > "$FINGERPRINT_FILE"
fi

1
env.example Normal file
View File

@@ -0,0 +1 @@
MAIL_DOMAIN=chat.example.com