docker: streamline overrides, rename datadirs, external TLS

also
- get rid of CHATMAIL_* in compose
- trim compose override example

doc/source/docker.rst

@@ -60,39 +60,32 @@ Either:
 - Create a service directory, e.g., `/srv/chatmail-relay`::
 
     mkdir -p /srv/chatmail-relay && cd /srv/chatmail-relay
-    wget https://raw.githubusercontent.com/chatmail/relay/refs/heads/main/docker-compose.yaml https://raw.githubusercontent.com/chatmail/relay/refs/heads/main/docker-compose.override.yaml.example
-    wget https://raw.githubusercontent.com/chatmail/relay/refs/heads/main/docker/env.example -O .env
-    
+    wget https://raw.githubusercontent.com/chatmail/relay/refs/heads/main/docker-compose.yaml
+    wget https://raw.githubusercontent.com/chatmail/relay/refs/heads/main/docker-compose.override.yaml.example -O docker-compose.override.yaml
 
 - or clone the chatmail repo ::
 
    git clone https://github.com/chatmail/relay
    cd relay
-   cp example.env .env
-
 
 
 Customize and start
 ^^^^^^^^^^^^^^^^^^^
 
-1. All local customizations (data paths, extra volumes, config mounts) go in
-   ``docker-compose.override.yaml``, which Compose merges automatically with
-   the base file. By default, all data is stored in docker volumes, you will
-   likely want to at least create and configure the mail storage location. Copy
-   the example to get started::
+1. Set the fully qualified domain name of the relay::
 
-       cp docker/docker-compose.override.yaml.example docker-compose.override.yaml
-       # and edit docker-compose.override.yaml
-
-
-
-2. Configure the ``.env`` file. Only ``MAIL_DOMAIN`` is required, the domain
-   name of the future server.
+       echo 'MAIL_DOMAIN=chat.example.org' > .env
 
    The container generates a ``chatmail.ini`` with defaults from
    ``MAIL_DOMAIN`` on first start. To customize chatmail settings, mount
    your own ``chatmail.ini`` instead (see `Custom chatmail.ini`_ below).
 
+2. All local customizations (data paths, extra volumes, config mounts) go in
+   ``docker-compose.override.yaml``, which Compose merges automatically with
+   the base file. By default, all data is stored in docker volumes, you will
+   likely want to at least create and configure the mail storage location, but
+   you might also want to configure external TLS certificates there.
+
 3. Start the container::
 
        docker compose up -d
@@ -101,26 +94,36 @@ Customize and start
 4. After installation is complete, open ``https://chat.example.org`` in
    your browser.
 
+Finish install and test
+-----------------------
 
-Managing the server
--------------------
+You can test the installation with::
 
-Use ``docker exec`` to run cmdeploy commands inside the container::
+        pip install cmping chat.example.org # or
+        uvx cmping chat.example.org # if you use https://docs.astral.sh/uv/
+
+You should check and extend your DNS records for better interoperability::
 
     # Show required DNS records
     docker exec chatmail /opt/cmdeploy/bin/cmdeploy dns --ssh-host @local
 
-    # Check server status
+You can check server status with::
+
     docker exec chatmail /opt/cmdeploy/bin/cmdeploy status --ssh-host @local
 
-    # Run benchmarks (can also run from any machine with cmdeploy installed)
+You can run some benchmarks (can also run from any machine with cmdeploy installed)
+
     docker exec chatmail /opt/cmdeploy/bin/cmdeploy bench chat.example.org
 
+You can run the test suite with
+
+    docker exec chatmail /opt/cmdeploy/bin/cmdeploy test chat.example.org --ssh-host localhost
+
 
 Customization
 -------------
 
-Custom website
+Website
 ^^^^^^^^^^^^^^
 
 You can customize the chatmail landing page by mounting a directory with
@@ -147,14 +150,8 @@ your own website source files.
 Custom chatmail.ini
 ^^^^^^^^^^^^^^^^^^^
 
-There are two configuration modes:
-
-**Simple (default):** Set ``MAIL_DOMAIN`` in ``.env``. The container
-auto-generates ``chatmail.ini`` with defaults on first start. This is
-sufficient for most deployments.
-
-**Advanced:** Generate a ``chatmail.ini``, edit it, and mount it into
-the container. This gives you full control over all chatmail settings.
+If you want to go beyond simply setting the ``MAIL_DOMAIN`` in ``.env``, you
+can use a regular `chatmail.ini` to give you full control.
 
 1. Extract the generated config from a running container::
 
@@ -174,6 +171,16 @@ the container. This gives you full control over all chatmail settings.
        docker compose down && docker compose up -d
 
 
+External TLS certificates
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If TLS certificates are managed outside the container (e.g. by certbot,
+acmetool, or Traefik on the host), mount them into the container and set
+``TLS_EXTERNAL_CERT_AND_KEY`` in ``docker-compose.override.yaml``.
+Changed certificates are picked up automatically via inotify.
+See the examples in the example override and :ref:`external-tls` in the getting started guide for details.
+
+
 Migrating from a bare-metal install
 ------------------------------------
 
@@ -196,16 +203,19 @@ switch to Docker:
 
 3. Copy persistent data into the ``./data/`` subdirectories (for example, as configured in `Customize and start`_) ::
 
-       mkdir -p data/chatmail-dkimkeys data/chatmail-acme data/chatmail
+       mkdir -p data/dkim data/certs data/mail
 
        # DKIM keys
-       cp -a /etc/dkimkeys/* data/chatmail-dkimkeys/
+       cp -a /etc/dkimkeys/* data/dkim/
 
-       # ACME certificates and account
-       rsync -a /var/lib/acme/ data/chatmail-acme/
+       # TLS certificates
+       rsync -a /var/lib/acme/ data/certs/
 
-       # Mail data
-       rsync -a /home/ data/chatmail/
+   Note that ownership of dkim and acme is adjusted on container start.
+
+   For the mail directory::
+
+       rsync -a /home/vmail/ data/mail/
 
    Alternatively, mount ``/home/vmail`` directly by changing the volume
    in ``docker-compose-override.yaml``::
@@ -226,8 +236,8 @@ Clone the repository and build the Docker image::
     docker compose build chatmail
 
 The build bakes all binaries, Python packages, and the install stage
-into the image. After building, only ``docker-compose.yaml`` and ``.env``
-are needed to run the container.
+into the image. After building, only ``docker-compose.yaml`` and a ``.env`` with
+``MAIL_DOMAIN`` are needed to run the container.
 
 You can transfer a locally built image to your server directly (pigz is parallel `gzip` which can be used instead as well) ::
 

docker-compose.override.yaml.example

@@ -1,26 +1,20 @@
-# Local overrides — copy to docker-compose.override.yaml in the repo root.
+# Local overrides: copy to docker-compose.override.yaml in the repo root.
 # Compose automatically merges this with docker-compose.yaml.
 #
-#   cp docker/docker-compose.override.yaml.example docker-compose.override.yaml
+#   cp docker-compose.override.yaml.example docker-compose.override.yaml
 #
-# Volumes listed here are APPENDED to the base file's volumes.
-# Scalar values (environment, image, etc.) are REPLACED.
+# Volumes are APPENDED to the base file's volumes list, environment and other scalar keys are MERGED by key.
 services:
   chatmail:
     volumes:
       ## Data paths — bind-mount to host directories for easy access/backup.
-      ## Uncomment and adjust paths as needed. These override the named
-      ## volumes in the base docker-compose.yaml.
-      # - ./data/chatmail:/home/vmail
-      # - ./data/chatmail-dkimkeys:/etc/dkimkeys
-      # - ./data/chatmail-acme:/var/lib/acme
 
-      ## Or mount data from an existing bare-metal install.
-      ## Note: DKIM key ownership is fixed automatically on startup
-      ## (the host's opendkim UID may differ from the container's).
+      # - ./data/dkim:/etc/dkimkeys
+      # - ./data/certs:/var/lib/acme
+
+      # - ./data/mail:/home/vmail
+      ## Or mount from an existing bare-metal install.
       # - /home/vmail:/home/vmail
-      # - /etc/dkimkeys:/etc/dkimkeys
-      # - /var/lib/acme:/var/lib/acme
 
       ## Mount your own chatmail.ini (skips auto-generation):
       # - ./chatmail.ini:/etc/chatmail/chatmail.ini
@@ -31,3 +25,15 @@ services:
       ## Debug — mount scripts from the repo for live editing:
       # - ./docker/files/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
       # - ./docker/files/entrypoint.sh:/entrypoint.sh
+
+    # environment:
+      ## Mount certs (above) and set TLS_EXTERNAL_CERT_AND_KEY to in-container paths.
+      ## Changed certs are picked up automatically (inotify via tls-cert-reload.path).
+      ##
+      ## Host acmetool (bare-metal migration): create mount above, and
+      ##   rsync -a /var/lib/acme/live data/certs
+      # TLS_EXTERNAL_CERT_AND_KEY: "/var/lib/acme/live/${MAIL_DOMAIN}/fullchain /var/lib/acme/live/${MAIL_DOMAIN}/privkey"
+      ##
+      ## (Untested) Traefik certs-dumper (see docker/docker-compose-traefik.yaml) - also add volume:
+      ##   - traefik-certs:/certs:ro
+      # TLS_EXTERNAL_CERT_AND_KEY: "/certs/${MAIL_DOMAIN}/certificate.crt /certs/${MAIL_DOMAIN}/privatekey.key"

docker-compose.yaml

@@ -35,17 +35,17 @@ services:
       CMDEPLOY_STAGES: ${CMDEPLOY_STAGES:-}
       CHATMAIL_NOSYSCTL: ${CHATMAIL_NOSYSCTL:-True}
       CHATMAIL_NOPORTCHECK: ${CHATMAIL_NOPORTCHECK:-True}
-      CHATMAIL_NOACME: ${CHATMAIL_NOACME:-}
+      TLS_EXTERNAL_CERT_AND_KEY: ${TLS_EXTERNAL_CERT_AND_KEY:-}
     network_mode: "host"
     volumes:
       ## system (required)
       - /sys/fs/cgroup:/sys/fs/cgroup:rw
       ## data (defaults — override in docker-compose.override.yaml)
-      - chatmail-data:/home/vmail
-      - chatmail-dkimkeys:/etc/dkimkeys
-      - chatmail-acme:/var/lib/acme
+      - mail:/home/vmail
+      - dkim:/etc/dkimkeys
+      - certs:/var/lib/acme
 
 volumes:
-  chatmail-data:
-  chatmail-dkimkeys:
-  chatmail-acme:
+  mail:
+  dkim:
+  certs:

docker/files/entrypoint.sh

@@ -6,7 +6,7 @@ SETUP_CHATMAIL_SERVICE_PATH="${SETUP_CHATMAIL_SERVICE_PATH:-/lib/systemd/system/
 # Whitelist only the env vars needed by setup_chatmail_docker.sh.
 # Forwarding all env vars (via printenv) would leak Docker internals,
 # orchestrator secrets, and other unrelated variables into systemd.
-env_vars="MAIL_DOMAIN CMDEPLOY_STAGES CHATMAIL_INI CHATMAIL_NOSYSCTL CHATMAIL_NOPORTCHECK CHATMAIL_NOACME PATH_TO_SSL PATH"
+env_vars="MAIL_DOMAIN CMDEPLOY_STAGES CHATMAIL_INI TLS_EXTERNAL_CERT_AND_KEY PATH"
 sed -i "s|<envs_list>|$env_vars|g" "$SETUP_CHATMAIL_SERVICE_PATH"
 
 exec /lib/systemd/systemd "$@"

docker/files/setup_chatmail_docker.sh

@@ -2,6 +2,8 @@
 
 set -euo pipefail
 export CHATMAIL_INI="${CHATMAIL_INI:-/etc/chatmail/chatmail.ini}"
+export CHATMAIL_NOSYSCTL=True
+export CHATMAIL_NOPORTCHECK=True
 
 CMDEPLOY=/opt/cmdeploy/bin/cmdeploy
 

env.example

@@ -1,7 +1 @@
-MAIL_DOMAIN="chat.example.com"
-
-# CMDEPLOY_STAGES - default: "configure,activate". Set to "install,configure,activate" to force full reinstall.
-# CMDEPLOY_STAGES="configure,activate"
-
-# Skip acmetool when using an external certificate manager (e.g. Traefik, Caddy).
-# CHATMAIL_NOACME="True"
+MAIL_DOMAIN=chat.example.com