try make opendkim reload on test1 faster

ci: replace staging workflows with LXC-local testing
Remove the staging VPS-based test-and-deploy workflows and replace them with LXC-local CI. Refactor image caching to use per-container aliases instead of a single relay image. Ensure postfix/dovecot/opendkim restarts after unbound restarts
2026-05-15 19:14:44 +00:00 · 2026-03-10 21:39:08 +01:00 · 2026-03-10 21:35:14 +01:00 · 2026-03-10 21:34:13 +01:00 · 2026-03-10 21:14:22 +01:00 · 2026-03-10 21:14:22 +01:00
22 changed files with 272 additions and 481 deletions
--- a/.github/workflows/ci.yaml
+++ b/.github/workflows/ci.yaml
@@ -15,28 +15,98 @@ jobs:
        with:
          ref: ${{ github.event.pull_request.head.sha }}
      - name: download filtermail
-        run: curl -L https://github.com/chatmail/filtermail/releases/download/v0.6.0/filtermail-x86_64 -o /usr/local/bin/filtermail && chmod +x /usr/local/bin/filtermail
-      - name: run chatmaild tests 
+        run: curl -L https://github.com/chatmail/filtermail/releases/download/v0.5.2/filtermail-x86_64 -o /usr/local/bin/filtermail && chmod +x /usr/local/bin/filtermail
+      - name: run chatmaild tests
        working-directory: chatmaild
        run: pipx run tox

  scripts:
-    name: deploy-chatmail tests 
+    name: deploy-chatmail tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

-      - name: initenv 
+      - name: initenv
        run: scripts/initenv.sh

      - name: append venv/bin to PATH
        run: echo venv/bin >>$GITHUB_PATH

-      - name: run formatting checks 
-        run: cmdeploy fmt -v 
+      - name: run formatting checks
+        run: cmdeploy fmt -v

-      - name: run deploy-chatmail offline tests 
-        run: pytest --pyargs cmdeploy 
+      - name: run deploy-chatmail offline tests
+        run: pytest --pyargs cmdeploy

-      # all other cmdeploy commands require a staging server 
-      # see https://github.com/deltachat/chatmail/issues/100
+  lxc-test:
+    name: LXC deploy and test
+    runs-on: ubuntu-24.04
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+
+      - name: install incus
+        run: sudo apt-get update && sudo apt-get install -y incus
+
+      - name: initialise incus
+        run: |
+          sudo systemctl stop docker.socket docker || true
+          sudo iptables -P FORWARD ACCEPT
+          sudo sysctl -w fs.inotify.max_user_instances=65535
+          sudo sysctl -w fs.inotify.max_user_watches=65535
+          sudo incus admin init --minimal --quiet
+          sudo usermod -aG incus-admin "$USER"
+
+      - name: initenv
+        run: scripts/initenv.sh
+
+      - name: append venv/bin to PATH
+        run: echo venv/bin >>$GITHUB_PATH
+
+      - name: restore cached images
+        id: cache-images
+        uses: actions/cache@v4
+        with:
+          path: |
+            /tmp/localchat-base.tar.gz
+            /tmp/localchat-ns.tar.gz
+            /tmp/localchat-test0.tar.gz
+            /tmp/localchat-test1.tar.gz
+            lxconfigs/id_localchat*
+          key: incus-contain-v2-${{ runner.os }}-${{ github.ref_name }}
+          restore-keys: |
+            incus-contain-v2-${{ runner.os }}-${{ github.ref_name }}-
+            incus-contain-v2-${{ runner.os }}-main-
+            incus-contain-v2-${{ runner.os }}-
+
+      - name: import cached images
+        run: |
+          for alias in localchat-base localchat-ns localchat-test0 localchat-test1; do
+            if [ -f /tmp/$alias.tar.gz ]; then
+              sg incus-admin -c "incus --quiet image import /tmp/$alias.tar.gz --alias $alias" || true
+            fi
+          done
+
+      - name: cmdeploy lxc-test
+        run: sg incus-admin -c 'cmdeploy lxc-test -vv'
+
+      - name: show container logs on failure
+        if: failure() || cancelled()
+        run: |
+          for c in test0-localchat test1-localchat; do
+            echo "::group::$c journal (warnings+errors)"
+            sg incus-admin -c "incus exec $c -- journalctl -p warning --no-pager -n200" 2>/dev/null || echo "no log"
+            echo "::endgroup::"
+            echo "::group::$c failed services"
+            sg incus-admin -c "incus exec $c -- systemctl --no-pager --failed" || true
+            echo "::endgroup::"
+          done
+
+      - name: export images for cache
+        if: always()
+        run: |
+          for alias in localchat-base localchat-ns localchat-test0 localchat-test1; do
+            sg incus-admin -c "incus --quiet image export $alias /tmp/$alias" || true
+          done
--- a/.github/workflows/staging-ipv4.testrun.org-default.zone
+++ b/.github/workflows/staging-ipv4.testrun.org-default.zone
@@ -1,20 +0,0 @@
-;; Zone file for staging-ipv4.testrun.org
-
-$ORIGIN staging-ipv4.testrun.org.
-$TTL 300
-
-@  IN  SOA     ns.testrun.org. root.nine.testrun.org (
-  2023010101 ; Serial
-  7200       ; Refresh
-  3600       ; Retry
-  1209600    ; Expire
-  3600       ; Negative response caching TTL
-)
-
-;; Nameservers.
-@  IN NS ns.testrun.org.
-
-;; DNS records.
-@       IN      A       37.27.95.249
-mta-sts.staging-ipv4.testrun.org.    CNAME   staging-ipv4.testrun.org.
-www.staging-ipv4.testrun.org.        CNAME   staging-ipv4.testrun.org.
--- a/.github/workflows/staging.testrun.org-default.zone
+++ b/.github/workflows/staging.testrun.org-default.zone
@@ -1,21 +0,0 @@
-;; Zone file for staging2.testrun.org
-
-$ORIGIN staging2.testrun.org.
-$TTL 300
-
-@  IN  SOA     ns.testrun.org. root.nine.testrun.org (
-  2023010101 ; Serial
-  7200       ; Refresh
-  3600       ; Retry
-  1209600    ; Expire
-  3600       ; Negative response caching TTL
-)
-
-;; Nameservers.
-@  IN NS ns.testrun.org.
-
-;; DNS records.
-@       IN      A       37.27.24.139
-mta-sts.staging2.testrun.org.    CNAME   staging2.testrun.org.
-www.staging2.testrun.org.        CNAME   staging2.testrun.org.
-
--- a/.github/workflows/test-and-deploy-ipv4only.yaml
+++ b/.github/workflows/test-and-deploy-ipv4only.yaml
@@ -1,104 +0,0 @@
-name: deploy on staging-ipv4.testrun.org, and run tests
-
-on:
-  push:
-    branches:
-      - main
-  pull_request:
-    paths-ignore:
-      - 'scripts/**'
-      - '**/README.md'
-      - 'CHANGELOG.md'
-      - 'LICENSE'
-
-jobs:
-  deploy:
-    name: deploy on staging-ipv4.testrun.org, and run tests
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    environment:
-      name: staging-ipv4.testrun.org
-      url: https://staging-ipv4.testrun.org/
-    concurrency: staging-ipv4.testrun.org
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: prepare SSH
-        run: |
-          mkdir ~/.ssh
-          echo "${{ secrets.STAGING_SSH_KEY }}" >> ~/.ssh/id_ed25519
-          chmod 600 ~/.ssh/id_ed25519
-          ssh-keyscan staging-ipv4.testrun.org > ~/.ssh/known_hosts
-          # save previous acme & dkim state
-          rsync -avz root@staging-ipv4.testrun.org:/var/lib/acme acme-ipv4 || true
-          rsync -avz root@staging-ipv4.testrun.org:/etc/dkimkeys dkimkeys-ipv4 || true
-          # store previous acme & dkim state on ns.testrun.org, if it contains useful certs
-          if [ -f dkimkeys-ipv4/dkimkeys/opendkim.private ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" dkimkeys-ipv4 root@ns.testrun.org:/tmp/ || true; fi
-          if [ "$(ls -A acme-ipv4/acme/certs)" ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" acme-ipv4 root@ns.testrun.org:/tmp/ || true; fi
-          # make sure CAA record isn't set
-          scp -o StrictHostKeyChecking=accept-new .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone
-          ssh root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging-ipv4.testrun.org.zone
-          ssh root@ns.testrun.org nsd-checkzone staging-ipv4.testrun.org /etc/nsd/staging-ipv4.testrun.org.zone
-          ssh root@ns.testrun.org systemctl reload nsd
-
-      - name: rebuild staging-ipv4.testrun.org to have a clean VPS
-        run: |
-            curl -X POST \
-            -H "Authorization: Bearer ${{ secrets.HETZNER_API_TOKEN }}" \
-            -H "Content-Type: application/json" \
-            -d '{"image":"debian-12"}' \
-            "https://api.hetzner.cloud/v1/servers/${{ secrets.STAGING_IPV4_SERVER_ID }}/actions/rebuild"
-
-      - run: scripts/initenv.sh
-
-      - name: append venv/bin to PATH
-        run: echo venv/bin >>$GITHUB_PATH
-
-      - name: upload TLS cert after rebuilding
-        run: |
-          echo " --- wait until staging-ipv4.testrun.org VPS is rebuilt --- "
-          rm ~/.ssh/known_hosts
-          while ! ssh -o ConnectTimeout=180 -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org id -u ; do sleep 1 ; done
-          ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org id -u
-          # download acme & dkim state from ns.testrun.org
-          rsync -e "ssh -o StrictHostKeyChecking=accept-new" -avz root@ns.testrun.org:/tmp/acme-ipv4/acme acme-restore || true
-          rsync -avz root@ns.testrun.org:/tmp/dkimkeys-ipv4/dkimkeys dkimkeys-restore || true
-          # restore acme & dkim state to staging2.testrun.org
-          rsync -avz acme-restore/acme root@staging-ipv4.testrun.org:/var/lib/ || true
-          rsync -avz dkimkeys-restore/dkimkeys root@staging-ipv4.testrun.org:/etc/ || true
-          ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org chown root:root -R /var/lib/acme || true
-
-      - name: run deploy-chatmail offline tests 
-        run: pytest --pyargs cmdeploy 
-
-      - name: setup dependencies
-        run: |
-          ssh root@staging-ipv4.testrun.org apt update
-          ssh root@staging-ipv4.testrun.org apt install -y git python3.11-venv python3-dev gcc
-          ssh root@staging-ipv4.testrun.org git clone https://github.com/chatmail/relay
-          ssh root@staging-ipv4.testrun.org "cd relay && git checkout " ${{ github.head_ref }}
-          ssh root@staging-ipv4.testrun.org "cd relay && scripts/initenv.sh"
-
-      - name: initialize config
-        run: |
-          ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy init staging-ipv4.testrun.org"
-          ssh root@staging-ipv4.testrun.org "sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' relay/chatmail.ini"
-          ssh root@staging-ipv4.testrun.org "sed -i 's/#\s*mtail_address/mtail_address/' relay/chatmail.ini"
-
-      - run: ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy run --verbose --skip-dns-check --ssh-host localhost"
-
-      - name: set DNS entries
-        run: |
-          ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy dns --zonefile staging-generated.zone --ssh-host localhost"
-          ssh root@staging-ipv4.testrun.org cat relay/staging-generated.zone >> .github/workflows/staging-ipv4.testrun.org-default.zone
-          cat .github/workflows/staging-ipv4.testrun.org-default.zone
-          scp .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone
-          ssh root@ns.testrun.org nsd-checkzone staging-ipv4.testrun.org /etc/nsd/staging-ipv4.testrun.org.zone
-          ssh root@ns.testrun.org systemctl reload nsd
-
-      - name: cmdeploy test
-        run: ssh root@staging-ipv4.testrun.org "cd relay && CHATMAIL_DOMAIN2=ci-chatmail.testrun.org scripts/cmdeploy test --slow --ssh-host localhost"
-
-      - name: cmdeploy dns
-        run: ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy dns -v --ssh-host localhost"
-
--- a/.github/workflows/test-and-deploy.yaml
+++ b/.github/workflows/test-and-deploy.yaml
@@ -1,97 +0,0 @@
-name: deploy on staging2.testrun.org, and run tests
-
-on:
-  push:
-    branches:
-      - main
-  pull_request:
-    paths-ignore:
-      - 'scripts/**'
-      - '**/README.md'
-      - 'CHANGELOG.md'
-      - 'LICENSE'
-
-jobs:
-  deploy:
-    name: deploy on staging2.testrun.org, and run tests
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    environment:
-      name: staging2.testrun.org
-      url: https://staging2.testrun.org/
-    concurrency: staging2.testrun.org
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: prepare SSH
-        run: |
-          mkdir ~/.ssh
-          echo "${{ secrets.STAGING_SSH_KEY }}" >> ~/.ssh/id_ed25519
-          chmod 600 ~/.ssh/id_ed25519
-          ssh-keyscan staging2.testrun.org > ~/.ssh/known_hosts
-          # save previous acme & dkim state
-          rsync -avz root@staging2.testrun.org:/var/lib/acme . || true
-          rsync -avz root@staging2.testrun.org:/etc/dkimkeys . || true
-          # store previous acme & dkim state on ns.testrun.org, if it contains useful certs
-          if [ -f dkimkeys/opendkim.private ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" dkimkeys root@ns.testrun.org:/tmp/ || true; fi
-          if [ "$(ls -A acme/certs)" ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" acme root@ns.testrun.org:/tmp/ || true; fi
-          # make sure CAA record isn't set
-          scp -o StrictHostKeyChecking=accept-new .github/workflows/staging.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging2.testrun.org.zone
-          ssh root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging2.testrun.org.zone
-          ssh root@ns.testrun.org nsd-checkzone staging2.testrun.org /etc/nsd/staging2.testrun.org.zone
-          ssh root@ns.testrun.org systemctl reload nsd
-
-      - name: rebuild staging2.testrun.org to have a clean VPS
-        run: |
-            curl -X POST \
-            -H "Authorization: Bearer ${{ secrets.HETZNER_API_TOKEN }}" \
-            -H "Content-Type: application/json" \
-            -d '{"image":"debian-12"}' \
-            "https://api.hetzner.cloud/v1/servers/${{ secrets.STAGING_SERVER_ID }}/actions/rebuild"
-
-      - run: scripts/initenv.sh
-
-      - name: append venv/bin to PATH
-        run: echo venv/bin >>$GITHUB_PATH
-
-      - name: upload TLS cert after rebuilding
-        run: |
-          echo " --- wait until staging2.testrun.org VPS is rebuilt --- "
-          rm ~/.ssh/known_hosts
-          while ! ssh -o ConnectTimeout=180 -o StrictHostKeyChecking=accept-new -v root@staging2.testrun.org id -u ; do sleep 1 ; done
-          ssh -o StrictHostKeyChecking=accept-new -v root@staging2.testrun.org id -u
-          # download acme & dkim state from ns.testrun.org
-          rsync -e "ssh -o StrictHostKeyChecking=accept-new" -avz root@ns.testrun.org:/tmp/acme acme-restore || true
-          rsync -avz root@ns.testrun.org:/tmp/dkimkeys dkimkeys-restore || true
-          # restore acme & dkim state to staging2.testrun.org
-          rsync -avz acme-restore/acme root@staging2.testrun.org:/var/lib/ || true
-          rsync -avz dkimkeys-restore/dkimkeys root@staging2.testrun.org:/etc/ || true
-          ssh -o StrictHostKeyChecking=accept-new -v root@staging2.testrun.org chown root:root -R /var/lib/acme || true
-
-      - name: add hpk42 key to staging server
-        run: ssh root@staging2.testrun.org 'curl -s https://github.com/hpk42.keys >> .ssh/authorized_keys'
-
-      - name: run deploy-chatmail offline tests 
-        run: pytest --pyargs cmdeploy 
-
-      - run: |
-          cmdeploy init staging2.testrun.org
-          sed -i 's/#\s*mtail_address/mtail_address/' chatmail.ini
-
-      - run: cmdeploy run --verbose --skip-dns-check
-
-      - name: set DNS entries
-        run: |
-          cmdeploy dns --zonefile staging-generated.zone --verbose
-          cat staging-generated.zone >> .github/workflows/staging.testrun.org-default.zone
-          cat .github/workflows/staging.testrun.org-default.zone
-          scp .github/workflows/staging.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging2.testrun.org.zone
-          ssh root@ns.testrun.org nsd-checkzone staging2.testrun.org /etc/nsd/staging2.testrun.org.zone
-          ssh root@ns.testrun.org systemctl reload nsd
-
-      - name: cmdeploy test
-        run: CHATMAIL_DOMAIN2=ci-chatmail.testrun.org cmdeploy test --slow
-
-      - name: cmdeploy dns
-        run: cmdeploy dns -v
-
--- a/chatmaild/pyproject.toml
+++ b/chatmaild/pyproject.toml
@@ -6,7 +6,10 @@ build-backend = "setuptools.build_meta"
 name = "chatmaild"
 version = "0.3"
 dependencies = [
+  "aiosmtpd",
  "iniconfig",
+  "deltachat-rpc-server",
+  "deltachat-rpc-client",
  "filelock",
  "requests",
  "crypt-r >= 3.13.1 ; python_version >= '3.11'",
@@ -67,7 +70,6 @@ commands =
 deps = pytest 
       pdbpp 
       pytest-localserver
-       aiosmtpd
       execnet
 commands = pytest -v -rsXx {posargs}
 """
--- a/cmdeploy/pyproject.toml
+++ b/cmdeploy/pyproject.toml
@@ -10,6 +10,7 @@ dependencies = [
  "pillow",
  "qrcode",
  "markdown",
+  "pytest",
  "setuptools>=68",
  "termcolor",
  "build",
@@ -20,7 +21,6 @@ dependencies = [
  "execnet",
  "imap_tools",
  "deltachat-rpc-client",
-  "deltachat-rpc-server",
 ]

 [project.scripts]
--- a/cmdeploy/src/cmdeploy/cmdeploy.py
+++ b/cmdeploy/src/cmdeploy/cmdeploy.py
@@ -260,10 +260,10 @@ def test_cmd(args, out):
    pytest_args = [
        pytest_path,
        "cmdeploy/src/",
-        "-n4",
        "-rs",
        "-x",
-        "-v",
+        "-vv" if args.verbose > 1 else "-v",
+        "-s",
        "--durations=5",
    ]
    if args.slow:
--- a/cmdeploy/src/cmdeploy/deployers.py
+++ b/cmdeploy/src/cmdeploy/deployers.py
@@ -488,6 +488,10 @@ class ChatmailDeployer(Deployer):
            name="Install rsync",
            packages=["rsync"],
        )
+        apt.packages(
+            name="Ensure cron is installed",
+            packages=["cron"],
+        )

    def configure(self):
        # Ensure the per-domain mailbox directory exists before
--- a/cmdeploy/src/cmdeploy/dns.py
+++ b/cmdeploy/src/cmdeploy/dns.py
@@ -47,40 +47,33 @@ def get_filled_zone_file(remote_data):
        remote_data["sts_id"] = datetime.datetime.now().strftime("%Y%m%d%H%M")

    d = remote_data["mail_domain"]
-
-    def rec(name, rtype, rdata, ttl=3600):
-        return f"{name:<40} {ttl:<6} IN  {rtype:<5}  {rdata}"
-
    lines = ["; Required DNS entries"]
    if remote_data.get("A"):
-        lines.append(rec(f"{d}.", "A", remote_data["A"]))
+        lines.append(f"{d}.  3600  IN  A  {remote_data['A']}")
    if remote_data.get("AAAA"):
-        lines.append(rec(f"{d}.", "AAAA", remote_data["AAAA"]))
-    lines.append(rec(f"{d}.", "MX", f"10 {d}."))
+        lines.append(f"{d}.  3600  IN  AAAA  {remote_data['AAAA']}")
+    lines.append(f"{d}.  3600  IN  MX  10 {d}.")
    if remote_data.get("strict_tls"):
        lines.append(
-            rec(f"_mta-sts.{d}.", "TXT", f'"v=STSv1; id={remote_data["sts_id"]}"')
+            f'_mta-sts.{d}.  3600  IN  TXT  "v=STSv1; id={remote_data["sts_id"]}"'
        )
-        lines.append(rec(f"mta-sts.{d}.", "CNAME", f"{d}."))
-    lines.append(rec(f"www.{d}.", "CNAME", f"{d}."))
+        lines.append(f"mta-sts.{d}.  3600  IN  CNAME  {d}.")
+    lines.append(f"www.{d}.  3600  IN  CNAME  {d}.")
    lines.append(remote_data["dkim_entry"])
    lines.append("")
    lines.append("; Recommended DNS entries")
-    lines.append(rec(f"{d}.", "TXT", '"v=spf1 a ~all"'))
-    lines.append(rec(f"_dmarc.{d}.", "TXT", '"v=DMARC1;p=reject;adkim=s;aspf=s"'))
+    lines.append(f'{d}.  3600  IN  TXT  "v=spf1 a ~all"')
+    lines.append(f'_dmarc.{d}.  3600  IN  TXT  "v=DMARC1;p=reject;adkim=s;aspf=s"')
    if remote_data.get("acme_account_url"):
        lines.append(
-            rec(
-                f"{d}.",
-                "CAA",
-                f'0 issue "letsencrypt.org;accounturi={remote_data["acme_account_url"]}"',
-            )
+            f"{d}.  3600  IN  CAA  0 issue"
+            f' "letsencrypt.org;accounturi={remote_data["acme_account_url"]}"'
        )
-    lines.append(rec(f"_adsp._domainkey.{d}.", "TXT", '"dkim=discardable"'))
-    lines.append(rec(f"_submission._tcp.{d}.", "SRV", f"0 1 587 {d}."))
-    lines.append(rec(f"_submissions._tcp.{d}.", "SRV", f"0 1 465 {d}."))
-    lines.append(rec(f"_imap._tcp.{d}.", "SRV", f"0 1 143 {d}."))
-    lines.append(rec(f"_imaps._tcp.{d}.", "SRV", f"0 1 993 {d}."))
+    lines.append(f'_adsp._domainkey.{d}.  3600  IN  TXT  "dkim=discardable"')
+    lines.append(f"_submission._tcp.{d}.  3600  IN  SRV  0 1 587 {d}.")
+    lines.append(f"_submissions._tcp.{d}.  3600  IN  SRV  0 1 465 {d}.")
+    lines.append(f"_imap._tcp.{d}.  3600  IN  SRV  0 1 143 {d}.")
+    lines.append(f"_imaps._tcp.{d}.  3600  IN  SRV  0 1 993 {d}.")
    lines.append("")
    return "\n".join(lines)

--- a/cmdeploy/src/cmdeploy/dovecot/deployer.py
+++ b/cmdeploy/src/cmdeploy/dovecot/deployer.py
@@ -2,8 +2,8 @@ import urllib.request

 from chatmaild.config import Config
 from pyinfra import host
-from pyinfra.facts.deb import DebPackages
 from pyinfra.facts.server import Arch, Command, Sysctl
+from pyinfra.facts.systemd import SystemdEnabled
 from pyinfra.operations import apt, files, server, systemd

 from cmdeploy.basedeploy import (
@@ -12,19 +12,9 @@ from cmdeploy.basedeploy import (
    blocked_service_startup,
    configure_remote_units,
    get_resource,
+    has_systemd,
 )

-DOVECOT_VERSION = "2.3.21+dfsg1-3"
-
-DOVECOT_SHA256 = {
-    ("core", "amd64"): "dd060706f52a306fa863d874717210b9fe10536c824afe1790eec247ded5b27d",
-    ("core", "arm64"): "e7548e8a82929722e973629ecc40fcfa886894cef3db88f23535149e7f730dc9",
-    ("imapd", "amd64"): "8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86",
-    ("imapd", "arm64"): "178fa877ddd5df9930e8308b518f4b07df10e759050725f8217a0c1fb3fd707f",
-    ("lmtpd", "amd64"): "2f69ba5e35363de50962d42cccbfe4ed8495265044e244007d7ccddad77513ab",
-    ("lmtpd", "arm64"): "89f52fb36524f5877a177dff4a713ba771fd3f91f22ed0af7238d495e143b38f",
-}
-

 class DovecotDeployer(Deployer):
    daemon_reload = False
@@ -36,22 +26,13 @@ class DovecotDeployer(Deployer):

    def install(self):
        arch = host.get_fact(Arch)
+        if has_systemd() and "dovecot.service" in host.get_fact(SystemdEnabled):
+            return  # already installed and running
+
        with blocked_service_startup():
-            debs = []
-            for pkg in ("core", "imapd", "lmtpd"):
-                deb = _download_dovecot_package(pkg, arch)
-                if deb:
-                    debs.append(deb)
-            if debs:
-                deb_list = " ".join(debs)
-                server.shell(
-                    name="Install dovecot packages",
-                    commands=[
-                        f"dpkg --force-confdef --force-confold -i {deb_list} 2> /dev/null || true",
-                        "DEBIAN_FRONTEND=noninteractive apt-get -y --fix-broken install",
-                        f"dpkg --force-confdef --force-confold -i {deb_list}",
-                    ],
-                )
+            _install_dovecot_package("core", arch)
+            _install_dovecot_package("imapd", arch)
+            _install_dovecot_package("lmtpd", arch)

    def configure(self):
        configure_remote_units(self.config.mail_domain, self.units)
@@ -84,37 +65,40 @@ def _pick_url(primary, fallback):
        return fallback


-def _download_dovecot_package(package: str, arch: str):
-    """Download a dovecot .deb if needed, return its path (or None)."""
+def _install_dovecot_package(package: str, arch: str):
    arch = "amd64" if arch == "x86_64" else arch
    arch = "arm64" if arch == "aarch64" else arch
-
-    pkg_name = f"dovecot-{package}"
-    sha256 = DOVECOT_SHA256.get((package, arch))
-    if sha256 is None:
-        apt.packages(packages=[pkg_name])
-        return None
-
-    installed_versions = host.get_fact(DebPackages).get(pkg_name, [])
-    if DOVECOT_VERSION in installed_versions:
-        return None
-
-    url_version = DOVECOT_VERSION.replace("+", "%2B")
-    deb_base = f"{pkg_name}_{url_version}_{arch}.deb"
-    primary_url = f"https://download.delta.chat/dovecot/{deb_base}"
-    fallback_url = f"https://github.com/chatmail/dovecot/releases/download/upstream%2F{url_version}/{deb_base}"
+    primary_url = f"https://download.delta.chat/dovecot/dovecot-{package}_2.3.21%2Bdfsg1-3_{arch}.deb"
+    fallback_url = f"https://github.com/chatmail/dovecot/releases/download/upstream%2F2.3.21%2Bdfsg1/dovecot-{package}_2.3.21%2Bdfsg1-3_{arch}.deb"
    url = _pick_url(primary_url, fallback_url)
-    deb_filename = f"/root/{deb_base}"
+    deb_filename = "/root/" + url.split("/")[-1]
+
+    match (package, arch):
+        case ("core", "amd64"):
+            sha256 = "dd060706f52a306fa863d874717210b9fe10536c824afe1790eec247ded5b27d"
+        case ("core", "arm64"):
+            sha256 = "e7548e8a82929722e973629ecc40fcfa886894cef3db88f23535149e7f730dc9"
+        case ("imapd", "amd64"):
+            sha256 = "8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86"
+        case ("imapd", "arm64"):
+            sha256 = "178fa877ddd5df9930e8308b518f4b07df10e759050725f8217a0c1fb3fd707f"
+        case ("lmtpd", "amd64"):
+            sha256 = "2f69ba5e35363de50962d42cccbfe4ed8495265044e244007d7ccddad77513ab"
+        case ("lmtpd", "arm64"):
+            sha256 = "89f52fb36524f5877a177dff4a713ba771fd3f91f22ed0af7238d495e143b38f"
+        case _:
+            apt.packages(packages=[f"dovecot-{package}"])
+            return

    files.download(
-        name=f"Download {pkg_name}",
+        name=f"Download dovecot-{package}",
        src=url,
        dest=deb_filename,
        sha256sum=sha256,
        cache_time=60 * 60 * 24 * 365 * 10,  # never redownload the package
    )

-    return deb_filename
+    apt.deb(name=f"Install dovecot-{package}", src=deb_filename)


 def _configure_dovecot(config: Config, debug: bool = False) -> (bool, bool):
@@ -161,7 +145,7 @@ def _configure_dovecot(config: Config, debug: bool = False) -> (bool, bool):
        if not can_modify:
            print(
                "\n!!!! refusing to attempt sysctl setting in shared-kernel containers\n"
-                f"!!!! dovecot: sysctl {key!r}={value}, should be >65534 for production setups\n"
+                f"!!!! dovecot: sysctl {key!r}={value}, should be >65535 for production setups\n"
                "!!!!"
            )
            continue
--- a/cmdeploy/src/cmdeploy/filtermail/deployer.py
+++ b/cmdeploy/src/cmdeploy/filtermail/deployer.py
@@ -14,10 +14,10 @@ class FiltermailDeployer(Deployer):

    def install(self):
        arch = host.get_fact(facts.server.Arch)
-        url = f"https://github.com/chatmail/filtermail/releases/download/v0.6.0/filtermail-{arch}"
+        url = f"https://github.com/chatmail/filtermail/releases/download/v0.5.2/filtermail-{arch}"
        sha256sum = {
-            "x86_64": "3fd8b18282252c75a5bbfa603d8c1b65f6563e5e920bddf3e64e451b7cdb43ce",
-            "aarch64": "2bd191de205f7fd60158dd8e3516ab7e3efb14627696f3d7dc186bdcd9e10a43",
+            "x86_64": "ce24ca0075aa445510291d775fb3aea8f4411818c7b885ae51a0fe18c5f789ce",
+            "aarch64": "c5d783eefa5332db3d97a0e6a23917d72849e3eb45da3d16ce908a9b4e5a797d",
        }[arch]
        self.need_restart |= files.download(
            name="Download filtermail",
--- a/cmdeploy/src/cmdeploy/lxc/init.py
+++ b/cmdeploy/src/cmdeploy/lxc/init.py
--- a/cmdeploy/src/cmdeploy/lxc/cli.py
+++ b/cmdeploy/src/cmdeploy/lxc/cli.py
@@ -4,7 +4,7 @@ import os
 import time

 from ..util import get_git_hash, get_version_string, shell
-from .incus import RELAY_IMAGE_ALIAS, Incus, RelayContainer
+from .incus import Incus, RelayContainer

 RELAY_NAMES = ("test0", "test1")

@@ -47,6 +47,7 @@ def _lxc_start_cmd(args, out):
    out.green("Ensuring DNS container (ns-localchat) ...")
    dns_ct = ix.get_dns_container()
    dns_ct.ensure()
+    dns_ct.ensure_cached_as_image()
    sub.print(f"DNS container IP: {dns_ct.ipv4}")

    names = args.names if args.names else RELAY_NAMES
@@ -116,17 +117,39 @@ def _lxc_start_cmd(args, out):

    # Optionally run cmdeploy run + dns on each relay
    if args.run:
+        local_hash = get_git_hash()
        for ct in relays:
+            status = _deploy_status(ct, local_hash, ix)
            with out.section(f"cmdeploy run: {ct.sname} ({ct.domain})"):
-                ret = _run_cmdeploy("run", ct, ix, out, extra=["--skip-dns-check"])
-                if ret:
-                    out.red(f"Deploy to {ct.sname} failed (exit {ret})")
-                    return ret
+                if "IN-SYNC" in status:
+                    out.print(f"{ct.sname} is {status}, skipping")
+                else:
+                    ret = _run_cmdeploy("run", ct, ix, out, extra=["--skip-dns-check"])
+                    if ret:
+                        out.red(f"Deploy to {ct.sname} failed (exit {ret})")
+                        return ret
+                    # Cache a per-relay image after each successful deploy
+                    # so the next run can launch directly from the deployed state.
+                    with out.section(f"lxc-test: caching {ct.sname} image"):
+                        ct.ensure_cached_as_image()

-        with out.section("loading DNS zones"):
-            for ct in relays:
+        # Restart mail services to flush stale DNS state.
+        # Cached container images boot with a resolv.conf
+        # pointing to the previous run's DNS IP;
+        # configure_dns() already restarted unbound,
+        # but postfix/dovecot may hold stale results
+        # from the window between boot and DNS fix.
+        for ct in relays:
+            out.print(f"Restarting mail services on {ct.name} ...")
+            ct.bash("systemctl restart postfix dovecot opendkim")
+
+        for ct in relays:
+            with out.section(f"cmdeploy dns: {ct.sname} ({ct.domain})"):
                ret = _run_cmdeploy(
-                    "dns", ct, ix, out,
+                    "dns",
+                    ct,
+                    ix,
+                    out,
                    extra=["--zonefile", str(ct.zone)],
                )
                if ret:
@@ -134,7 +157,10 @@ def _lxc_start_cmd(args, out):
                    return ret
                if ct.zone.exists():
                    dns_ct.set_dns_records(ct.zone.read_text())
-                out.print(f"Restarting filtermail-incoming on {ct.name}")
+                # Restart filtermail so its in-process DNS cache
+                # does not hold stale negative DKIM responses
+                # from before the zones were loaded.
+                out.print(f"Restarting filtermail-incoming on {ct.name} ...")
                ct.bash("systemctl restart filtermail-incoming")


@@ -209,71 +235,26 @@ def lxc_test_cmd(args, out):
    """
    ix = Incus(out)
    t_total = time.time()
-    relay_names = list(RELAY_NAMES)
-    if args.one:
-        relay_names = relay_names[:1]
+    v_flag = " -" + "v" * out.verbosity if out.verbosity > 0 else ""

-    local_hash = get_git_hash()
+    ret = out.shell(f"cmdeploy lxc-start{v_flag} --run test0", cwd=str(ix.project_root))
+    if ret:
+        return ret

-    # Per-relay: start, deploy, then snapshot the first relay as a
-    # reusable image so the second relay launches pre-deployed.
-    ipv4_only_flags = {RELAY_NAMES[0]: False, RELAY_NAMES[1]: True}
-
-    for ct in map(ix.get_container, relay_names):
-        name = ct.sname
-        ipv4_only = ipv4_only_flags.get(name, False)
-        v_flag = " -" + "v" * out.verbosity if out.verbosity > 0 else ""
-        start_cmd = f"cmdeploy lxc-start{v_flag} {name}"
-        if ipv4_only:
-            start_cmd += " --ipv4-only"
-        with out.section(f"cmdeploy lxc-start: {name}"):
-            ret = out.shell(start_cmd, cwd=str(ix.project_root))
-            if ret:
-                return ret
-
-        status = _deploy_status(ct, local_hash, ix)
-        with out.section(f"cmdeploy run: {name}"):
-            if "IN-SYNC" in status:
-                out.print(f"{name} is {status}, skipping")
-            else:
-                ret = _run_cmdeploy("run", ct, ix, out, extra=["--skip-dns-check"])
-                if ret:
-                    out.red(f"Deploy to {name} failed (exit {ret})")
-                    return ret
-
-        # Snapshot the first relay so subsequent ones launch pre-deployed
-        if not ix.find_image([RELAY_IMAGE_ALIAS]):
-            with out.section("lxc-test: caching relay image"):
-                ct.publish_as_relay_image()
-
-    for ct in map(ix.get_container, relay_names):
-        with out.section(f"cmdeploy dns: {ct.sname} ({ct.domain})"):
-            ret = _run_cmdeploy("dns", ct, ix, out, extra=["--zonefile", str(ct.zone)])
-            if ret:
-                out.red(f"DNS for {ct.sname} failed (exit {ret})")
-                return ret
-
-    with out.section(f"lxc-test: loading DNS zones {' & '.join(relay_names)}"):
-        dns_ct = ix.get_dns_container()
-        for ct in map(ix.get_container, relay_names):
-            if ct.zone.exists():
-                zone_data = ct.zone.read_text()
-                out.print(f"Loading {ct.zone} into PowerDNS ...")
-                dns_ct.set_dns_records(zone_data)
-
-        # Restart filtermail so its in-process DNS cache
-        # does not hold stale negative DKIM responses
-        # from before the zones were loaded.
-        for ct in map(ix.get_container, relay_names):
-            out.print(f"Restarting filtermail-incoming on {ct.name} ...")
-            ct.bash("systemctl restart filtermail-incoming")
+    if not args.one:
+        ret = out.shell(
+            f"cmdeploy lxc-start{v_flag} --run test1 --ipv4-only",
+            cwd=str(ix.project_root),
+        )
+        if ret:
+            return ret

    with out.section("cmdeploy test"):
-        first = ix.get_container(relay_names[0])
+        first = ix.get_container("test0")
        env = None
-        if len(relay_names) > 1:
+        if not args.one:
            env = os.environ.copy()
-            env["CHATMAIL_DOMAIN2"] = ix.get_container(relay_names[1]).domain
+            env["CHATMAIL_DOMAIN2"] = ix.get_container("test1").domain
        ret = _run_cmdeploy("test", first, ix, out, **({"env": env} if env else {}))
        if ret:
            out.red(f"Tests failed (exit {ret})")
--- a/cmdeploy/src/cmdeploy/lxc/incus.py
+++ b/cmdeploy/src/cmdeploy/lxc/incus.py
@@ -14,9 +14,9 @@ DOMAIN_SUFFIX = ".localchat"
 UPSTREAM_IMAGE = "images:debian/12"
 BASE_IMAGE_ALIAS = "localchat-base"
 BASE_SETUP_NAME = "localchat-base-setup"
-RELAY_IMAGE_ALIAS = "localchat-relay"

 DNS_CONTAINER_NAME = "ns-localchat"
+DNS_IMAGE_ALIAS = "localchat-ns"
 DNS_DOMAIN = "ns.localchat"


@@ -184,7 +184,16 @@ class Incus:
        )
        if result.returncode != 0:
            return None
-        return json.loads(result.stdout)
+        try:
+            return json.loads(result.stdout)
+        except json.JSONDecodeError as e:
+            msg = f"Incus JSON processing failed for {args!r}: {e!s}"
+            self.out.red(msg)
+            self.out.red(f"Captured stdout: {result.stdout!r}")
+            self.out.red(f"Captured stderr: {result.stderr!r}")
+            if check:
+                raise
+            return None

    def run_output(self, args, check=True):
        """Run an incus command and return its stripped stdout.
@@ -207,8 +216,13 @@ class Incus:
        return None

    def delete_images(self):
-        """Delete the cached base and relay images."""
-        for alias in (RELAY_IMAGE_ALIAS, BASE_IMAGE_ALIAS):
+        """Delete localchat-base and per-container images."""
+        for alias in [
+            BASE_IMAGE_ALIAS,
+            DNS_IMAGE_ALIAS,
+            "localchat-test0",
+            "localchat-test1",
+        ]:
            self.run(["image", "delete", alias], check=False)  # ok if absent

    def list_managed(self):
@@ -238,7 +252,7 @@ class Incus:
    def ensure_base_image(self):
        """Build and cache a base image with openssh and the SSH key.

-        The image is published as a local incus image with alias
+        The image is cached as a local incus image with alias
        'localchat-base'.  Subsequent container launches use this
        image instead of the upstream Debian 12, skipping the
        slow apt-get install step.
@@ -338,7 +352,7 @@ class Container:

    def launch(self):
        """Launch from the best available image, return the alias used."""
-        image = self.incus.find_image([RELAY_IMAGE_ALIAS, BASE_IMAGE_ALIAS])
+        image = self.incus.find_image([BASE_IMAGE_ALIAS])
        if not image:
            raise RuntimeError(
                f"No base image '{BASE_IMAGE_ALIAS}' found. "
@@ -419,6 +433,18 @@ class Container:
                    parts = line.split()
                    return int(parts[2]), int(parts[1])

+    def ensure_cached_as_image(self):
+        """Cache this container as a respective image."""
+        alias = self.image_alias
+        if self.incus.find_image([alias]):
+            return
+        self.out.print("  Cleaning apt cache before caching image ...")
+        self.bash("apt-get clean")
+        self.out.print(f"  Caching {self.name!r} as '{alias}' ...")
+        self.incus.run(["publish", self.name, f"--alias={alias}", "--force"])
+        self.out.print(f"  Image '{alias}' cached.")
+        self.wait_ready()
+

 class RelayContainer(Container):
    """Container handle for a chatmail relay.
@@ -434,12 +460,22 @@ class RelayContainer(Container):
            domain=f"_{name}{DOMAIN_SUFFIX}",
        )
        self.sname = name
+        self.image_alias = f"localchat-{name}"
        self.ini = incus.lxconfigs_dir / f"chatmail-{name}.ini"
        self.zone = incus.lxconfigs_dir / f"{name}.zone"

    def launch(self):
-        """Launch (from a potentially cached image) and clear inherited chatmail-version."""
-        image = super().launch()
+        """Launch from localchat-{sname} if cached, else localchat-base."""
+
+        candidates = [self.image_alias]
+        candidates.append(BASE_IMAGE_ALIAS)
+        image = self.incus.find_image(candidates)
+        assert image, f"No deployment base, candidates: {','.join(candidates)}"
+        self.out.print(f"  Launching from '{image}' image ...")
+        cfg = []
+        cfg += ("-c", f"{LABEL_KEY}=true")
+        cfg += ("-c", f"user.localchat-domain={self.domain}")
+        self.incus.run(["launch", image, self.name, *cfg])
        self.bash("rm -f /etc/chatmail-version")
        return image

@@ -474,22 +510,6 @@ class RelayContainer(Container):
            echo '{ip} {self.name} {self.domain}' >> /etc/hosts
        """)

-    def publish_as_relay_image(self):
-        """Publish this container as a reusable relay image.
-
-        Stops the container, 'publishes' it as 'localchat-relay', then restarts it.
-        """
-        if self.incus.find_image([RELAY_IMAGE_ALIAS]):
-            return
-        self.out.print(
-            f"  Locally caching {self.name!r} as '{RELAY_IMAGE_ALIAS}' image ..."
-        )
-        self.incus.run(
-            ["publish", self.name, f"--alias={RELAY_IMAGE_ALIAS}", "--force"]
-        )
-        self.wait_ready()
-        self.out.print(f"  Relay image '{RELAY_IMAGE_ALIAS}' ready.")
-
    def deployed_version(self):
        """Read /etc/chatmail-version, or None if absent."""
        return self.bash("cat /etc/chatmail-version", check=False)
@@ -572,6 +592,21 @@ class DNSContainer(Container):

    def __init__(self, incus):
        super().__init__(incus, DNS_CONTAINER_NAME, domain=DNS_DOMAIN)
+        self.image_alias = DNS_IMAGE_ALIAS
+
+    def launch(self):
+        """Launch from localchat-ns if cached, else localchat-base."""
+        image = self.incus.find_image([DNS_IMAGE_ALIAS, BASE_IMAGE_ALIAS])
+        if not image:
+            raise RuntimeError(
+                f"No base image '{BASE_IMAGE_ALIAS}' found. "
+                "Call ensure_base_image() before launching containers."
+            )
+        self.out.print(f"  Launching from '{image}' image ...")
+        cfg = []
+        cfg += ("-c", f"{LABEL_KEY}=true")
+        cfg += ("-c", f"user.localchat-domain={self.domain}")
+        self.incus.run(["launch", image, self.name, *cfg])

    def pdnsutil(self, *args, check=True):
        """Run ``pdnsutil <args>`` inside the DNS container."""
--- a/cmdeploy/src/cmdeploy/opendkim/opendkim.conf
+++ b/cmdeploy/src/cmdeploy/opendkim/opendkim.conf
@@ -60,6 +60,10 @@ PidFile			/run/opendkim/opendkim.pid
 # by the package dns-root-data.
 TrustAnchorFile		/usr/share/dns/root.key

+# Use the local unbound resolver rather than querying root servers directly.
+# on IPv4-only hosts opendkim may otherwise try ipv6 requests and time out. 
+Nameservers		127.0.0.1
+
 # Sign messages when `-o milter_macro_daemon_name=ORIGINATING` is set.
 MTA ORIGINATING

--- a/cmdeploy/src/cmdeploy/postfix/main.cf.j2
+++ b/cmdeploy/src/cmdeploy/postfix/main.cf.j2
@@ -20,7 +20,7 @@ smtpd_tls_key_file={{ config.tls_key_path }}
 smtpd_tls_security_level=may

 smtp_tls_CApath=/etc/ssl/certs
-smtp_tls_security_level=verify
+smtp_tls_security_level={{ "verify" if config.tls_cert_mode == "acme" else "encrypt" }}
 # Send SNI extension when connecting to other servers.
 # <https://www.postfix.org/postconf.5.html#smtp_tls_servername>
 smtp_tls_servername = hostname
@@ -88,22 +88,6 @@ inet_protocols = ipv4
 inet_protocols = all
 {% endif %}

-# Postfix does not try IPv4 and IPv6 connections
-# concurrently as of version 3.7.11.
-#
-# When relay has both A (IPv4) and AAAA (IPv6) records,
-# but broken IPv6 connectivity,
-# every second message is delayed by the connection timeout
-# <https://www.postfix.org/postconf.5.html#smtp_connect_timeout>
-# which defaults to 30 seconds. Reducing timeouts is not a solution
-# as this will result in a failure to connect to slow servers.
-#
-# As a workaround we always prefer IPv4 when it is available.
-# 
-# The setting is documented at
-# <https://www.postfix.org/postconf.5.html#smtp_address_preference>
-smtp_address_preference=ipv4
-
 virtual_transport = lmtp:unix:private/dovecot-lmtp
 virtual_mailbox_domains = {{ config.mail_domain }}
 lmtp_header_checks = regexp:/etc/postfix/lmtp_header_cleanup
--- a/cmdeploy/src/cmdeploy/remote/rdns.py
+++ b/cmdeploy/src/cmdeploy/remote/rdns.py
@@ -57,10 +57,9 @@ def get_dkim_entry(mail_domain, pre_command, dkim_selector):
    dkim_value_raw = f"v=DKIM1;k=rsa;p={dkim_pubkey};s=email;t=s"
    dkim_value = '" "'.join(re.findall(".{1,255}", dkim_value_raw))
    web_dkim_value = "".join(re.findall(".{1,255}", dkim_value_raw))
-    name = f"{dkim_selector}._domainkey.{mail_domain}."
    return (
-        f'{name:<40} 3600   IN  TXT    "{dkim_value}"',
-        f'{name:<40} 3600   IN  TXT    "{web_dkim_value}"',
+        f'{dkim_selector}._domainkey.{mail_domain}.  3600  IN  TXT  "{dkim_value}"',
+        f'{dkim_selector}._domainkey.{mail_domain}.  3600  IN  TXT  "{web_dkim_value}"',
    )


--- a/cmdeploy/src/cmdeploy/tests/data/zftest.zone
+++ b/cmdeploy/src/cmdeploy/tests/data/zftest.zone
@@ -1,18 +1,18 @@
 ; Required DNS entries
-zftest.testrun.org.                      3600   IN  A      135.181.204.127
-zftest.testrun.org.                      3600   IN  AAAA   2a01:4f9:c012:52f4::1
-zftest.testrun.org.                      3600   IN  MX     10 zftest.testrun.org.
-_mta-sts.zftest.testrun.org.             3600   IN  TXT    "v=STSv1; id=202403211706"
-mta-sts.zftest.testrun.org.              3600   IN  CNAME  zftest.testrun.org.
-www.zftest.testrun.org.                  3600   IN  CNAME  zftest.testrun.org.
-opendkim._domainkey.zftest.testrun.org.  3600   IN  TXT    "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoYt82CVUyz2ouaqjX2kB+5J80knAyoOU3MGU5aWppmwUwwTvj/oSTSpkc5JMtVTRmKKr8NUDWAL1Yw7dfGqqPHdHfwwjS3BIvDzYx+hzgtz62RnfNgV+/2MAoNpfX7cAFIHdRzEHNtwugc3RDLquqPoupAE3Y2YRw2T5zG5fILh4vwIcJZL5Uq6B92j8wwJqOex" "33n+vm1NKQ9rxo/UsHAmZlJzpooXcG/4igTBxJyJlamVSRR6N7Nul1v//YJb7J6v2o0iPHW6uE0StzKaPPNC2IVosSRFbD9H2oqppltptFSNPlI0E+t0JBWHem6YK7xcugiO3ImMCaaU8g6Jt/wIDAQAB;s=email;t=s"
+zftest.testrun.org.  3600  IN  A  135.181.204.127
+zftest.testrun.org.  3600  IN  AAAA  2a01:4f9:c012:52f4::1
+zftest.testrun.org.  3600  IN  MX  10 zftest.testrun.org.
+_mta-sts.zftest.testrun.org.  3600  IN  TXT  "v=STSv1; id=202403211706"
+mta-sts.zftest.testrun.org.  3600  IN  CNAME  zftest.testrun.org.
+www.zftest.testrun.org.  3600  IN  CNAME  zftest.testrun.org.
+opendkim._domainkey.zftest.testrun.org.  3600  IN  TXT  "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoYt82CVUyz2ouaqjX2kB+5J80knAyoOU3MGU5aWppmwUwwTvj/oSTSpkc5JMtVTRmKKr8NUDWAL1Yw7dfGqqPHdHfwwjS3BIvDzYx+hzgtz62RnfNgV+/2MAoNpfX7cAFIHdRzEHNtwugc3RDLquqPoupAE3Y2YRw2T5zG5fILh4vwIcJZL5Uq6B92j8wwJqOex" "33n+vm1NKQ9rxo/UsHAmZlJzpooXcG/4igTBxJyJlamVSRR6N7Nul1v//YJb7J6v2o0iPHW6uE0StzKaPPNC2IVosSRFbD9H2oqppltptFSNPlI0E+t0JBWHem6YK7xcugiO3ImMCaaU8g6Jt/wIDAQAB;s=email;t=s"

 ; Recommended DNS entries
-zftest.testrun.org.                      3600   IN  TXT    "v=spf1 a ~all"
-_dmarc.zftest.testrun.org.               3600   IN  TXT    "v=DMARC1;p=reject;adkim=s;aspf=s"
-zftest.testrun.org.                      3600   IN  CAA    0 issue "letsencrypt.org;accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/1371472956"
-_adsp._domainkey.zftest.testrun.org.     3600   IN  TXT    "dkim=discardable"
-_submission._tcp.zftest.testrun.org.     3600   IN  SRV    0 1 587 zftest.testrun.org.
-_submissions._tcp.zftest.testrun.org.    3600   IN  SRV    0 1 465 zftest.testrun.org.
-_imap._tcp.zftest.testrun.org.           3600   IN  SRV    0 1 143 zftest.testrun.org.
-_imaps._tcp.zftest.testrun.org.          3600   IN  SRV    0 1 993 zftest.testrun.org.
+zftest.testrun.org.  3600  IN  TXT  "v=spf1 a ~all"
+_dmarc.zftest.testrun.org.  3600  IN  TXT  "v=DMARC1;p=reject;adkim=s;aspf=s"
+zftest.testrun.org.  3600  IN  CAA  0 issue "letsencrypt.org;accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/1371472956"
+_adsp._domainkey.zftest.testrun.org.  3600  IN  TXT  "dkim=discardable"
+_submission._tcp.zftest.testrun.org.  3600  IN  SRV  0 1 587 zftest.testrun.org.
+_submissions._tcp.zftest.testrun.org.  3600  IN  SRV  0 1 465 zftest.testrun.org.
+_imap._tcp.zftest.testrun.org.  3600  IN  SRV  0 1 143 zftest.testrun.org.
+_imaps._tcp.zftest.testrun.org.  3600  IN  SRV  0 1 993 zftest.testrun.org.
--- a/cmdeploy/src/cmdeploy/tests/online/test_0_login.py
+++ b/cmdeploy/src/cmdeploy/tests/online/test_0_login.py
@@ -89,9 +89,7 @@ def test_concurrent_logins_same_account(
        assert login_results.get()


-def test_no_vrfy(cmfactory, chatmail_config):
-    ac = cmfactory.get_online_account()
-    addr = ac.get_config("addr")
+def test_no_vrfy(chatmail_config):
    domain = chatmail_config.mail_domain

    s = smtplib.SMTP(domain)
@@ -100,7 +98,7 @@ def test_no_vrfy(cmfactory, chatmail_config):
    s.putcmd("vrfy", f"wrongaddress@{chatmail_config.mail_domain}")
    result = s.getreply()
    print(result)
-    s.putcmd("vrfy", addr)
+    s.putcmd("vrfy", f"echo@{chatmail_config.mail_domain}")
    result2 = s.getreply()
    print(result2)
    assert result[0] == result2[0] == 252
--- a/cmdeploy/src/cmdeploy/tests/test_dns.py
+++ b/cmdeploy/src/cmdeploy/tests/test_dns.py
@@ -132,28 +132,11 @@ def test_parse_zone_records():

    ; Another comment
    www.some.domain. 3600 IN CNAME some.domain.
-
-    ; Multi-word rdata
-    some.domain. 3600 IN MX 10 mail.some.domain.
-
-    ; DKIM record (single line, multi-word TXT rdata)
-    dkim._domainkey.some.domain. 3600 IN TXT "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG" "9w0BAQEFAAOCAQ8AMIIBCgKCAQEA"
-
-    ; Another TXT record
-    _dmarc.some.domain. 3600 IN TXT "v=DMARC1;p=reject"
    """
    records = list(parse_zone_records(text))
    assert records == [
        ("some.domain", "3600", "A", "1.1.1.1"),
        ("www.some.domain", "3600", "CNAME", "some.domain."),
-        ("some.domain", "3600", "MX", "10 mail.some.domain."),
-        (
-            "dkim._domainkey.some.domain",
-            "3600",
-            "TXT",
-            '"v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG" "9w0BAQEFAAOCAQ8AMIIBCgKCAQEA"',
-        ),
-        ("_dmarc.some.domain", "3600", "TXT", '"v=DMARC1;p=reject"'),
    ]


--- a/doc/source/lxc.rst
+++ b/doc/source/lxc.rst
@@ -15,10 +15,6 @@ as they would on a real Debian server or cloud VPS.
 Prerequisites
 -------------

- Around 4-5 GiB free disk space
- `systemd-networkd` for the automagic hostname resolution
- No other service occupying Port 53
-
 Install `Incus <https://linuxcontainers.org/incus/>`_
 (LXC container manager).
 See the `official installation guide
Author	SHA1	Message	Date
holger krekel	ddc36902c8	try make opendkim reload on test1 faster	2026-03-10 21:39:08 +01:00
holger krekel	8a1c1d9288	ci: replace staging workflows with LXC-local testing Remove the staging VPS-based test-and-deploy workflows and replace them with LXC-local CI. Refactor image caching to use per-container aliases instead of a single relay image. Ensure postfix/dovecot/opendkim restarts after unbound restarts	2026-03-10 21:35:14 +01:00
holger krekel	c3fd52e5dd	fix: use push_file_content for unbound and sysctl config files fixes silent fail on invalid configuration file.	2026-03-10 21:34:13 +01:00
holger krekel	233d442bca	fix: run cmdeploy dns and restart filtermail in lxc-start --run lxc-start --run deployed the relay but never loaded DNS zones or restarted filtermail-incoming, so DKIM verification and outgoing mail failed.	2026-03-10 21:14:22 +01:00
holger krekel	8aabb9a955	fix: pass full config to ChatmailDeployer and ensure mailboxes_dir ChatmailDeployer now receives the full config object so it can access mailboxes_dir and create this directory during deployment, preventing mail delivery failures.	2026-03-10 21:14:22 +01:00
holger krekel	9fc05646e0	fix: separate stderr capture in incus and suppress in test plugin Capture stderr separately in Incus.run() instead of merging it into stdout, which corrupted JSON parsing in run_json(). Add --quiet flag to reduce incus noise. Suppress stderr in Remote subprocess to prevent pytest-xdist communication issues.	2026-03-10 21:14:22 +01:00
holger krekel	155c1221b8	remove superflous sepchar argument	2026-03-10 19:12:31 +01:00
holger krekel	3393d071e5	remove memory limits in containers	2026-03-10 14:14:40 +01:00
holger krekel	52c73658f9	fix a potential hang because "journalctl -f" never finishes by itself, and handles might stay open if we don't close them during test teardown.	2026-03-10 13:31:53 +01:00
holger krekel	b022a61955	try to discover and use host dns settings instead of 9.9.9.9	2026-03-10 13:25:52 +01:00
holger krekel	2e383a8e94	make sure subprocesses by defaulit don't inherit stdin FD	2026-03-10 10:54:27 +01:00
holger krekel	9908a4c88c	try fix some test hangs potentially caused by filtermail's DNS cache	2026-03-09 17:33:52 +01:00
holger krekel	c04f4a4b44	force tcp connections for dns resolution	2026-03-09 16:44:33 +01:00
holger krekel	6d0ce061bc	change all timeouts to 60	2026-03-09 16:40:18 +01:00
holger krekel	a48c525455	remove superflous """\	2026-03-09 00:51:33 +01:00
holger krekel	2786b60658	only use explicit server settings if the host resolves to ip address via ssh config	2026-03-08 23:40:37 +01:00
holger krekel	fe46b573a6	move --verbose option back to subcommands	2026-03-08 23:40:37 +01:00
holger krekel	674e496a53	cmdeploy: replace globals() subcommand scan with explicit SUBCOMMANDS list The get_parser() loop that scanned globals() for *_cmd names was fragile and forced # noqa: F401 on all lxc imports (ruff couldn't see they were used dynamically). Replace it with an explicit SUBCOMMANDS list of (cmd_func, options_func, needs_config) tuples. This makes the full set of subcommands visible at a glance, their registration order defined, and the imports unconditionally used (no more noqa suppressions).	2026-03-08 23:40:37 +01:00
holger krekel	86e5708709	lxc: dovecot sysctl: warn but skip when running in shared-kernel container Replace the CHATMAIL_NOSYSCTL guard with an explicit systemd-detect-virt -c check.	2026-03-08 23:40:37 +01:00
holger krekel	04ac2cf700	lxc: code cleanup and docs polish from review Code: - Fix check_ssh_config_include() to do a case-insensitive line match (read_text().splitlines() instead of filter(None, map(..., open()))). - Drop the default help_text= from _add_name_args(); callers now must supply an explicit string. - Expand the DNSContainer class docstring. - Fix sysctl comment: note that incus provides net.* virtualization so the sysctl only affects the container's network namespace. Docs (doc/source/lxc.rst): - Remove double blank line after page title; fix missing comma. - Replace the plain-text root-access note with a .. caution:: block. - Tighten the Quick-start section and lxc-test CLI entry.	2026-03-08 23:40:37 +01:00
holger krekel	e2ec0cf2c5	lxc: simplify to a single find_image(aliases) method Replace the two-function find_relay_image / _find_relay_image pair with Incus.find_image(aliases), which returns the first alias that exists in the local image store, or None. Container.launch() passes [RELAY_IMAGE_ALIAS, BASE_IMAGE_ALIAS] and ensure_base_image() passes [BASE_IMAGE_ALIAS].	2026-03-08 23:40:37 +01:00
holger krekel	ee9d54f7d6	lxc: extract blocked_service_startup() context manager into basedeploy Move the policy-rc.d install/remove boilerplate into a shared context manager in basedeploy.py so both UnboundDeployer and DovecotDeployer use the same abstraction, and the DNS container's _install_powerdns() inline shell uses the same pattern. DovecotDeployer now wraps its three package installs in blocked_service_startup() to prevent Dovecot from auto-starting on initial install — avoiding bind conflicts on IPv4-only systems.	2026-03-08 23:40:37 +01:00
holger krekel	1604321d5b	lxc: add Out class with --verbose, section timing, and coloured shell output Move the Out output-printer class to cmdeploy/util.py so it is shared across CLI modules. All print/shell calls in lxc/cli.py, lxc/incus.py, and dns.py now route through Out instead of bare print(). Key additions: - Out.section() / Out.section_line(): coloured section headers scaled to the current terminal width (or $_CMDEPLOY_WIDTH for sub-processes). - Out.shell(): merges stdout/stderr, prefixes each output line, and prints a red error line with the exit code on failure. - Out.new_prefixed_out(): indented sub-printer that shares section_timings. - 'cmdeploy -v / -vv' exposes the verbosity levels. - Tests for Out added to test_util.py.	2026-03-08 23:40:37 +01:00
holger krekel	4ab04fa6c4	lxc: poll until DNS answers before continuing; reset bridge on --destroy-all After restarting pdns/pdns-recursor, wait up to 10 s for the recursor to actually answer a query before proceeding. Likewise, after configure_dns(), poll from inside the relay container until the configured DNS IP responds. On --destroy-all, unset the incusbr0 dns.mode and raw.dnsmasq network options so the next lxc-start starts from a clean bridge state. DNSConfigurationError (caught in main()) is raised on timeout so the CLI prints a clean error instead of failing later with a cryptic message.	2026-03-08 23:40:37 +01:00
holger krekel	8b6829b906	ignore sysctl permission problems (likely in containers)	2026-03-08 23:40:37 +01:00
holger krekel	cf2cb57cca	address link2xt review comments	2026-03-08 23:40:37 +01:00
holger krekel	693c3f8555	address link2xt comments (zone parsing and turn v0.4 release	2026-03-08 23:40:37 +01:00
holger krekel	371efdfafb	simplify start instructions	2026-03-08 23:40:37 +01:00
holger krekel	23765a5ed8	make helpers testable and test them, also streamline intro of docs	2026-03-08 23:40:37 +01:00
holger krekel	0adeefbdd7	fix lxc-test to not re-run deploy when nothing changed + some other beautifications	2026-03-08 23:40:37 +01:00
holger krekel	624838eedd	don't use env vars but explicit pytest options to pass ssh info around.	2026-03-08 23:40:37 +01:00
holger krekel	1abdc407af	feat: add LXC container support for local chatmail development Add cmdeploy "lxc-test" command to run cmdeploy against local containers, with supplementary lxc-start, lxc-stop and lxc-status subcommands. See doc/source/lxc.rst for full documentation including prerequisites, DNS setup, TLS handling, DNS-free testing, and known limitations. Apart from adding lxc-specific docs, tests, and implementation files in the cmdeploy/lxc directory, this PR adds the --ssh-config option to cmdeploy run/dns/status/test commands and pyinfra invocations, and also to sshexec (Execnet) handling. This allows for the host to need no DNS entries for a relay, and route all resolution through ssh-config. This is used by the "lxc-test" command, which performs a completely local setup -- again, see docs for more details. While working on DNS/SSH things i also unified all zone-file handling to use actual BIND format as it is easy enough to parse back.	2026-03-08 23:40:37 +01:00