doc: cmdeploy command makes manual configuration obsolete

fix #604
related to #629
pulled out of https://github.com/Keonik1/relay/pull/3

.github/workflows/test-and-deploy-ipv4only.yaml

@@ -77,7 +77,7 @@ jobs:
           cmdeploy init staging-ipv4.testrun.org
           sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' chatmail.ini
 
-      - run: cmdeploy run --verbose --skip-dns-check
+      - run: cmdeploy run
 
       - name: set DNS entries
         run: |

.github/workflows/test-and-deploy.yaml

@@ -75,7 +75,7 @@ jobs:
 
       - run: cmdeploy init staging2.testrun.org
 
-      - run: cmdeploy run --verbose --skip-dns-check
+      - run: cmdeploy run --verbose
 
       - name: set DNS entries
         run: |

ARCHITECTURE.md

@@ -48,19 +48,3 @@ graph LR;
 The edges in this graph should not be taken too literally; they
 reflect some sort of communication path or dependency relationship
 between components of the chatmail server.
-
-## Message between users on the same relay
-
-```mermaid
-graph LR;
-    chatmail core --> |465|smtps/smtpd;
-    chatmail core --> |587|submission/smtpd;
-    smtps/smtpd --> |10080|filtermail;
-    submission/smtpd --> |10080|filtermail;
-    filtermail --> |10025|smtpd reinject;
-    smtpd reinject --> cleanup;
-    cleanup --> qmgr;
-    qmgr --> smtpd accepts message;
-    qmgr --> |lmtp|dovecot;
-    dovecot --> chatmail core;
-```

CHANGELOG.md

@@ -2,48 +2,24 @@
 
 ## untagged
 
-- Setup TURN server
-  ([#621](https://github.com/chatmail/relay/pull/621))
-
 - cmdeploy: make --ssh-host work with localhost
   ([#659](https://github.com/chatmail/relay/pull/659))
 
 - Update iroh-relay to 0.35.0
   ([#650](https://github.com/chatmail/relay/pull/650))
 
-- postfix: accept whole mail before passing it to filtermail
-  ([#673](https://github.com/chatmail/relay/pull/673))
-
-- filtermail: accept mails from Protonmail
-  ([#616](https://github.com/chatmail/relay/pull/655))
-
 - Ignore all RCPT TO: parameters
   ([#651](https://github.com/chatmail/relay/pull/651))
 
-- Increase opendkim DNS Timeout from 5 to 60 seconds
-  ([#672](https://github.com/chatmail/relay/pull/672))
-
-- Add config parameter for Let's Encrypt ACME email
-  ([#663](https://github.com/chatmail/relay/pull/663))
-
 - Use max username length in newemail.py, not min
   ([#648](https://github.com/chatmail/relay/pull/648))
 
-- Add startup for `fcgiwrap.service` because sometimes it did not start automatically.
-  ([#657](https://github.com/chatmail/relay/pull/657))
-
-- Add `cmdeploy init --force` command for recreating chatmail.ini
-  ([#656](https://github.com/chatmail/relay/pull/656))
-
 - Increase maxproc for reinjecting ports from 10 to 100
   ([#646](https://github.com/chatmail/relay/pull/646))
 
 - Allow ports 143 and 993 to be used by `dovecot` process
   ([#639](https://github.com/chatmail/relay/pull/639))
 
-- Add `--skip-dns-check` argument to `cmdeploy run` command, which disables DNS record checking before installation.
-  ([#661](https://github.com/chatmail/relay/pull/661))
-
 ## 1.7.0 2025-09-11
 
 - Make www upload path configurable

README.md

@@ -456,94 +456,15 @@ to send messages outside.
 
 To setup a reverse proxy
 (or rather Destination NAT, DNAT)
-for your chatmail relay,
+for your chatmail relay, run:
-put the following configuration in `/etc/nftables.conf`:
-```
-#!/usr/sbin/nft -f
-
-flush ruleset
-
-define wan = eth0
-
-# Which ports to proxy.
-#
-# Note that SSH is not proxied
-# so it is possible to log into the proxy server 
-# and not the original one.
-define ports = { smtp, http, https, imap, imaps, submission, submissions }
-
-# The host we want to proxy to.
-define ipv4_address = AAA.BBB.CCC.DDD
-define ipv6_address = [XXX::1]
-
-table ip nat {
-        chain prerouting {
-                type nat hook prerouting priority dstnat; policy accept;
-                iif $wan tcp dport $ports dnat to $ipv4_address
-        }
-
-        chain postrouting {
-                type nat hook postrouting priority 0;
-
-                oifname $wan masquerade
-        }
-}
-
-table ip6 nat {
-        chain prerouting {
-                type nat hook prerouting priority dstnat; policy accept;
-                iif $wan tcp dport $ports dnat to $ipv6_address
-        }
-
-        chain postrouting {
-                type nat hook postrouting priority 0;
-
-                oifname $wan masquerade
-        }
-}
-
-table inet filter {
-        chain input {
-                type filter hook input priority filter; policy drop;
-
-                # Accept ICMP.
-                # It is especially important to accept ICMPv6 ND messages,
-                # otherwise IPv6 connectivity breaks.
-                icmp type { echo-request } accept
-                icmpv6 type { echo-request, nd-neighbor-solicit, nd-router-advert, nd-neighbor-advert } accept
-
-                # Allow incoming SSH connections.
-                tcp dport { ssh } accept
-
-                ct state established accept
-        }
-        chain forward {
-                type filter hook forward priority filter; policy drop;
-
-                ct state established accept
-                ip daddr $ipv4_address counter accept
-                ip6 daddr $ipv6_address counter accept
-        }
-        chain output {
-                type filter hook output priority filter;
-        }
-}
-```
-
-Run `systemctl enable nftables.service`
-to ensure configuration is reloaded when the proxy relay reboots.
-
-Uncomment in `/etc/sysctl.conf` the following two lines:
 
 ```
-net.ipv4.ip_forward=1
+scripts/cmdeploy proxy <proxy_ip_address> --relay-ipv4 <relay_ipv4_address> --relay-ipv6 <relay_ipv6_address>
-net.ipv6.conf.all.forwarding=1
 ```
 
-Then reboot the relay or do `sysctl -p` and `nft -f /etc/nftables.conf`.
-
 Once proxy relay is set up,
-you can add its IP address to the DNS.
+you can add its IP address to the DNS,
+or distribute it as you wish.
 
 ## Neighbors and Acquaintances
 

chatmaild/pyproject.toml

@@ -29,7 +29,6 @@ echobot = "chatmaild.echo:main"
 chatmail-metrics = "chatmaild.metrics:main"
 delete_inactive_users = "chatmaild.delete_inactive_users:main"
 lastlogin = "chatmaild.lastlogin:main"
-turnserver = "chatmaild.turnserver:main"
 
 [project.entry-points.pytest11]
 "chatmaild.testplugin" = "chatmaild.tests.plugin"

chatmaild/src/chatmaild/config.py

@@ -44,7 +44,6 @@ class Config:
         )
         self.mtail_address = params.get("mtail_address")
         self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
-        self.acme_email = params.get("acme_email", "")
         self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
         if "iroh_relay" not in params:
             self.iroh_relay = "https://" + params["mail_domain"]

chatmaild/src/chatmaild/filtermail.py

@@ -83,14 +83,8 @@ def check_openpgp_payload(payload: bytes):
     return False
 
 
-def check_armored_payload(payload: str, outgoing: bool):
+def check_armored_payload(payload: str):
-    """Check the armored PGP message for invalid content.
+    prefix = "-----BEGIN PGP MESSAGE-----\r\n\r\n"
-
-    :param payload: the armored PGP message
-    :param outgoing: whether the message is outgoing or incoming
-    :return: whether the message is a valid PGP message
-    """
-    prefix = "-----BEGIN PGP MESSAGE-----\r\n"
     if not payload.startswith(prefix):
         return False
     payload = payload.removeprefix(prefix)
@@ -102,17 +96,6 @@ def check_armored_payload(payload: str, outgoing: bool):
         return False
     payload = payload.removesuffix(suffix)
 
-    # Disallow comments in outgoing messages
-    version_comment = "Version: "
-    if payload.startswith(version_comment):
-        version_line = payload.splitlines()[0]
-        payload = payload.removeprefix(version_line)
-        if outgoing:
-            return False
-
-    while payload.startswith("\r\n"):
-        payload = payload.removeprefix("\r\n")
-
     # Remove CRC24.
     payload = payload.rpartition("=")[0]
 
@@ -148,7 +131,7 @@ def is_securejoin(message):
     return True
 
 
-def check_encrypted(message, outgoing=True):
+def check_encrypted(message):
     """Check that the message is an OpenPGP-encrypted message.
 
     MIME structure of the message must correspond to <https://www.rfc-editor.org/rfc/rfc3156>.
@@ -175,7 +158,7 @@ def check_encrypted(message, outgoing=True):
             if part.get_content_type() != "application/octet-stream":
                 return False
 
-            if not check_armored_payload(part.get_payload(), outgoing=outgoing):
+            if not check_armored_payload(part.get_payload()):
                 return False
         else:
             return False
@@ -258,7 +241,7 @@ class OutgoingBeforeQueueHandler:
         logging.info(f"Processing DATA message from {envelope.mail_from}")
 
         message = BytesParser(policy=policy.default).parsebytes(envelope.content)
-        mail_encrypted = check_encrypted(message, outgoing=True)
+        mail_encrypted = check_encrypted(message)
 
         _, from_addr = parseaddr(message.get("from").strip())
 
@@ -318,7 +301,7 @@ class IncomingBeforeQueueHandler:
         logging.info(f"Processing DATA message from {envelope.mail_from}")
 
         message = BytesParser(policy=policy.default).parsebytes(envelope.content)
-        mail_encrypted = check_encrypted(message, outgoing=False)
+        mail_encrypted = check_encrypted(message)
 
         if mail_encrypted or is_securejoin(message):
             print("Incoming: Filtering encrypted mail.", file=sys.stderr)

chatmaild/src/chatmaild/ini/chatmail.ini.f

@@ -45,9 +45,6 @@ passthrough_senders =
 # (space-separated, item may start with "@" to whitelist whole recipient domains)
 passthrough_recipients = xstore@testrun.org echo@{mail_domain}
 
-# path to www directory - documented here: https://github.com/chatmail/relay/#custom-web-pages
-#www_folder = www
-
 #
 # Deployment Details
 #
@@ -63,9 +60,6 @@ postfix_reinject_port_incoming = 10026
 # if set to "True" IPv6 is disabled
 disable_ipv6 = False
 
-# Your email adress, which will be used in acmetool to manage Let's Encrypt SSL certificates
-acme_email = 
-
 # Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail
 # service.
 # If you set it to anything else, the service will be disabled

chatmaild/src/chatmaild/metadata.py

@@ -7,7 +7,6 @@ from .config import read_config
 from .dictproxy import DictProxy
 from .filedict import FileDict
 from .notifier import Notifier
-from .turnserver import turn_credentials
 
 
 def _is_valid_token_timestamp(timestamp, now):
@@ -76,12 +75,11 @@ class Metadata:
 
 
 class MetadataDictProxy(DictProxy):
-    def __init__(self, notifier, metadata, iroh_relay=None, turn_hostname=None):
+    def __init__(self, notifier, metadata, iroh_relay=None):
         super().__init__()
         self.notifier = notifier
         self.metadata = metadata
         self.iroh_relay = iroh_relay
-        self.turn_hostname = turn_hostname
 
     def handle_lookup(self, parts):
         # Lpriv/43f5f508a7ea0366dff30200c15250e3/devicetoken\tlkj123poi@c2.testrun.org
@@ -100,11 +98,6 @@ class MetadataDictProxy(DictProxy):
             ):
                 # Handle `GETMETADATA "" /shared/vendor/deltachat/irohrelay`
                 return f"O{self.iroh_relay}\n"
-            elif keyname == "vendor/vendor.dovecot/pvt/server/vendor/deltachat/turn":
-                res = turn_credentials()
-                port = 3478
-                return f"O{self.turn_hostname}:{port}:{res}\n"
-
         logging.warning(f"lookup ignored: {parts!r}")
         return "N\n"
 
@@ -128,7 +121,6 @@ def main():
 
     config = read_config(config_path)
     iroh_relay = config.iroh_relay
-    mail_domain = config.mail_domain
 
     vmail_dir = config.mailboxes_dir
     if not vmail_dir.exists():
@@ -142,10 +134,7 @@ def main():
     notifier.start_notification_threads(metadata.remove_token_from_addr)
 
     dictproxy = MetadataDictProxy(
-        notifier=notifier,
+        notifier=notifier, metadata=metadata, iroh_relay=iroh_relay
-        metadata=metadata,
-        iroh_relay=iroh_relay,
-        turn_hostname=mail_domain,
     )
 
     dictproxy.serve_forever_from_socket(socket)

chatmaild/src/chatmaild/tests/test_filtermail.py

@@ -241,9 +241,8 @@ def test_cleartext_passthrough_senders(gencreds, handler, maildata):
 
 
 def test_check_armored_payload():
-    prefix = "-----BEGIN PGP MESSAGE-----\r\n"
+    payload = """-----BEGIN PGP MESSAGE-----\r
-    comment = "Version: ProtonMail\r\n"
+\r
-    payload = """\r
 wU4DSqFx0d1yqAoSAQdAYkX/ZN/Az4B0k7X47zKyWrXxlDEdS3WOy0Yf2+GJTFgg\r
 Zk5ql0mLG8Ze+ZifCS0XMO4otlemSyJ0K1ZPdFMGzUDBTgNqzkFabxXoXRIBB0AM\r
 755wlX41X6Ay3KhnwBq7yEqSykVH6F3x11iHPKraLCAGZoaS8bKKNy/zg5slda1X\r
@@ -279,25 +278,16 @@ UN4fiB0KR9JyG2ayUdNJVkXZSZLnHyRgiaadlpUo16LVvw==\r
 \r
 """
 
-    commented_payload = prefix + comment + payload
+    assert check_armored_payload(payload) == True
-    assert check_armored_payload(commented_payload, outgoing=False) == True
-    assert check_armored_payload(commented_payload, outgoing=True) == False
-
-    payload = prefix + payload
-    assert check_armored_payload(payload, outgoing=False) == True
-    assert check_armored_payload(payload, outgoing=True) == True
 
     payload = payload.removesuffix("\r\n")
-    assert check_armored_payload(payload, outgoing=False) == True
+    assert check_armored_payload(payload) == True
-    assert check_armored_payload(payload, outgoing=True) == True
 
     payload = payload.removesuffix("\r\n")
-    assert check_armored_payload(payload, outgoing=False) == True
+    assert check_armored_payload(payload) == True
-    assert check_armored_payload(payload, outgoing=True) == True
 
     payload = payload.removesuffix("\r\n")
-    assert check_armored_payload(payload, outgoing=False) == True
+    assert check_armored_payload(payload) == True
-    assert check_armored_payload(payload, outgoing=True) == True
 
     payload = """-----BEGIN PGP MESSAGE-----\r
 \r
@@ -305,8 +295,7 @@ HELLOWORLD
 -----END PGP MESSAGE-----\r
 \r
 """
-    assert check_armored_payload(payload, outgoing=False) == False
+    assert check_armored_payload(payload) == False
-    assert check_armored_payload(payload, outgoing=True) == False
 
     payload = """-----BEGIN PGP MESSAGE-----\r
 \r
@@ -314,8 +303,7 @@ HELLOWORLD
 -----END PGP MESSAGE-----\r
 \r
 """
-    assert check_armored_payload(payload, outgoing=False) == False
+    assert check_armored_payload(payload) == False
-    assert check_armored_payload(payload, outgoing=True) == False
 
     # Test payload using partial body length
     # as generated by GopenPGP.
@@ -357,5 +345,4 @@ myLbG7cJB787QjplEyVe2P/JBO6xYvbkJLf9Q+HaviTO25rugRSrYsoKMDfO8VlQ\r
 =6iHb\r
 -----END PGP MESSAGE-----\r
 """
-    assert check_armored_payload(payload, outgoing=False) == True
+    assert check_armored_payload(payload) == True
-    assert check_armored_payload(payload, outgoing=True) == True

chatmaild/src/chatmaild/turnserver.py

@@ -1,9 +0,0 @@
-#!/usr/bin/env python3
-import socket
-
-
-def turn_credentials() -> str:
-    with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client_socket:
-        client_socket.connect("/run/chatmail-turn/turn.socket")
-        with client_socket.makefile("rb") as file:
-            return file.readline().decode("utf-8")

cmdeploy/src/cmdeploy/__init__.py

@@ -128,7 +128,6 @@ def _install_remote_venv_with_chatmaild(config) -> None:
         "echobot",
         "chatmail-metadata",
         "lastlogin",
-        "turnserver",
     ):
         execpath = fn if fn != "filtermail-incoming" else "filtermail"
         params = dict(
@@ -498,56 +497,6 @@ def check_config(config):
     return config
 
 
-def deploy_turn_server(config):
-    (url, sha256sum) = {
-        "x86_64": (
-            "https://github.com/chatmail/chatmail-turn/releases/download/v0.3/chatmail-turn-x86_64-linux",
-            "841e527c15fdc2940b0469e206188ea8f0af48533be12ecb8098520f813d41e4",
-        ),
-        "aarch64": (
-            "https://github.com/chatmail/chatmail-turn/releases/download/v0.3/chatmail-turn-aarch64-linux",
-            "a5fc2d06d937b56a34e098d2cd72a82d3e89967518d159bf246dc69b65e81b42",
-        ),
-    }[host.get_fact(facts.server.Arch)]
-
-    need_restart = False
-
-    existing_sha256sum = host.get_fact(Sha256File, "/usr/local/bin/chatmail-turn")
-    if existing_sha256sum != sha256sum:
-        server.shell(
-            name="Download chatmail-turn",
-            commands=[
-                f"(curl -L {url} >/usr/local/bin/chatmail-turn.new && (echo '{sha256sum} /usr/local/bin/chatmail-turn.new' | sha256sum -c) && mv /usr/local/bin/chatmail-turn.new /usr/local/bin/chatmail-turn)",
-                "chmod 755 /usr/local/bin/chatmail-turn",
-            ],
-        )
-        need_restart = True
-
-    source_path = importlib.resources.files(__package__).joinpath(
-        "service", "turnserver.service.f"
-    )
-    content = source_path.read_text().format(mail_domain=config.mail_domain).encode()
-
-    systemd_unit = files.put(
-        name="Upload turnserver.service",
-        src=io.BytesIO(content),
-        dest="/etc/systemd/system/turnserver.service",
-        user="root",
-        group="root",
-        mode="644",
-    )
-    need_restart |= systemd_unit.changed
-
-    systemd.service(
-        name="Setup turnserver service",
-        service="turnserver.service",
-        running=True,
-        enabled=True,
-        restarted=need_restart,
-        daemon_reload=systemd_unit.changed,
-    )
-
-
 def deploy_mtail(config):
     # Uninstall mtail package, we are going to install a static binary.
     apt.packages(name="Uninstall mtail", packages=["mtail"], present=False)
@@ -724,8 +673,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
         packages=["rsync"],
     )
 
-    deploy_turn_server(config)
-
     # Run local DNS resolver `unbound`.
     # `resolvconf` takes care of setting up /etc/resolv.conf
     # to use 127.0.0.1 as the resolver.
@@ -780,7 +727,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
     # Deploy acmetool to have TLS certificates.
     tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"]
     deploy_acmetool(
-        email=config.acme_email,
         domains=tls_domains,
     )
 
@@ -819,7 +765,7 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
         if build_dir:
             www_path = build_webpages(src_dir, build_dir, config)
         # if it is not a hugo page, upload it as is
-        files.rsync(f"{www_path}/", "/var/www/html", flags=["-avz", "--chown=www-data"])
+        files.rsync(f"{www_path}/", "/var/www/html", flags=["-avz"])
 
     _install_remote_venv_with_chatmaild(config)
     debug = False
@@ -867,13 +813,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
         restarted=nginx_need_restart,
     )
 
-    systemd.service(
-        name="Start and enable fcgiwrap",
-        service="fcgiwrap.service",
-        running=True,
-        enabled=True,
-    )
-
     systemd.service(
         name="Restart echobot if postfix and dovecot were just started",
         service="echobot.service",

cmdeploy/src/cmdeploy/cmdeploy.py

@@ -19,7 +19,7 @@ from packaging import version
 from termcolor import colored
 
 from . import dns, remote
-from .sshexec import SSHExec, LocalExec
+from .sshexec import SSHExec
 
 #
 # cmdeploy sub commands and options
@@ -32,30 +32,17 @@ def init_cmd_options(parser):
         action="store",
         help="fully qualified DNS domain name for your chatmail instance",
     )
-    parser.add_argument(
-        "--force",
-        dest="recreate_ini",
-        action="store_true",
-        help="force reacreate ini file",
-    )
 
 
 def init_cmd(args, out):
     """Initialize chatmail config file."""
     mail_domain = args.chatmail_domain
-    inipath = args.inipath
     if args.inipath.exists():
-        if not args.recreate_ini:
+        print(f"Path exists, not modifying: {args.inipath}")
-            print(f"[WARNING] Path exists, not modifying: {inipath}")
         return 1
     else:
-            print(
+        write_initial_config(args.inipath, mail_domain, overrides={})
-                f"[WARNING] Force argument was provided, deleting config file: {inipath}"
+        out.green(f"created config file for {mail_domain} in {args.inipath}")
-            )
-            inipath.unlink()
-
-    write_initial_config(inipath, mail_domain, overrides={})
-    out.green(f"created config file for {mail_domain} in {inipath}")
 
 
 def run_cmd_options(parser):
@@ -72,12 +59,10 @@ def run_cmd_options(parser):
         help="install/upgrade the server, but disable postfix & dovecot for now",
     )
     parser.add_argument(
-        "--skip-dns-check",
+        "--ssh-host",
-        dest="dns_check_disabled",
+        dest="ssh_host",
-        action="store_true",
+        help="Deploy to 'localhost' or to a specific SSH host",
-        help="disable checks nslookup for dns",
     )
-    add_ssh_host_option(parser)
 
 
 def run_cmd(args, out):
@@ -86,7 +71,6 @@ def run_cmd(args, out):
     ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
     sshexec = get_sshexec(ssh_host)
     require_iroh = args.config.enable_iroh_relay
-    if not args.dns_check_disabled:
     remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
     if not dns.check_initial_remote_data(remote_data, print=out.red):
         return 1
@@ -99,7 +83,7 @@ def run_cmd(args, out):
     pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
 
     cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
-    if ssh_host in ["localhost", "@docker"]:
+    if ssh_host == "localhost":
         cmd = f"{pyinf} @local {deploy_path} -y"
 
     if version.parse(pyinfra.__version__) < version.parse("3"):
@@ -109,7 +93,6 @@ def run_cmd(args, out):
     try:
         retcode = out.check_call(cmd, env=env)
         if retcode == 0:
-            if not args.disable_mail:
             print("\nYou can try out the relay by talking to this echo bot: ")
             sshexec = SSHExec(args.config.mail_domain, verbose=args.verbose)
             print(
@@ -139,7 +122,11 @@ def dns_cmd_options(parser):
         default=None,
         help="write out a zonefile",
     )
-    add_ssh_host_option(parser)
+    parser.add_argument(
+        "--ssh-host",
+        dest="ssh_host",
+        help="Run the DNS queries on 'localhost' or on a specific SSH host",
+    )
 
 
 def dns_cmd(args, out):
@@ -221,6 +208,61 @@ def test_cmd(args, out):
     return ret
 
 
+def proxy_cmd_options(parser: argparse.ArgumentParser):
+    parser.add_argument(
+        "ip_address",
+        help="specify a server to deploy to; can also be an inventory.py file",
+    )
+    parser.add_argument(
+        "--relay-ipv4",
+        dest="relay_ipv4",
+        help="The ipv4 address of the relay you want to forward traffic to",
+    )
+    parser.add_argument(
+        "--relay-ipv6",
+        dest="relay_ipv6",
+        help="The ipv6 address of the relay you want to forward traffic to",
+    )
+    parser.add_argument(
+        "--dry-run",
+        dest="dry_run",
+        action="store_true",
+        help="don't actually modify the server",
+    )
+
+
+def proxy_cmd(args, out):
+    """Deploy reverse proxy on a second server."""
+    env = os.environ.copy()
+    env["RELAY_IPV4"] = args.relay_ipv4
+    env["RELAY_IPV6"] = args.relay_ipv6
+    deploy_path = importlib.resources.files(__package__).joinpath("proxy-deploy.py").resolve()
+    pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
+
+    sshexec = args.get_sshexec()
+    # :todo make sure relay is deployed to args.relay_ipv4 and args.relay_ipv6
+
+    # abort if IP address == the chatmail relay itself: if port 22 is open AND /etc/chatmail-version exists
+    if sshexec.logged(call=remote.rshell.get_port_service, args=[22]):
+        if sshexec.logged(call=remote.rshell.chatmail_version):
+            out.red("Can not deploy proxy on the chatmail relay itself, use another server")
+            return 1
+        cmd = f"{pyinf} --ssh-user root {args.ip_address} {deploy_path} -y"
+        out.check_call(cmd, env=env)  # during first try, only set SSH port to 2222
+
+    cmd = f"{pyinf} --ssh-port 2222 --ssh-user root {args.ip_address} {deploy_path} -y"
+    try:
+        retcode = out.check_call(cmd, env=env)
+        if retcode == 0:
+            out.green("Reverse proxy deployed - you can distribute the IP address now.")
+        else:
+            out.red("Deploying reverse proxy failed")
+    except subprocess.CalledProcessError:
+        out.red("Deploying reverse proxy failed")
+        retcode = 1
+    return retcode
+
+
 def fmt_cmd_options(parser):
     parser.add_argument(
         "--check",
@@ -299,15 +341,6 @@ class Out:
         return proc.returncode
 
 
-def add_ssh_host_option(parser):
-    parser.add_argument(
-        "--ssh-host",
-        dest="ssh_host",
-        help="Run commands on 'localhost', via '@docker', or on a specific SSH host "
-             "instead of chatmail.ini's mail_domain.",
-    )
-
-
 def add_config_option(parser):
     parser.add_argument(
         "--config",
@@ -365,9 +398,7 @@ def get_parser():
 
 def get_sshexec(ssh_host: str, verbose=True):
     if ssh_host in ["localhost", "@local"]:
-        return LocalExec(verbose, docker=False)
+        return "localhost"
-    elif ssh_host == "@docker":
-        return LocalExec(verbose, docker=True)
     if verbose:
         print(f"[ssh] login to {ssh_host}")
     return SSHExec(ssh_host, verbose=verbose)

cmdeploy/src/cmdeploy/dns.py

@@ -7,9 +7,13 @@ from . import remote
 
 
 def get_initial_remote_data(sshexec, mail_domain):
-    return sshexec.logged(
+    if sshexec == "localhost":
+        result = remote.rdns.perform_initial_checks(mail_domain)
+    else:
+        result = sshexec.logged(
             call=remote.rdns.perform_initial_checks, kwargs=dict(mail_domain=mail_domain)
         )
+    return result
 
 
 def check_initial_remote_data(remote_data, *, print=print):
@@ -44,6 +48,11 @@ def check_full_zone(sshexec, remote_data, out, zonefile) -> int:
     """Check existing DNS records, optionally write them to zone file
     and return (exitcode, remote_data) tuple."""
 
+    if sshexec == "localhost":
+        required_diff, recommended_diff = remote.rdns.check_zonefile(
+            zonefile=zonefile, verbose=False
+        )
+    else:
         required_diff, recommended_diff = sshexec.logged(
             remote.rdns.check_zonefile, kwargs=dict(zonefile=zonefile, verbose=False),
         )

cmdeploy/src/cmdeploy/iroh-relay.toml

@@ -1,11 +1,5 @@
 enable_relay = true
 http_bind_addr = "[::]:3340"
-
+enable_stun = true
-# Disable built-in STUN server in iroh-relay 0.35
-# as we deploy our own TURN server instead.
-# STUN server is going to be removed in iroh-relay 1.0
-# and this line can be removed after upgrade.
-enable_stun = false
-
 enable_metrics = false
 metrics_bind_addr = "127.0.0.1:9092"

cmdeploy/src/cmdeploy/opendkim/opendkim.conf

@@ -13,7 +13,6 @@ OversignHeaders		From
 On-BadSignature         reject
 On-KeyNotFound          reject
 On-NoSignature          reject
-DNSTimeout          60
 
 # Signing domain, selector, and key (required). For example, perform signing
 # for domain "example.com" with selector "2020" (2020._domainkey.example.com),

cmdeploy/src/cmdeploy/postfix/master.cf.j2

@@ -31,7 +31,6 @@ submission inet n       -       y       -       5000    smtpd
   -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
   -o milter_macro_daemon_name=ORIGINATING
   -o smtpd_client_connection_count_limit=1000
-  -o smtpd_proxy_options=speed_adjust
   -o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
 smtps     inet  n       -       y       -       5000    smtpd
   -o syslog_name=postfix/smtps
@@ -49,7 +48,6 @@ smtps     inet  n       -       y       -       5000    smtpd
   -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
   -o smtpd_client_connection_count_limit=1000
   -o milter_macro_daemon_name=ORIGINATING
-  -o smtpd_proxy_options=speed_adjust
   -o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
 #628       inet  n       -       y       -       -       qmqpd
 pickup    unix  n       -       y       60      1       pickup

cmdeploy/src/cmdeploy/proxy-deploy.py

@@ -0,0 +1,19 @@
+import os
+
+import pyinfra
+from pyinfra import host
+
+from proxy import configure_ssh, configure_proxy
+
+
+def main():
+    ipv4_relay = os.getenv("IPV4_RELAY")
+    ipv6_relay = os.getenv("IPV6_RELAY")
+
+    configure_ssh()
+    if host.data.get("ssh_port") not in (None, 22):
+        configure_proxy(ipv4_relay, ipv6_relay)
+
+
+if pyinfra.is_cli:
+    main()

cmdeploy/src/cmdeploy/proxy.py

@@ -0,0 +1,63 @@
+import importlib
+
+from pyinfra import host
+from pyinfra.operations import files, server, apt, systemd
+
+def configure_ssh():
+    files.replace(
+        name="Configure sshd to use port 2222",
+        path="/etc/ssh/sshd_config",
+        text="Port 22\n",
+        replace="Port 2222\n",
+    )
+    systemd.service(
+        name="apply SSH config",
+        service="ssh",
+        reloaded=True,
+    )
+    apt.update()
+
+
+def configure_proxy(ipv4_relay, ipv6_relay):
+    files.put(
+        name="Configure nftables",
+        src=importlib.resources.files(__package__).joinpath("proxy_files/nftables.conf.j2"),
+        dest="/etc/nftables.conf",
+        ipv4_address=ipv4_relay,  # :todo what if only one of them is specified?
+        ipv6_address=ipv6_relay,
+    )
+
+    server.sysctl(name="enable IPv4 forwarding", key="net.ipv4.ip_forward", value=1, persist=True)
+
+    server.sysctl(
+        name="enable IPv6 forwarding",
+        key="net.ipv6.conf.all.forwarding",
+        value=1,
+        persist=True,
+    )
+
+    server.shell(
+        name="apply forwarding configuration",
+        commands=[
+            "sysctl -p",
+            "nft -f /etc/nftables.conf",
+        ],
+    )
+
+    if host.data.get("floating_ips"):
+        i = 0
+        for floating_ip in host.data.get("floating_ips"):
+            i += 1
+            files.template(
+                name="Add floating IPs",
+                src="servers/proxy-nine/files/60-floating.ip.cfg.j2",
+                dest=f"/etc/network/interfaces.d/{59 + i}-floating.ip.cfg",
+                ip_address=floating_ip,
+                i=i,
+            )
+
+        systemd.service(
+            name="apply floating IPs",
+            service="networking",
+            restarted=True,
+        )

cmdeploy/src/cmdeploy/proxy_files/60-floating.ip.cfg.j2

@@ -0,0 +1,4 @@
+auto eth0:{{ i }}
+iface eth0:{{ i }} inet static
+    address {{ ip_address }}
+    netmask 32

cmdeploy/src/cmdeploy/proxy_files/nftables.conf.j2

@@ -0,0 +1,67 @@
+#!/usr/sbin/nft -f
+
+flush ruleset
+
+define wan = eth0
+
+# which ports to proxy
+define ports = { smtp, http, https, imap, imaps, submission, submissions }
+
+# the host we want to proxy to
+define ipv4_address = {{ ipv4_address }}
+define ipv6_address = [{{ ipv6_address }}]
+
+table ip nat {
+        chain prerouting {
+                type nat hook prerouting priority dstnat; policy accept;
+                iif $wan tcp dport $ports dnat to $ipv4_address
+        }
+
+        chain postrouting {
+                type nat hook postrouting priority 0;
+
+                oifname $wan masquerade
+        }
+}
+
+table ip6 nat {
+        chain prerouting {
+                type nat hook prerouting priority dstnat; policy accept;
+                iif $wan tcp dport $ports dnat to $ipv6_address
+        }
+
+        chain postrouting {
+                type nat hook postrouting priority 0;
+
+                oifname $wan masquerade
+        }
+}
+
+table inet filter {
+        chain input {
+                type filter hook input priority filter; policy drop;
+
+                # Accept ICMP.
+                # It is especially important to accept ICMPv6 ND messages,
+                # otherwise IPv6 connectivity breaks.
+                icmp type { echo-request } accept
+                icmpv6 type { echo-request, nd-neighbor-solicit, nd-router-advert, nd-neighbor-advert } accept
+
+                # Allow incoming SSH connections.
+                tcp dport { 22, 2222 } accept
+                # Allow incoming shadowsocks connections.
+                tcp dport { 8388 } accept
+
+                ct state established accept
+        }
+        chain forward {
+                type filter hook forward priority filter; policy drop;
+
+                ct state established accept
+                ip daddr $ipv4_address counter accept
+                ip6 daddr $ipv6_address counter accept
+        }
+        chain output {
+                type filter hook output priority filter;
+        }
+}

cmdeploy/src/cmdeploy/remote/rshell.py

@@ -21,6 +21,20 @@ def shell(command, fail_ok=False, print=print):
         return ""
 
 
+def get_port_service(port: int) -> str:
+    return shell(
+        "ss -lptn 'src :%d' | awk 'NR>1 {print $6,$7}' | sed 's/users:((\"//;s/\".*//'"
+        % (port,)
+    )
+
+
+def chatmail_version():
+    version = shell("cat /etc/chatmail-version")
+    if "cat: /etc/chatmail-version:" in version:
+        version = None
+    return version
+
+
 def get_systemd_running():
     lines = shell("systemctl --type=service --state=running").split("\n")
     return [line for line in lines if line.startswith("  ")]

cmdeploy/src/cmdeploy/service/turnserver.service.f

@@ -1,16 +0,0 @@
-[Unit]
-Description=A wrapper for the TURN server
-After=network.target
-
-[Service]
-Type=simple
-Restart=always
-ExecStart=/usr/local/bin/chatmail-turn --realm {mail_domain} --socket /run/chatmail-turn/turn.socket
-
-# Create /run/chatmail-turn
-RuntimeDirectory=chatmail-turn
-User=vmail
-Group=vmail
-
-[Install]
-WantedBy=multi-user.target

cmdeploy/src/cmdeploy/sshexec.py

@@ -82,19 +82,3 @@ class SSHExec:
             res = self(call, kwargs, log_callback=remote.rshell.log_progress)
             print_stderr()
             return res
-
-
-class LocalExec:
-    def __init__(self, verbose=False, docker=False):
-        self.verbose = verbose
-        self.docker = docker
-
-    def logged(self, call, kwargs: dict):
-        where = "locally"
-        if self.docker:
-            if call == remote.rdns.perform_initial_checks:
-                kwargs['pre_command'] = "docker exec chatmail "
-                where = "in docker"
-        if self.verbose:
-            print(f"Running {where}: {call.__name__}(**{kwargs})")
-        return call(**kwargs)

cmdeploy/src/cmdeploy/tests/online/test_1_basic.py

@@ -2,7 +2,6 @@ import datetime
 import smtplib
 import socket
 import subprocess
-import time
 
 import pytest
 
@@ -32,7 +31,6 @@ class TestSSHExecutor:
         )
         out, err = capsys.readouterr()
         assert err.startswith("Collecting")
-        # XXX could not figure out how capturing can be made to work properly
         #assert err.endswith("....\n")
         assert err.count("\n") == 1
 
@@ -42,7 +40,6 @@ class TestSSHExecutor:
         )
         out, err = capsys.readouterr()
         lines = err.split("\n")
-        # XXX could not figure out how capturing can be made to work properly
         #assert len(lines) > 4
         assert remote.rdns.perform_initial_checks.__doc__ in lines[0]
 
@@ -72,7 +69,7 @@ def test_timezone_env(remote):
     for line in remote.iter_output("env"):
         print(line)
         if line == "tz=:/etc/localtime":
-            return
+            return True
     pytest.fail("TZ is not set")
 
 
@@ -149,16 +146,6 @@ def test_reject_missing_dkim(cmsetup, maildata, from_addr):
             s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg)
 
 
-def try_n_times(n, f):
-    for _ in range(n - 1):
-        try:
-            return f()
-        except Exception:
-            time.sleep(1)
-
-    return f()
-
-
 def test_rewrite_subject(cmsetup, maildata):
     """Test that subject gets replaced with [...]."""
     user1, user2 = cmsetup.gen_users(2)
@@ -171,8 +158,7 @@ def test_rewrite_subject(cmsetup, maildata):
     ).as_string()
     user1.smtp.sendmail(from_addr=user1.addr, to_addrs=[user2.addr], msg=sent_msg)
 
-    # The message may need some time to get delivered by postfix.
+    messages = user2.imap.fetch_all_messages()
-    messages = try_n_times(5, user2.imap.fetch_all_messages)
     assert len(messages) == 1
     rcvd_msg = messages[0]
     assert "Subject: [...]" not in sent_msg
@@ -195,8 +181,9 @@ def test_exceed_rate_limit(cmsetup, gencreds, maildata, chatmail_config):
         except smtplib.SMTPException as e:
             if i < chatmail_config.max_user_send_per_minute:
                 pytest.fail(f"rate limit was exceeded too early with msg {i}")
-            assert e.smtp_code == 450
+            outcome = e.recipients[user2.addr]
-            assert b"4.7.1: Too much mail from" in e.smtp_error
+            assert outcome[0] == 450
+            assert b"4.7.1: Too much mail from" in outcome[1]
             return
     pytest.fail("Rate limit was not exceeded")
 
@@ -222,14 +209,8 @@ def test_expunged(remote, chatmail_config):
 
 
 def test_deployed_state(remote):
-    try:
     git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode()
-    except Exception:
-        git_hash = "unknown\n"
-    try:
     git_diff = subprocess.check_output(["git", "diff"]).decode()
-    except Exception:
-        git_diff = ""
     git_status = [git_hash.strip()]
     for line in git_diff.splitlines():
         git_status.append(line.strip().lower())

cmdeploy/src/cmdeploy/tests/test_cmdeploy.py

@@ -26,15 +26,10 @@ class TestCmdline:
     def test_init_not_overwrite(self, capsys):
         assert main(["init", "chat.example.org"]) == 0
         capsys.readouterr()
-
         assert main(["init", "chat.example.org"]) == 1
         out, err = capsys.readouterr()
         assert "path exists" in out.lower()
 
-        assert main(["init", "chat.example.org", "--force"]) == 0
-        out, err = capsys.readouterr()
-        assert "deleting config file" in out.lower()
-
 
 def test_www_folder(example_config, tmp_path):
     reporoot = importlib.resources.files(__package__).joinpath("../../../../").resolve()