deploy chatmaild in a virtualenv to make it easier to add dependencies

.github/CODE_OF_CONDUCT.md

@@ -1,4 +0,0 @@
-
-Please refer to 
-[Delta Chat community standards and practices](https://delta.chat/en/community-standards)
-which also apply for all chatmail developments. 

.github/workflows/ci.yaml

@@ -6,32 +6,26 @@ on:
 
 jobs:
   tox:
-    name: isolated chatmaild tests
+    name: chatmail tests
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v3
-
       - name: run chatmaild tests 
         working-directory: chatmaild
         run: pipx run tox
+      - name: run deploy-chatmail offline tests 
+        working-directory: deploy-chatmail
+        run: pipx run tox
+      - name: run deploy-chatmail offline tests 
+        working-directory: deploy-chatmail
+        run: pipx run tox
 
   scripts:
-    name: deploy-chatmail tests 
+    name: chatmail script invocations 
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v3
-
+      - name: run init.sh 
-      - name: initenv 
+        run: ./scripts/init.sh 
-        run: scripts/initenv.sh
+      - name: run test.sh 
-
+        run: ./scripts/test.sh 
-      - name: append venv/bin to PATH
-        run: echo venv/bin >>$GITHUB_PATH
-
-      - name: run formatting checks 
-        run: cmdeploy fmt -v 
-
-      - name: run deploy-chatmail offline tests 
-        run: pytest --pyargs cmdeploy 
-
-      # all other cmdeploy commands require a staging server 
-      # see https://github.com/deltachat/chatmail/issues/100

.gitignore

@@ -3,8 +3,10 @@ __pycache__/
 *.py[cod]
 *$py.class
 *.swp
+www/privacy.html*
+www/index.html*
+www/info.html*
 *qr-*.png
-chatmail.ini
 
 
 # C extensions

LICENSE

@@ -1,21 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2023, chatmail and delta chat teams
-
-Permission is hereby granted, free of charge, to any person obtaining a copy of
-this software and associated documentation files (the "Software"), to deal in
-the Software without restriction, including without limitation the rights to
-use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
-of the Software, and to permit persons to whom the Software is furnished to do
-so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

README.md

@@ -1,9 +1,9 @@
 
 <img width="800px" src="www/src/collage-top.png"/>
 
-# Chatmail services optimized for Delta Chat apps 
+# Chatmail instances optimized for Delta Chat apps 
 
-This repository helps to setup a ready-to-use chatmail server
+This repository helps to setup a ready-to-use chatmail instance
 comprised of a minimal setup of the battle-tested 
 [postfix smtp](https://www.postfix.org) and [dovecot imap](https://www.dovecot.org) services. 
 
@@ -13,150 +13,119 @@ for use by [Delta Chat apps](https://delta.chat).
 Chatmail accounts are automatically created by a first login, 
 after which the initially specified password is required for using them. 
 
-## Deploying your own chatmail server 
+## Getting Started deploying your own chatmail instance
 
-We use `chat.example.org` as the chatmail domain in the following steps. 
+1. Prepare your local (presumably Linux) system:
-Please substitute it with your own domain. 
 
-1. Install the `cmdeploy` command in a virtualenv
+    scripts/init.sh 
 
-   ```
+2. Setup a domain with `A` and `AAAA` records for your chatmail server.
-    git clone https://github.com/deltachat/chatmail
-    cd chatmail
-    scripts/initenv.sh
-   ```
 
-2. Create chatmail configuration file `chatmail.ini`:
+3. Set environment variable to the chatmail domain you want to setup:
 
-   ```
+    export CHATMAIL_DOMAIN=c1.testrun.org   # replace with your host
-    scripts/cmdeploy init chat.example.org  # <-- use your domain 
-   ```
 
-3. Setup first DNS records for your chatmail domain, 
+4. Fill in privacy contact data into the `chatmail.ini` file
-   according to the hints provided by `cmdeploy init`.
-   Verify that SSH root login works:
 
-   ```
+5. Deploy the chat mail instance to your chatmail server: 
-    ssh root@chat.example.org   # <-- use your domain 
-   ```
 
-4. Deploy to the remote chatmail server:
+    scripts/deploy.sh 
 
-   ```
+   This script remotely sets up packages and configures the chatmail provider. 
-    scripts/cmdeploy run
-   ```
-   This script will also show you additional DNS records
-   which you should configure at your DNS provider
-   (it can take some time until they are public).
 
-### Other helpful commands:
+6. Run `scripts/generate-dns-zone.sh` and 
+   transfer the generated DNS records at your DNS provider
 
-To check the status of your remotely running chatmail service:
 
-```
+### Home page and getting started for users 
-scripts/cmdeploy status
+
+The `deploy.sh` script deploys 
+
+- a default `index.html` along with a QR code that users can click to 
+  create accounts on your chatmail provider,
+
+- a default `info.html` that is linked from the home page,
+
+- a default `policy.html` that is linked from the home page. 
+
+All files are generated by the according markdown `.md` file in the `www` directory.
+
+
+### Refining the web pages 
+
+The `scripts/webdev.sh` script supports live development of the chatmail web presence:
+
+``` 
+    scripts/init.sh  # to locally initialize python virtual environments etc. 
+    scripts/webdev.sh
 ```
 
-To check whether your DNS records are correct:
+- uses the `www/src/page-layout.html` file for producing html documents
+  from `www/src/*.md` files. 
 
-```
+- continously builds the web presence reading files from `www/src` directory
-scripts/cmdeploy dns
+  and generating html files and copying assets to the `www/build` directory. 
-```
 
-To test whether your chatmail service is working correctly:
+- Starts a browser window automatically where you can "refresh" as needed. 
 
-```
+Note that this script is not needed for running `scripts/deploy.sh" 
-scripts/cmdeploy test
+which deploys the whole chatmail setup remotely. 
-```
+The code that generates the web pages is identical
+which means that `webdev.sh` gives a pretty good preview. 
 
-To measure the performance of your chatmail service:
+### Ports
 
-```
+Postfix listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
-scripts/cmdeploy bench
+Dovecot listens on ports 143(imap) and 993 (imaps).
-```
 
-## Overview of this repository
+Delta Chat will, however, discover all ports and configurations 
+automatically by reading the `autoconfig.xml` file from the chatmail instance. 
 
-This repository drives the development of chatmail services, 
-comprised of minimal setups of
 
-- [postfix smtp server](https://www.postfix.org)
+## Emergency Commands to disable automatic account creation 
-- [dovecot imap server](https://www.dovecot.org)
 
-as well as custom services that are integrated with these two:
+If you need to stop account creation,
+e.g. because some script is wildly creating accounts, run:
+
+    touch /etc/chatmail-nocreate
+
+While this file is present, account creation will be blocked. 
+
+
+## Running tests and benchmarks (offline and online) 
+
+1. Set `CHATMAIL_SSH` so that `ssh root@$CHATMAIL_SSH` allows 
+   to login to the chatmail instance server. 
+
+2. To run local and online tests: 
+
+    scripts/test.sh 
+
+3. To run benchmarks against your chatmail instance: 
+
+    scripts/bench.sh 
+
+
+## Development Background for chatmail instances 
+
+This repository drives the development of "chatmail instances", 
+comprised of minimal setups of 
+
+- [postfix smtp server](https://www.postfix.org) 
+- [dovecot imap server](https://www.dovecot.org) 
+
+as well as two custom services that are integrated with these two: 
 
 - `chatmaild/src/chatmaild/doveauth.py` implements
   create-on-login account creation semantics and is used
   by Dovecot during login authentication and by Postfix
   which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)
   to authenticate users
-  to send mails for them.
+  to send mails for them. 
 
-- `chatmaild/src/chatmaild/filtermail.py` prevents
+- `chatmaild/src/chatmaild/filtermail.py` prevents 
-  unencrypted e-mail from leaving the chatmail service
+  unencrypted e-mail from leaving the chatmail instance
-  and is integrated into postfix's outbound mail pipelines.
+  and is integrated into postfix's outbound mail pipelines. 
-
-There is also the `cmdeploy/src/cmdeploy/cmdeploy.py` command line tool
-which helps with setting up and managing the chatmail service.
-`cmdeploy run` uses [pyinfra-based scripting](https://pyinfra.com/)
-in `cmdeploy/src/cmdeploy/__init__.py`
-to automatically install all chatmail components on a server.
 
 
-### Home page and getting started for users
-
-`cmdeploy run` also creates default static Web pages and deploys them
-to a nginx web server with: 
-
-- a default `index.html` along with a QR code that users can click to
-  create accounts on your chatmail provider,
-
-- a default `info.html` that is linked from the home page,
-
-- a default `policy.html` that is linked from the home page.
-
-All `.html` files are generated
-by the according markdown `.md` file in the `www/src` directory.
-
-
-### Refining the web pages
-
-
-```
-scripts/cmdeploy webdev
-```
-
-This starts a local live development cycle for chatmail Web pages:
-
-- uses the `www/src/page-layout.html` file for producing static
-  HTML pages from `www/src/*.md` files
-
-- continously builds the web presence reading files from `www/src` directory
-  and generating html files and copying assets to the `www/build` directory.
-
-- Starts a browser window automatically where you can "refresh" as needed.
-
-
-## Emergency Commands to disable automatic account creation
-
-If you need to stop account creation,
-e.g. because some script is wildly creating accounts,
-login to the server with ssh and run:
-
-```
-    touch /etc/chatmail-nocreate
-```
-
-While this file is present, account creation will be blocked.
-
-### Ports
-
-[Postfix](http://www.postfix.org/) listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
-[Dovecot](https://www.dovecot.org/) listens on ports 143 (imap) and 993 (imaps).
-[nginx](https://www.nginx.com/) listens on port 443 (https).
-[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (http).
-
-Delta Chat apps will, however, discover all ports and configurations
-automatically by reading the [autoconfig XML file](https://web.archive.org/web/20210624004729/https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration) from the chatmail service.
-
 

chatmail.ini

@@ -1,16 +1,15 @@
-
+[config]
-[privacy]
-
-passthrough_recipients = privacy@testrun.org xstore@testrun.org groupsbot@hispanilandia.net
 
 privacy_postal =
     Merlinux GmbH, Represented by the managing director H. Krekel,
     Reichgrafen Str. 20, 79102 Freiburg, Germany
 
-privacy_mail = privacy@testrun.org
+privacy_mail = delta-privacy@merlinux.eu
+
 privacy_pdo =
     Prof. Dr. Fabian Schmieder, lexICT UG (limited), Ostfeldstr. 49, 30559 Hannover.
     You can contact him at *delta-privacy@merlinux.eu* (Keyword: DPO)
+
 privacy_supervisor =
     State Commissioner for Data Protection and Freedom of Information of
     Baden-Württemberg in 70173 Stuttgart, Germany.

chatmaild/MANIFEST.in

@@ -1,4 +0,0 @@
-include src/chatmaild/*.f 
-include src/chatmaild/ini/*.ini.f
-include src/chatmaild/ini/*.ini
-include src/chatmaild/tests/mail-data/*

chatmaild/pyproject.toml

@@ -1,31 +1,17 @@
 [build-system]
-requires = ["setuptools>=61"]
+requires = ["setuptools>=45"]
 build-backend = "setuptools.build_meta"
 
 [project]
 name = "chatmaild"
-version = "0.2"
+version = "0.1"
 dependencies = [
   "aiosmtpd",
-  "iniconfig",
-  "deltachat-rpc-server",
-  "deltachat-rpc-client",
 ]
 
-[tool.setuptools]
-include-package-data = true
-
-[tool.setuptools.packages.find]
-where = ['src']
-
 [project.scripts]
 doveauth = "chatmaild.doveauth:main"
 filtermail = "chatmaild.filtermail:main"
-echobot = "chatmaild.echo:main"
-chatmail-metrics = "chatmaild.metrics:main"
-
-[project.entry-points.pytest11]
-"chatmaild.testplugin" = "chatmaild.tests.plugin"
 
 [tool.pytest.ini_options]
 addopts = "-v -ra --strict-markers"
@@ -50,7 +36,8 @@ commands =
   ruff src/
 
 [testenv]
+passenv = CHATMAIL_DOMAIN
 deps = pytest 
        pdbpp 
-commands = pytest -v -rsXx {posargs}
+commands = pytest -v -rsXx {posargs: ../tests/chatmaild}
 """

chatmaild/src/chatmaild/config.py

@@ -1,59 +0,0 @@
-import iniconfig
-
-
-def read_config(inipath):
-    cfg = iniconfig.IniConfig(inipath)
-    return Config(inipath, params=cfg.sections["params"])
-
-
-class Config:
-    def __init__(self, inipath, params):
-        self._inipath = inipath
-        self.mail_domain = params["mail_domain"]
-        self.max_user_send_per_minute = int(params["max_user_send_per_minute"])
-        self.max_mailbox_size = params["max_mailbox_size"]
-        self.delete_mails_after = params["delete_mails_after"]
-        self.username_min_length = int(params["username_min_length"])
-        self.username_max_length = int(params["username_max_length"])
-        self.password_min_length = int(params["password_min_length"])
-        self.passthrough_senders = params["passthrough_senders"].split()
-        self.passthrough_recipients = params["passthrough_recipients"].split()
-        self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
-        self.postfix_reinject_port = int(params["postfix_reinject_port"])
-        self.privacy_postal = params.get("privacy_postal")
-        self.privacy_mail = params.get("privacy_mail")
-        self.privacy_pdo = params.get("privacy_pdo")
-        self.privacy_supervisor = params.get("privacy_supervisor")
-
-    def _getbytefile(self):
-        return open(self._inipath, "rb")
-
-
-def write_initial_config(inipath, mail_domain):
-    from importlib.resources import files
-
-    inidir = files(__package__).joinpath("ini")
-    content = (
-        inidir.joinpath("chatmail.ini.f").read_text().format(mail_domain=mail_domain)
-    )
-    if mail_domain.endswith(".testrun.org"):
-        override_inipath = inidir.joinpath("override-testrun.ini")
-        privacy = iniconfig.IniConfig(override_inipath)["privacy"]
-        lines = []
-        for line in content.split("\n"):
-            for key, value in privacy.items():
-                value_lines = value.strip().split("\n")
-                if not line.startswith(f"{key} =") or not value_lines:
-                    continue
-                if len(value_lines) == 1:
-                    lines.append(f"{key} = {value}")
-                else:
-                    lines.append(f"{key} =")
-                    for vl in value_lines:
-                        lines.append(f"    {vl}")
-                break
-            else:
-                lines.append(line)
-        content = "\n".join(lines)
-
-    inipath.write_text(content)

chatmaild/src/chatmaild/doveauth.py

@@ -12,7 +12,6 @@ from socketserver import (
 import pwd
 
 from .database import Database
-from .config import read_config, Config
 
 NOCREATE_FILE = "/etc/chatmail-nocreate"
 
@@ -23,17 +22,14 @@ def encrypt_password(password: str):
     return "{SHA512-CRYPT}" + passhash
 
 
-def is_allowed_to_create(config: Config, user, cleartext_password) -> bool:
+def is_allowed_to_create(user, cleartext_password) -> bool:
     """Return True if user and password are admissable."""
     if os.path.exists(NOCREATE_FILE):
         logging.warning(f"blocked account creation because {NOCREATE_FILE!r} exists.")
         return False
 
-    if len(cleartext_password) < config.password_min_length:
+    if len(cleartext_password) < 9:
-        logging.warning(
+        logging.warning("Password needs to be at least 9 characters long")
-            "Password needs to be at least %s characters long",
-            config.password_min_length,
-        )
         return False
 
     parts = user.split("@")
@@ -42,17 +38,10 @@ def is_allowed_to_create(config: Config, user, cleartext_password) -> bool:
         return False
     localpart, domain = parts
 
-    if (
+    if domain == "nine.testrun.org":
-        len(localpart) > config.username_max_length
+        # nine.testrun.org policy, username has to be exactly nine chars
-        or len(localpart) < config.username_min_length
+        if len(localpart) != 9:
-    ):
+            logging.warning(f"localpart {localpart!r} has not exactly nine chars")
-        if localpart != "echo":
-            logging.warning(
-                "localpart %s has to be between %s and %s chars long",
-                localpart,
-                config.username_min_length,
-                config.username_max_length,
-            )
             return False
 
     return True
@@ -71,7 +60,7 @@ def lookup_userdb(db, user):
     return get_user_data(db, user)
 
 
-def lookup_passdb(db, config: Config, user, cleartext_password):
+def lookup_passdb(db, user, cleartext_password):
     with db.write_transaction() as conn:
         userdata = conn.get_user(user)
         if userdata:
@@ -83,7 +72,7 @@ def lookup_passdb(db, config: Config, user, cleartext_password):
             userdata["uid"] = "vmail"
             userdata["gid"] = "vmail"
             return userdata
-        if not is_allowed_to_create(config, user, cleartext_password):
+        if not is_allowed_to_create(user, cleartext_password):
             return
 
         encrypted_password = encrypt_password(cleartext_password)
@@ -98,59 +87,25 @@ def lookup_passdb(db, config: Config, user, cleartext_password):
         )
 
 
-def split_and_unescape(s):
+def handle_dovecot_request(msg, db, mail_domain):
-    """Split strings using double quote as a separator and backslash as escape character
-    into parts."""
-
-    out = ""
-    i = 0
-    while i < len(s):
-        c = s[i]
-        if c == "\\":
-            # Skip escape character.
-            i += 1
-
-            # This will raise IndexError if there is no character
-            # after escape character. This is expected
-            # as this is an invalid input.
-            out += s[i]
-        elif c == '"':
-            # Separator
-            yield out
-            out = ""
-        else:
-            out += c
-        i += 1
-    yield out
-
-
-def handle_dovecot_request(msg, db, config: Config):
     short_command = msg[0]
     if short_command == "L":  # LOOKUP
         parts = msg[1:].split("\t")
-
+        keyname, user = parts[:2]
-        # Dovecot <2.3.17 has only one part,
+        namespace, type, *args = keyname.split("/")
-        # do not attempt to read any other parts for compatibility.
-        keyname = parts[0]
-
-        namespace, type, args = keyname.split("/", 2)
-        args = list(split_and_unescape(args))
-
         reply_command = "F"
         res = ""
         if namespace == "shared":
             if type == "userdb":
-                user = args[0]
+                if user.endswith(f"@{mail_domain}"):
-                if user.endswith(f"@{config.mail_domain}"):
                     res = lookup_userdb(db, user)
                 if res:
                     reply_command = "O"
                 else:
                     reply_command = "N"
             elif type == "passdb":
-                user = args[1]
+                if user.endswith(f"@{mail_domain}"):
-                if user.endswith(f"@{config.mail_domain}"):
+                    res = lookup_passdb(db, user, cleartext_password=args[0])
-                    res = lookup_passdb(db, config, user, cleartext_password=args[0])
                 if res:
                     reply_command = "O"
                 else:
@@ -168,7 +123,8 @@ def main():
     socket = sys.argv[1]
     passwd_entry = pwd.getpwnam(sys.argv[2])
     db = Database(sys.argv[3])
-    config = read_config(sys.argv[4])
+    with open("/etc/mailname", "r") as fp:
+        mail_domain = fp.read().strip()
 
     class Handler(StreamRequestHandler):
         def handle(self):
@@ -177,7 +133,7 @@ def main():
                     msg = self.rfile.readline().strip().decode()
                     if not msg:
                         break
-                    res = handle_dovecot_request(msg, db, config)
+                    res = handle_dovecot_request(msg, db, mail_domain)
                     if res:
                         self.wfile.write(res.encode("ascii"))
                         self.wfile.flush()

chatmaild/src/chatmaild/doveauth.service.f

@@ -1,8 +1,8 @@
 [Unit]
-Description=Chatmail dict authentication proxy for dovecot
+Description=Dict authentication proxy for dovecot
 
 [Service]
-ExecStart={execpath} /run/dovecot/doveauth.socket vmail /home/vmail/passdb.sqlite {config_path}
+ExecStart={execpath} /run/dovecot/doveauth.socket vmail /home/vmail/passdb.sqlite
 Restart=always
 RestartSec=30
 

chatmaild/src/chatmaild/echo.py

@@ -1,84 +0,0 @@
-#!/usr/bin/env python3
-"""Advanced echo bot example.
-
-it will echo back any message that has non-empty text and also supports the /help command.
-"""
-import logging
-import os
-import sys
-
-from deltachat_rpc_client import Bot, DeltaChat, EventType, Rpc, events
-
-from chatmaild.newemail import create_newemail_dict
-from chatmaild.config import read_config
-
-hooks = events.HookCollection()
-
-
-@hooks.on(events.RawEvent)
-def log_event(event):
-    if event.kind == EventType.INFO:
-        logging.info(event.msg)
-    elif event.kind == EventType.WARNING:
-        logging.warning(event.msg)
-
-
-@hooks.on(events.RawEvent(EventType.ERROR))
-def log_error(event):
-    logging.error(event.msg)
-
-
-@hooks.on(events.MemberListChanged)
-def on_memberlist_changed(event):
-    logging.info(
-        "member %s was %s", event.member, "added" if event.member_added else "removed"
-    )
-
-
-@hooks.on(events.GroupImageChanged)
-def on_group_image_changed(event):
-    logging.info("group image %s", "deleted" if event.image_deleted else "changed")
-
-
-@hooks.on(events.GroupNameChanged)
-def on_group_name_changed(event):
-    logging.info("group name changed, old name: %s", event.old_name)
-
-
-@hooks.on(events.NewMessage(func=lambda e: not e.command))
-def echo(event):
-    snapshot = event.message_snapshot
-    if snapshot.text or snapshot.file:
-        snapshot.chat.send_message(text=snapshot.text, file=snapshot.file)
-
-
-@hooks.on(events.NewMessage(command="/help"))
-def help_command(event):
-    snapshot = event.message_snapshot
-    snapshot.chat.send_text("Send me any message and I will echo it back")
-
-
-def main():
-    path = os.environ.get("PATH")
-    venv_path = sys.argv[0].strip("echobot")
-    os.environ["PATH"] = path + ":" + venv_path
-    with Rpc() as rpc:
-        deltachat = DeltaChat(rpc)
-        system_info = deltachat.get_system_info()
-        logging.info("Running deltachat core %s", system_info.deltachat_core_version)
-
-        accounts = deltachat.get_all_accounts()
-        account = accounts[0] if accounts else deltachat.add_account()
-
-        bot = Bot(account, hooks)
-        if not bot.is_configured():
-            config = read_config(sys.argv[1])
-            password = create_newemail_dict(config).get("password")
-            email = "echo@" + config.mail_domain
-            bot.configure(email, password)
-        bot.run_forever()
-
-
-if __name__ == "__main__":
-    logging.basicConfig(level=logging.INFO)
-    main()

chatmaild/src/chatmaild/echobot.service.f

@@ -1,11 +0,0 @@
-[Unit]
-Description=Chatmail echo bot for testing it works
-
-[Service]
-ExecStart={execpath} {config_path}
-Environment="PATH={remote_venv_dir}:$PATH"
-Restart=always
-RestartSec=30
-
-[Install]
-WantedBy=multi-user.target

chatmaild/src/chatmaild/filtermail.py

@@ -7,11 +7,10 @@ from email.parser import BytesParser
 from email import policy
 from email.utils import parseaddr
 
+from aiosmtpd.smtp import SMTP
 from aiosmtpd.controller import Controller
 from smtplib import SMTP as SMTPClient
 
-from .config import read_config
-
 
 def check_encrypted(message):
     """Check that the message is an OpenPGP-encrypted message."""
@@ -35,6 +34,14 @@ def check_encrypted(message):
     return True
 
 
+def is_passthrough_recipient(recipient):
+    """Check whether a recipient is configured as passthrough."""
+    passthroughlist = ["privacy@testrun.org"]
+    if recipient in passthroughlist:
+        return True
+    return False
+
+
 def check_mdn(message, envelope):
     if len(envelope.rcpt_tos) != 1:
         return False
@@ -63,21 +70,19 @@ def check_mdn(message, envelope):
     return True
 
 
-async def asyncmain_beforequeue(config):
+class SMTPController(Controller):
-    port = config.filtermail_smtp_port
+    def factory(self):
-    Controller(BeforeQueueHandler(config), hostname="127.0.0.1", port=port).start()
+        return SMTP(self.handler, **self.SMTP_kwargs)
 
 
 class BeforeQueueHandler:
-    def __init__(self, config):
+    def __init__(self):
-        self.config = config
         self.send_rate_limiter = SendRateLimiter()
 
     async def handle_MAIL(self, server, session, envelope, address, mail_options):
         logging.info(f"handle_MAIL from {address}")
         envelope.mail_from = address
-        max_sent = self.config.max_user_send_per_minute
+        if not self.send_rate_limiter.is_sending_allowed(address):
-        if not self.send_rate_limiter.is_sending_allowed(address, max_sent):
             return f"450 4.7.1: Too much mail from {address}"
 
         parts = envelope.mail_from.split("@")
@@ -88,64 +93,65 @@ class BeforeQueueHandler:
 
     async def handle_DATA(self, server, session, envelope):
         logging.info("handle_DATA before-queue")
-        error = self.check_DATA(envelope)
+        error = check_DATA(envelope)
         if error:
             return error
         logging.info("re-injecting the mail that passed checks")
-        client = SMTPClient("localhost", self.config.postfix_reinject_port)
+        client = SMTPClient("localhost", "10025")
         client.sendmail(envelope.mail_from, envelope.rcpt_tos, envelope.content)
         return "250 OK"
 
-    def check_DATA(self, envelope):
-        """the central filtering function for e-mails."""
-        logging.info(f"Processing DATA message from {envelope.mail_from}")
 
-        message = BytesParser(policy=policy.default).parsebytes(envelope.content)
+async def asyncmain_beforequeue(port):
-        mail_encrypted = check_encrypted(message)
+    Controller(BeforeQueueHandler(), hostname="127.0.0.1", port=port).start()
 
-        _, from_addr = parseaddr(message.get("from").strip())
-        logging.info(f"mime-from: {from_addr} envelope-from: {envelope.mail_from!r}")
-        if envelope.mail_from.lower() != from_addr.lower():
-            return f"500 Invalid FROM <{from_addr!r}> for <{envelope.mail_from!r}>"
 
-        if not mail_encrypted and check_mdn(message, envelope):
+def check_DATA(envelope):
-            return
+    """the central filtering function for e-mails."""
+    logging.info(f"Processing DATA message from {envelope.mail_from}")
 
-        if envelope.mail_from in self.config.passthrough_senders:
+    message = BytesParser(policy=policy.default).parsebytes(envelope.content)
-            return
+    mail_encrypted = check_encrypted(message)
 
-        passthrough_recipients = self.config.passthrough_recipients
+    _, from_addr = parseaddr(message.get("from").strip())
-        envelope_from_domain = from_addr.split("@").pop()
+    logging.info(f"mime-from: {from_addr} envelope-from: {envelope.mail_from!r}")
-        for recipient in envelope.rcpt_tos:
+    if envelope.mail_from.lower() != from_addr.lower():
-            if envelope.mail_from == recipient:
+        return f"500 Invalid FROM <{from_addr!r}> for <{envelope.mail_from!r}>"
-                # Always allow sending emails to self.
-                continue
-            if recipient in passthrough_recipients:
-                continue
-            res = recipient.split("@")
-            if len(res) != 2:
-                return f"500 Invalid address <{recipient}>"
-            _recipient_addr, recipient_domain = res
 
-            is_outgoing = recipient_domain != envelope_from_domain
+    if not mail_encrypted and check_mdn(message, envelope):
-            if is_outgoing and not mail_encrypted:
+        return
-                is_securejoin = message.get("secure-join") in [
+
-                    "vc-request",
+    envelope_from_domain = from_addr.split("@").pop()
-                    "vg-request",
+    for recipient in envelope.rcpt_tos:
-                ]
+        if envelope.mail_from == recipient:
-                if not is_securejoin:
+            # Always allow sending emails to self.
-                    return f"500 Invalid unencrypted mail to <{recipient}>"
+            continue
+        if is_passthrough_recipient(recipient):
+            # Always allow recipients marked as passthrough
+            continue
+        res = recipient.split("@")
+        if len(res) != 2:
+            return f"500 Invalid address <{recipient}>"
+        _recipient_addr, recipient_domain = res
+
+        is_outgoing = recipient_domain != envelope_from_domain
+        if is_outgoing and not mail_encrypted:
+            is_securejoin = message.get("secure-join") in ["vc-request", "vg-request"]
+            if not is_securejoin:
+                return f"500 Invalid unencrypted mail to <{recipient}>"
 
 
 class SendRateLimiter:
+    MAX_USER_SEND_PER_MINUTE = 80
+
     def __init__(self):
         self.addr2timestamps = {}
 
-    def is_sending_allowed(self, mail_from, max_send_per_minute):
+    def is_sending_allowed(self, mail_from):
         last = self.addr2timestamps.setdefault(mail_from, [])
         now = time.time()
         last[:] = [ts for ts in last if ts >= (now - 60)]
-        if len(last) <= max_send_per_minute:
+        if len(last) <= self.MAX_USER_SEND_PER_MINUTE:
             last.append(now)
             return True
         return False
@@ -154,10 +160,9 @@ class SendRateLimiter:
 def main():
     args = sys.argv[1:]
     assert len(args) == 1
-    config = read_config(args[0])
     logging.basicConfig(level=logging.WARN)
     loop = asyncio.new_event_loop()
     asyncio.set_event_loop(loop)
-    task = asyncmain_beforequeue(config)
+    task = asyncmain_beforequeue(port=int(args[0]))
     loop.create_task(task)
     loop.run_forever()

chatmaild/src/chatmaild/filtermail.service.f

@@ -2,7 +2,7 @@
 Description=Chatmail Postfix BeforeQeue filter 
 
 [Service]
-ExecStart={execpath} {config_path}
+ExecStart={execpath} 10080
 Restart=always
 RestartSec=30
 

chatmaild/src/chatmaild/ini/chatmail.ini.f

@@ -1,63 +0,0 @@
-[params]
-
-# mail domain (MUST be set to fully qualified chat mail domain)
-mail_domain = {mail_domain}
-
-#
-# If you only do private test deploys, you don't need to modify any settings below
-#
-
-#
-# Account Restrictions
-#
-
-# how many mails a user can send out per minute
-max_user_send_per_minute = 60
-
-# maximum mailbox size of a chatmail account
-max_mailbox_size = 100M
-
-# days after which mails are unconditionally deleted
-delete_mails_after = 40
-
-# minimum length a username must have
-username_min_length = 9
-
-# maximum length a username can have
-username_max_length = 9
-
-# minimum length a password must have
-password_min_length = 9
-
-# list of chatmail accounts which can send outbound un-encrypted mail
-passthrough_senders =
-
-# list of e-mail recipients for which to accept outbound un-encrypted mails
-passthrough_recipients = xstore@testrun.org groupsbot@hispanilandia.net
-
-#
-# Deployment Details
-#
-
-# where the filtermail SMTP service listens
-filtermail_smtp_port = 10080
-
-# postfix accepts on the localhost reinject SMTP port
-postfix_reinject_port = 10025
-
-#
-# Privacy Policy
-#
-
-# postal address of privacy contact
-privacy_postal =
-
-# email address of privacy contact
-privacy_mail =
-
-# postal address of the privacy data officer
-privacy_pdo =
-
-# postal address of the privacy supervisor
-privacy_supervisor =
-

chatmaild/src/chatmaild/metrics.py

@@ -1,25 +0,0 @@
-#!/usr/bin/env python3
-from pathlib import Path
-import time
-import sys
-
-
-def main(vmail_dir=None):
-    if vmail_dir is None:
-        vmail_dir = sys.argv[1]
-
-    accounts = 0
-    ci_accounts = 0
-
-    for path in Path(vmail_dir).iterdir():
-        accounts += 1
-        if path.name[:3] in ("ci-", "ac_"):
-            ci_accounts += 1
-
-    timestamp = int(time.time() * 1000)
-    print(f"accounts {accounts} {timestamp}")
-    print(f"ci_accounts {ci_accounts} {timestamp}")
-
-
-if __name__ == "__main__":
-    main()

chatmaild/src/chatmaild/newemail.py

@@ -1,31 +1,23 @@
-#!/usr/local/lib/chatmaild/venv/bin/python3
+#!/usr/bin/python3
 
 """ CGI script for creating new accounts. """
 
 import json
 import random
-import secrets
-import string
 
-from chatmaild.config import read_config, Config
+mailname_path = "/etc/mailname"
-
-CONFIG_PATH = "/usr/local/lib/chatmaild/chatmail.ini"
-ALPHANUMERIC = string.ascii_lowercase + string.digits
-ALPHANUMERIC_PUNCT = string.ascii_letters + string.digits + string.punctuation
 
 
-def create_newemail_dict(config: Config):
+def create_newemail_dict(domain):
-    user = "".join(random.choices(ALPHANUMERIC, k=config.username_min_length))
+    alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
-    password = "".join(
+    user = "".join(random.choices(alphanumeric, k=9))
-        secrets.choice(ALPHANUMERIC_PUNCT)
+    password = "".join(random.choices(alphanumeric, k=12))
-        for _ in range(config.password_min_length + 3)
+    return dict(email=f"{user}@{domain}", password=f"{password}")
-    )
-    return dict(email=f"{user}@{config.mail_domain}", password=f"{password}")
 
 
 def print_new_account():
-    config = read_config(CONFIG_PATH)
+    domain = open(mailname_path).read().strip()
-    creds = create_newemail_dict(config)
+    creds = create_newemail_dict(domain=domain)
 
     print("Content-Type: application/json")
     print("")

chatmaild/src/chatmaild/tests/plugin.py

@@ -1,68 +0,0 @@
-import random
-import importlib.resources
-import itertools
-from email.parser import BytesParser
-from email import policy
-import pytest
-
-from chatmaild.database import Database
-from chatmaild.config import read_config, write_initial_config
-
-
-@pytest.fixture
-def make_config(tmp_path):
-    inipath = tmp_path.joinpath("chatmail.ini")
-
-    def make_conf(mail_domain):
-        write_initial_config(inipath, mail_domain=mail_domain)
-        return read_config(inipath)
-
-    return make_conf
-
-
-@pytest.fixture
-def example_config(make_config):
-    return make_config("chat.example.org")
-
-
-@pytest.fixture
-def maildomain(example_config):
-    return example_config.mail_domain
-
-
-@pytest.fixture
-def gencreds(maildomain):
-    count = itertools.count()
-    next(count)
-
-    def gen(domain=None):
-        domain = domain if domain else maildomain
-        while 1:
-            num = next(count)
-            alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
-            user = "".join(random.choices(alphanumeric, k=10))
-            user = f"ac{num}_{user}"[:9]
-            password = "".join(random.choices(alphanumeric, k=12))
-            yield f"{user}@{domain}", f"{password}"
-
-    return lambda domain=None: next(gen(domain))
-
-
-@pytest.fixture()
-def db(tmpdir):
-    db_path = tmpdir / "passdb.sqlite"
-    print("database path:", db_path)
-    return Database(db_path)
-
-
-@pytest.fixture
-def maildata(request):
-    datadir = importlib.resources.files(__package__).joinpath("mail-data")
-    assert datadir.exists(), datadir
-
-    def maildata(name, from_addr, to_addr):
-        data = datadir.joinpath(name).read_text()
-        text = data.format(from_addr=from_addr, to_addr=to_addr)
-        return BytesParser(policy=policy.default).parsebytes(text.encode())
-
-    return maildata

chatmaild/src/chatmaild/tests/test_config.py

@@ -1,32 +0,0 @@
-from chatmaild.config import read_config
-
-
-def test_read_config_basic(example_config):
-    assert example_config.mail_domain == "chat.example.org"
-    assert not example_config.privacy_supervisor and not example_config.privacy_mail
-    assert not example_config.privacy_pdo and not example_config.privacy_postal
-
-    inipath = example_config._inipath
-    inipath.write_text(inipath.read_text().replace("60", "37"))
-    example_config = read_config(inipath)
-    assert example_config.max_user_send_per_minute == 37
-    assert example_config.mail_domain == "chat.example.org"
-
-
-def test_read_config_testrun(make_config):
-    config = make_config("something.testrun.org")
-    assert config.mail_domain == "something.testrun.org"
-    assert len(config.privacy_postal.split("\n")) > 1
-    assert len(config.privacy_supervisor.split("\n")) > 1
-    assert len(config.privacy_pdo.split("\n")) > 1
-    assert config.privacy_mail == "privacy@testrun.org"
-    assert config.filtermail_smtp_port == 10080
-    assert config.postfix_reinject_port == 10025
-    assert config.max_user_send_per_minute == 60
-    assert config.max_mailbox_size == "100M"
-    assert config.delete_mails_after == "40"
-    assert config.username_min_length == 9
-    assert config.username_max_length == 9
-    assert config.password_min_length == 9
-    assert "privacy@testrun.org" in config.passthrough_recipients
-    assert config.passthrough_senders == []

chatmaild/src/chatmaild/tests/test_metrics.py

@@ -1,16 +0,0 @@
-from chatmaild.metrics import main
-
-
-def test_main(tmp_path, capsys):
-    for x in ("ci-asllkj", "ac_12l3kj", "qweqwe", "ci-l1k2j31l2k3"):
-        tmp_path.joinpath(x).mkdir()
-    main(tmp_path)
-    out, _ = capsys.readouterr()
-    d = {}
-    for line in out.split("\n"):
-        if line.strip():
-            name, num, _ = line.split()
-            d[name] = int(num)
-
-    assert d["accounts"] == 4
-    assert d["ci_accounts"] == 3

cmdeploy/pyproject.toml

@@ -1,32 +0,0 @@
-[build-system]
-requires = ["setuptools>=68"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "cmdeploy"
-version = "0.2"
-dependencies = [
-  "pyinfra",
-  "pillow",
-  "qrcode",
-  "markdown",
-  "pytest",
-  "setuptools>=68",
-  "termcolor",
-  "build",
-  "tox",
-  "ruff",
-  "black",
-  "pytest",
-  "pytest-xdist", 
-]
-
-[project.scripts]
-cmdeploy = "cmdeploy.cmdeploy:main"
-
-[project.entry-points.pytest11]
-"chatmaild.testplugin" = "chatmaild.tests.plugin"
-"cmdeploy.testplugin" = "cmdeploy.tests.plugin"
-
-[tool.pytest.ini_options]
-addopts = "-v -ra --strict-markers"

cmdeploy/src/cmdeploy/acmetool/acmetool-redirector.service

@@ -1,11 +0,0 @@
-[Unit]
-Description=acmetool HTTP redirector
-
-[Service]
-Type=notify
-ExecStart=/usr/bin/acmetool redirector --service.uid=daemon
-Restart=always
-RestartSec=30
-
-[Install]
-WantedBy=multi-user.target

cmdeploy/src/cmdeploy/chatmail.zone.f

@@ -1,14 +0,0 @@
-{chatmail_domain}.                   A     {ipv4}
-{chatmail_domain}.                   AAAA  {ipv6}
-{chatmail_domain}.                   MX 10 {chatmail_domain}.
-_submission._tcp.{chatmail_domain}.  SRV 0 1 587 {chatmail_domain}.
-_submissions._tcp.{chatmail_domain}. SRV 0 1 465 {chatmail_domain}.
-_imap._tcp.{chatmail_domain}.        SRV 0 1 143 {chatmail_domain}.
-_imaps._tcp.{chatmail_domain}.       SRV 0 1 993 {chatmail_domain}.
-{chatmail_domain}.                   CAA 128 issue "letsencrypt.org;accounturi={acme_account_url}"
-{chatmail_domain}.                   TXT "v=spf1 a:{chatmail_domain} -all"
-_dmarc.{chatmail_domain}.            TXT "v=DMARC1;p=reject;rua=mailto:{email};ruf=mailto:{email};fo=1;adkim=r;aspf=r"
-_mta-sts.{chatmail_domain}.          TXT "v=STSv1; id={sts_id}"
-mta-sts.{chatmail_domain}.           CNAME {chatmail_domain}.
-_smtp._tls.{chatmail_domain}.        TXT "v=TLSRPTv1;rua=mailto:{email}"
-{dkim_entry}

cmdeploy/src/cmdeploy/cmdeploy.py

@@ -1,314 +0,0 @@
-"""
-Provides the `cmdeploy` entry point function,
-along with command line option and subcommand parsing.
-"""
-import argparse
-import shutil
-import subprocess
-import importlib.resources
-import importlib.util
-import os
-import sys
-from pathlib import Path
-
-
-from termcolor import colored
-from chatmaild.config import read_config, write_initial_config
-from cmdeploy.dns import show_dns, check_necessary_dns
-
-
-#
-# cmdeploy sub commands and options
-#
-
-
-def init_cmd_options(parser):
-    parser.add_argument(
-        "chatmail_domain",
-        action="store",
-        help="fully qualified DNS domain name for your chatmail instance",
-    )
-
-
-def init_cmd(args, out):
-    """Initialize chatmail config file."""
-    mail_domain = args.chatmail_domain
-    if args.inipath.exists():
-        print(f"Path exists, not modifying: {args.inipath}")
-    else:
-        write_initial_config(args.inipath, mail_domain)
-        out.green(f"created config file for {mail_domain} in {args.inipath}")
-    check_necessary_dns(
-        out,
-        mail_domain,
-    )
-
-
-def run_cmd_options(parser):
-    parser.add_argument(
-        "--dry-run",
-        dest="dry_run",
-        action="store_true",
-        help="don't actually modify the server",
-    )
-
-
-def run_cmd(args, out):
-    """Deploy chatmail services on the remote server."""
-    mail_domain = args.config.mail_domain
-    if not check_necessary_dns(
-        out,
-        mail_domain,
-    ):
-        sys.exit(1)
-
-    env = os.environ.copy()
-    env["CHATMAIL_INI"] = args.inipath
-    deploy_path = importlib.resources.files(__package__).joinpath("deploy.py").resolve()
-    pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
-    cmd = f"{pyinf} --ssh-user root {args.config.mail_domain} {deploy_path}"
-
-    out.check_call(cmd, env=env)
-    print("Deploy completed, call `cmdeploy dns` next.")
-
-
-def dns_cmd_options(parser):
-    parser.add_argument(
-        "--zonefile",
-        dest="zonefile",
-        help="print the whole zonefile for deploying directly",
-    )
-
-
-def dns_cmd(args, out):
-    """Generate dns zone file."""
-    show_dns(args, out)
-
-
-def status_cmd(args, out):
-    """Display status for online chatmail instance."""
-
-    ssh = f"ssh root@{args.config.mail_domain}"
-
-    out.green(f"chatmail domain: {args.config.mail_domain}")
-    if args.config.privacy_mail:
-        out.green("privacy settings: present")
-    else:
-        out.red("no privacy settings")
-
-    s1 = "systemctl --type=service --state=running"
-    for line in out.shell_output(f"{ssh} -- {s1}").split("\n"):
-        if line.startswith("  "):
-            print(line)
-
-
-def test_cmd_options(parser):
-    parser.add_argument(
-        "--slow",
-        dest="slow",
-        action="store_true",
-        help="also run slow tests",
-    )
-
-
-def test_cmd(args, out):
-    """Run local and online tests for chatmail deployment.
-
-    This will automatically pip-install 'deltachat' if it's not available.
-    """
-
-    x = importlib.util.find_spec("deltachat")
-    if x is None:
-        out.check_call(f"{sys.executable} -m pip install deltachat")
-
-    pytest_path = shutil.which("pytest")
-    pytest_args = [
-        pytest_path,
-        "cmdeploy/src/",
-        "-n4",
-        "-rs",
-        "-x",
-        "-vrx",
-        "--durations=5",
-    ]
-    if args.slow:
-        pytest_args.append("--slow")
-    ret = out.run_ret(pytest_args)
-    return ret
-
-
-def fmt_cmd_options(parser):
-    parser.add_argument(
-        "--verbose",
-        "-v",
-        dest="verbose",
-        action="store_true",
-        help="provide information on invocations",
-    )
-
-    parser.add_argument(
-        "--check",
-        "-c",
-        action="store_true",
-        help="only check but don't fix problems",
-    )
-
-
-def fmt_cmd(args, out):
-    """Run formattting fixes (ruff and black) on all chatmail source code."""
-
-    sources = [str(importlib.resources.files(x)) for x in ("chatmaild", "cmdeploy")]
-    black_args = [shutil.which("black")]
-    ruff_args = [shutil.which("ruff")]
-
-    if args.check:
-        black_args.append("--check")
-    else:
-        ruff_args.append("--fix")
-
-    if not args.verbose:
-        black_args.append("-q")
-        ruff_args.append("-q")
-
-    black_args.extend(sources)
-    ruff_args.extend(sources)
-
-    out.check_call(" ".join(black_args), quiet=not args.verbose)
-    out.check_call(" ".join(ruff_args), quiet=not args.verbose)
-    return 0
-
-
-def bench_cmd(args, out):
-    """Run benchmarks against an online chatmail instance."""
-    args = ["pytest", "--pyargs", "cmdeploy.tests.online.benchmark", "-vrx"]
-    cmdstring = " ".join(args)
-    out.green(f"[$ {cmdstring}]")
-    subprocess.check_call(args)
-
-
-def webdev_cmd(args, out):
-    """Run local web development loop for static web pages."""
-    from .www import main
-
-    main()
-
-
-#
-# Parsing command line options and starting commands
-#
-
-
-class Out:
-    """Convenience output printer providing coloring."""
-
-    def red(self, msg, file=sys.stderr):
-        print(colored(msg, "red"), file=file)
-
-    def green(self, msg, file=sys.stderr):
-        print(colored(msg, "green"), file=file)
-
-    def __call__(self, msg, red=False, green=False, file=sys.stdout):
-        color = "red" if red else ("green" if green else None)
-        print(colored(msg, color), file=file)
-
-    def shell_output(self, arg, no_print=False, timeout=10):
-        if not no_print:
-            self(f"[$ {arg}]", file=sys.stderr)
-            output = subprocess.STDOUT
-        else:
-            output = subprocess.DEVNULL
-        return subprocess.check_output(
-            arg, shell=True, timeout=timeout, stderr=output
-        ).decode()
-
-    def check_call(self, arg, env=None, quiet=False):
-        if not quiet:
-            self(f"[$ {arg}]", file=sys.stderr)
-        return subprocess.check_call(arg, shell=True, env=env)
-
-    def run_ret(self, args, env=None, quiet=False):
-        if not quiet:
-            cmdstring = " ".join(args)
-            self(f"[$ {cmdstring}]", file=sys.stderr)
-        proc = subprocess.run(args, env=env)
-        return proc.returncode
-
-
-def add_config_option(parser):
-    parser.add_argument(
-        "--config",
-        dest="inipath",
-        action="store",
-        default=Path("chatmail.ini"),
-        type=Path,
-        help="path to the chatmail.ini file",
-    )
-
-
-def add_subcommand(subparsers, func):
-    name = func.__name__
-    assert name.endswith("_cmd")
-    name = name[:-4]
-    doc = func.__doc__.strip()
-    help = doc.split("\n")[0].strip(".")
-    p = subparsers.add_parser(name, description=doc, help=help)
-    p.set_defaults(func=func)
-    add_config_option(p)
-    return p
-
-
-description = """
-Setup your chatmail server configuration and
-deploy it via SSH to your remote location.
-"""
-
-
-def get_parser():
-    """Return an ArgumentParser for the 'cmdeploy' CLI"""
-
-    parser = argparse.ArgumentParser(description=description.strip())
-    subparsers = parser.add_subparsers(title="subcommands")
-
-    # find all subcommands in the module namespace
-    glob = globals()
-    for name, func in glob.items():
-        if name.endswith("_cmd"):
-            subparser = add_subcommand(subparsers, func)
-            addopts = glob.get(name + "_options")
-            if addopts is not None:
-                addopts(subparser)
-
-    return parser
-
-
-def main(args=None):
-    """Provide main entry point for 'xdcget' CLI invocation."""
-    parser = get_parser()
-    args = parser.parse_args(args=args)
-    if not hasattr(args, "func"):
-        return parser.parse_args(["-h"])
-    out = Out()
-    kwargs = {}
-    if args.func.__name__ not in ("init_cmd", "fmt_cmd"):
-        if not args.inipath.exists():
-            out.red(f"expecting {args.inipath} to exist, run init first?")
-            raise SystemExit(1)
-        try:
-            args.config = read_config(args.inipath)
-        except Exception as ex:
-            out.red(ex)
-            raise SystemExit(1)
-
-    try:
-        res = args.func(args, out, **kwargs)
-        if res is None:
-            res = 0
-        return res
-
-    except KeyboardInterrupt:
-        out.red("KeyboardInterrupt")
-        sys.exit(130)
-
-
-if __name__ == "__main__":
-    main()

cmdeploy/src/cmdeploy/deploy.py

@@ -1,17 +0,0 @@
-import os
-import importlib.resources
-import pyinfra
-from cmdeploy import deploy_chatmail
-
-
-def main():
-    config_path = os.getenv(
-        "CHATMAIL_INI",
-        importlib.resources.files("cmdeploy").joinpath("../../../chatmail.ini"),
-    )
-
-    deploy_chatmail(config_path)
-
-
-if pyinfra.is_cli:
-    main()

cmdeploy/src/cmdeploy/dns.py

@@ -1,205 +0,0 @@
-import sys
-
-import requests
-import importlib
-import subprocess
-import datetime
-from ipaddress import ip_address
-
-
-class DNS:
-    def __init__(self, out, mail_domain):
-        self.session = requests.Session()
-        self.out = out
-        self.ssh = f"ssh root@{mail_domain} -- "
-        try:
-            self.shell(f"unbound-control flush_zone {mail_domain}")
-        except subprocess.CalledProcessError:
-            pass
-
-    def shell(self, cmd):
-        try:
-            return self.out.shell_output(f"{self.ssh}{cmd}", no_print=True)
-        except (subprocess.CalledProcessError, subprocess.TimeoutExpired) as e:
-            if "exit status 255" in str(e) or "timed out" in str(e):
-                self.out.red(f"Error: can't reach the server with: {self.ssh[:-4]}")
-                sys.exit(1)
-            else:
-                raise
-
-    def get_ipv4(self):
-        cmd = "ip a | grep 'inet ' | grep 'scope global' | grep -oE '[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}' | head -1"
-        return self.shell(cmd).strip()
-
-    def get_ipv6(self):
-        cmd = "ip a | grep inet6 | grep 'scope global' | sed -e 's#/64 scope global##' | sed -e 's#inet6##'"
-        return self.shell(cmd).strip()
-
-    def get(self, typ: str, domain: str) -> str | None:
-        """Get a DNS entry"""
-        dig_result = self.shell(f"dig -r -q {domain} -t {typ} +short")
-        line = dig_result.partition("\n")[0]
-        if line:
-            return line
-
-    def check_ptr_record(self, ip: str, mail_domain) -> bool:
-        """Check the PTR record for an IPv4 or IPv6 address."""
-        result = self.shell(f"dig -r -x {ip} +short").rstrip()
-        return result == f"{mail_domain}."
-
-
-def show_dns(args, out):
-    template = importlib.resources.files(__package__).joinpath("chatmail.zone.f")
-    mail_domain = args.config.mail_domain
-    ssh = f"ssh root@{mail_domain}"
-    dns = DNS(out, mail_domain)
-
-    def read_dkim_entries(entry):
-        lines = []
-        for line in entry.split("\n"):
-            if line.startswith(";") or not line.strip():
-                continue
-            line = line.replace("\t", " ")
-            lines.append(line)
-        return "\n".join(lines)
-
-    print("Checking your DKIM keys and DNS entries...")
-    try:
-        acme_account_url = out.shell_output(f"{ssh} -- acmetool account-url")
-    except subprocess.CalledProcessError:
-        print("Please run `cmdeploy run` first.")
-        return
-    dkim_entry = read_dkim_entries(out.shell_output(f"{ssh} -- opendkim-genzone -F"))
-
-    ipv6 = dns.get_ipv6()
-    reverse_ipv6 = dns.check_ptr_record(ipv6, mail_domain)
-    ipv4 = dns.get_ipv4()
-    reverse_ipv4 = dns.check_ptr_record(ipv4, mail_domain)
-    to_print = []
-
-    with open(template, "r") as f:
-        zonefile = (
-            f.read()
-            .format(
-                acme_account_url=acme_account_url,
-                email=f"root@{args.config.mail_domain}",
-                sts_id=datetime.datetime.now().strftime("%Y%m%d%H%M"),
-                chatmail_domain=args.config.mail_domain,
-                dkim_entry=dkim_entry,
-                ipv6=ipv6,
-                ipv4=ipv4,
-            )
-            .strip()
-        )
-        try:
-            with open(args.zonefile, "w+") as zf:
-                zf.write(zonefile)
-                print(f"DNS records successfully written to: {args.zonefile}")
-            return
-        except TypeError:
-            pass
-        started_dkim_parsing = False
-        for line in zonefile.splitlines():
-            line = line.format(
-                acme_account_url=acme_account_url,
-                email=f"root@{args.config.mail_domain}",
-                sts_id=datetime.datetime.now().strftime("%Y%m%d%H%M"),
-                chatmail_domain=args.config.mail_domain,
-                dkim_entry=dkim_entry,
-                ipv6=ipv6,
-            ).strip()
-            for typ in ["A", "AAAA", "CNAME", "CAA"]:
-                if f" {typ} " in line:
-                    domain, value = line.split(f" {typ} ")
-                    current = dns.get(typ, domain.strip()[:-1])
-                    if current != value.strip():
-                        to_print.append(line)
-            if " MX " in line:
-                domain, typ, prio, value = line.split()
-                current = dns.get(typ, domain[:-1])
-                if not current:
-                    to_print.append(line)
-                elif current.split()[1] != value:
-                    print(line.replace(prio, str(int(current[0]) + 1)))
-            if " SRV " in line:
-                domain, typ, prio, weight, port, value = line.split()
-                current = dns.get("SRV", domain[:-1])
-                if current != f"{prio} {weight} {port} {value}":
-                    to_print.append(line)
-            if "  TXT " in line:
-                domain, value = line.split(" TXT ")
-                current = dns.get("TXT", domain.strip()[:-1])
-                if domain.startswith("_mta-sts."):
-                    if current:
-                        if current.split("id=")[0] == value.split("id=")[0]:
-                            continue
-                if current != value:
-                    to_print.append(line)
-            if " IN TXT ( " in line:
-                started_dkim_parsing = True
-                dkim_lines = [line]
-            if started_dkim_parsing and line.startswith('"'):
-                dkim_lines.append(" " + line)
-        domain, data = "\n".join(dkim_lines).split(" IN TXT ")
-        current = dns.get("TXT", domain.strip()[:-1])
-        if current:
-            current = "( %s )" % (current.replace('" "', '"\n "'))
-            if current.replace(";", "\\;") != data:
-                to_print.append(dkim_entry)
-        else:
-            to_print.append(dkim_entry)
-
-    if to_print:
-        to_print.insert(
-            0, "You should configure the following DNS entries at your provider:\n"
-        )
-        to_print.append(
-            "\nIf you already configured the DNS entries, wait a bit until the DNS entries propagate to the Internet."
-        )
-        print("\n".join(to_print))
-    else:
-        out.green("Great! All your DNS entries are correct.")
-
-    to_print = []
-    if not reverse_ipv4:
-        to_print.append(f"\tIPv4:\t{ipv4}\t{args.config.mail_domain}")
-    if not reverse_ipv6:
-        to_print.append(f"\tIPv6:\t{ipv6}\t{args.config.mail_domain}")
-    if len(to_print) > 0:
-        if len(to_print) == 1:
-            warning = "You should add the following PTR/reverse DNS entry:"
-        else:
-            warning = "You should add the following PTR/reverse DNS entries:"
-        out.red(warning)
-        for entry in to_print:
-            print(entry)
-        print(
-            "You can do so at your hosting provider (maybe this isn't your DNS provider)."
-        )
-
-
-def check_necessary_dns(out, mail_domain):
-    """Check whether $mail_domain and mta-sts.$mail_domain resolve."""
-    dns = DNS(out, mail_domain)
-    ipv4 = dns.get("A", mail_domain)
-    ipv6 = dns.get("AAAA", mail_domain)
-    mta_entry = dns.get("CNAME", "mta-sts." + mail_domain)
-    mta_ip = dns.get("A", mta_entry)
-    if not mta_ip:
-        mta_ip = dns.get("AAAA", mta_entry)
-    to_print = []
-    if not (ipv4 or ipv6):
-        to_print.append(f"\t{mail_domain}.\t\t\tA<your server's IPv4 address>")
-    if not mta_ip or not (mta_ip == ipv4 or mta_ip == ipv6):
-        to_print.append(f"\tmta-sts.{mail_domain}.\tCNAME\t{mail_domain}.")
-    if to_print:
-        to_print.insert(
-            0,
-            "\nFor chatmail to work, you need to configure this at your DNS provider:\n",
-        )
-        for line in to_print:
-            print(line)
-        print()
-    else:
-        dns.out.green("\nAll necessary DNS entries seem to be set.")
-        return True

cmdeploy/src/cmdeploy/dovecot/auth.conf

@@ -1,10 +0,0 @@
-uri = proxy:/run/dovecot/doveauth.socket:auth
-iterate_disable = yes
-default_pass_scheme = plain
-# %E escapes characters " (double quote), ' (single quote) and \ (backslash) with \ (backslash).
-# See <https://doc.dovecot.org/configuration_manual/config_file/config_variables/#modifiers>
-# for documentation.
-#
-# We escape user-provided input and use double quote as a separator.
-password_key = passdb/%Ew"%Eu
-user_key = userdb/%Eu

cmdeploy/src/cmdeploy/dovecot/expunge.cron.j2

@@ -1,10 +0,0 @@
-# delete all mails after {{ config.delete_mails_after }} days, in the Inbox
-2 0 * * * dovecot find /home/vmail/mail/{{ config.mail_domain }}/*/cur -mtime +{{ config.delete_mails_after }} -type f -delete
-# or in any IMAP subfolder
-2 0 * * * dovecot find /home/vmail/mail/{{ config.mail_domain }}/*/.*/cur -mtime +{{ config.delete_mails_after }} -type f -delete
-# even if they are unseen
-2 0 * * * dovecot find /home/vmail/mail/{{ config.mail_domain }}/*/new -mtime +{{ config.delete_mails_after }} -type f -delete
-2 0 * * * dovecot find /home/vmail/mail/{{ config.mail_domain }}/*/.*/new -mtime +{{ config.delete_mails_after }} -type f -delete
-# or only temporary (but then they shouldn't be around after {{ config.delete_mails_after }} days anyway).
-2 0 * * * dovecot find /home/vmail/mail/{{ config.mail_domain }}/*/tmp -mtime +{{ config.delete_mails_after }} -type f -delete
-2 0 * * * dovecot find /home/vmail/mail/{{ config.mail_domain }}/*/.*/tmp -mtime +{{ config.delete_mails_after }} -type f -delete

cmdeploy/src/cmdeploy/metrics.cron.j2

@@ -1 +0,0 @@
-*/5 * * * * root {{ config.execpath }} /home/vmail/mail/{{ config.mail_domain }} >/var/www/html/metrics

cmdeploy/src/cmdeploy/tests/online/test_0_qr.py

@@ -1,25 +0,0 @@
-import requests
-
-from cmdeploy.genqr import gen_qr_png_data
-
-
-def test_gen_qr_png_data(maildomain):
-    data = gen_qr_png_data(maildomain)
-    assert data
-
-
-def test_fastcgi_working(maildomain, chatmail_config):
-    url = f"https://{maildomain}/cgi-bin/newemail.py"
-    print(url)
-    res = requests.post(url)
-    assert maildomain in res.json().get("email")
-    assert len(res.json().get("password")) > chatmail_config.password_min_length
-
-
-def test_newemail_configure(maildomain, rpc):
-    """Test configuring accounts by scanning a QR code works."""
-    url = f"DCACCOUNT:https://{maildomain}/cgi-bin/newemail.py"
-    for i in range(3):
-        account_id = rpc.add_account()
-        rpc.set_config_from_qr(account_id, url)
-        rpc.configure(account_id)

cmdeploy/src/cmdeploy/tests/test_cmdeploy.py

@@ -1,27 +0,0 @@
-import os
-
-import pytest
-from cmdeploy.cmdeploy import get_parser, main
-
-
-@pytest.fixture(autouse=True)
-def _chdir(tmp_path):
-    old = os.getcwd()
-    os.chdir(tmp_path)
-    yield
-    os.chdir(old)
-
-
-class TestCmdline:
-    def test_parser(self, capsys):
-        parser = get_parser()
-        parser.parse_args([])
-        init = parser.parse_args(["init", "chat.example.org"])
-        run = parser.parse_args(["run"])
-        assert init and run
-
-    @pytest.mark.xfail(reason="init doesn't exit anymore, check for CLI output instead")
-    def test_init_not_overwrite(self):
-        main(["init", "chat.example.org"])
-        with pytest.raises(SystemExit):
-            main(["init", "chat.example.org"])

cmdeploy/src/cmdeploy/tests/test_helpers.py

@@ -1,13 +0,0 @@
-import importlib.resources
-
-from cmdeploy.www import build_webpages
-
-
-def test_build_webpages(tmp_path, make_config):
-    pkgroot = importlib.resources.files("cmdeploy")
-    src_dir = pkgroot.joinpath("../../../www/src").resolve()
-    assert src_dir.exists(), src_dir
-    config = make_config("chat.example.org")
-    build_dir = tmp_path.joinpath("build")
-    build_webpages(src_dir, build_dir, config)
-    assert len([x for x in build_dir.iterdir() if x.suffix == ".html"]) >= 3

deploy-chatmail/pyproject.toml

@@ -0,0 +1,33 @@
+[build-system]
+requires = ["setuptools>=45"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "deploy-chatmail"
+version = "0.1"
+dependencies = [
+  "pyinfra",
+  "pillow",
+  "qrcode",
+  "markdown",
+]
+
+[tool.pytest.ini_options]
+addopts = "-v -ra --strict-markers"
+
+[tool.tox]
+legacy_tox_ini = """
+[tox]
+isolated_build = true
+envlist = lint
+
+[testenv:lint]
+skipdist = True
+skip_install = True
+deps =
+  ruff
+  black
+commands =
+  black --quiet --check --diff src/
+  ruff src/
+"""

deploy-chatmail/src/deploy_chatmail/__init__.py

@@ -6,6 +6,7 @@ import importlib.resources
 import subprocess
 import shutil
 import io
+import configparser
 from pathlib import Path
 
 from pyinfra import host
@@ -14,8 +15,6 @@ from pyinfra.facts.files import File
 from pyinfra.facts.systemd import SystemdEnabled
 from .acmetool import deploy_acmetool
 
-from chatmaild.config import read_config, Config
-
 
 def _build_chatmaild(dist_dir) -> None:
     dist_dir = Path(dist_dir).resolve()
@@ -31,24 +30,11 @@ def _build_chatmaild(dist_dir) -> None:
     return entries[0]
 
 
-def remove_legacy_artifacts():
+def _install_remote_venv_with_chatmaild() -> None:
-    # disable legacy doveauth-dictproxy.service
-    if host.get_fact(SystemdEnabled).get("doveauth-dictproxy.service"):
-        systemd.service(
-            name="Disable legacy doveauth-dictproxy.service",
-            service="doveauth-dictproxy.service",
-            running=False,
-            enabled=False,
-        )
-
-
-def _install_remote_venv_with_chatmaild(config) -> None:
-    remove_legacy_artifacts()
     dist_file = _build_chatmaild(dist_dir=Path("chatmaild/dist"))
     remote_base_dir = "/usr/local/lib/chatmaild"
     remote_dist_file = f"{remote_base_dir}/dist/{dist_file.name}"
     remote_venv_dir = f"{remote_base_dir}/venv"
-    remote_chatmail_inipath = f"{remote_base_dir}/chatmail.ini"
     root_owned = dict(user="root", group="root", mode="644")
 
     apt.packages(
@@ -64,13 +50,6 @@ def _install_remote_venv_with_chatmaild(config) -> None:
         **root_owned,
     )
 
-    files.put(
-        name=f"Upload {remote_chatmail_inipath}",
-        src=config._getbytefile(),
-        dest=remote_chatmail_inipath,
-        **root_owned,
-    )
-
     pip.virtualenv(
         name=f"chatmaild virtualenv {remote_venv_dir}",
         path=remote_venv_dir,
@@ -84,31 +63,24 @@ def _install_remote_venv_with_chatmaild(config) -> None:
         ],
     )
 
-    files.template(
+    # disable legacy doveauth-dictproxy.service
-        src=importlib.resources.files(__package__).joinpath("metrics.cron.j2"),
+    if host.get_fact(SystemdEnabled).get("doveauth-dictproxy.service"):
-        dest="/etc/cron.d/chatmail-metrics",
+        systemd.service(
-        user="root",
+            name="Disable legacy doveauth-dictproxy.service",
-        group="root",
+            service="doveauth-dictproxy.service",
-        mode="644",
+            running=False,
-        config={
+            enabled=False,
-            "mail_domain": config.mail_domain,
+        )
-            "execpath": f"{remote_venv_dir}/bin/chatmail-metrics",
-        },
-    )
 
     # install systemd units
+
     for fn in (
         "doveauth",
         "filtermail",
-        "echobot",
     ):
-        params = dict(
+        execpath = f"{remote_venv_dir}/bin/{fn}"
-            execpath=f"{remote_venv_dir}/bin/{fn}",
-            config_path=remote_chatmail_inipath,
-            remote_venv_dir=remote_venv_dir,
-        )
         source_path = importlib.resources.files("chatmaild").joinpath(f"{fn}.service.f")
-        content = source_path.read_text().format(**params).encode()
+        content = source_path.read_text().format(execpath=execpath).encode()
 
         files.put(
             name=f"Upload {fn}.service",
@@ -126,7 +98,7 @@ def _install_remote_venv_with_chatmaild(config) -> None:
         )
 
 
-def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
+def _configure_opendkim(domain: str, dkim_selector: str) -> bool:
     """Configures OpenDKIM"""
     need_restart = False
 
@@ -209,7 +181,7 @@ def _install_mta_sts_daemon() -> bool:
     server.shell(
         name="install postfix-mta-sts-resolver with pip",
         commands=[
-            "python3 -m virtualenv /usr/local/lib/postfix-mta-sts-resolver",
+            "python3 -m venv /usr/local/lib/postfix-mta-sts-resolver",
             "/usr/local/lib/postfix-mta-sts-resolver/bin/pip install postfix-mta-sts-resolver",
         ],
     )
@@ -229,7 +201,7 @@ def _install_mta_sts_daemon() -> bool:
     return need_restart
 
 
-def _configure_postfix(config: Config, debug: bool = False) -> bool:
+def _configure_postfix(domain: str, debug: bool = False) -> bool:
     """Configures Postfix SMTP server."""
     need_restart = False
 
@@ -239,7 +211,7 @@ def _configure_postfix(config: Config, debug: bool = False) -> bool:
         user="root",
         group="root",
         mode="644",
-        config=config,
+        config={"domain_name": domain},
     )
     need_restart |= main_config.changed
 
@@ -250,14 +222,13 @@ def _configure_postfix(config: Config, debug: bool = False) -> bool:
         group="root",
         mode="644",
         debug=debug,
-        config=config,
     )
     need_restart |= master_config.changed
 
     return need_restart
 
 
-def _configure_dovecot(config: Config, debug: bool = False) -> bool:
+def _configure_dovecot(mail_server: str, debug: bool = False) -> bool:
     """Configures Dovecot IMAP server."""
     need_restart = False
 
@@ -267,7 +238,7 @@ def _configure_dovecot(config: Config, debug: bool = False) -> bool:
         user="root",
         group="root",
         mode="644",
-        config=config,
+        config={"hostname": mail_server},
         debug=debug,
     )
     need_restart |= main_config.changed
@@ -280,13 +251,14 @@ def _configure_dovecot(config: Config, debug: bool = False) -> bool:
     )
     need_restart |= auth_config.changed
 
-    files.template(
+    files.put(
-        src=importlib.resources.files(__package__).joinpath("dovecot/expunge.cron.j2"),
+        src=importlib.resources.files(__package__)
+        .joinpath("dovecot/expunge.cron")
+        .open("rb"),
         dest="/etc/cron.d/expunge",
         user="root",
         group="root",
         mode="644",
-        config=config,
     )
 
     # as per https://doc.dovecot.org/configuration_manual/os/
@@ -359,27 +331,28 @@ def _configure_nginx(domain: str, debug: bool = False) -> bool:
     return need_restart
 
 
-def check_config(config):
+def get_ini_settings(mail_domain, inipath):
-    mail_domain = config.mail_domain
+    parser = configparser.ConfigParser()
+    parser.read(inipath)
+    settings = {key: value.strip() for (key, value) in parser["config"].items()}
     if mail_domain != "testrun.org" and not mail_domain.endswith(".testrun.org"):
-        blocked_words = "merlinux schmieder testrun.org".split()
+        for value in settings.values():
-        for value in config.__dict__.values():
+            value = value.lower()
-            if any(x in str(value) for x in blocked_words):
+            if "merlinux" in value or "schmieder" in value or "@testrun.org" in value:
                 raise ValueError(
-                    f"please set your own privacy contacts/addresses in {config._inipath}"
+                    f"please set your own privacy contacts/addresses in {inipath}"
                 )
-    return config
+    settings["mail_domain"] = mail_domain
+    return settings
 
 
-def deploy_chatmail(config_path: Path) -> None:
+def deploy_chatmail(mail_domain: str, mail_server: str, dkim_selector: str) -> None:
     """Deploy a chat-mail instance.
 
-    :param config_path: path to chatmail.ini
+    :param mail_domain: domain part of your future email addresses
+    :param mail_server: the DNS name under which your mail server is reachable
+    :param dkim_selector:
     """
-    config = read_config(config_path)
-    check_config(config)
-    mail_domain = config.mail_domain
-
     from .www import build_webpages
 
     apt.update(name="apt update", cache_time=24 * 3600)
@@ -394,26 +367,8 @@ def deploy_chatmail(config_path: Path) -> None:
         system=True,
     )
 
-    # Run local DNS resolver `unbound`.
-    # `resolvconf` takes care of setting up /etc/resolv.conf
-    # to use 127.0.0.1 as the resolver.
-    apt.packages(
-        name="Install unbound",
-        packages=["unbound", "unbound-anchor", "dnsutils"],
-    )
-    server.shell(
-        name="Generate root keys for validating DNSSEC",
-        commands=["unbound-anchor -a /var/lib/unbound/root.key || true"],
-    )
-    systemd.service(
-        name="Start and enable unbound",
-        service="unbound.service",
-        running=True,
-        enabled=True,
-    )
-
     # Deploy acmetool to have TLS certificates.
-    deploy_acmetool(nginx_hook=True, domains=[mail_domain, f"mta-sts.{mail_domain}"])
+    deploy_acmetool(nginx_hook=True, domains=[mail_server, f"mta-sts.{mail_server}"])
 
     apt.packages(
         name="Install Postfix",
@@ -443,18 +398,21 @@ def deploy_chatmail(config_path: Path) -> None:
         packages=["fcgiwrap"],
     )
 
-    www_path = importlib.resources.files(__package__).joinpath("../../../www").resolve()
+    pkg_root = importlib.resources.files(__package__)
+    chatmail_ini = pkg_root.joinpath("../../../chatmail.ini").resolve()
+    config = get_ini_settings(mail_domain, chatmail_ini)
+    www_path = pkg_root.joinpath("../../../www").resolve()
 
     build_dir = www_path.joinpath("build")
     src_dir = www_path.joinpath("src")
     build_webpages(src_dir, build_dir, config)
     files.rsync(f"{build_dir}/", "/var/www/html", flags=["-avz"])
 
-    _install_remote_venv_with_chatmaild(config)
+    _install_remote_venv_with_chatmaild()
     debug = False
-    dovecot_need_restart = _configure_dovecot(config, debug=debug)
+    dovecot_need_restart = _configure_dovecot(mail_server, debug=debug)
-    postfix_need_restart = _configure_postfix(config, debug=debug)
+    postfix_need_restart = _configure_postfix(mail_domain, debug=debug)
-    opendkim_need_restart = _configure_opendkim(mail_domain)
+    opendkim_need_restart = _configure_opendkim(mail_domain, dkim_selector)
     mta_sts_need_restart = _install_mta_sts_daemon()
     nginx_need_restart = _configure_nginx(mail_domain)
 

deploy-chatmail/src/deploy_chatmail/acmetool/__init__.py

@@ -1,8 +1,6 @@
 import importlib.resources
 
-from pyinfra.operations import apt, files, systemd, server
+from pyinfra.operations import apt, files, server
-from pyinfra import host
-from pyinfra.facts.systemd import SystemdStatus
 
 
 def deploy_acmetool(nginx_hook=False, email="", domains=[]):
@@ -48,30 +46,6 @@ def deploy_acmetool(nginx_hook=False, email="", domains=[]):
         mode="644",
     )
 
-    service_file = files.put(
-        src=importlib.resources.files(__package__).joinpath(
-            "acmetool-redirector.service"
-        ),
-        dest="/etc/systemd/system/acmetool-redirector.service",
-        user="root",
-        group="root",
-        mode="644",
-    )
-    if host.get_fact(SystemdStatus).get("nginx.service"):
-        systemd.service(
-            name="Stop nginx service to free port 80",
-            service="nginx",
-            running=False,
-        )
-
-    systemd.service(
-        name="Setup acmetool-redirector service",
-        service="acmetool-redirector.service",
-        running=True,
-        enabled=True,
-        restarted=service_file.changed,
-    )
-
     server.shell(
         name=f"Request certificate for: { ', '.join(domains) }",
         commands=[f"acmetool want { ' '.join(domains)}"],

deploy-chatmail/src/deploy_chatmail/deploy.py

@@ -0,0 +1,19 @@
+import os
+import pyinfra
+from deploy_chatmail import deploy_chatmail
+
+
+def main():
+    mail_domain = os.getenv("CHATMAIL_DOMAIN")
+    mail_server = os.getenv("CHATMAIL_SERVER", mail_domain)
+    dkim_selector = os.getenv("CHATMAIL_DKIM_SELECTOR", "dkim")
+
+    assert mail_domain
+    assert mail_server
+    assert dkim_selector
+
+    deploy_chatmail(mail_domain, mail_server, dkim_selector)
+
+
+if pyinfra.is_cli:
+    main()

deploy-chatmail/src/deploy_chatmail/dovecot/auth.conf

@@ -0,0 +1,5 @@
+uri = proxy:/run/dovecot/doveauth.socket:auth
+iterate_disable = yes
+default_pass_scheme = plain
+password_key = passdb/%w/%u
+user_key = userdb/%u

deploy-chatmail/src/deploy_chatmail/dovecot/dovecot.conf.j2

@@ -86,7 +86,7 @@ plugin {
 plugin {
   # for now we define static quota-rules for all users 
   quota = maildir:User quota
-  quota_rule = *:storage={{ config.max_mailbox_size }}
+  quota_rule = *:storage=100M
   quota_max_mail_size=30M
   quota_grace = 0
   # quota_over_flag_value = TRUE
@@ -137,8 +137,8 @@ service imap-login {
 }
 
 ssl = required
-ssl_cert = </var/lib/acme/live/{{ config.mail_domain }}/fullchain
+ssl_cert = </var/lib/acme/live/{{ config.hostname }}/fullchain
-ssl_key = </var/lib/acme/live/{{ config.mail_domain }}/privkey
+ssl_key = </var/lib/acme/live/{{ config.hostname }}/privkey
 ssl_dh = </usr/share/dovecot/dh.pem
 ssl_min_protocol = TLSv1.2
 ssl_prefer_server_ciphers = yes

deploy-chatmail/src/deploy_chatmail/dovecot/expunge.cron

@@ -0,0 +1,4 @@
+2 0 * * * dovecot doveadm expunge -A SEEN BEFORE 40d INBOX
+2 0 * * * dovecot doveadm expunge -A SEEN BEFORE 40d Deltachat
+2 0 * * * dovecot doveadm expunge -A SEEN BEFORE 40d Trash
+2 30 * * * dovecot doveadm purge -A

deploy-chatmail/src/deploy_chatmail/nginx/nginx.conf.j2

@@ -41,11 +41,15 @@ http {
 			try_files $uri $uri/ =404;
 		}
 
-                location /metrics {
-                        default_type text/plain;
-                }
-
         # add cgi-bin support
         include /usr/share/doc/fcgiwrap/examples/nginx.conf;
 	}
+    server {
+        listen 80 default_server;
+        listen [::]:80 default_server;
+        server_name _;
+
+        return 301 https://$host$request_uri;
+    }
 }
+

deploy-chatmail/src/deploy_chatmail/postfix/main.cf.j2

@@ -1,4 +1,4 @@
-myorigin = {{ config.mail_domain }}
+myorigin = {{ config.domain_name }}
 
 smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
 biff = no
@@ -11,12 +11,13 @@ append_dot_mydomain = no
 
 readme_directory = no
 
-# See http://www.postfix.org/COMPATIBILITY_README.html
+# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 2 on
-compatibility_level = 3.6
+# fresh installs.
+compatibility_level = 2
 
 # TLS parameters
-smtpd_tls_cert_file=/var/lib/acme/live/{{ config.mail_domain }}/fullchain
+smtpd_tls_cert_file=/var/lib/acme/live/{{ config.domain_name }}/fullchain
-smtpd_tls_key_file=/var/lib/acme/live/{{ config.mail_domain }}/privkey
+smtpd_tls_key_file=/var/lib/acme/live/{{ config.domain_name }}/privkey
 smtpd_tls_security_level=may
 
 smtp_tls_CApath=/etc/ssl/certs
@@ -25,7 +26,7 @@ smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
 smtp_tls_policy_maps = socketmap:inet:127.0.0.1:8461:postfix
 
 smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
-myhostname = {{ config.mail_domain }}
+myhostname = {{ config.domain_name }}
 alias_maps = hash:/etc/aliases
 alias_database = hash:/etc/aliases
 
@@ -44,7 +45,7 @@ inet_interfaces = all
 inet_protocols = all
 
 virtual_transport = lmtp:unix:private/dovecot-lmtp
-virtual_mailbox_domains = {{ config.mail_domain }}
+virtual_mailbox_domains = {{ config.domain_name }}
 
 smtpd_milters = unix:opendkim/opendkim.sock
 non_smtpd_milters = $smtpd_milters

deploy-chatmail/src/deploy_chatmail/postfix/master.cf.j2

@@ -33,7 +33,7 @@ submission inet n       -       y       -       -       smtpd
   -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
   -o milter_macro_daemon_name=ORIGINATING
   -o smtpd_client_connection_count_limit=1000
-  -o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
+  -o smtpd_proxy_filter=127.0.0.1:10080
 smtps     inet  n       -       y       -       -       smtpd
   -o syslog_name=postfix/smtps
   -o smtpd_tls_wrappermode=yes
@@ -49,7 +49,7 @@ smtps     inet  n       -       y       -       -       smtpd
   -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
   -o smtpd_client_connection_count_limit=1000
   -o milter_macro_daemon_name=ORIGINATING
-  -o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
+  -o smtpd_proxy_filter=127.0.0.1:10080
 #628       inet  n       -       y       -       -       qmqpd
 pickup    unix  n       -       y       60      1       pickup
 cleanup   unix  n       -       y       -       0       cleanup
@@ -78,5 +78,5 @@ scache    unix  -       -       y       -       1       scache
 postlog   unix-dgram n  -       n       -       1       postlogd
 filter    unix -        n       n       -       -       lmtp
 # Local SMTP server for reinjecting filered mail.
-localhost:{{ config.postfix_reinject_port }} inet  n       -       n       -       10      smtpd
+localhost:10025 inet  n       -       n       -       10      smtpd
   -o syslog_name=postfix/reinject

deploy-chatmail/src/deploy_chatmail/www.py

@@ -7,7 +7,7 @@ import traceback
 import markdown
 from jinja2 import Template
 from .genqr import gen_qr_png_data
-from chatmaild.config import read_config
+from deploy_chatmail import get_ini_settings
 
 
 def snapshot_dir_stats(somedir):
@@ -36,32 +36,8 @@ def build_webpages(src_dir, build_dir, config):
         print(traceback.format_exc())
 
 
-def int_to_english(number):
-    if number >= 0 and number <= 12:
-        a = [
-            "zero",
-            "one",
-            "two",
-            "three",
-            "four",
-            "five",
-            "six",
-            "seven",
-            "eight",
-            "nine",
-            "ten",
-            "eleven",
-            "twelve",
-        ]
-        return a[number]
-    elif number <= 50:
-        return str(number)
-    if number > 50:
-        return "more"
-
-
 def _build_webpages(src_dir, build_dir, config):
-    mail_domain = config.mail_domain
+    mail_domain = config["mail_domain"]
     assert src_dir.exists(), src_dir
     if not build_dir.exists():
         build_dir.mkdir()
@@ -72,15 +48,6 @@ def _build_webpages(src_dir, build_dir, config):
     for path in src_dir.iterdir():
         if path.suffix == ".md":
             render_vars, content = prepare_template(path)
-            render_vars["username_min_length"] = int_to_english(
-                config.username_min_length
-            )
-            render_vars["username_max_length"] = int_to_english(
-                config.username_max_length
-            )
-            render_vars["password_min_length"] = int_to_english(
-                config.password_min_length
-            )
             target = build_dir.joinpath(path.stem + ".html")
 
             # recursive jinja2 rendering
@@ -99,12 +66,12 @@ def _build_webpages(src_dir, build_dir, config):
 
 
 def main():
+    chatmail_domain = "example.testrun.org"
     path = importlib.resources.files(__package__)
     reporoot = path.joinpath("../../../").resolve()
     inipath = reporoot.joinpath("chatmail.ini")
-    config = read_config(inipath)
+    config = get_ini_settings(chatmail_domain, inipath)
-    config.webdev = True
+    config["webdev"] = True
-    assert config.mail_domain
     www_path = reporoot.joinpath("www")
     src_path = www_path.joinpath("src")
     stats = None

plan.txt

@@ -0,0 +1,65 @@
+# Chat-mail server development (up until Oct 18th)
+
+## Dovecot goals/steps
+
+- automatic expiry of messages older than M days
+   - also expunge unread messages
+
+- limit: configure max-connections per account
+
+
+## nami: send out rate limit / rspamd
+
+- basic outgoing send rate/limits (depending on "account-rating")
+  use rspamd in a minimal way, check support dkim-signing
+  (including an online test exceeding rate limit)
+
+
+## doveauth questions/futures
+
+- bcrypt-password scheme is slow: require long passwords, use faster hashing
+
+- define user-name and password policies, and implement them
+  (be very restrictive at the beginning, we can relax later)
+
+- password is part of the dictproxy-lookup key, is it safe to use auth-caching?
+
+
+## How to limit creation of accounts?
+
+attack: a 3-line bash script to fill the chatmail db with millions of unused accouts
+
+- make it computationally expensive (somehow try to except our tests from it)
+  1st pass instant onboarding: create userid + cheap password -- if it fails then
+  2nd pass instant onboarding: create userdid + comput. expensive password
+
+- probably also do firewall: limit number of new tcp-connections per IP address per duration
+
+
+## Open/deferred questions
+
+- automatic expiry of users that haven't logged in for N days
+  Is it neccessary?  If all messages are gone, does the existence of
+  an e-mail address bother anybody?
+
+
+## web page for chat-mail servers?
+
+- documentation for users, privacy policy etc.
+  (probably also with provider-messages ...)
+
+
+## online tests (first with plain python/pytest)
+
+- write tests for dovecot login (exists)
+- write tests for postfix logins (exists)
+- write A<>B send/receive tests (exists)
+
+
+## Delta Chat
+
+1. qr code that defines access to a chatmail instance (like mailadm but without http etc.)
+
+2. support for creating username/password and verifying login works
+
+

scripts/bench.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+set -e
+
+venv/bin/pytest tests/online/benchmark.py -vrx

scripts/cmdeploy

@@ -1,6 +0,0 @@
-#!/usr/bin/env bash
-#
-# Wrapper for cmdelpoy to run it in activated virtualenv.
-set -e
-. venv/bin/activate
-cmdeploy "$@"

scripts/deploy.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+
+echo -----------------------------------------
+echo deploying to $CHATMAIL_DOMAIN 
+echo -----------------------------------------
+
+venv/bin/pyinfra --ssh-user root "$CHATMAIL_DOMAIN" \
+    deploy-chatmail/src/deploy_chatmail/deploy.py

scripts/generate-dns-zone.sh

@@ -0,0 +1,23 @@
+#!/bin/sh
+: ${CHATMAIL_DOMAIN:=c1.testrun.org}
+: ${CHATMAIL_SSH:=$CHATMAIL_DOMAIN}
+
+set -e
+SSH="ssh root@$CHATMAIL_SSH"
+EMAIL="root@$CHATMAIL_DOMAIN"
+ACME_ACCOUNT_URL="$($SSH -- acmetool account-url)"
+
+cat <<EOF
+$CHATMAIL_DOMAIN. MX 10 $CHATMAIL_DOMAIN.
+$CHATMAIL_DOMAIN. TXT "v=spf1 a:$CHATMAIL_DOMAIN -all"
+_dmarc.$CHATMAIL_DOMAIN. TXT "v=DMARC1;p=reject;rua=mailto:$EMAIL;ruf=mailto:$EMAIL;fo=1;adkim=r;aspf=r"
+_submission._tcp.$CHATMAIL_DOMAIN.  SRV 0 1 587 $CHATMAIL_DOMAIN.
+_submissions._tcp.$CHATMAIL_DOMAIN. SRV 0 1 465 $CHATMAIL_DOMAIN.
+_imap._tcp.$CHATMAIL_DOMAIN.        SRV 0 1 143 $CHATMAIL_DOMAIN.
+_imaps._tcp.$CHATMAIL_DOMAIN.       SRV 0 1 993 $CHATMAIL_DOMAIN.
+$CHATMAIL_DOMAIN. IN CAA 128 issue "letsencrypt.org;accounturi=$ACME_ACCOUNT_URL"
+_mta-sts.$CHATMAIL_DOMAIN. IN TXT "v=STSv1; id=$(date -u '+%Y%m%d%H%M')"
+mta-sts.$CHATMAIL_DOMAIN. IN CNAME $CHATMAIL_DOMAIN.
+_smtp._tls.$CHATMAIL_DOMAIN. IN TXT "v=TLSRPTv1;rua=mailto:$EMAIL"
+EOF
+$SSH opendkim-genzone -F | sed 's/^;.*$//;/^$/d'

scripts/get_imap_capabilities.py

@@ -0,0 +1,14 @@
+import os
+import time
+import imaplib
+
+domain = os.environ.get("CHATMAIL_DOMAIN", "c3.testrun.org")
+
+print("connecting")
+conn = imaplib.IMAP4_SSL(domain)
+print("logging in")
+conn.login(f"imapcapa", "pass")
+status, res = conn.capability()
+for capa in sorted(res[0].decode().split()):
+    print(capa)
+

scripts/init.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+set -e
+python3 -m venv venv
+pip=venv/bin/pip
+
+$pip install pyinfra pytest build 'setuptools>=68' tox 
+$pip install -e deploy-chatmail 
+$pip install -e chatmaild 

scripts/initenv.sh

@@ -1,6 +0,0 @@
-#!/bin/bash
-set -e
-python3 -m venv venv
-
-venv/bin/pip install -e chatmaild 
-venv/bin/pip install -e cmdeploy

scripts/measure_tls_and_logins.py

@@ -0,0 +1,28 @@
+#!/usr/bin/env python3
+import os
+import time
+import imaplib
+
+domain = os.environ.get("CHATMAIL_DOMAIN", "c3.testrun.org")
+
+NUM_CONNECTIONS=10
+
+conns = []
+
+start = time.time()
+for i in range(NUM_CONNECTIONS):
+    print(f"opening connection {i} to {domain}")
+    conn = imaplib.IMAP4_SSL(domain)
+    conns.append(conn)
+
+tlsdone = time.time()
+duration = tlsdone-start
+print(f"{duration}: TLS connections opening TLS connections")
+
+for i, conn in enumerate(conns):
+    print(f"logging into connection {i}")
+    conn.login(f"measure{i}", "pass")
+
+logindone = time.time()
+duration = logindone - tlsdone
+print(f"{duration}: LOGINS done")

scripts/remote-deploy.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+set -e
+
+: ${CHATMAIL_DOMAIN:=c1.testrun.org}
+: ${CHATMAIL_SSH:=$CHATMAIL_DOMAIN}
+
+rsync -avz . "root@$CHATMAIL_SSH:/root/chatmail" --exclude='/.git' --filter="dir-merge,- .gitignore"
+ssh "root@$CHATMAIL_SSH" "cd /root/chatmail; apt install -y python3-venv; python3 -m venv venv; venv/bin/pip install pyinfra build; venv/bin/python3 -m build -n --sdist chatmaild --outdir dist; venv/bin/pip install -e ./deploy-chatmail -e ./chatmaild; export CHATMAIL_DOMAIN=$CHATMAIL_DOMAIN; venv/bin/pyinfra @local deploy.py"

scripts/test.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+venv/bin/tox -c chatmaild
+venv/bin/tox -c deploy-chatmail
+venv/bin/pytest tests/online -rs -vrx --durations=5 $@

scripts/webdev.sh

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+echo -----------------------------------------
+echo starting local webdev 
+echo -----------------------------------------
+
+venv/bin/python3 -m deploy_chatmail.www 
+
+

tests/chatmaild/test_doveauth.py

@@ -1,4 +1,5 @@
 import json
+import sys
 import pytest
 import threading
 import queue
@@ -6,38 +7,32 @@ import traceback
 
 import chatmaild.doveauth
 from chatmaild.doveauth import get_user_data, lookup_passdb, handle_dovecot_request
-from chatmaild.database import DBError
+from chatmaild.database import Database, DBError
 
 
-def test_basic(db, example_config):
+def test_basic(db):
-    lookup_passdb(db, example_config, "asdf12345@chat.example.org", "q9mr3faue")
+    lookup_passdb(db, "link2xt@c1.testrun.org", "Pieg9aeToe3eghuthe5u")
-    data = get_user_data(db, "asdf12345@chat.example.org")
+    data = get_user_data(db, "link2xt@c1.testrun.org")
     assert data
-    data2 = lookup_passdb(
+    data2 = lookup_passdb(db, "link2xt@c1.testrun.org", "Pieg9aeToe3eghuthe5u")
-        db, example_config, "asdf12345@chat.example.org", "q9mr3jewvadsfaue"
-    )
     assert data == data2
 
 
-def test_dont_overwrite_password_on_wrong_login(db, example_config):
+def test_dont_overwrite_password_on_wrong_login(db):
     """Test that logging in with a different password doesn't create a new user"""
-    res = lookup_passdb(
+    res = lookup_passdb(db, "newuser1@something.org", "kajdlkajsldk12l3kj1983")
-        db, example_config, "newuser12@chat.example.org", "kajdlkajsldk12l3kj1983"
-    )
     assert res["password"]
-    res2 = lookup_passdb(db, example_config, "newuser12@chat.example.org", "kajdslqwe")
+    res2 = lookup_passdb(db, "newuser1@something.org", "kajdlqweqwe")
     # this function always returns a password hash, which is actually compared by dovecot.
     assert res["password"] == res2["password"]
 
 
-def test_nocreate_file(db, monkeypatch, tmpdir, example_config):
+def test_nocreate_file(db, monkeypatch, tmpdir):
     p = tmpdir.join("nocreate")
     p.write("")
     monkeypatch.setattr(chatmaild.doveauth, "NOCREATE_FILE", str(p))
-    lookup_passdb(
+    lookup_passdb(db, "newuser1@something.org", "zequ0Aimuchoodaechik")
-        db, example_config, "newuser12@chat.example.org", "zequ0Aimuchoodaechik"
+    assert not get_user_data(db, "newuser1@something.org")
-    )
-    assert not get_user_data(db, "newuser12@chat.example.org")
 
 
 def test_db_version(db):
@@ -51,23 +46,22 @@ def test_too_high_db_version(db):
         db.ensure_tables()
 
 
-def test_handle_dovecot_request(db, example_config):
+def test_handle_dovecot_request(db):
-    # Test that password can contain ", ', \ and /
     msg = (
-        'Lshared/passdb/laksjdlaksjdlak\\\\sjdlk\\"12j\\\'3l1/k2j3123"'
+        "Lshared/passdb/laksjdlaksjdlaksjdlk12j3l1k2j3123/"
-        "some42123@chat.example.org\tsome42123@chat.example.org"
+        "some42@c3.testrun.org\tsome42@c3.testrun.org"
     )
-    res = handle_dovecot_request(msg, db, example_config)
+    res = handle_dovecot_request(msg, db, "c3.testrun.org")
     assert res
     assert res[0] == "O" and res.endswith("\n")
     userdata = json.loads(res[1:].strip())
-    assert userdata["home"] == "/home/vmail/some42123@chat.example.org"
+    assert userdata["home"] == "/home/vmail/some42@c3.testrun.org"
     assert userdata["uid"] == userdata["gid"] == "vmail"
     assert userdata["password"].startswith("{SHA512-CRYPT}")
 
 
-def test_50_concurrent_lookups_different_accounts(db, gencreds, example_config):
+def test_100_concurrent_lookups_different_accounts(db, gencreds):
-    num_threads = 50
+    num_threads = 100
     req_per_thread = 5
     results = queue.Queue()
 
@@ -75,7 +69,7 @@ def test_50_concurrent_lookups_different_accounts(db, gencreds, example_config):
         for i in range(req_per_thread):
             addr, password = gencreds()
             try:
-                lookup_passdb(db, example_config, addr, password)
+                lookup_passdb(db, addr, password)
             except Exception:
                 results.put(traceback.format_exc())
             else:

tests/chatmaild/test_filtermail.py

@@ -1,10 +1,4 @@
-from chatmaild.filtermail import (
+from chatmaild.filtermail import check_encrypted, check_DATA, SendRateLimiter, check_mdn, is_passthrough_recipient
-    check_encrypted,
-    BeforeQueueHandler,
-    SendRateLimiter,
-    check_mdn,
-)
-
 import pytest
 
 
@@ -14,48 +8,32 @@ def maildomain():
     return "chatmail.example.org"
 
 
-@pytest.fixture
+def test_reject_forged_from(maildata, gencreds):
-def handler(make_config, maildomain):
-    config = make_config(maildomain)
-    return BeforeQueueHandler(config)
-
-
-def test_reject_forged_from(maildata, gencreds, handler):
     class env:
         mail_from = gencreds()[0]
         rcpt_tos = [gencreds()[0]]
 
     # test that the filter lets good mail through
-    to_addr = gencreds()[0]
+    env.content = maildata("plain.eml", from_addr=env.mail_from).as_bytes()
-    env.content = maildata(
+    assert not check_DATA(envelope=env)
-        "plain.eml", from_addr=env.mail_from, to_addr=to_addr
-    ).as_bytes()
-
-    assert not handler.check_DATA(envelope=env)
 
     # test that the filter rejects forged mail
-    env.content = maildata(
+    env.content = maildata("plain.eml", from_addr="forged@c3.testrun.org").as_bytes()
-        "plain.eml", from_addr="forged@c3.testrun.org", to_addr=to_addr
+    error = check_DATA(envelope=env)
-    ).as_bytes()
-    error = handler.check_DATA(envelope=env)
     assert "500" in error
 
 
 def test_filtermail_no_encryption_detection(maildata):
-    msg = maildata(
+    msg = maildata("plain.eml")
-        "plain.eml", from_addr="some@example.org", to_addr="other@example.org"
-    )
     assert not check_encrypted(msg)
 
     # https://xkcd.com/1181/
-    msg = maildata(
+    msg = maildata("fake-encrypted.eml")
-        "fake-encrypted.eml", from_addr="some@example.org", to_addr="other@example.org"
-    )
     assert not check_encrypted(msg)
 
 
 def test_filtermail_encryption_detection(maildata):
-    msg = maildata("encrypted.eml", from_addr="1@example.org", to_addr="2@example.org")
+    msg = maildata("encrypted.eml")
     assert check_encrypted(msg)
 
     # if the subject is not "..." it is not considered ac-encrypted
@@ -63,7 +41,7 @@ def test_filtermail_encryption_detection(maildata):
     assert not check_encrypted(msg)
 
 
-def test_filtermail_is_mdn(maildata, gencreds, handler):
+def test_filtermail_is_mdn(maildata, gencreds):
     from_addr = gencreds()[0]
     to_addr = gencreds()[0] + ".other"
     msg = maildata("mdn.eml", from_addr, to_addr)
@@ -75,8 +53,7 @@ def test_filtermail_is_mdn(maildata, gencreds, handler):
 
     assert check_mdn(msg, env)
     print(msg.as_string())
-
+    assert not check_DATA(env)
-    assert not handler.check_DATA(env)
 
 
 def test_filtermail_to_multiple_recipients_no_mdn(maildata, gencreds):
@@ -96,20 +73,21 @@ def test_filtermail_to_multiple_recipients_no_mdn(maildata, gencreds):
 def test_send_rate_limiter():
     limiter = SendRateLimiter()
     for i in range(100):
-        if limiter.is_sending_allowed("some@example.org", 10):
+        if limiter.is_sending_allowed("some@example.org"):
-            if i <= 10:
+            if i <= SendRateLimiter.MAX_USER_SEND_PER_MINUTE:
                 continue
             pytest.fail("limiter didn't work")
         else:
-            assert i == 11
+            assert i == SendRateLimiter.MAX_USER_SEND_PER_MINUTE + 1
             break
 
 
-def test_excempt_privacy(maildata, gencreds, handler):
+def test_excempt_privacy(maildata, gencreds):
     from_addr = gencreds()[0]
     to_addr = "privacy@testrun.org"
-    handler.config.passthrough_recipients = [to_addr]
+    false_to = "privacy@tstrn.org"
-    false_to = "privacy@something.org"
+    false_to2 = "prvcy@testrun.org"
+    assert is_passthrough_recipient(to_addr)
 
     msg = maildata("plain.eml", from_addr, to_addr)
 
@@ -119,27 +97,11 @@ def test_excempt_privacy(maildata, gencreds, handler):
         content = msg.as_bytes()
 
     # assert that None/no error is returned
-    assert not handler.check_DATA(envelope=env)
+    assert not check_DATA(envelope=env)
 
     class env2:
         mail_from = from_addr
-        rcpt_tos = [to_addr, false_to]
+        rcpt_tos = [to_addr, false_to, false_to2]
         content = msg.as_bytes()
 
-    assert "500" in handler.check_DATA(envelope=env2)
+    assert "500" in check_DATA(envelope=env2)
-
-
-def test_passthrough_senders(gencreds, handler, maildata):
-    acc1 = gencreds()[0]
-    to_addr = "recipient@something.org"
-    handler.config.passthrough_senders = [acc1]
-
-    msg = maildata("plain.eml", acc1, to_addr)
-
-    class env:
-        mail_from = acc1
-        rcpt_tos = to_addr
-        content = msg.as_bytes()
-
-    # assert that None/no error is returned
-    assert not handler.check_DATA(envelope=env)

tests/chatmaild/test_newmail.py

@@ -4,24 +4,26 @@ import chatmaild
 from chatmaild.newemail import create_newemail_dict, print_new_account
 
 
-def test_create_newemail_dict(example_config):
+def test_create_newemail_dict():
-    ac1 = create_newemail_dict(example_config)
+    ac1 = create_newemail_dict(domain="example.org")
     assert "@" in ac1["email"]
     assert len(ac1["password"]) >= 10
 
-    ac2 = create_newemail_dict(example_config)
+    ac2 = create_newemail_dict(domain="example.org")
 
     assert ac1["email"] != ac2["email"]
     assert ac1["password"] != ac2["password"]
 
 
-def test_print_new_account(capsys, monkeypatch, maildomain, tmpdir, example_config):
+def test_print_new_account(capsys, monkeypatch, maildomain, tmpdir):
-    monkeypatch.setattr(chatmaild.newemail, "CONFIG_PATH", str(example_config._inipath))
+    p = tmpdir.join("mailname")
+    p.write(maildomain)
+    monkeypatch.setattr(chatmaild.newemail, "mailname_path", str(p))
     print_new_account()
     out, err = capsys.readouterr()
     lines = out.split("\n")
     assert lines[0] == "Content-Type: application/json"
     assert not lines[1]
     dic = json.loads(lines[2])
-    assert dic["email"].endswith(f"@{example_config.mail_domain}")
+    assert dic["email"].endswith(f"@{maildomain}")
     assert len(dic["password"]) >= 10

tests/conftest.py

@@ -6,11 +6,12 @@ import subprocess
 import imaplib
 import smtplib
 import itertools
+from email.parser import BytesParser
+from email import policy
 from pathlib import Path
 import pytest
 
 from chatmaild.database import Database
-from chatmaild.config import read_config
 
 
 conftestdir = Path(__file__).parent
@@ -37,22 +38,11 @@ def pytest_runtest_setup(item):
 
 
 @pytest.fixture
-def chatmail_config(pytestconfig):
+def maildomain():
-    current = basedir = Path().resolve()
+    domain = os.environ.get("CHATMAIL_DOMAIN")
-    while 1:
+    if not domain:
-        path = current.joinpath("chatmail.ini").resolve()
+        pytest.skip("set CHATMAIL_DOMAIN to a ssh-reachable chatmail instance")
-        if path.exists():
+    return domain
-            return read_config(path)
-        if current == current.parent:
-            break
-        current = current.parent
-
-    pytest.skip(f"no chatmail.ini file found in {basedir} or parent dirs")
-
-
-@pytest.fixture
-def maildomain(chatmail_config):
-    return chatmail_config.mail_domain
 
 
 @pytest.fixture
@@ -228,25 +218,18 @@ def imap_or_smtp(request):
 
 
 @pytest.fixture
-def gencreds(chatmail_config):
+def gencreds(maildomain):
     count = itertools.count()
     next(count)
 
     def gen(domain=None):
-        domain = domain if domain else chatmail_config.mail_domain
+        domain = domain if domain else maildomain
         while 1:
             num = next(count)
             alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
-            user = "".join(
+            user = "".join(random.choices(alphanumeric, k=10))
-                random.choices(alphanumeric, k=chatmail_config.username_max_length)
+            user = f"ac{num}_{user}"[:9]
-            )
+            password = "".join(random.choices(alphanumeric, k=12))
-            if domain == "nine.testrun.org":
-                user = f"ac{num}_{user}"[:9]
-            else:
-                user = f"ac{num}_{user}"[: chatmail_config.username_max_length]
-            password = "".join(
-                random.choices(alphanumeric, k=chatmail_config.password_min_length)
-            )
             yield f"{user}@{domain}", f"{password}"
 
     return lambda domain=None: next(gen(domain))
@@ -357,6 +340,22 @@ def lp(request):
     return LP()
 
 
+@pytest.fixture
+def maildata(request, gencreds):
+    datadir = conftestdir.joinpath("mail-data")
+
+    def maildata(name, from_addr=None, to_addr=None):
+        if from_addr is None:
+            from_addr = gencreds()[0]
+        if to_addr is None:
+            to_addr = gencreds()[0]
+        data = datadir.joinpath(name).read_text()
+        text = data.format(from_addr=from_addr, to_addr=to_addr)
+        return BytesParser(policy=policy.default).parsebytes(text.encode())
+
+    return maildata
+
+
 @pytest.fixture
 def cmsetup(maildomain, gencreds):
     return CMSetup(maildomain, gencreds)

tests/online/benchmark.py

@@ -30,31 +30,31 @@ def test_login_smtp(benchmark, smtp, gencreds):
 
 class TestDC:
     def test_autoconfigure(self, benchmark, cmfactory):
-        def dc_autoconfig_and_idle_ready():
+        def autoconfig_and_idle_ready():
             cmfactory.get_online_accounts(1)
 
-        benchmark(dc_autoconfig_and_idle_ready, 5)
+        benchmark(autoconfig_and_idle_ready, 5)
 
     def test_ping_pong(self, benchmark, cmfactory):
         ac1, ac2 = cmfactory.get_online_accounts(2)
         chat = cmfactory.get_accepted_chat(ac1, ac2)
 
-        def dc_ping_pong():
+        def ping_pong():
             chat.send_text("ping")
             msg = ac2.wait_next_incoming_message()
             msg.chat.send_text("pong")
             ac1.wait_next_incoming_message()
 
-        benchmark(dc_ping_pong, 5)
+        benchmark(ping_pong, 5)
 
     def test_send_10_receive_10(self, benchmark, cmfactory, lp):
         ac1, ac2 = cmfactory.get_online_accounts(2)
         chat = cmfactory.get_accepted_chat(ac1, ac2)
 
-        def dc_send_10_receive_10():
+        def send_10_receive_10():
             for i in range(10):
                 chat.send_text(f"hello {i}")
             for i in range(10):
                 ac2.wait_next_incoming_message()
 
-        benchmark(dc_send_10_receive_10, 5)
+        benchmark(send_10_receive_10, 5)

tests/online/test_0_login.py

@@ -2,16 +2,6 @@ import pytest
 import threading
 import queue
 
-from chatmaild.config import read_config
-from cmdeploy.cmdeploy import main
-
-
-def test_init(tmp_path, maildomain):
-    inipath = tmp_path.joinpath("chatmail.ini")
-    main(["init", "--config", str(inipath), maildomain])
-    config = read_config(inipath)
-    assert config.mail_domain == maildomain
-
 
 def test_login_basic_functioning(imap_or_smtp, gencreds, lp):
     """Test a) that an initial login creates a user automatically

tests/online/test_0_qr.py

@@ -0,0 +1,6 @@
+from deploy_chatmail.genqr import gen_qr_png_data
+
+
+def test_gen_qr_png_data(maildomain):
+    data = gen_qr_png_data(maildomain)
+    assert data

tests/online/test_1_basic.py

@@ -20,7 +20,7 @@ def test_use_two_chatmailservers(cmfactory, maildomain2):
 
 
 @pytest.mark.parametrize("forgeaddr", ["internal", "someone@example.org"])
-def test_reject_forged_from(cmsetup, maildata, gencreds, lp, forgeaddr):
+def test_reject_forged_from(cmsetup, maildata, lp, forgeaddr):
     user1, user3 = cmsetup.gen_users(2)
 
     lp.sec("send encrypted message with forged from")
@@ -43,18 +43,18 @@ def test_reject_forged_from(cmsetup, maildata, gencreds, lp, forgeaddr):
 
 
 @pytest.mark.slow
-def test_exceed_rate_limit(cmsetup, gencreds, maildata, chatmail_config):
+def test_exceed_rate_limit(cmsetup, gencreds, maildata):
     """Test that the per-account send-mail limit is exceeded."""
     user1, user2 = cmsetup.gen_users(2)
     mail = maildata(
         "encrypted.eml", from_addr=user1.addr, to_addr=user2.addr
     ).as_string()
-    for i in range(chatmail_config.max_user_send_per_minute + 5):
+    for i in range(100):
         print("Sending mail", str(i))
         try:
             user1.smtp.sendmail(user1.addr, [user2.addr], mail)
         except smtplib.SMTPException as e:
-            if i < chatmail_config.max_user_send_per_minute:
+            if i < 80:
                 pytest.fail(f"rate limit was exceeded too early with msg {i}")
             outcome = e.recipients[user2.addr]
             assert outcome[0] == 450

tests/online/test_2_deltachat.py

@@ -1,5 +1,4 @@
 import time
-import re
 import random
 import pytest
 
@@ -21,24 +20,14 @@ class TestEndToEndDeltaChat:
         assert msg2.text == "message0"
 
     @pytest.mark.slow
-    def test_exceed_quota(self, cmfactory, lp, tmpdir, remote, chatmail_config):
+    def test_exceed_quota(self, cmfactory, lp, tmpdir, remote):
         """This is a very slow test as it needs to upload >100MB of mail data
         before quota is exceeded, and thus depends on the speed of the upload.
         """
         ac1, ac2 = cmfactory.get_online_accounts(2)
         chat = cmfactory.get_accepted_chat(ac1, ac2)
 
-        def parse_size_limit(limit: str) -> int:
+        quota = 1024 * 1024 * 100
-            """Parse a size limit and return the number of bytes as integer.
-
-            Example input: 100M, 2.4T, 500 K
-            """
-            units = {"B": 1, "K": 2**10, "M": 2**20, "G": 2**30, "T": 2**40}
-            size = re.sub(r"([KMGT])", r" \1", limit.upper())
-            number, unit = [string.strip() for string in size.split()]
-            return int(float(number) * units[unit])
-
-        quota = parse_size_limit(chatmail_config.max_mailbox_size)
         attachsize = 1 * 1024 * 1024
         num_to_send = quota // attachsize + 2
         lp.sec(f"ac1: send {num_to_send} large files to ac2")
@@ -102,9 +91,9 @@ class TestEndToEndDeltaChat:
 
         lp.sec("setup encrypted comms between ac1 and ac2 on different instances")
         qr = ac1.get_setup_contact_qr()
-        ch = ac2.qr_setup_contact(qr)
+        ac2.qr_setup_contact(qr)
-        assert ch.id >= 10
+        msg = ac2.wait_next_incoming_message()
-        ac1._evtracker.wait_securejoin_inviter_progress(1000)
+        assert "verified" in msg.text
 
         lp.sec("ac1 sends a message and ac2 marks it as seen")
         chat = ac1.create_chat(ac2)

tests/test_helpers.py

@@ -0,0 +1,47 @@
+import textwrap
+import importlib.resources
+
+from deploy_chatmail.www import build_webpages
+from deploy_chatmail import get_ini_settings
+
+
+def create_ini(inipath):
+    inipath.write_text(
+        textwrap.dedent(
+            """\
+        [config]
+
+        privacy_postal =
+            address-line1
+            address-line2
+
+        privacy_mail = privacy@example.org
+
+        privacy_pdo =
+            address-line3
+    """
+        )
+    )
+
+
+def test_build_webpages(tmp_path):
+    pkgroot = importlib.resources.files("deploy_chatmail")
+    src_dir = pkgroot.joinpath("../../../www/src").resolve()
+    assert src_dir.exists(), src_dir
+
+    inipath = tmp_path.joinpath("chatmail.ini")
+    create_ini(inipath)
+    config = get_ini_settings("example.org", inipath)
+    build_dir = tmp_path.joinpath("build")
+    build_webpages(src_dir, build_dir, config)
+
+
+def test_get_settings(tmp_path):
+    inipath = tmp_path.joinpath("chatmail.ini")
+    create_ini(inipath)
+
+    d = get_ini_settings("x.testrun.org", inipath)
+    assert d["privacy_postal"] == "address-line1\naddress-line2"
+    assert d["privacy_mail"] == "privacy@example.org"
+    assert d["privacy_pdo"] == "address-line3"
+    assert d["mail_domain"] == "x.testrun.org"

www/src/index.md

@@ -1,5 +1,5 @@
 
-<img class="banner" src="collage-top.png"/>
+<img width="800px" src="collage-top.png"/>
 
 ## Dear [Delta Chat](https://get.delta.chat) users and newcomers, 
 
@@ -14,6 +14,6 @@ Welcome to instant, interoperable and [privacy-preserving](privacy.html) messagi
 
 💬 **Start** chatting with any Delta Chat contacts using [QR invite codes](https://delta.chat/en/help#howtoe2ee)
 
-{% if config.mail_domain != "nine.testrun.org" %}
+
-<div class="experimental">Note: this is only a temporary development chatmail service</div>
+## ⚡ Note: this is an experimental service ⚡
-{% endif %}
+

www/src/info.md

@@ -1,34 +1,18 @@
 
-<img class="banner" src="collage-info.png"/>
+<img width="800px" src="collage-info.png"/>
 
 ## More information 
 
-`nine.testrun.org` provides a low-maintenance, resource efficient and 
-interoperable e-mail service for everyone. What's behind a `chatmail` is 
-effectively a normal e-mail address just like any other but optimized 
-for the usage in chats, especially DeltaChat.
-
 ### Choosing a chatmail address instead of using a random one
 
 In the Delta Chat account setup 
 you may tap `LOG INTO YOUR E-MAIL ACCOUNT` 
 and fill the two fields like this: 
 
-- `Address`: invent a word with
+- `Address`: invent a word with *exactly* nine characters 
-{% if username_min_length == username_max_length %}
+   and append `@{{config.mail_domain}}` to it. 
-  *exactly* {{ username_min_length }}
-{% else %}
-  {{ username_min_length}}
-  {% if username_max_length == "more" %}
-    or more
-  {% else %}
-    to {{ username_max_length }}
-  {% endif %}
-{% endif %}
-  characters
-  and append `@{{config.mail_domain}}` to it.
 
-- `Password`: invent at least {{ password_min_length }} characters.
+- `Password`: invent at least 9 characters. 
 
 If the e-mail address is not yet taken, you'll get that account. 
 The first login sets your password. 
@@ -40,11 +24,11 @@ The first login sets your password.
   {{config.mail_domain}} but setting up contact via [QR invite codes](https://delta.chat/en/help#howtoe2ee) 
   allows your messages to pass freely to any outside recipients.
 
-- You may send up to {{ config.max_user_send_per_minute }} messages per minute.
+- You may send up to 60 messages per minute
 
-- Messages are unconditionally removed {{ config.delete_mails_after }} days after arriving on the server.
+- Messages are unconditionally removed 40 days after arriving on the server
 
-- You can store up to [{{ config.max_mailbox_size }} messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server).
+- You can store up to [100MB messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server)
 
 
 ### Who are the operators? Which software is running? 

www/src/logo.svg

@@ -1,44 +0,0 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
-<svg
-   width="145"
-   height="145"
-   version="1.1"
-   id="svg4"
-   sodipodi:docname="At_sign.svg"
-   inkscape:version="1.2.2 (b0a84865, 2022-12-01)"
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
-   xmlns="http://www.w3.org/2000/svg"
-   xmlns:svg="http://www.w3.org/2000/svg">
-  <defs
-     id="defs8" />
-  <sodipodi:namedview
-     id="namedview6"
-     pagecolor="#ffffff"
-     bordercolor="#000000"
-     borderopacity="0.25"
-     inkscape:showpageshadow="2"
-     inkscape:pageopacity="0.0"
-     inkscape:pagecheckerboard="0"
-     inkscape:deskcolor="#d1d1d1"
-     showgrid="false"
-     inkscape:zoom="3.0241379"
-     inkscape:cx="67.622577"
-     inkscape:cy="72.913341"
-     inkscape:window-width="1390"
-     inkscape:window-height="1027"
-     inkscape:window-x="55"
-     inkscape:window-y="25"
-     inkscape:window-maximized="0"
-     inkscape:current-layer="svg4" />
-  <g
-     aria-label="@"
-     id="text2"
-     style="font-size:144px;font-family:Arial">
-    <path
-       d="m 79.927878,94.422406 c -2.704286,3.120332 -5.741407,5.637394 -9.111364,7.551194 -3.328352,1.87221 -6.677506,2.80831 -10.047463,2.80831 -3.702792,0 -7.301573,-1.08172 -10.796342,-3.24515 -3.49477,-2.163426 -6.344671,-5.491779 -8.549704,-9.985058 -2.163429,-4.493275 -3.245144,-9.423397 -3.245144,-14.790365 0,-6.615099 1.684978,-13.230199 5.054935,-19.845299 3.411561,-6.656705 7.634407,-11.649233 12.66854,-14.977585 5.034133,-3.328352 9.92265,-4.992528 14.665552,-4.992528 3.619583,0 7.072748,0.956901 10.359496,2.870704 3.286748,1.872198 6.115847,4.742902 8.487297,8.612111 l 2.121825,-9.673023 h 11.170784 l -8.986557,41.87483 c -1.248129,5.824616 -1.872194,9.048957 -1.872194,9.673023 0,1.123319 0.416044,2.101022 1.248132,2.93311 0.873692,0.790484 1.913802,1.185726 3.120332,1.185726 2.20503,0 5.096537,-1.268934 8.674517,-3.806803 4.7429,-3.328352 8.4873,-7.780023 11.23319,-13.355013 2.78749,-5.616594 4.18124,-11.399606 4.18124,-17.349035 0,-6.947935 -1.78899,-13.438222 -5.36697,-19.47086 -3.53637,-6.032638 -8.84094,-10.858749 -15.913687,-14.478332 -7.03114,-3.619583 -14.811161,-5.429374 -23.340064,-5.429374 -9.73543,0 -18.638772,2.288242 -26.710026,6.864726 -8.029649,4.534879 -14.27031,11.06677 -18.721981,19.595673 -4.410066,8.487298 -6.615099,17.598662 -6.615099,27.334092 0,10.193078 2.205033,18.971607 6.615099,26.33559 2.290454,3.78888 -7.136335,18.96983 -3.810585,21.73443 3.138096,2.60861 18.971963,-7.14297 23.031819,-5.44631 8.404089,3.53637 17.702673,5.30456 27.895752,5.30456 10.90035,0 20.032515,-1.83059 27.396492,-5.49178 7.36399,-3.66119 12.87657,-8.11286 16.53776,-13.35501 l 9.29559,4 c -2.12183,4.36846 -3.76221,4.82013 -8.92116,9.35501 -5.15895,4.53488 -11.2956,8.11286 -18.40995,10.73393 -7.114346,2.66268 -15.684851,3.99402 -25.711512,3.99402 -9.236177,0 -17.76508,-1.18572 -25.586707,-3.55717 -7.780023,-2.37145 -29.296198,9.26152 -34.78798,4.47701 -5.49178,-4.7429 5.248856,-25.42482 2.461361,-31.62388 -3.49477,-7.863231 -5.242155,-16.350531 -5.242155,-25.461894 0,-10.151474 2.08022,-19.824498 6.240661,-29.019071 5.075736,-11.274793 12.273297,-19.907706 21.592683,-25.898739 9.360991,-5.991034 20.69819,-8.986551 34.011599,-8.986551 10.317891,0 19.574873,2.121824 27.77093,6.365473 8.23767,4.202045 14.72796,10.484309 19.47086,18.846794 4.03563,7.197561 6.05344,15.019189 6.05344,23.464883 0,12.065277 -4.24365,22.77841 -12.73094,32.1394 -7.572,8.404095 -15.85128,12.606135 -24.837827,12.606135 -2.870704,0 -5.200551,-0.43684 -6.98954,-1.31053 -1.747385,-0.8737 -3.037121,-2.12183 -3.869209,-3.744402 -0.540857,-1.040114 -0.936099,-2.829105 -1.185726,-5.366972 z M 49.723082,77.510217 c 0,5.699803 1.352143,10.130671 4.05643,13.292606 2.704286,3.161935 5.803814,4.742902 9.298583,4.742902 2.329847,0 4.784506,-0.686473 7.363979,-2.059418 2.579473,-1.41455 5.034133,-3.49477 7.363979,-6.240661 2.371451,-2.74589 4.306056,-6.219857 5.803815,-10.421902 1.497759,-4.243649 2.246638,-8.487298 2.246638,-12.730947 0,-5.658198 -1.41455,-10.047462 -4.243649,-13.167793 -2.787495,-3.12033 -6.199056,-4.680495 -10.234683,-4.680495 -2.662682,0 -5.179749,0.686473 -7.5512,2.059418 -2.329846,1.331341 -4.597286,3.494769 -6.802319,6.490286 -2.205033,2.995517 -3.97322,6.635903 -5.304561,10.921156 -1.331341,4.285253 -1.997012,8.216869 -1.997012,11.794848 z"
-       id="path347"
-       style="stroke-width:0.887561"
-       sodipodi:nodetypes="ccsscscsscccccscsccsccsccscscssccscscccsccsccscscccssscccscscsss" />
-  </g>
-</svg>

www/src/main.css

@@ -1,74 +0,0 @@
-
-body {
-  font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', 'Segoe UI Emoji', 'Apple Color Emoji', 'Noto Color Emoji', sans-serif;
-  line-height: 1.4;
-  font-size: 1.2em;
-  max-width: 800px;
-  margin: 20px auto;
-  padding: 0 10px;
-  color: #363636;
-  background: #fff;
-}
-
-h1 {
-  font-size: 2.2em;
-  margin-top: 0;
-}
-
-h1, h2, h3, h4, h5, h6 {
-  color: #000;
-  margin-bottom: 12px;
-  margin-top: 24px;
-  font-weight: 600;
-}
-
-a {
-  text-decoration: none;
-  color: #0076d1;
-}
-
-a:hover {
-  text-decoration: underline;
-}
-
-img, video {
-  max-width: 100%;
-  height: auto;
-}
-
-code {
-  background: #efefef;
-  padding: 2.5px 5px;
-  border-radius: 6px;
-}
-
-#menu {
-  display: flex;
-  flex-wrap: wrap;
-  padding: 0;
-}
-
-#menu li {
-  display: inline-block;
-  padding-right: 0.5em;
-}
-
-#domain {
-  margin-left: auto;
-}
-
-#domain a {
-  color: #888;
-}
-
-.banner {
-  width: 100%;
-}
-
-.experimental {
-  margin: 3em 0;
-  padding: 1em;
-  border: 4px dashed red;
-  color: red;
-  font-weight: bold;
-}

www/src/page-layout.html

@@ -5,23 +5,16 @@
     {% if config.webdev %}
     <meta http-equiv="refresh" content="3">
     {% endif %}
-    <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">
     <title>{{ config.mail_domain }} {{ pagename }}</title>
-    <link rel="stylesheet" href="./main.css">
+    <link rel="stylesheet" href="./water.css">
-    <link rel="icon" href="/logo.svg">
-    <link rel=”mask-icon” href=”/logo.svg” color=”#000000">
 </head>
 <body>
-
-<ul id="menu">
-    <li><a href="index.html">home</a></li>
-    <li><a href="info.html">info</a></li>
-    <li><a href="privacy.html">privacy</a></li>
-    <li><a href="https://github.com/deltachat/chatmail">public code ↗</a></li>
-    <li id="domain"><a href="index.html">{{ config.mail_domain }}</a></li>
-</ul>
-
 {{ markdown_html }}
-
+<footer>
+    <a href="index.html">home</a> |
+    <a href="info.html">more info</a> |
+    <a href="privacy.html">privacy</a> |
+    <a href="https://github.com/deltachat/chatmail">-> public development </a>
+</footer>
 </body>
 </html>

www/src/privacy.md

@@ -1,4 +1,4 @@
-<img class="banner" src="collage-privacy.png"/>
+<img width="800px" src="collage-privacy.png"/>
 
 # Privacy Policy for {{ config.mail_domain }} 
 
@@ -62,7 +62,7 @@ we process the following data and details:
 Creating an account happens in one of two ways on our mail servers: 
 
 - with a QR invitation token 
-  which is scanned using the Delta Chat app
+  which is scanned using the DeltaChat app
   and then the account is created.
 
 - by letting Delta Chat otherwise create an account 
@@ -218,16 +218,97 @@ on these or possible objections.
 A deletion can be made
 directly in the Delta Chat email messenger.
 
-If you have any questions or complaints, 
+a) request information about your personal data processed by us
-please feel free to contact us by email:  
+in accordance with Art. 15 GDPR.
-{{ config.privacy_mail }}
+In particular,
+you can request information about the processing purposes,
+the category of personal data,
+the categories of recipients to whom your data have been or will be disclosed,
+the planned storage period,
+the existence of a right to rectification, erasure, restriction of processing or objection,
+the existence of a right of complaint,
+the origin of your data if it has not been collected by us,
+as well as the existence of automated decision-making including profiling
+and, if applicable,
+meaningful information about its details;  
 
-As a rule, you can contact the supervisory authority of your usual place of residence
+b) in accordance with Art. 16 of the GDPR,
+immediately request the correction
+of inaccurate or incomplete personal data stored by us;  
+
+c) pursuant to Article 17 of the GDPR,
+to request the erasure of your personal data stored by us,
+unless the processing is necessary
+for the exercise of the right to freedom of expression and information,
+for compliance with a legal obligation,
+for reasons of public interest,
+or for the establishment, exercise or defence of legal claims;  
+
+d) pursuant to Art. 18 GDPR,
+to request the restriction of the processing of your personal data,
+insofar as the accuracy of the data is disputed by you,
+the processing is unlawful,
+but you object to its erasure
+and we no longer require the data,
+but you need it for the assertion, exercise or defence of legal claims
+or you have objected to the processing pursuant to Art. 21 GDPR;  
+
+e) pursuant to Art. 20 GDPR,
+to receive your personal data that you have provided to us
+in a structured, common and machine-readable format
+or to request that it be transferred to another controller;  
+
+f) in accordance with Art. 7 (3) of the GDPR,
+to revoke your consent given to us at any time.
+This has the consequence that we may no longer continue the data processing
+based on this consent in the future; and  
+
+g) complain to a supervisory authority
+in accordance with Article 77 of the GDPR.
+As a rule,
+you can contact the supervisory authority of your usual place of residence
 or workplace
 or our registered office for this purpose.
 The supervisory authority responsible for our place of business
 is the `{{ config.privacy_supervisor }}`.
 
+If you have any questions or complaints, please feel free to contact us by email:  
+{{ config.privacy_mail }}
+
+
+### 5.1 Right to object
+
+If your personal data is processed on the basis of our legitimate interests
+in accordance with Art. 6 (1) lit. f GDPR,
+you have the right to object to the processing of your personal data
+in accordance with Art. 21 GDPR,
+provided that there are grounds for this based on your particular situation
+or the objection is directed against direct advertising.
+In the latter case,
+you have a general right of objection,
+which will be implemented by us
+without specifying a particular situation.
+
+If you wish to exercise your right of objection,
+simply send an e-mail to: {{ config.privacy_mail }}
+
+### 5.2 Right to withdraw
+
+If your personal data is processed on the basis of your consent
+in accordance with Art. 6 (1) lit. a GDPR
+(e.g. via the mailing list),
+you can withdraw your consent at any time
+and without any disadvantages.
+As a result,
+we may no longer continue the data processing
+that was based on this consent for the future.
+However,
+the withdrawal of your consent
+does not affect the lawfulness of the processing
+carried out on the basis of the consent until the withdrawal.
+
+If you wish to make use of your right of withdrawal,
+simply send an e-mail to: {{ config.privacy_mail }}
 
 ## 6. Validity of this privacy policy