strike this weird CHATMAIL_DOMAIN variable

This expands the character set used for passwords generated for new
accounts.  The set it taken from the set used by the pass tool.  The
special characters is the full GNU grep [:punct:] set.

.github/CODE_OF_CONDUCT.md

@@ -0,0 +1,4 @@
+
+Please refer to 
+[Delta Chat community standards and practices](https://delta.chat/en/community-standards)
+which also apply for all chatmail developments. 

.github/workflows/ci.yaml

@@ -31,7 +31,7 @@ jobs:
         run: cmdeploy fmt -v 
 
       - name: run deploy-chatmail offline tests 
-        run: pytest tests
+        run: pytest --pyargs cmdeploy 
 
       - name: initialize with chatmail domain
         run: cmdeploy init chat.example.org  

LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2023, chatmail and delta chat teams
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
+of the Software, and to permit persons to whom the Software is furnished to do
+so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -15,146 +15,144 @@ after which the initially specified password is required for using them.
 
 ## Deploying your own chatmail server 
 
-We subsequently use `CHATMAIL_DOMAIN` as a placeholder for your fully qualified 
-DNS domain name (FQDN), for example `chat.example.org`.
+We use `chat.example.org` as the chatmail domain in the following steps. 
+Please substitute it with your own domain. 
 
-1. Setup DNS `A` and `AAAA` records for your `CHATMAIL_DOMAIN`. 
-   Verify that DNS is set and SSH root login works:
+1. Install the `cmdeploy` command in a virtualenv
 
    ```
-        ssh root@CHATMAIL_DOMAIN 
+    git clone https://github.com/deltachat/chatmail
+    cd chatmail
+    scripts/initenv.sh
    ```
 
-2. Install the `cmdeploy` command in a virtualenv 
+2. Create chatmail configuration file `chatmail.ini`:
 
    ```
-    source scripts/initenv.sh
+    scripts/cmdeploy init chat.example.org  # <-- use your domain 
    ```
-  
-3. Create chatmail configuration file `chatmail.ini`:
+
+3. Setup first DNS records for your chatmail domain, 
+   according to the hints provided by `cmdeploy init`.
+   Verify that SSH root login works:
 
    ```
-    cmdeploy init CHATMAIL_DOMAIN
+    ssh root@chat.example.org   # <-- use your domain 
    ```
 
 4. Deploy to the remote chatmail server:
 
    ```
-    cmdeploy run 
+    scripts/cmdeploy run
    ```
 
 5. To output a DNS zone file from which you can transfer DNS records 
    to your DNS provider:
 
    ```
-    cmdeploy dns
+    scripts/cmdeploy dns
    ```
 
-6. To check status of your remotely running chatmail service: 
+### Other helpful commands:
 
-   ```
-    cmdeploy status
-   ```
+To check the status of your remotely running chatmail service:
 
-7. To test your chatmail service: 
-
-   ```
-    cmdeploy test
-   ```
-
-8. To benchmark your chatmail service: 
-
-   ```
-    cmdeploy bench
-   ```
-
-### Refining the web pages 
-
-
-``` 
-    cmdeploy webdev 
+```
+scripts/cmdeploy status
 ```
 
-This starts a local live development cycle for chatmail Web pages: 
+To test whether your chatmail service is working correctly:
 
-- uses the `www/src/page-layout.html` file for producing static 
-  HTML pages from `www/src/*.md` files
+```
+scripts/cmdeploy test
+```
 
-- continously builds the web presence reading files from `www/src` directory
-  and generating html files and copying assets to the `www/build` directory. 
+To measure the performance of your chatmail service:
 
-- Starts a browser window automatically where you can "refresh" as needed. 
+```
+scripts/cmdeploy bench
+```
 
+## Overview of this repository
 
-### Home page and getting started for users 
+This repository drives the development of chatmail services, 
+comprised of minimal setups of
 
-`cmdeploy run` sets up mail services,
-and also creates default static Web pages and deploys them: 
+- [postfix smtp server](https://www.postfix.org)
+- [dovecot imap server](https://www.dovecot.org)
 
-- a default `index.html` along with a QR code that users can click to 
-  create accounts on your chatmail provider,
-
-- a default `info.html` that is linked from the home page,
-
-- a default `policy.html` that is linked from the home page. 
-
-All `.html` files are generated 
-by the according markdown `.md` file in the `www/src` directory.
-
-
-### Ports
-
-Postfix listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
-Dovecot listens on ports 143(imap) and 993 (imaps).
-
-Delta Chat will, however, discover all ports and configurations 
-automatically by reading the `autoconfig.xml` file from the chatmail instance. 
-
-
-## Emergency Commands to disable automatic account creation 
-
-If you need to stop account creation,
-e.g. because some script is wildly creating accounts, run:
-
-    touch /etc/chatmail-nocreate
-
-While this file is present, account creation will be blocked. 
-
-
-## Running tests and benchmarks (offline and online) 
-
-1. Set `CHATMAIL_SSH` so that `ssh root@$CHATMAIL_SSH` allows 
-   to login to the chatmail instance server. 
-
-2. To run local and online tests: 
-
-    scripts/test.sh 
-
-3. To run benchmarks against your chatmail instance: 
-
-    scripts/bench.sh 
-
-
-## Development Background for chatmail instances 
-
-This repository drives the development of "chatmail instances", 
-comprised of minimal setups of 
-
-- [postfix smtp server](https://www.postfix.org) 
-- [dovecot imap server](https://www.dovecot.org) 
-
-as well as two custom services that are integrated with these two: 
+as well as custom services that are integrated with these two:
 
 - `chatmaild/src/chatmaild/doveauth.py` implements
   create-on-login account creation semantics and is used
   by Dovecot during login authentication and by Postfix
   which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)
   to authenticate users
-  to send mails for them. 
+  to send mails for them.
 
-- `chatmaild/src/chatmaild/filtermail.py` prevents 
-  unencrypted e-mail from leaving the chatmail instance
-  and is integrated into postfix's outbound mail pipelines. 
+- `chatmaild/src/chatmaild/filtermail.py` prevents
+  unencrypted e-mail from leaving the chatmail service
+  and is integrated into postfix's outbound mail pipelines.
+
+There is also the `cmdeploy/src/cmdeploy/cmdeploy.py` command line tool
+which helps with setting up and managing the chatmail service.
+`cmdeploy run` uses [pyinfra-based scripting](https://pyinfra.com/)
+in `cmdeploy/src/cmdeploy/__init__.py`
+to automatically install all chatmail components on a server.
 
 
+### Home page and getting started for users
+
+`cmdeploy run` also creates default static Web pages and deploys them
+to a nginx web server with: 
+
+- a default `index.html` along with a QR code that users can click to
+  create accounts on your chatmail provider,
+
+- a default `info.html` that is linked from the home page,
+
+- a default `policy.html` that is linked from the home page.
+
+All `.html` files are generated
+by the according markdown `.md` file in the `www/src` directory.
+
+
+### Refining the web pages
+
+
+```
+scripts/cmdeploy webdev
+```
+
+This starts a local live development cycle for chatmail Web pages:
+
+- uses the `www/src/page-layout.html` file for producing static
+  HTML pages from `www/src/*.md` files
+
+- continously builds the web presence reading files from `www/src` directory
+  and generating html files and copying assets to the `www/build` directory.
+
+- Starts a browser window automatically where you can "refresh" as needed.
+
+
+## Emergency Commands to disable automatic account creation
+
+If you need to stop account creation,
+e.g. because some script is wildly creating accounts,
+login to the server with ssh and run:
+
+```
+    touch /etc/chatmail-nocreate
+```
+
+While this file is present, account creation will be blocked.
+
+### Ports
+
+Postfix listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
+Dovecot listens on ports 143(imap) and 993 (imaps).
+
+Delta Chat apps will, however, discover all ports and configurations
+automatically by reading the `autoconfig.xml` file from the chatmail service.
+
 

chatmaild/pyproject.toml

@@ -8,6 +8,8 @@ version = "0.2"
 dependencies = [
   "aiosmtpd",
   "iniconfig",
+  "deltachat-rpc-server",
+  "deltachat-rpc-client",
 ]
 
 [tool.setuptools]
@@ -19,6 +21,7 @@ where = ['src']
 [project.scripts]
 doveauth = "chatmaild.doveauth:main"
 filtermail = "chatmaild.filtermail:main"
+echobot = "chatmaild.echo:main"
 
 [project.entry-points.pytest11]
 "chatmaild.testplugin" = "chatmaild.tests.plugin"

chatmaild/src/chatmaild/config.py

@@ -9,11 +9,17 @@ def read_config(inipath):
 class Config:
     def __init__(self, inipath, params):
         self._inipath = inipath
-        self.mailname = self.mail_domain = params["mailname"]
+        self.mail_domain = params["mail_domain"]
         self.max_user_send_per_minute = int(params["max_user_send_per_minute"])
+        self.max_mailbox_size = params["max_mailbox_size"]
+        self.delete_mails_after = params["delete_mails_after"]
+        self.username_min_length = int(params["username_min_length"])
+        self.username_max_length = int(params["username_max_length"])
+        self.password_min_length = int(params["password_min_length"])
+        self.passthrough_senders = params["passthrough_senders"].split()
+        self.passthrough_recipients = params["passthrough_recipients"].split()
         self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
         self.postfix_reinject_port = int(params["postfix_reinject_port"])
-        self.passthrough_recipients = params["passthrough_recipients"].split()
         self.privacy_postal = params.get("privacy_postal")
         self.privacy_mail = params.get("privacy_mail")
         self.privacy_pdo = params.get("privacy_pdo")
@@ -23,12 +29,14 @@ class Config:
         return open(self._inipath, "rb")
 
 
-def write_initial_config(inipath, mailname):
+def write_initial_config(inipath, mail_domain):
     from importlib.resources import files
 
     inidir = files(__package__).joinpath("ini")
-    content = inidir.joinpath("chatmail.ini.f").read_text().format(mailname=mailname)
-    if mailname.endswith(".testrun.org"):
+    content = (
+        inidir.joinpath("chatmail.ini.f").read_text().format(mail_domain=mail_domain)
+    )
+    if mail_domain.endswith(".testrun.org"):
         override_inipath = inidir.joinpath("override-testrun.ini")
         privacy = iniconfig.IniConfig(override_inipath)["privacy"]
         lines = []

chatmaild/src/chatmaild/doveauth.py

@@ -12,6 +12,7 @@ from socketserver import (
 import pwd
 
 from .database import Database
+from .config import read_config, Config
 
 NOCREATE_FILE = "/etc/chatmail-nocreate"
 
@@ -22,14 +23,17 @@ def encrypt_password(password: str):
     return "{SHA512-CRYPT}" + passhash
 
 
-def is_allowed_to_create(user, cleartext_password) -> bool:
+def is_allowed_to_create(config: Config, user, cleartext_password) -> bool:
     """Return True if user and password are admissable."""
     if os.path.exists(NOCREATE_FILE):
         logging.warning(f"blocked account creation because {NOCREATE_FILE!r} exists.")
         return False
 
-    if len(cleartext_password) < 9:
-        logging.warning("Password needs to be at least 9 characters long")
+    if len(cleartext_password) < config.password_min_length:
+        logging.warning(
+            "Password needs to be at least %s characters long",
+            config.password_min_length,
+        )
         return False
 
     parts = user.split("@")
@@ -38,10 +42,17 @@ def is_allowed_to_create(user, cleartext_password) -> bool:
         return False
     localpart, domain = parts
 
-    if domain == "nine.testrun.org":
-        # nine.testrun.org policy, username has to be exactly nine chars
-        if len(localpart) != 9:
-            logging.warning(f"localpart {localpart!r} has not exactly nine chars")
+    if (
+        len(localpart) > config.username_max_length
+        or len(localpart) < config.username_min_length
+    ):
+        if localpart != "echo":
+            logging.warning(
+                "localpart %s has to be between %s and %s chars long",
+                localpart,
+                config.username_min_length,
+                config.username_max_length,
+            )
             return False
 
     return True
@@ -60,7 +71,7 @@ def lookup_userdb(db, user):
     return get_user_data(db, user)
 
 
-def lookup_passdb(db, user, cleartext_password):
+def lookup_passdb(db, config: Config, user, cleartext_password):
     with db.write_transaction() as conn:
         userdata = conn.get_user(user)
         if userdata:
@@ -72,7 +83,7 @@ def lookup_passdb(db, user, cleartext_password):
             userdata["uid"] = "vmail"
             userdata["gid"] = "vmail"
             return userdata
-        if not is_allowed_to_create(user, cleartext_password):
+        if not is_allowed_to_create(config, user, cleartext_password):
             return
 
         encrypted_password = encrypt_password(cleartext_password)
@@ -87,7 +98,7 @@ def lookup_passdb(db, user, cleartext_password):
         )
 
 
-def handle_dovecot_request(msg, db, mail_domain):
+def handle_dovecot_request(msg, db, config: Config):
     short_command = msg[0]
     if short_command == "L":  # LOOKUP
         parts = msg[1:].split("\t")
@@ -97,15 +108,15 @@ def handle_dovecot_request(msg, db, mail_domain):
         res = ""
         if namespace == "shared":
             if type == "userdb":
-                if user.endswith(f"@{mail_domain}"):
+                if user.endswith(f"@{config.mail_domain}"):
                     res = lookup_userdb(db, user)
                 if res:
                     reply_command = "O"
                 else:
                     reply_command = "N"
             elif type == "passdb":
-                if user.endswith(f"@{mail_domain}"):
-                    res = lookup_passdb(db, user, cleartext_password=args[0])
+                if user.endswith(f"@{config.mail_domain}"):
+                    res = lookup_passdb(db, config, user, cleartext_password=args[0])
                 if res:
                     reply_command = "O"
                 else:
@@ -123,8 +134,7 @@ def main():
     socket = sys.argv[1]
     passwd_entry = pwd.getpwnam(sys.argv[2])
     db = Database(sys.argv[3])
-    with open("/etc/mailname", "r") as fp:
-        mail_domain = fp.read().strip()
+    config = read_config(sys.argv[4])
 
     class Handler(StreamRequestHandler):
         def handle(self):
@@ -133,7 +143,7 @@ def main():
                     msg = self.rfile.readline().strip().decode()
                     if not msg:
                         break
-                    res = handle_dovecot_request(msg, db, mail_domain)
+                    res = handle_dovecot_request(msg, db, config)
                     if res:
                         self.wfile.write(res.encode("ascii"))
                         self.wfile.flush()

chatmaild/src/chatmaild/doveauth.service.f

@@ -2,7 +2,7 @@
 Description=Chatmail dict authentication proxy for dovecot
 
 [Service]
-ExecStart={execpath} /run/dovecot/doveauth.socket vmail /home/vmail/passdb.sqlite
+ExecStart={execpath} /run/dovecot/doveauth.socket vmail /home/vmail/passdb.sqlite {config_path}
 Restart=always
 RestartSec=30
 

chatmaild/src/chatmaild/echo.py

@@ -0,0 +1,88 @@
+#!/usr/bin/env python3
+"""Advanced echo bot example.
+
+it will echo back any message that has non-empty text and also supports the /help command.
+"""
+import logging
+import os
+import sys
+from threading import Thread
+
+from deltachat_rpc_client import Bot, DeltaChat, EventType, Rpc, events
+
+from chatmaild.newemail import create_newemail_dict
+from chatmaild.config import read_config
+
+hooks = events.HookCollection()
+
+
+@hooks.on(events.RawEvent)
+def log_event(event):
+    if event.kind == EventType.INFO:
+        logging.info(event.msg)
+    elif event.kind == EventType.WARNING:
+        logging.warning(event.msg)
+
+
+@hooks.on(events.RawEvent(EventType.ERROR))
+def log_error(event):
+    logging.error(event.msg)
+
+
+@hooks.on(events.MemberListChanged)
+def on_memberlist_changed(event):
+    logging.info(
+        "member %s was %s", event.member, "added" if event.member_added else "removed"
+    )
+
+
+@hooks.on(events.GroupImageChanged)
+def on_group_image_changed(event):
+    logging.info("group image %s", "deleted" if event.image_deleted else "changed")
+
+
+@hooks.on(events.GroupNameChanged)
+def on_group_name_changed(event):
+    logging.info("group name changed, old name: %s", event.old_name)
+
+
+@hooks.on(events.NewMessage(func=lambda e: not e.command))
+def echo(event):
+    snapshot = event.message_snapshot
+    if snapshot.text or snapshot.file:
+        snapshot.chat.send_message(text=snapshot.text, file=snapshot.file)
+
+
+@hooks.on(events.NewMessage(command="/help"))
+def help_command(event):
+    snapshot = event.message_snapshot
+    snapshot.chat.send_text("Send me any message and I will echo it back")
+
+
+def main():
+    path = os.environ.get("PATH")
+    venv_path = sys.argv[0].strip("echobot")
+    os.environ["PATH"] = path + ":" + venv_path
+    with Rpc() as rpc:
+        deltachat = DeltaChat(rpc)
+        system_info = deltachat.get_system_info()
+        logging.info("Running deltachat core %s", system_info.deltachat_core_version)
+
+        accounts = deltachat.get_all_accounts()
+        account = accounts[0] if accounts else deltachat.add_account()
+
+        bot = Bot(account, hooks)
+        if not bot.is_configured():
+            config = read_config(sys.argv[1])
+            password = create_newemail_dict(config).get("password")
+            email = "echo@" + config.mail_domain
+            configure_thread = Thread(
+                target=bot.configure, kwargs={"email": email, "password": password}
+            )
+            configure_thread.start()
+        bot.run_forever()
+
+
+if __name__ == "__main__":
+    logging.basicConfig(level=logging.INFO)
+    main()

chatmaild/src/chatmaild/echobot.service.f

@@ -0,0 +1,11 @@
+[Unit]
+Description=Chatmail echo bot for testing it works
+
+[Service]
+ExecStart={execpath} {config_path}
+Environment="PATH={remote_venv_dir}:$PATH"
+Restart=always
+RestartSec=30
+
+[Install]
+WantedBy=multi-user.target

chatmaild/src/chatmaild/filtermail.py

@@ -111,6 +111,9 @@ class BeforeQueueHandler:
         if not mail_encrypted and check_mdn(message, envelope):
             return
 
+        if envelope.mail_from in self.config.passthrough_senders:
+            return
+
         passthrough_recipients = self.config.passthrough_recipients
         envelope_from_domain = from_addr.split("@").pop()
         for recipient in envelope.rcpt_tos:

chatmaild/src/chatmaild/ini/chatmail.ini.f

@@ -1,24 +1,54 @@
 [params]
 
 # mail domain (MUST be set to fully qualified chat mail domain)
-mailname = {mailname}
+mail_domain = {mail_domain}
 
 #
 # If you only do private test deploys, you don't need to modify any settings below
 #
 
+#
+# Account Restrictions
+#
+
 # how many mails a user can send out per minute
 max_user_send_per_minute = 60
 
+# maximum mailbox size of a chatmail account
+max_mailbox_size = 100M
+
+# time after which seen mails are deleted
+delete_mails_after = 40d
+
+# minimum length a username must have
+username_min_length = 9
+
+# maximum length a username can have
+username_max_length = 9
+
+# minimum length a password must have
+password_min_length = 9
+
+# list of chatmail accounts which can send outbound un-encrypted mail
+passthrough_senders =
+
 # list of e-mail recipients for which to accept outbound un-encrypted mails
 passthrough_recipients =
 
+#
+# Deployment Details
+#
+
 # where the filtermail SMTP service listens
 filtermail_smtp_port = 10080
 
 # postfix accepts on the localhost reinject SMTP port
 postfix_reinject_port = 10025
 
+#
+# Privacy Policy
+#
+
 # postal address of privacy contact
 privacy_postal =
 

chatmaild/src/chatmaild/newemail.py

@@ -1,23 +1,31 @@
-#!/usr/bin/python3
+#!/usr/local/lib/chatmaild/venv/bin/python3
 
 """ CGI script for creating new accounts. """
 
 import json
 import random
+import secrets
+import string
 
-mailname_path = "/etc/mailname"
+from chatmaild.config import read_config, Config
+
+CONFIG_PATH = "/usr/local/lib/chatmaild/chatmail.ini"
+ALPHANUMERIC = string.ascii_lowercase + string.digits
+ALPHANUMERIC_PUNCT = string.ascii_letters + string.digits + string.punctuation
 
 
-def create_newemail_dict(domain):
-    alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
-    user = "".join(random.choices(alphanumeric, k=9))
-    password = "".join(random.choices(alphanumeric, k=12))
-    return dict(email=f"{user}@{domain}", password=f"{password}")
+def create_newemail_dict(config: Config):
+    user = "".join(random.choices(ALPHANUMERIC, k=config.username_min_length))
+    password = "".join(
+        secrets.choice(ALPHANUMERIC_PUNCT)
+        for _ in range(config.password_min_length + 3)
+    )
+    return dict(email=f"{user}@{config.mail_domain}", password=f"{password}")
 
 
 def print_new_account():
-    domain = open(mailname_path).read().strip()
-    creds = create_newemail_dict(domain=domain)
+    config = read_config(CONFIG_PATH)
+    creds = create_newemail_dict(config)
 
     print("Content-Type: application/json")
     print("")

chatmaild/src/chatmaild/tests/plugin.py

@@ -13,8 +13,8 @@ from chatmaild.config import read_config, write_initial_config
 def make_config(tmp_path):
     inipath = tmp_path.joinpath("chatmail.ini")
 
-    def make_conf(mailname):
-        write_initial_config(inipath, mailname=mailname)
+    def make_conf(mail_domain):
+        write_initial_config(inipath, mail_domain=mail_domain)
         return read_config(inipath)
 
     return make_conf
@@ -27,7 +27,7 @@ def example_config(make_config):
 
 @pytest.fixture
 def maildomain(example_config):
-    return example_config.mailname
+    return example_config.mail_domain
 
 
 @pytest.fixture

chatmaild/src/chatmaild/tests/test_config.py

@@ -1,22 +1,21 @@
 from chatmaild.config import read_config
 
 
-def test_read_config_basic(make_config):
-    config = make_config("chat.example.org")
-    assert config.mailname == "chat.example.org"
-    assert not config.privacy_supervisor and not config.privacy_mail
-    assert not config.privacy_pdo and not config.privacy_postal
+def test_read_config_basic(example_config):
+    assert example_config.mail_domain == "chat.example.org"
+    assert not example_config.privacy_supervisor and not example_config.privacy_mail
+    assert not example_config.privacy_pdo and not example_config.privacy_postal
 
-    inipath = config._inipath
+    inipath = example_config._inipath
     inipath.write_text(inipath.read_text().replace("60", "37"))
-    config = read_config(inipath)
-    assert config.max_user_send_per_minute == 37
-    assert config.mailname == "chat.example.org"
+    example_config = read_config(inipath)
+    assert example_config.max_user_send_per_minute == 37
+    assert example_config.mail_domain == "chat.example.org"
 
 
 def test_read_config_testrun(make_config):
     config = make_config("something.testrun.org")
-    assert config.mailname == "something.testrun.org"
+    assert config.mail_domain == "something.testrun.org"
     assert len(config.privacy_postal.split("\n")) > 1
     assert len(config.privacy_supervisor.split("\n")) > 1
     assert len(config.privacy_pdo.split("\n")) > 1
@@ -24,4 +23,10 @@ def test_read_config_testrun(make_config):
     assert config.filtermail_smtp_port == 10080
     assert config.postfix_reinject_port == 10025
     assert config.max_user_send_per_minute == 60
-    assert config.passthrough_recipients
+    assert config.max_mailbox_size == "100M"
+    assert config.delete_mails_after == "40d"
+    assert config.username_min_length == 9
+    assert config.username_max_length == 9
+    assert config.password_min_length == 9
+    assert config.passthrough_recipients == ["privacy@testrun.org"]
+    assert config.passthrough_senders == []

chatmaild/src/chatmaild/tests/test_doveauth.py

@@ -9,29 +9,35 @@ from chatmaild.doveauth import get_user_data, lookup_passdb, handle_dovecot_requ
 from chatmaild.database import DBError
 
 
-def test_basic(db):
-    lookup_passdb(db, "link2xt@c1.testrun.org", "Pieg9aeToe3eghuthe5u")
-    data = get_user_data(db, "link2xt@c1.testrun.org")
+def test_basic(db, example_config):
+    lookup_passdb(db, example_config, "asdf12345@chat.example.org", "q9mr3faue")
+    data = get_user_data(db, "asdf12345@chat.example.org")
     assert data
-    data2 = lookup_passdb(db, "link2xt@c1.testrun.org", "Pieg9aeToe3eghuthe5u")
+    data2 = lookup_passdb(
+        db, example_config, "asdf12345@chat.example.org", "q9mr3jewvadsfaue"
+    )
     assert data == data2
 
 
-def test_dont_overwrite_password_on_wrong_login(db):
+def test_dont_overwrite_password_on_wrong_login(db, example_config):
     """Test that logging in with a different password doesn't create a new user"""
-    res = lookup_passdb(db, "newuser1@something.org", "kajdlkajsldk12l3kj1983")
+    res = lookup_passdb(
+        db, example_config, "newuser12@chat.example.org", "kajdlkajsldk12l3kj1983"
+    )
     assert res["password"]
-    res2 = lookup_passdb(db, "newuser1@something.org", "kajdlqweqwe")
+    res2 = lookup_passdb(db, example_config, "newuser12@chat.example.org", "kajdslqwe")
     # this function always returns a password hash, which is actually compared by dovecot.
     assert res["password"] == res2["password"]
 
 
-def test_nocreate_file(db, monkeypatch, tmpdir):
+def test_nocreate_file(db, monkeypatch, tmpdir, example_config):
     p = tmpdir.join("nocreate")
     p.write("")
     monkeypatch.setattr(chatmaild.doveauth, "NOCREATE_FILE", str(p))
-    lookup_passdb(db, "newuser1@something.org", "zequ0Aimuchoodaechik")
-    assert not get_user_data(db, "newuser1@something.org")
+    lookup_passdb(
+        db, example_config, "newuser12@chat.example.org", "zequ0Aimuchoodaechik"
+    )
+    assert not get_user_data(db, "newuser12@chat.example.org")
 
 
 def test_db_version(db):
@@ -45,21 +51,21 @@ def test_too_high_db_version(db):
         db.ensure_tables()
 
 
-def test_handle_dovecot_request(db):
+def test_handle_dovecot_request(db, example_config):
     msg = (
         "Lshared/passdb/laksjdlaksjdlaksjdlk12j3l1k2j3123/"
-        "some42@c3.testrun.org\tsome42@c3.testrun.org"
+        "some42123@chat.example.org\tsome42123@chat.example.org"
     )
-    res = handle_dovecot_request(msg, db, "c3.testrun.org")
+    res = handle_dovecot_request(msg, db, example_config)
     assert res
     assert res[0] == "O" and res.endswith("\n")
     userdata = json.loads(res[1:].strip())
-    assert userdata["home"] == "/home/vmail/some42@c3.testrun.org"
+    assert userdata["home"] == "/home/vmail/some42123@chat.example.org"
     assert userdata["uid"] == userdata["gid"] == "vmail"
     assert userdata["password"].startswith("{SHA512-CRYPT}")
 
 
-def test_50_concurrent_lookups_different_accounts(db, gencreds):
+def test_50_concurrent_lookups_different_accounts(db, gencreds, example_config):
     num_threads = 50
     req_per_thread = 5
     results = queue.Queue()
@@ -68,7 +74,7 @@ def test_50_concurrent_lookups_different_accounts(db, gencreds):
         for i in range(req_per_thread):
             addr, password = gencreds()
             try:
-                lookup_passdb(db, addr, password)
+                lookup_passdb(db, example_config, addr, password)
             except Exception:
                 results.put(traceback.format_exc())
             else:

chatmaild/src/chatmaild/tests/test_filtermail.py

@@ -127,3 +127,19 @@ def test_excempt_privacy(maildata, gencreds, handler):
         content = msg.as_bytes()
 
     assert "500" in handler.check_DATA(envelope=env2)
+
+
+def test_passthrough_senders(gencreds, handler, maildata):
+    acc1 = gencreds()[0]
+    to_addr = "recipient@something.org"
+    handler.config.passthrough_senders = [acc1]
+
+    msg = maildata("plain.eml", acc1, to_addr)
+
+    class env:
+        mail_from = acc1
+        rcpt_tos = to_addr
+        content = msg.as_bytes()
+
+    # assert that None/no error is returned
+    assert not handler.check_DATA(envelope=env)

chatmaild/src/chatmaild/tests/test_newmail.py

@@ -4,26 +4,24 @@ import chatmaild
 from chatmaild.newemail import create_newemail_dict, print_new_account
 
 
-def test_create_newemail_dict():
-    ac1 = create_newemail_dict(domain="example.org")
+def test_create_newemail_dict(example_config):
+    ac1 = create_newemail_dict(example_config)
     assert "@" in ac1["email"]
     assert len(ac1["password"]) >= 10
 
-    ac2 = create_newemail_dict(domain="example.org")
+    ac2 = create_newemail_dict(example_config)
 
     assert ac1["email"] != ac2["email"]
     assert ac1["password"] != ac2["password"]
 
 
-def test_print_new_account(capsys, monkeypatch, maildomain, tmpdir):
-    p = tmpdir.join("mailname")
-    p.write(maildomain)
-    monkeypatch.setattr(chatmaild.newemail, "mailname_path", str(p))
+def test_print_new_account(capsys, monkeypatch, maildomain, tmpdir, example_config):
+    monkeypatch.setattr(chatmaild.newemail, "CONFIG_PATH", str(example_config._inipath))
     print_new_account()
     out, err = capsys.readouterr()
     lines = out.split("\n")
     assert lines[0] == "Content-Type: application/json"
     assert not lines[1]
     dic = json.loads(lines[2])
-    assert dic["email"].endswith(f"@{maildomain}")
+    assert dic["email"].endswith(f"@{example_config.mail_domain}")
     assert len(dic["password"]) >= 10

cmdeploy/pyproject.toml

@@ -3,8 +3,8 @@ requires = ["setuptools>=68"]
 build-backend = "setuptools.build_meta"
 
 [project]
-name = "deploy-chatmail"
-version = "0.1"
+name = "cmdeploy"
+version = "0.2"
 dependencies = [
   "pyinfra",
   "pillow",
@@ -22,10 +22,11 @@ dependencies = [
 ]
 
 [project.scripts]
-cmdeploy = "deploy_chatmail.cmdeploy:main"
+cmdeploy = "cmdeploy.cmdeploy:main"
 
 [project.entry-points.pytest11]
 "chatmaild.testplugin" = "chatmaild.tests.plugin"
+"cmdeploy.testplugin" = "cmdeploy.tests.plugin"
 
 [tool.pytest.ini_options]
 addopts = "-v -ra --strict-markers"

cmdeploy/src/cmdeploy/__init__.py

@@ -88,10 +88,12 @@ def _install_remote_venv_with_chatmaild(config) -> None:
     for fn in (
         "doveauth",
         "filtermail",
+        "echobot",
     ):
         params = dict(
             execpath=f"{remote_venv_dir}/bin/{fn}",
             config_path=remote_chatmail_inipath,
+            remote_venv_dir=remote_venv_dir,
         )
         source_path = importlib.resources.files("chatmaild").joinpath(f"{fn}.service.f")
         content = source_path.read_text().format(**params).encode()
@@ -195,7 +197,7 @@ def _install_mta_sts_daemon() -> bool:
     server.shell(
         name="install postfix-mta-sts-resolver with pip",
         commands=[
-            "python3 -m venv /usr/local/lib/postfix-mta-sts-resolver",
+            "python3 -m virtualenv /usr/local/lib/postfix-mta-sts-resolver",
             "/usr/local/lib/postfix-mta-sts-resolver/bin/pip install postfix-mta-sts-resolver",
         ],
     )
@@ -243,7 +245,7 @@ def _configure_postfix(config: Config, debug: bool = False) -> bool:
     return need_restart
 
 
-def _configure_dovecot(mail_server: str, debug: bool = False) -> bool:
+def _configure_dovecot(config: Config, debug: bool = False) -> bool:
     """Configures Dovecot IMAP server."""
     need_restart = False
 
@@ -253,7 +255,7 @@ def _configure_dovecot(mail_server: str, debug: bool = False) -> bool:
         user="root",
         group="root",
         mode="644",
-        config={"hostname": mail_server},
+        config=config,
         debug=debug,
     )
     need_restart |= main_config.changed
@@ -266,14 +268,13 @@ def _configure_dovecot(mail_server: str, debug: bool = False) -> bool:
     )
     need_restart |= auth_config.changed
 
-    files.put(
-        src=importlib.resources.files(__package__)
-        .joinpath("dovecot/expunge.cron")
-        .open("rb"),
+    files.template(
+        src=importlib.resources.files(__package__).joinpath("dovecot/expunge.cron.j2"),
         dest="/etc/cron.d/expunge",
         user="root",
         group="root",
         mode="644",
+        config=config,
     )
 
     # as per https://doc.dovecot.org/configuration_manual/os/
@@ -347,8 +348,8 @@ def _configure_nginx(domain: str, debug: bool = False) -> bool:
 
 
 def check_config(config):
-    mailname = config.mailname
-    if mailname != "testrun.org" and not mailname.endswith(".testrun.org"):
+    mail_domain = config.mail_domain
+    if mail_domain != "testrun.org" and not mail_domain.endswith(".testrun.org"):
         blocked_words = "merlinux schmieder testrun.org".split()
         for value in config.__dict__.values():
             if any(x in value for x in blocked_words):
@@ -379,6 +380,20 @@ def deploy_chatmail(mail_domain: str, mail_server: str, dkim_selector: str) -> N
         system=True,
     )
 
+    # Run local DNS resolver `unbound`.
+    # `resolvconf` takes care of setting up /etc/resolv.conf
+    # to use 127.0.0.1 as the resolver.
+    apt.packages(
+        name="Install unbound",
+        packages="unbound",
+    )
+    systemd.service(
+        name="Start and enable unbound",
+        service="unbound.service",
+        running=True,
+        enabled=True,
+    )
+
     # Deploy acmetool to have TLS certificates.
     deploy_acmetool(nginx_hook=True, domains=[mail_server, f"mta-sts.{mail_server}"])
 
@@ -423,7 +438,7 @@ def deploy_chatmail(mail_domain: str, mail_server: str, dkim_selector: str) -> N
 
     _install_remote_venv_with_chatmaild(config)
     debug = False
-    dovecot_need_restart = _configure_dovecot(mail_server, debug=debug)
+    dovecot_need_restart = _configure_dovecot(config, debug=debug)
     postfix_need_restart = _configure_postfix(config, debug=debug)
     opendkim_need_restart = _configure_opendkim(mail_domain, dkim_selector)
     mta_sts_need_restart = _install_mta_sts_daemon()

cmdeploy/src/cmdeploy/cmdeploy.py

@@ -52,17 +52,17 @@ def run_cmd(args, out):
     """Deploy chatmail services on the remote server."""
 
     env = os.environ.copy()
-    env["CHATMAIL_DOMAIN"] = args.config.mailname
-    deploypy = "deploy-chatmail/src/deploy_chatmail/deploy.py"
+    env["CHATMAIL_DOMAIN"] = args.config.mail_domain
+    deploy_path = "cmdeploy/src/cmdeploy/deploy.py"
     pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
-    cmd = f"{pyinf} --ssh-user root {args.config.mailname} {deploypy}"
+    cmd = f"{pyinf} --ssh-user root {args.config.mail_domain} {deploy_path}"
     out.check_call(cmd, env=env)
 
 
 def dns_cmd(args, out):
     """Generate dns zone file."""
     template = importlib.resources.files(__package__).joinpath("chatmail.zone.f")
-    ssh = f"ssh root@{args.config.mailname}"
+    ssh = f"ssh root@{args.config.mail_domain}"
 
     def read_dkim_entries(entry):
         lines = []
@@ -77,16 +77,16 @@ def dns_cmd(args, out):
     dkim_entry = read_dkim_entries(out.shell_output(f"{ssh} -- opendkim-genzone -F"))
 
     out(
-        f"[writing {args.config.mailname} zone data (using space as separator) to stdout output]",
+        f"[writing {args.config.mail_domain} zone data (using space as separator) to stdout output]",
         green=True,
     )
     print(
         template.read_text()
         .format(
             acme_account_url=acme_account_url,
-            email=f"root@{args.config.mailname}",
+            email=f"root@{args.config.mail_domain}",
             sts_id=datetime.datetime.now().strftime("%Y%m%d%H%M"),
-            chatmail_domain=args.config.mailname,
+            chatmail_domain=args.config.mail_domain,
             dkim_entry=dkim_entry,
         )
         .strip()
@@ -96,9 +96,9 @@ def dns_cmd(args, out):
 def status_cmd(args, out):
     """Display status for online chatmail instance."""
 
-    ssh = f"ssh root@{args.config.mailname}"
+    ssh = f"ssh root@{args.config.mail_domain}"
 
-    out.green(f"chatmail domain: {args.config.mailname}")
+    out.green(f"chatmail domain: {args.config.mail_domain}")
     if args.config.privacy_mail:
         out.green("privacy settings: present")
     else:
@@ -110,6 +110,15 @@ def status_cmd(args, out):
             print(line)
 
 
+def test_cmd_options(parser):
+    parser.add_argument(
+        "--slow",
+        dest="slow",
+        action="store_true",
+        help="also run slow tests",
+    )
+
+
 def test_cmd(args, out):
     """Run local and online tests for chatmail deployment.
 
@@ -121,9 +130,18 @@ def test_cmd(args, out):
         out.check_call(f"{sys.executable} -m pip install deltachat")
 
     pytest_path = shutil.which("pytest")
-    ret = out.run_ret(
-        [pytest_path, "tests/", "-n4", "-rs", "-x", "-vrx", "--durations=5"]
-    )
+    pytest_args = [
+        pytest_path,
+        "cmdeploy/src/",
+        "-n4",
+        "-rs",
+        "-x",
+        "-vrx",
+        "--durations=5",
+    ]
+    if args.slow:
+        pytest_args.append("--slow")
+    ret = out.run_ret(pytest_args)
     return ret
 
 
@@ -145,13 +163,9 @@ def fmt_cmd_options(parser):
 
 
 def fmt_cmd(args, out):
-    """Run formattting fixes (fuff and black) on all chatmail source code."""
-
-    chatmaild = importlib.resources.files("chatmaild")
-    deploy_chatmail = importlib.resources.files("deploy_chatmail")
-    tests = deploy_chatmail.joinpath("../../../tests")
-    sources = list(str(x) for x in [chatmaild, deploy_chatmail, tests])
+    """Run formattting fixes (ruff and black) on all chatmail source code."""
 
+    sources = [str(importlib.resources.files(x)) for x in ("chatmaild", "cmdeploy")]
     black_args = [shutil.which("black")]
     ruff_args = [shutil.which("ruff")]
 
@@ -174,9 +188,10 @@ def fmt_cmd(args, out):
 
 def bench_cmd(args, out):
     """Run benchmarks against an online chatmail instance."""
-    pytest_path = shutil.which("pytest")
-    benchmark = "tests/online/benchmark.py"
-    subprocess.check_call([pytest_path, benchmark, "-vrx"])
+    args = ["pytest", "--pyargs", "cmdeploy.tests.online.benchmark", "-vrx"]
+    cmdstring = " ".join(args)
+    out.green(f"[$ {cmdstring}]")
+    subprocess.check_call(args)
 
 
 def webdev_cmd(args, out):

cmdeploy/src/cmdeploy/deploy.py

@@ -1,6 +1,6 @@
 import os
 import pyinfra
-from deploy_chatmail import deploy_chatmail
+from cmdeploy import deploy_chatmail
 
 
 def main():

cmdeploy/src/cmdeploy/dovecot/dovecot.conf.j2

@@ -86,7 +86,7 @@ plugin {
 plugin {
   # for now we define static quota-rules for all users 
   quota = maildir:User quota
-  quota_rule = *:storage=100M
+  quota_rule = *:storage={{ config.max_mailbox_size }}
   quota_max_mail_size=30M
   quota_grace = 0
   # quota_over_flag_value = TRUE
@@ -137,8 +137,8 @@ service imap-login {
 }
 
 ssl = required
-ssl_cert = </var/lib/acme/live/{{ config.hostname }}/fullchain
-ssl_key = </var/lib/acme/live/{{ config.hostname }}/privkey
+ssl_cert = </var/lib/acme/live/{{ config.mail_domain }}/fullchain
+ssl_key = </var/lib/acme/live/{{ config.mail_domain }}/privkey
 ssl_dh = </usr/share/dovecot/dh.pem
 ssl_min_protocol = TLSv1.2
 ssl_prefer_server_ciphers = yes

cmdeploy/src/cmdeploy/dovecot/expunge.cron.j2

@@ -0,0 +1,4 @@
+2 0 * * * dovecot doveadm expunge -A SEEN BEFORE {{ config.delete_mails_after }} INBOX
+2 0 * * * dovecot doveadm expunge -A SEEN BEFORE {{ config.delete_mails_after }} Deltachat
+2 0 * * * dovecot doveadm expunge -A SEEN BEFORE {{ config.delete_mails_after }} Trash
+2 30 * * * dovecot doveadm purge -A

cmdeploy/src/cmdeploy/postfix/main.cf.j2

@@ -1,4 +1,4 @@
-myorigin = {{ config.mailname }}
+myorigin = {{ config.mail_domain }}
 
 smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
 biff = no
@@ -16,8 +16,8 @@ readme_directory = no
 compatibility_level = 2
 
 # TLS parameters
-smtpd_tls_cert_file=/var/lib/acme/live/{{ config.mailname }}/fullchain
-smtpd_tls_key_file=/var/lib/acme/live/{{ config.mailname }}/privkey
+smtpd_tls_cert_file=/var/lib/acme/live/{{ config.mail_domain }}/fullchain
+smtpd_tls_key_file=/var/lib/acme/live/{{ config.mail_domain }}/privkey
 smtpd_tls_security_level=may
 
 smtp_tls_CApath=/etc/ssl/certs
@@ -26,7 +26,7 @@ smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
 smtp_tls_policy_maps = socketmap:inet:127.0.0.1:8461:postfix
 
 smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
-myhostname = {{ config.mailname }}
+myhostname = {{ config.mail_domain }}
 alias_maps = hash:/etc/aliases
 alias_database = hash:/etc/aliases
 
@@ -45,7 +45,7 @@ inet_interfaces = all
 inet_protocols = all
 
 virtual_transport = lmtp:unix:private/dovecot-lmtp
-virtual_mailbox_domains = {{ config.mailname }}
+virtual_mailbox_domains = {{ config.mail_domain }}
 
 smtpd_milters = unix:opendkim/opendkim.sock
 non_smtpd_milters = $smtpd_milters

cmdeploy/src/cmdeploy/tests/online/benchmark.py

@@ -30,31 +30,31 @@ def test_login_smtp(benchmark, smtp, gencreds):
 
 class TestDC:
     def test_autoconfigure(self, benchmark, cmfactory):
-        def autoconfig_and_idle_ready():
+        def dc_autoconfig_and_idle_ready():
             cmfactory.get_online_accounts(1)
 
-        benchmark(autoconfig_and_idle_ready, 5)
+        benchmark(dc_autoconfig_and_idle_ready, 5)
 
     def test_ping_pong(self, benchmark, cmfactory):
         ac1, ac2 = cmfactory.get_online_accounts(2)
         chat = cmfactory.get_accepted_chat(ac1, ac2)
 
-        def ping_pong():
+        def dc_ping_pong():
             chat.send_text("ping")
             msg = ac2.wait_next_incoming_message()
             msg.chat.send_text("pong")
             ac1.wait_next_incoming_message()
 
-        benchmark(ping_pong, 5)
+        benchmark(dc_ping_pong, 5)
 
     def test_send_10_receive_10(self, benchmark, cmfactory, lp):
         ac1, ac2 = cmfactory.get_online_accounts(2)
         chat = cmfactory.get_accepted_chat(ac1, ac2)
 
-        def send_10_receive_10():
+        def dc_send_10_receive_10():
             for i in range(10):
                 chat.send_text(f"hello {i}")
             for i in range(10):
                 ac2.wait_next_incoming_message()
 
-        benchmark(send_10_receive_10, 5)
+        benchmark(dc_send_10_receive_10, 5)

cmdeploy/src/cmdeploy/tests/online/test_0_qr.py

@@ -0,0 +1,16 @@
+import requests
+
+from cmdeploy.genqr import gen_qr_png_data
+
+
+def test_gen_qr_png_data(maildomain):
+    data = gen_qr_png_data(maildomain)
+    assert data
+
+
+def test_fastcgi_working(maildomain, chatmail_config):
+    url = f"https://{maildomain}/cgi-bin/newemail.py"
+    print(url)
+    res = requests.post(url)
+    assert maildomain in res.json().get("email")
+    assert len(res.json().get("password")) > chatmail_config.password_min_length

cmdeploy/src/cmdeploy/tests/online/test_1_basic.py

@@ -43,18 +43,18 @@ def test_reject_forged_from(cmsetup, maildata, gencreds, lp, forgeaddr):
 
 
 @pytest.mark.slow
-def test_exceed_rate_limit(cmsetup, gencreds, maildata):
+def test_exceed_rate_limit(cmsetup, gencreds, maildata, chatmail_config):
     """Test that the per-account send-mail limit is exceeded."""
     user1, user2 = cmsetup.gen_users(2)
     mail = maildata(
         "encrypted.eml", from_addr=user1.addr, to_addr=user2.addr
     ).as_string()
-    for i in range(100):
+    for i in range(chatmail_config.max_user_send_per_minute + 5):
         print("Sending mail", str(i))
         try:
             user1.smtp.sendmail(user1.addr, [user2.addr], mail)
         except smtplib.SMTPException as e:
-            if i < 60:
+            if i < chatmail_config.max_user_send_per_minute:
                 pytest.fail(f"rate limit was exceeded too early with msg {i}")
             outcome = e.recipients[user2.addr]
             assert outcome[0] == 450

cmdeploy/src/cmdeploy/tests/online/test_2_deltachat.py

@@ -1,4 +1,5 @@
 import time
+import re
 import random
 import pytest
 
@@ -20,14 +21,24 @@ class TestEndToEndDeltaChat:
         assert msg2.text == "message0"
 
     @pytest.mark.slow
-    def test_exceed_quota(self, cmfactory, lp, tmpdir, remote):
+    def test_exceed_quota(self, cmfactory, lp, tmpdir, remote, chatmail_config):
         """This is a very slow test as it needs to upload >100MB of mail data
         before quota is exceeded, and thus depends on the speed of the upload.
         """
         ac1, ac2 = cmfactory.get_online_accounts(2)
         chat = cmfactory.get_accepted_chat(ac1, ac2)
 
-        quota = 1024 * 1024 * 100
+        def parse_size_limit(limit: str) -> int:
+            """Parse a size limit and return the number of bytes as integer.
+
+            Example input: 100M, 2.4T, 500 K
+            """
+            units = {"B": 1, "K": 2**10, "M": 2**20, "G": 2**30, "T": 2**40}
+            size = re.sub(r"([KMGT])", r" \1", limit.upper())
+            number, unit = [string.strip() for string in size.split()]
+            return int(float(number) * units[unit])
+
+        quota = parse_size_limit(chatmail_config.max_mailbox_size)
         attachsize = 1 * 1024 * 1024
         num_to_send = quota // attachsize + 2
         lp.sec(f"ac1: send {num_to_send} large files to ac2")
@@ -91,9 +102,9 @@ class TestEndToEndDeltaChat:
 
         lp.sec("setup encrypted comms between ac1 and ac2 on different instances")
         qr = ac1.get_setup_contact_qr()
-        ac2.qr_setup_contact(qr)
-        msg = ac2.wait_next_incoming_message()
-        assert "verified" in msg.text
+        ch = ac2.qr_setup_contact(qr)
+        assert ch.id >= 10
+        ac1._evtracker.wait_securejoin_inviter_progress(1000)
 
         lp.sec("ac1 sends a message and ac2 marks it as seen")
         chat = ac1.create_chat(ac2)

cmdeploy/src/cmdeploy/tests/plugin.py

@@ -38,7 +38,7 @@ def pytest_runtest_setup(item):
 
 @pytest.fixture
 def chatmail_config(pytestconfig):
-    current = basedir = Path()
+    current = basedir = Path().resolve()
     while 1:
         path = current.joinpath("chatmail.ini").resolve()
         if path.exists():
@@ -52,7 +52,7 @@ def chatmail_config(pytestconfig):
 
 @pytest.fixture
 def maildomain(chatmail_config):
-    return chatmail_config.mailname
+    return chatmail_config.mail_domain
 
 
 @pytest.fixture
@@ -228,18 +228,25 @@ def imap_or_smtp(request):
 
 
 @pytest.fixture
-def gencreds(maildomain):
+def gencreds(chatmail_config):
     count = itertools.count()
     next(count)
 
     def gen(domain=None):
-        domain = domain if domain else maildomain
+        domain = domain if domain else chatmail_config.mail_domain
         while 1:
             num = next(count)
             alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
-            user = "".join(random.choices(alphanumeric, k=10))
-            user = f"ac{num}_{user}"[:9]
-            password = "".join(random.choices(alphanumeric, k=12))
+            user = "".join(
+                random.choices(alphanumeric, k=chatmail_config.username_max_length)
+            )
+            if domain == "nine.testrun.org":
+                user = f"ac{num}_{user}"[:9]
+            else:
+                user = f"ac{num}_{user}"[: chatmail_config.username_max_length]
+            password = "".join(
+                random.choices(alphanumeric, k=chatmail_config.password_min_length)
+            )
             yield f"{user}@{domain}", f"{password}"
 
     return lambda domain=None: next(gen(domain))

cmdeploy/src/cmdeploy/tests/test_cmdeploy.py

@@ -1,7 +1,7 @@
 import os
 
 import pytest
-from deploy_chatmail.cmdeploy import get_parser, main
+from cmdeploy.cmdeploy import get_parser, main
 from chatmaild.config import read_config
 
 
@@ -25,7 +25,7 @@ class TestCmdline:
         main(["init", "chat.example.org"])
         inipath = tmp_path.joinpath("chatmail.ini")
         config = read_config(inipath)
-        assert config.mailname == "chat.example.org"
+        assert config.mail_domain == "chat.example.org"
 
     def test_init_not_overwrite(self):
         main(["init", "chat.example.org"])

cmdeploy/src/cmdeploy/tests/test_helpers.py

@@ -1,10 +1,10 @@
 import importlib.resources
 
-from deploy_chatmail.www import build_webpages
+from cmdeploy.www import build_webpages
 
 
 def test_build_webpages(tmp_path, make_config):
-    pkgroot = importlib.resources.files("deploy_chatmail")
+    pkgroot = importlib.resources.files("cmdeploy")
     src_dir = pkgroot.joinpath("../../../www/src").resolve()
     assert src_dir.exists(), src_dir
     config = make_config("chat.example.org")

cmdeploy/src/cmdeploy/www.py

@@ -36,8 +36,55 @@ def build_webpages(src_dir, build_dir, config):
         print(traceback.format_exc())
 
 
+def timespan_to_english(timespan):
+    val = int(timespan[:-1])
+    c = timespan[-1].lower()
+    match c:
+        case "y":
+            return f"{val} years"
+        case "m":
+            return f"{val} months"
+        case "w":
+            return f"{val} weeks"
+        case "d":
+            return f"{val} days"
+        case "h":
+            return f"{val} hours"
+        case "c":
+            return f"{val} seconds"
+        case _:
+            raise ValueError(
+                c
+                + " is not a valid time unit. Try [y]ears, [w]eeks, [d]ays, or [h]ours"
+            )
+
+
+def int_to_english(number):
+    if number >= 0 and number <= 12:
+        a = [
+            "zero",
+            "one",
+            "two",
+            "three",
+            "four",
+            "five",
+            "six",
+            "seven",
+            "eight",
+            "nine",
+            "ten",
+            "eleven",
+            "twelve",
+        ]
+        return a[number]
+    elif number <= 50:
+        return str(number)
+    if number > 50:
+        return "more"
+
+
 def _build_webpages(src_dir, build_dir, config):
-    mail_domain = config.mailname
+    mail_domain = config.mail_domain
     assert src_dir.exists(), src_dir
     if not build_dir.exists():
         build_dir.mkdir()
@@ -48,6 +95,18 @@ def _build_webpages(src_dir, build_dir, config):
     for path in src_dir.iterdir():
         if path.suffix == ".md":
             render_vars, content = prepare_template(path)
+            render_vars["username_min_length"] = int_to_english(
+                config.username_min_length
+            )
+            render_vars["username_max_length"] = int_to_english(
+                config.username_max_length
+            )
+            render_vars["password_min_length"] = int_to_english(
+                config.password_min_length
+            )
+            render_vars["delete_mails_after"] = timespan_to_english(
+                config.delete_mails_after
+            )
             target = build_dir.joinpath(path.stem + ".html")
 
             # recursive jinja2 rendering
@@ -71,7 +130,7 @@ def main():
     inipath = reporoot.joinpath("chatmail.ini")
     config = read_config(inipath)
     config.webdev = True
-    assert config.mailname
+    assert config.mail_domain
     www_path = reporoot.joinpath("www")
     src_path = www_path.joinpath("src")
     stats = None

deploy-chatmail/src/deploy_chatmail/dovecot/expunge.cron

@@ -1,4 +0,0 @@
-2 0 * * * dovecot doveadm expunge -A SEEN BEFORE 40d INBOX
-2 0 * * * dovecot doveadm expunge -A SEEN BEFORE 40d Deltachat
-2 0 * * * dovecot doveadm expunge -A SEEN BEFORE 40d Trash
-2 30 * * * dovecot doveadm purge -A

scripts/cmdeploy

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+#
+# Wrapper for cmdelpoy to run it in activated virtualenv.
+set -e
+. venv/bin/activate
+cmdeploy "$@"

scripts/initenv.sh

@@ -2,8 +2,5 @@
 set -e
 python3 -m venv venv
 
-venv/bin/pip install -e deploy-chatmail 
 venv/bin/pip install -e chatmaild 
-
-source venv/bin/activate
-echo activated 'venv' python virtualenv environment containing "cmdeploy" tool
+venv/bin/pip install -e cmdeploy

tests/online/test_0_qr.py

@@ -1,6 +0,0 @@
-from deploy_chatmail.genqr import gen_qr_png_data
-
-
-def test_gen_qr_png_data(maildomain):
-    data = gen_qr_png_data(maildomain)
-    assert data

www/src/index.md

@@ -1,5 +1,5 @@
 
-<img width="800px" src="collage-top.png"/>
+<img class="banner" src="collage-top.png"/>
 
 ## Dear [Delta Chat](https://get.delta.chat) users and newcomers, 
 
@@ -14,6 +14,6 @@ Welcome to instant, interoperable and [privacy-preserving](privacy.html) messagi
 
 💬 **Start** chatting with any Delta Chat contacts using [QR invite codes](https://delta.chat/en/help#howtoe2ee)
 
-
-## ⚡ Note: this is an experimental service ⚡
-
+{% if config.mail_domain != "nine.testrun.org" %}
+<div class="experimental">Note: this is only a temporary development chatmail service</div>
+{% endif %}

www/src/info.md

@@ -1,5 +1,5 @@
 
-<img width="800px" src="collage-info.png"/>
+<img class="banner" src="collage-info.png"/>
 
 ## More information 
 
@@ -9,10 +9,21 @@ In the Delta Chat account setup
 you may tap `LOG INTO YOUR E-MAIL ACCOUNT` 
 and fill the two fields like this: 
 
-- `Address`: invent a word with *exactly* nine characters 
-   and append `@{{config.mail_domain}}` to it. 
+- `Address`: invent a word with
+{% if username_min_length == username_max_length %}
+  *exactly* {{ username_min_length }}
+{% else %}
+  {{ username_min_length}}
+  {% if username_max_length == "more" %}
+    or more
+  {% else %}
+    to {{ username_max_length }}
+  {% endif %}
+{% endif %}
+  characters
+  and append `@{{config.mail_domain}}` to it.
 
-- `Password`: invent at least 9 characters. 
+- `Password`: invent at least {{ password_min_length }} characters.
 
 If the e-mail address is not yet taken, you'll get that account. 
 The first login sets your password. 
@@ -24,11 +35,11 @@ The first login sets your password.
   {{config.mail_domain}} but setting up contact via [QR invite codes](https://delta.chat/en/help#howtoe2ee) 
   allows your messages to pass freely to any outside recipients.
 
-- You may send up to 60 messages per minute
+- You may send up to {{ config.max_user_send_per_minute }} messages per minute.
 
-- Messages are unconditionally removed 40 days after arriving on the server
+- Seen messages are removed {{ delete_mails_after }} after arriving on the server.
 
-- You can store up to [100MB messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server)
+- You can store up to [{{ config.max_mailbox_size }} messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server).
 
 
 ### Who are the operators? Which software is running? 

www/src/logo.svg

@@ -0,0 +1,44 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<svg
+   width="145"
+   height="145"
+   version="1.1"
+   id="svg4"
+   sodipodi:docname="At_sign.svg"
+   inkscape:version="1.2.2 (b0a84865, 2022-12-01)"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:svg="http://www.w3.org/2000/svg">
+  <defs
+     id="defs8" />
+  <sodipodi:namedview
+     id="namedview6"
+     pagecolor="#ffffff"
+     bordercolor="#000000"
+     borderopacity="0.25"
+     inkscape:showpageshadow="2"
+     inkscape:pageopacity="0.0"
+     inkscape:pagecheckerboard="0"
+     inkscape:deskcolor="#d1d1d1"
+     showgrid="false"
+     inkscape:zoom="3.0241379"
+     inkscape:cx="67.622577"
+     inkscape:cy="72.913341"
+     inkscape:window-width="1390"
+     inkscape:window-height="1027"
+     inkscape:window-x="55"
+     inkscape:window-y="25"
+     inkscape:window-maximized="0"
+     inkscape:current-layer="svg4" />
+  <g
+     aria-label="@"
+     id="text2"
+     style="font-size:144px;font-family:Arial">
+    <path
+       d="m 79.927878,94.422406 c -2.704286,3.120332 -5.741407,5.637394 -9.111364,7.551194 -3.328352,1.87221 -6.677506,2.80831 -10.047463,2.80831 -3.702792,0 -7.301573,-1.08172 -10.796342,-3.24515 -3.49477,-2.163426 -6.344671,-5.491779 -8.549704,-9.985058 -2.163429,-4.493275 -3.245144,-9.423397 -3.245144,-14.790365 0,-6.615099 1.684978,-13.230199 5.054935,-19.845299 3.411561,-6.656705 7.634407,-11.649233 12.66854,-14.977585 5.034133,-3.328352 9.92265,-4.992528 14.665552,-4.992528 3.619583,0 7.072748,0.956901 10.359496,2.870704 3.286748,1.872198 6.115847,4.742902 8.487297,8.612111 l 2.121825,-9.673023 h 11.170784 l -8.986557,41.87483 c -1.248129,5.824616 -1.872194,9.048957 -1.872194,9.673023 0,1.123319 0.416044,2.101022 1.248132,2.93311 0.873692,0.790484 1.913802,1.185726 3.120332,1.185726 2.20503,0 5.096537,-1.268934 8.674517,-3.806803 4.7429,-3.328352 8.4873,-7.780023 11.23319,-13.355013 2.78749,-5.616594 4.18124,-11.399606 4.18124,-17.349035 0,-6.947935 -1.78899,-13.438222 -5.36697,-19.47086 -3.53637,-6.032638 -8.84094,-10.858749 -15.913687,-14.478332 -7.03114,-3.619583 -14.811161,-5.429374 -23.340064,-5.429374 -9.73543,0 -18.638772,2.288242 -26.710026,6.864726 -8.029649,4.534879 -14.27031,11.06677 -18.721981,19.595673 -4.410066,8.487298 -6.615099,17.598662 -6.615099,27.334092 0,10.193078 2.205033,18.971607 6.615099,26.33559 2.290454,3.78888 -7.136335,18.96983 -3.810585,21.73443 3.138096,2.60861 18.971963,-7.14297 23.031819,-5.44631 8.404089,3.53637 17.702673,5.30456 27.895752,5.30456 10.90035,0 20.032515,-1.83059 27.396492,-5.49178 7.36399,-3.66119 12.87657,-8.11286 16.53776,-13.35501 l 9.29559,4 c -2.12183,4.36846 -3.76221,4.82013 -8.92116,9.35501 -5.15895,4.53488 -11.2956,8.11286 -18.40995,10.73393 -7.114346,2.66268 -15.684851,3.99402 -25.711512,3.99402 -9.236177,0 -17.76508,-1.18572 -25.586707,-3.55717 -7.780023,-2.37145 -29.296198,9.26152 -34.78798,4.47701 -5.49178,-4.7429 5.248856,-25.42482 2.461361,-31.62388 -3.49477,-7.863231 -5.242155,-16.350531 -5.242155,-25.461894 0,-10.151474 2.08022,-19.824498 6.240661,-29.019071 5.075736,-11.274793 12.273297,-19.907706 21.592683,-25.898739 9.360991,-5.991034 20.69819,-8.986551 34.011599,-8.986551 10.317891,0 19.574873,2.121824 27.77093,6.365473 8.23767,4.202045 14.72796,10.484309 19.47086,18.846794 4.03563,7.197561 6.05344,15.019189 6.05344,23.464883 0,12.065277 -4.24365,22.77841 -12.73094,32.1394 -7.572,8.404095 -15.85128,12.606135 -24.837827,12.606135 -2.870704,0 -5.200551,-0.43684 -6.98954,-1.31053 -1.747385,-0.8737 -3.037121,-2.12183 -3.869209,-3.744402 -0.540857,-1.040114 -0.936099,-2.829105 -1.185726,-5.366972 z M 49.723082,77.510217 c 0,5.699803 1.352143,10.130671 4.05643,13.292606 2.704286,3.161935 5.803814,4.742902 9.298583,4.742902 2.329847,0 4.784506,-0.686473 7.363979,-2.059418 2.579473,-1.41455 5.034133,-3.49477 7.363979,-6.240661 2.371451,-2.74589 4.306056,-6.219857 5.803815,-10.421902 1.497759,-4.243649 2.246638,-8.487298 2.246638,-12.730947 0,-5.658198 -1.41455,-10.047462 -4.243649,-13.167793 -2.787495,-3.12033 -6.199056,-4.680495 -10.234683,-4.680495 -2.662682,0 -5.179749,0.686473 -7.5512,2.059418 -2.329846,1.331341 -4.597286,3.494769 -6.802319,6.490286 -2.205033,2.995517 -3.97322,6.635903 -5.304561,10.921156 -1.331341,4.285253 -1.997012,8.216869 -1.997012,11.794848 z"
+       id="path347"
+       style="stroke-width:0.887561"
+       sodipodi:nodetypes="ccsscscsscccccscsccsccsccscscssccscscccsccsccscscccssscccscscsss" />
+  </g>
+</svg>

www/src/main.css

@@ -0,0 +1,74 @@
+
+body {
+  font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', 'Segoe UI Emoji', 'Apple Color Emoji', 'Noto Color Emoji', sans-serif;
+  line-height: 1.4;
+  font-size: 1.2em;
+  max-width: 800px;
+  margin: 20px auto;
+  padding: 0 10px;
+  color: #363636;
+  background: #fff;
+}
+
+h1 {
+  font-size: 2.2em;
+  margin-top: 0;
+}
+
+h1, h2, h3, h4, h5, h6 {
+  color: #000;
+  margin-bottom: 12px;
+  margin-top: 24px;
+  font-weight: 600;
+}
+
+a {
+  text-decoration: none;
+  color: #0076d1;
+}
+
+a:hover {
+  text-decoration: underline;
+}
+
+img, video {
+  max-width: 100%;
+  height: auto;
+}
+
+code {
+  background: #efefef;
+  padding: 2.5px 5px;
+  border-radius: 6px;
+}
+
+#menu {
+  display: flex;
+  flex-wrap: wrap;
+  padding: 0;
+}
+
+#menu li {
+  display: inline-block;
+  padding-right: 0.5em;
+}
+
+#domain {
+  margin-left: auto;
+}
+
+#domain a {
+  color: #888;
+}
+
+.banner {
+  width: 100%;
+}
+
+.experimental {
+  margin: 3em 0;
+  padding: 1em;
+  border: 4px dashed red;
+  color: red;
+  font-weight: bold;
+}

www/src/page-layout.html

@@ -5,16 +5,23 @@
     {% if config.webdev %}
     <meta http-equiv="refresh" content="3">
     {% endif %}
+    <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">
     <title>{{ config.mail_domain }} {{ pagename }}</title>
-    <link rel="stylesheet" href="./water.css">
+    <link rel="stylesheet" href="./main.css">
+    <link rel="icon" href="/logo.svg">
+    <link rel=”mask-icon” href=”/logo.svg” color=”#000000">
 </head>
 <body>
+
+<ul id="menu">
+    <li><a href="index.html">home</a></li>
+    <li><a href="info.html">info</a></li>
+    <li><a href="privacy.html">privacy</a></li>
+    <li><a href="https://github.com/deltachat/chatmail">public code ↗</a></li>
+    <li id="domain"><a href="index.html">{{ config.mail_domain }}</a></li>
+</ul>
+
 {{ markdown_html }}
-<footer>
-    <a href="index.html">home</a> |
-    <a href="info.html">more info</a> |
-    <a href="privacy.html">privacy</a> |
-    <a href="https://github.com/deltachat/chatmail">-> public development </a>
-</footer>
+
 </body>
 </html>

www/src/privacy.md

@@ -1,4 +1,4 @@
-<img width="800px" src="collage-privacy.png"/>
+<img class="banner" src="collage-privacy.png"/>
 
 # Privacy Policy for {{ config.mail_domain }} 
 
@@ -62,7 +62,7 @@ we process the following data and details:
 Creating an account happens in one of two ways on our mail servers: 
 
 - with a QR invitation token 
-  which is scanned using the DeltaChat app
+  which is scanned using the Delta Chat app
   and then the account is created.
 
 - by letting Delta Chat otherwise create an account 
@@ -218,97 +218,16 @@ on these or possible objections.
 A deletion can be made
 directly in the Delta Chat email messenger.
 
-a) request information about your personal data processed by us
-in accordance with Art. 15 GDPR.
-In particular,
-you can request information about the processing purposes,
-the category of personal data,
-the categories of recipients to whom your data have been or will be disclosed,
-the planned storage period,
-the existence of a right to rectification, erasure, restriction of processing or objection,
-the existence of a right of complaint,
-the origin of your data if it has not been collected by us,
-as well as the existence of automated decision-making including profiling
-and, if applicable,
-meaningful information about its details;  
+If you have any questions or complaints, 
+please feel free to contact us by email:  
+{{ config.privacy_mail }}
 
-b) in accordance with Art. 16 of the GDPR,
-immediately request the correction
-of inaccurate or incomplete personal data stored by us;  
-
-c) pursuant to Article 17 of the GDPR,
-to request the erasure of your personal data stored by us,
-unless the processing is necessary
-for the exercise of the right to freedom of expression and information,
-for compliance with a legal obligation,
-for reasons of public interest,
-or for the establishment, exercise or defence of legal claims;  
-
-d) pursuant to Art. 18 GDPR,
-to request the restriction of the processing of your personal data,
-insofar as the accuracy of the data is disputed by you,
-the processing is unlawful,
-but you object to its erasure
-and we no longer require the data,
-but you need it for the assertion, exercise or defence of legal claims
-or you have objected to the processing pursuant to Art. 21 GDPR;  
-
-e) pursuant to Art. 20 GDPR,
-to receive your personal data that you have provided to us
-in a structured, common and machine-readable format
-or to request that it be transferred to another controller;  
-
-f) in accordance with Art. 7 (3) of the GDPR,
-to revoke your consent given to us at any time.
-This has the consequence that we may no longer continue the data processing
-based on this consent in the future; and  
-
-g) complain to a supervisory authority
-in accordance with Article 77 of the GDPR.
-As a rule,
-you can contact the supervisory authority of your usual place of residence
+As a rule, you can contact the supervisory authority of your usual place of residence
 or workplace
 or our registered office for this purpose.
 The supervisory authority responsible for our place of business
 is the `{{ config.privacy_supervisor }}`.
 
-If you have any questions or complaints, please feel free to contact us by email:  
-{{ config.privacy_mail }}
-
-
-### 5.1 Right to object
-
-If your personal data is processed on the basis of our legitimate interests
-in accordance with Art. 6 (1) lit. f GDPR,
-you have the right to object to the processing of your personal data
-in accordance with Art. 21 GDPR,
-provided that there are grounds for this based on your particular situation
-or the objection is directed against direct advertising.
-In the latter case,
-you have a general right of objection,
-which will be implemented by us
-without specifying a particular situation.
-
-If you wish to exercise your right of objection,
-simply send an e-mail to: {{ config.privacy_mail }}
-
-### 5.2 Right to withdraw
-
-If your personal data is processed on the basis of your consent
-in accordance with Art. 6 (1) lit. a GDPR
-(e.g. via the mailing list),
-you can withdraw your consent at any time
-and without any disadvantages.
-As a result,
-we may no longer continue the data processing
-that was based on this consent for the future.
-However,
-the withdrawal of your consent
-does not affect the lawfulness of the processing
-carried out on the basis of the consent until the withdrawal.
-
-If you wish to make use of your right of withdrawal,
-simply send an e-mail to: {{ config.privacy_mail }}
 
 ## 6. Validity of this privacy policy