refine warnings for experimental service,

only show for non-nine domains.

README.md

@@ -18,7 +18,14 @@ after which the initially specified password is required for using them.
 We subsequently use `CHATMAIL_DOMAIN` as a placeholder for your fully qualified 
 DNS domain name (FQDN), for example `chat.example.org`.
 
-1. Install the `cmdeploy` command in a virtualenv
+1. Setup DNS `A` and `AAAA` records for your `CHATMAIL_DOMAIN`. 
+   Verify that DNS is set and SSH root login works:
+
+   ```
+        ssh root@CHATMAIL_DOMAIN 
+   ```
+
+2. Install the `cmdeploy` command in a virtualenv
 
    ```
     git clone https://github.com/deltachat/chatmail
@@ -26,20 +33,12 @@ DNS domain name (FQDN), for example `chat.example.org`.
     scripts/initenv.sh
    ```
   
-2. Create chatmail configuration file `chatmail.ini`:
+3. Create chatmail configuration file `chatmail.ini`:
 
    ```
     scripts/cmdeploy init CHATMAIL_DOMAIN
    ```
 
-3. Setup first DNS records for your `CHATMAIL_DOMAIN`,
-   according to the hints provided by `cmdeploy init`.
-   Verify that SSH root login works:
-
-   ```
-    ssh root@CHATMAIL_DOMAIN
-   ```
-
 4. Deploy to the remote chatmail server:
 
    ```
@@ -53,70 +52,24 @@ DNS domain name (FQDN), for example `chat.example.org`.
     scripts/cmdeploy dns
    ```
 
-### Other helpful commands:
+6. To check status of your remotely running chatmail service: 
-
-To check the status of your remotely running chatmail service:
 
    ```
     scripts/cmdeploy status
    ```
 
-To test whether your chatmail service is working correctly:
+7. To test your chatmail service: 
 
    ```
     scripts/cmdeploy test
    ```
 
-To measure the performance of your chatmail service:
+8. To benchmark your chatmail service: 
 
    ```
     scripts/cmdeploy bench
    ```
 
-## Overview of this repository
-
-This repository drives the development of "chatmail instances",
-comprised of minimal setups of
-
-- [postfix smtp server](https://www.postfix.org)
-- [dovecot imap server](https://www.dovecot.org)
-
-as well as custom services that are integrated with these two:
-
-- `chatmaild/src/chatmaild/doveauth.py` implements
-  create-on-login account creation semantics and is used
-  by Dovecot during login authentication and by Postfix
-  which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)
-  to authenticate users
-  to send mails for them.
-
-- `chatmaild/src/chatmaild/filtermail.py` prevents
-  unencrypted e-mail from leaving the chatmail instance
-  and is integrated into postfix's outbound mail pipelines.
-
-There is also the `cmdeploy/src/cmdeploy/cmdeploy.py` command line tool
-which helps with setting up and managing the chatmail service.
-`cmdeploy run` uses [pyinfra-based scripting](https://pyinfra.com/)
-in `cmdeploy/src/cmdeploy/__init__.py`
-to automatically install all chatmail components on a server.
-
-
-### Home page and getting started for users
-
-`cmdeploy run` also creates default static Web pages and deploys them
-to an nginx web server under `https://CHATMAIL_DOMAIN`.
-
-- a default `index.html` along with a QR code that users can click to
-  create accounts on your chatmail provider,
-
-- a default `info.html` that is linked from the home page,
-
-- a default `policy.html` that is linked from the home page.
-
-All `.html` files are generated
-by the according markdown `.md` file in the `www/src` directory.
-
-
 ### Refining the web pages 
 
 
@@ -135,24 +88,75 @@ This starts a local live development cycle for chatmail Web pages:
 - Starts a browser window automatically where you can "refresh" as needed. 
 
 
-## Emergency Commands to disable automatic account creation
+### Home page and getting started for users 
 
-If you need to stop account creation,
+`cmdeploy run` sets up mail services,
-e.g. because some script is wildly creating accounts,
+and also creates default static Web pages and deploys them: 
-login to the server with ssh and run:
 
-```
+- a default `index.html` along with a QR code that users can click to 
-    touch /etc/chatmail-nocreate
+  create accounts on your chatmail provider,
-```
+
+- a default `info.html` that is linked from the home page,
+
+- a default `policy.html` that is linked from the home page. 
+
+All `.html` files are generated 
+by the according markdown `.md` file in the `www/src` directory.
 
-While this file is present, account creation will be blocked.
 
 ### Ports
 
 Postfix listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
 Dovecot listens on ports 143(imap) and 993 (imaps).
 
-Delta Chat apps will, however, discover all ports and configurations
+Delta Chat will, however, discover all ports and configurations 
-automatically by reading the `autoconfig.xml` file from the chatmail service.
+automatically by reading the `autoconfig.xml` file from the chatmail instance. 
+
+
+## Emergency Commands to disable automatic account creation 
+
+If you need to stop account creation,
+e.g. because some script is wildly creating accounts, run:
+
+    touch /etc/chatmail-nocreate
+
+While this file is present, account creation will be blocked. 
+
+
+## Running tests and benchmarks (offline and online) 
+
+1. Set `CHATMAIL_SSH` so that `ssh root@$CHATMAIL_SSH` allows 
+   to login to the chatmail instance server. 
+
+2. To run local and online tests: 
+
+    scripts/test.sh 
+
+3. To run benchmarks against your chatmail instance: 
+
+    scripts/bench.sh 
+
+
+## Development Background for chatmail instances 
+
+This repository drives the development of "chatmail instances", 
+comprised of minimal setups of 
+
+- [postfix smtp server](https://www.postfix.org) 
+- [dovecot imap server](https://www.dovecot.org) 
+
+as well as two custom services that are integrated with these two: 
+
+- `chatmaild/src/chatmaild/doveauth.py` implements
+  create-on-login account creation semantics and is used
+  by Dovecot during login authentication and by Postfix
+  which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)
+  to authenticate users
+  to send mails for them. 
+
+- `chatmaild/src/chatmaild/filtermail.py` prevents 
+  unencrypted e-mail from leaving the chatmail instance
+  and is integrated into postfix's outbound mail pipelines. 
+
 
 

chatmaild/pyproject.toml

@@ -8,8 +8,6 @@ version = "0.2"
 dependencies = [
   "aiosmtpd",
   "iniconfig",
-  "deltachat-rpc-server",
-  "deltachat-rpc-client",
 ]
 
 [tool.setuptools]
@@ -21,8 +19,6 @@ where = ['src']
 [project.scripts]
 doveauth = "chatmaild.doveauth:main"
 filtermail = "chatmaild.filtermail:main"
-echobot = "chatmaild.echo:main"
-chatmail-metrics = "chatmaild.metrics:main"
 
 [project.entry-points.pytest11]
 "chatmaild.testplugin" = "chatmaild.tests.plugin"

chatmaild/src/chatmaild/doveauth.py

@@ -46,7 +46,6 @@ def is_allowed_to_create(config: Config, user, cleartext_password) -> bool:
         len(localpart) > config.username_max_length
         or len(localpart) < config.username_min_length
     ):
-        if localpart != "echo":
         logging.warning(
             "localpart %s has to be between %s and %s chars long",
             localpart,

chatmaild/src/chatmaild/echo.py

@@ -1,88 +0,0 @@
-#!/usr/bin/env python3
-"""Advanced echo bot example.
-
-it will echo back any message that has non-empty text and also supports the /help command.
-"""
-import logging
-import os
-import sys
-from threading import Thread
-
-from deltachat_rpc_client import Bot, DeltaChat, EventType, Rpc, events
-
-from chatmaild.newemail import create_newemail_dict
-from chatmaild.config import read_config
-
-hooks = events.HookCollection()
-
-
-@hooks.on(events.RawEvent)
-def log_event(event):
-    if event.kind == EventType.INFO:
-        logging.info(event.msg)
-    elif event.kind == EventType.WARNING:
-        logging.warning(event.msg)
-
-
-@hooks.on(events.RawEvent(EventType.ERROR))
-def log_error(event):
-    logging.error(event.msg)
-
-
-@hooks.on(events.MemberListChanged)
-def on_memberlist_changed(event):
-    logging.info(
-        "member %s was %s", event.member, "added" if event.member_added else "removed"
-    )
-
-
-@hooks.on(events.GroupImageChanged)
-def on_group_image_changed(event):
-    logging.info("group image %s", "deleted" if event.image_deleted else "changed")
-
-
-@hooks.on(events.GroupNameChanged)
-def on_group_name_changed(event):
-    logging.info("group name changed, old name: %s", event.old_name)
-
-
-@hooks.on(events.NewMessage(func=lambda e: not e.command))
-def echo(event):
-    snapshot = event.message_snapshot
-    if snapshot.text or snapshot.file:
-        snapshot.chat.send_message(text=snapshot.text, file=snapshot.file)
-
-
-@hooks.on(events.NewMessage(command="/help"))
-def help_command(event):
-    snapshot = event.message_snapshot
-    snapshot.chat.send_text("Send me any message and I will echo it back")
-
-
-def main():
-    path = os.environ.get("PATH")
-    venv_path = sys.argv[0].strip("echobot")
-    os.environ["PATH"] = path + ":" + venv_path
-    with Rpc() as rpc:
-        deltachat = DeltaChat(rpc)
-        system_info = deltachat.get_system_info()
-        logging.info("Running deltachat core %s", system_info.deltachat_core_version)
-
-        accounts = deltachat.get_all_accounts()
-        account = accounts[0] if accounts else deltachat.add_account()
-
-        bot = Bot(account, hooks)
-        if not bot.is_configured():
-            config = read_config(sys.argv[1])
-            password = create_newemail_dict(config).get("password")
-            email = "echo@" + config.mail_domain
-            configure_thread = Thread(
-                target=bot.configure, kwargs={"email": email, "password": password}
-            )
-            configure_thread.start()
-        bot.run_forever()
-
-
-if __name__ == "__main__":
-    logging.basicConfig(level=logging.INFO)
-    main()

chatmaild/src/chatmaild/echobot.service.f

@@ -1,11 +0,0 @@
-[Unit]
-Description=Chatmail echo bot for testing it works
-
-[Service]
-ExecStart={execpath} {config_path}
-Environment="PATH={remote_venv_dir}:$PATH"
-Restart=always
-RestartSec=30
-
-[Install]
-WantedBy=multi-user.target

chatmaild/src/chatmaild/metrics.py

@@ -1,23 +0,0 @@
-#!/usr/bin/env python3
-from pathlib import Path
-import time
-import sys
-
-
-def main():
-    vmail_dir = sys.argv[1]
-    accounts = 0
-    ci_accounts = 0
-
-    for path in Path(vmail_dir).iterdir():
-        accounts += 1
-        if path.name.startswith("ci-"):
-            ci_accounts += 1
-
-    timestamp = int(time.time() * 1000)
-    print(f"accounts {accounts} {timestamp}")
-    print(f"ci_accounts {ci_accounts} {timestamp}")
-
-
-if __name__ == "__main__":
-    main()

chatmaild/src/chatmaild/newemail.py

@@ -4,22 +4,16 @@
 
 import json
 import random
-import secrets
-import string
 
 from chatmaild.config import read_config, Config
 
 CONFIG_PATH = "/usr/local/lib/chatmaild/chatmail.ini"
-ALPHANUMERIC = string.ascii_lowercase + string.digits
-ALPHANUMERIC_PUNCT = string.ascii_letters + string.digits + string.punctuation
 
 
 def create_newemail_dict(config: Config):
-    user = "".join(random.choices(ALPHANUMERIC, k=config.username_min_length))
+    alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
-    password = "".join(
+    user = "".join(random.choices(alphanumeric, k=config.username_min_length))
-        secrets.choice(ALPHANUMERIC_PUNCT)
+    password = "".join(random.choices(alphanumeric, k=config.password_min_length + 3))
-        for _ in range(config.password_min_length + 3)
-    )
     return dict(email=f"{user}@{config.mail_domain}", password=f"{password}")
 
 

cmdeploy/src/cmdeploy/__init__.py

@@ -84,28 +84,14 @@ def _install_remote_venv_with_chatmaild(config) -> None:
         ],
     )
 
-    files.template(
-        src=importlib.resources.files(__package__).joinpath("metrics.cron.j2"),
-        dest="/etc/cron.d/chatmail-metrics",
-        user="root",
-        group="root",
-        mode="644",
-        config={
-            "mail_domain": config.mail_domain,
-            "execpath": f"{remote_venv_dir}/bin/chatmail-metrics",
-        },
-    )
-
     # install systemd units
     for fn in (
         "doveauth",
         "filtermail",
-        "echobot",
     ):
         params = dict(
             execpath=f"{remote_venv_dir}/bin/{fn}",
             config_path=remote_chatmail_inipath,
-            remote_venv_dir=remote_venv_dir,
         )
         source_path = importlib.resources.files("chatmaild").joinpath(f"{fn}.service.f")
         content = source_path.read_text().format(**params).encode()

cmdeploy/src/cmdeploy/metrics.cron.j2

@@ -1 +0,0 @@
-*/5 * * * * root {{ config.execpath }} /home/vmail/mail/{{ config.mail_domain }} >/var/www/html/metrics

cmdeploy/src/cmdeploy/nginx/nginx.conf.j2

@@ -41,10 +41,6 @@ http {
 			try_files $uri $uri/ =404;
 		}
 
-                location /metrics {
-                        default_type text/plain;
-                }
-
         # add cgi-bin support
         include /usr/share/doc/fcgiwrap/examples/nginx.conf;
 	}

www/src/info.md

@@ -3,11 +3,6 @@
 
 ## More information 
 
-`nine.testrun.org` provides a low-maintenance, resource efficient and 
-interoperable e-mail service for everyone. What's behind a `chatmail` is 
-effectively a normal e-mail address just like any other but optimized 
-for the usage in chats, especially DeltaChat.
-
 ### Choosing a chatmail address instead of using a random one
 
 In the Delta Chat account setup