Compare commits

..

4 Commits

Author SHA1 Message Date
B. Petersen
9667fe661c make 'experimental' more outstanding 2023-12-12 19:46:26 +01:00
B. Petersen
3141431ee3 use a class for the banner, make sure it is always width of page 2023-12-12 19:39:30 +01:00
B. Petersen
ac2517df4b add domain right of the menu, standardize menu help-code 2023-12-12 19:27:39 +01:00
B. Petersen
f467d7bcd1 move menu up 2023-12-12 18:48:24 +01:00
67 changed files with 2148 additions and 1862 deletions

View File

@@ -1,4 +0,0 @@
Please refer to
[Delta Chat community standards and practices](https://delta.chat/en/community-standards)
which also apply for all chatmail developments.

View File

@@ -33,5 +33,8 @@ jobs:
- name: run deploy-chatmail offline tests
run: pytest --pyargs cmdeploy
- name: initialize with chatmail domain
run: cmdeploy init chat.example.org
# all other cmdeploy commands require a staging server
# see https://github.com/deltachat/chatmail/issues/100

View File

@@ -1,20 +0,0 @@
;; Zone file for staging.testrun.org
$ORIGIN staging.testrun.org.
$TTL 300
@ IN SOA ns.testrun.org. root.nine.testrun.org (
2023010101 ; Serial
7200 ; Refresh
3600 ; Retry
1209600 ; Expire
3600 ; Negative response caching TTL
)
;; Nameservers.
@ IN NS ns.testrun.org.
;; DNS records.
@ IN A 37.27.37.98
mta-sts.staging.testrun.org. CNAME staging.testrun.org.
www.staging.testrun.org. CNAME staging.testrun.org.

View File

@@ -1,72 +0,0 @@
name: deploy on staging.testrun.org, and run tests
on:
push:
branches:
- main
- staging-ci
jobs:
deploy:
name: deploy on staging.testrun.org, and run tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: prepare SSH
run: |
mkdir ~/.ssh
echo "${{ secrets.STAGING_SSH_KEY }}" >> ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
ssh-keyscan staging.testrun.org > ~/.ssh/known_hosts
# rsync -avz root@staging.testrun.org:/var/lib/acme . || true
# rsync -avz root@staging.testrun.org:/var/lib/rspamd/dkim . || true
#- name: rebuild staging.testrun.org to have a clean VPS
# run: |
# curl -X POST \
# -H "Authorization: Bearer ${{ secrets.HETZNER_API_TOKEN }}" \
# -H "Content-Type: application/json" \
# -d '{"image":"debian-12"}' \
# "https://api.hetzner.cloud/v1/servers/${{ secrets.STAGING_SERVER_ID }}/actions/rebuild"
- run: scripts/initenv.sh
- name: append venv/bin to PATH
run: echo venv/bin >>$GITHUB_PATH
- name: run formatting checks
run: cmdeploy fmt -v
- name: run deploy-chatmail offline tests
run: pytest --pyargs cmdeploy
#- name: upload TLS cert after rebuilding
# run: |
# echo " --- wait until staging.testrun.org VPS is rebuilt --- "
# rm ~/.ssh/known_hosts
# while ! ssh -o ConnectTimeout=180 -o StrictHostKeyChecking=accept-new -v root@staging.testrun.org id -u ; do sleep 1 ; done
# ssh -o StrictHostKeyChecking=accept-new -v root@staging.testrun.org id -u
# rsync -avz acme root@staging.testrun.org:/var/lib/ || true
# rsync -avz dkim root@staging.testrun.org:/var/lib/rspamd/ || true
- run: cmdeploy init staging.testrun.org
- run: cmdeploy run
- name: set DNS entries
run: |
#ssh -o StrictHostKeyChecking=accept-new -v root@staging.testrun.org chown _rspamd:_rspamd -R /var/lib/rspamd/dkim
cmdeploy dns --zonefile staging-generated.zone
cat staging-generated.zone >> .github/workflows/staging.testrun.org-default.zone
cat .github/workflows/staging.testrun.org-default.zone
scp -o StrictHostKeyChecking=accept-new .github/workflows/staging.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging.testrun.org.zone
ssh root@ns.testrun.org nsd-checkzone staging.testrun.org /etc/nsd/staging.testrun.org.zone
ssh root@ns.testrun.org systemctl reload nsd
- name: cmdeploy test
run: CHATMAIL_DOMAIN2=nine.testrun.org cmdeploy test --slow
- name: cmdeploy dns (try 3 times)
run: cmdeploy dns || cmdeploy dns || cmdeploy dns

21
LICENSE
View File

@@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2023, chatmail and delta chat teams
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

208
README.md
View File

@@ -15,148 +15,146 @@ after which the initially specified password is required for using them.
## Deploying your own chatmail server
We use `chat.example.org` as the chatmail domain in the following steps.
Please substitute it with your own domain.
We subsequently use `CHATMAIL_DOMAIN` as a placeholder for your fully qualified
DNS domain name (FQDN), for example `chat.example.org`.
1. Install the `cmdeploy` command in a virtualenv
1. Setup DNS `A` and `AAAA` records for your `CHATMAIL_DOMAIN`.
Verify that DNS is set and SSH root login works:
```
git clone https://github.com/deltachat/chatmail
cd chatmail
scripts/initenv.sh
ssh root@CHATMAIL_DOMAIN
```
2. Create chatmail configuration file `chatmail.ini`:
2. Install the `cmdeploy` command in a virtualenv
```
scripts/cmdeploy init chat.example.org # <-- use your domain
source scripts/initenv.sh
```
3. Setup first DNS records for your chatmail domain,
according to the hints provided by `cmdeploy init`.
Verify that SSH root login works:
3. Create chatmail configuration file `chatmail.ini`:
```
ssh root@chat.example.org # <-- use your domain
cmdeploy init CHATMAIL_DOMAIN
```
4. Deploy to the remote chatmail server:
```
scripts/cmdeploy run
cmdeploy run
```
This script will also show you additional DNS records
which you should configure at your DNS provider
(it can take some time until they are public).
### Other helpful commands:
5. To output a DNS zone file from which you can transfer DNS records
to your DNS provider:
To check the status of your remotely running chatmail service:
```
cmdeploy dns
```
```
scripts/cmdeploy status
6. To check status of your remotely running chatmail service:
```
cmdeploy status
```
7. To test your chatmail service:
```
cmdeploy test
```
8. To benchmark your chatmail service:
```
cmdeploy bench
```
### Refining the web pages
```
cmdeploy webdev
```
To check whether your DNS records are correct:
This starts a local live development cycle for chatmail Web pages:
```
scripts/cmdeploy dns
```
- uses the `www/src/page-layout.html` file for producing static
HTML pages from `www/src/*.md` files
To test whether your chatmail service is working correctly:
- continously builds the web presence reading files from `www/src` directory
and generating html files and copying assets to the `www/build` directory.
```
scripts/cmdeploy test
```
- Starts a browser window automatically where you can "refresh" as needed.
To measure the performance of your chatmail service:
```
scripts/cmdeploy bench
```
### Home page and getting started for users
## Overview of this repository
`cmdeploy run` sets up mail services,
and also creates default static Web pages and deploys them:
This repository drives the development of chatmail services,
comprised of minimal setups of
- a default `index.html` along with a QR code that users can click to
create accounts on your chatmail provider,
- [postfix smtp server](https://www.postfix.org)
- [dovecot imap server](https://www.dovecot.org)
- a default `info.html` that is linked from the home page,
as well as custom services that are integrated with these two:
- a default `policy.html` that is linked from the home page.
All `.html` files are generated
by the according markdown `.md` file in the `www/src` directory.
### Ports
Postfix listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
Dovecot listens on ports 143(imap) and 993 (imaps).
Delta Chat will, however, discover all ports and configurations
automatically by reading the `autoconfig.xml` file from the chatmail instance.
## Emergency Commands to disable automatic account creation
If you need to stop account creation,
e.g. because some script is wildly creating accounts, run:
touch /etc/chatmail-nocreate
While this file is present, account creation will be blocked.
## Running tests and benchmarks (offline and online)
1. Set `CHATMAIL_SSH` so that `ssh root@$CHATMAIL_SSH` allows
to login to the chatmail instance server.
2. To run local and online tests:
scripts/test.sh
3. To run benchmarks against your chatmail instance:
scripts/bench.sh
## Development Background for chatmail instances
This repository drives the development of "chatmail instances",
comprised of minimal setups of
- [postfix smtp server](https://www.postfix.org)
- [dovecot imap server](https://www.dovecot.org)
as well as two custom services that are integrated with these two:
- `chatmaild/src/chatmaild/doveauth.py` implements
create-on-login account creation semantics and is used
by Dovecot during login authentication and by Postfix
which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)
to authenticate users
to send mails for them.
to send mails for them.
- `chatmaild/src/chatmaild/filtermail.py` prevents
unencrypted e-mail from leaving the chatmail service
and is integrated into postfix's outbound mail pipelines.
There is also the `cmdeploy/src/cmdeploy/cmdeploy.py` command line tool
which helps with setting up and managing the chatmail service.
`cmdeploy run` uses [pyinfra-based scripting](https://pyinfra.com/)
in `cmdeploy/src/cmdeploy/__init__.py`
to automatically install all chatmail components on a server.
- `chatmaild/src/chatmaild/filtermail.py` prevents
unencrypted e-mail from leaving the chatmail instance
and is integrated into postfix's outbound mail pipelines.
### Home page and getting started for users
`cmdeploy run` also creates default static Web pages and deploys them
to a nginx web server with:
- a default `index.html` along with a QR code that users can click to
create accounts on your chatmail provider,
- a default `info.html` that is linked from the home page,
- a default `policy.html` that is linked from the home page.
All `.html` files are generated
by the according markdown `.md` file in the `www/src` directory.
### Refining the web pages
```
scripts/cmdeploy webdev
```
This starts a local live development cycle for chatmail Web pages:
- uses the `www/src/page-layout.html` file for producing static
HTML pages from `www/src/*.md` files
- continously builds the web presence reading files from `www/src` directory
and generating html files and copying assets to the `www/build` directory.
- Starts a browser window automatically where you can "refresh" as needed.
## Emergency Commands to disable automatic account creation
If you need to stop account creation,
e.g. because some script is wildly creating accounts,
login to the server with ssh and run:
```
touch /etc/chatmail-nocreate
```
While this file is present, account creation will be blocked.
### Ports
[Postfix](http://www.postfix.org/) listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
[Dovecot](https://www.dovecot.org/) listens on ports 143 (imap) and 993 (imaps).
[nginx](https://www.nginx.com/) listens on port 443 (https).
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (http).
Delta Chat apps will, however, discover all ports and configurations
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail service.

View File

@@ -8,9 +8,6 @@ version = "0.2"
dependencies = [
"aiosmtpd",
"iniconfig",
"deltachat-rpc-server",
"deltachat-rpc-client",
"requests",
]
[tool.setuptools]
@@ -21,10 +18,7 @@ where = ['src']
[project.scripts]
doveauth = "chatmaild.doveauth:main"
chatmail-metadata = "chatmaild.metadata:main"
filtermail = "chatmaild.filtermail:main"
echobot = "chatmaild.echo:main"
chatmail-metrics = "chatmaild.metrics:main"
[project.entry-points.pytest11]
"chatmaild.testplugin" = "chatmaild.tests.plugin"

View File

@@ -1,10 +0,0 @@
[Unit]
Description=Chatmail dict proxy for IMAP METADATA
[Service]
ExecStart={execpath} /run/dovecot/metadata.socket vmail {config_path}
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

View File

@@ -11,15 +11,9 @@ class Config:
self._inipath = inipath
self.mail_domain = params["mail_domain"]
self.max_user_send_per_minute = int(params["max_user_send_per_minute"])
self.max_mailbox_size = params["max_mailbox_size"]
self.delete_mails_after = params["delete_mails_after"]
self.username_min_length = int(params["username_min_length"])
self.username_max_length = int(params["username_max_length"])
self.password_min_length = int(params["password_min_length"])
self.passthrough_senders = params["passthrough_senders"].split()
self.passthrough_recipients = params["passthrough_recipients"].split()
self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
self.postfix_reinject_port = int(params["postfix_reinject_port"])
self.passthrough_recipients = params["passthrough_recipients"].split()
self.privacy_postal = params.get("privacy_postal")
self.privacy_mail = params.get("privacy_mail")
self.privacy_pdo = params.get("privacy_pdo")

View File

@@ -12,7 +12,6 @@ from socketserver import (
import pwd
from .database import Database
from .config import read_config, Config
NOCREATE_FILE = "/etc/chatmail-nocreate"
@@ -23,17 +22,14 @@ def encrypt_password(password: str):
return "{SHA512-CRYPT}" + passhash
def is_allowed_to_create(config: Config, user, cleartext_password) -> bool:
def is_allowed_to_create(user, cleartext_password) -> bool:
"""Return True if user and password are admissable."""
if os.path.exists(NOCREATE_FILE):
logging.warning(f"blocked account creation because {NOCREATE_FILE!r} exists.")
return False
if len(cleartext_password) < config.password_min_length:
logging.warning(
"Password needs to be at least %s characters long",
config.password_min_length,
)
if len(cleartext_password) < 9:
logging.warning("Password needs to be at least 9 characters long")
return False
parts = user.split("@")
@@ -42,37 +38,29 @@ def is_allowed_to_create(config: Config, user, cleartext_password) -> bool:
return False
localpart, domain = parts
if (
len(localpart) > config.username_max_length
or len(localpart) < config.username_min_length
):
if localpart != "echo":
logging.warning(
"localpart %s has to be between %s and %s chars long",
localpart,
config.username_min_length,
config.username_max_length,
)
if domain == "nine.testrun.org":
# nine.testrun.org policy, username has to be exactly nine chars
if len(localpart) != 9:
logging.warning(f"localpart {localpart!r} has not exactly nine chars")
return False
return True
def get_user_data(db, config: Config, user):
def get_user_data(db, user):
with db.read_connection() as conn:
result = conn.get_user(user)
if result:
result["home"] = f"/home/vmail/mail/{config.mail_domain}/{user}"
result["uid"] = "vmail"
result["gid"] = "vmail"
return result
def lookup_userdb(db, config: Config, user):
return get_user_data(db, config, user)
def lookup_userdb(db, user):
return get_user_data(db, user)
def lookup_passdb(db, config: Config, user, cleartext_password):
def lookup_passdb(db, user, cleartext_password):
with db.write_transaction() as conn:
userdata = conn.get_user(user)
if userdata:
@@ -81,11 +69,10 @@ def lookup_passdb(db, config: Config, user, cleartext_password):
"UPDATE users SET last_login=? WHERE addr=?", (int(time.time()), user)
)
userdata["home"] = f"/home/vmail/mail/{config.mail_domain}/{user}"
userdata["uid"] = "vmail"
userdata["gid"] = "vmail"
return userdata
if not is_allowed_to_create(config, user, cleartext_password):
if not is_allowed_to_create(user, cleartext_password):
return
encrypted_password = encrypt_password(cleartext_password)
@@ -93,66 +80,32 @@ def lookup_passdb(db, config: Config, user, cleartext_password):
VALUES (?, ?, ?)"""
conn.execute(q, (user, encrypted_password, int(time.time())))
return dict(
home=f"/home/vmail/mail/{config.mail_domain}/{user}",
home=f"/home/vmail/{user}",
uid="vmail",
gid="vmail",
password=encrypted_password,
)
def split_and_unescape(s):
"""Split strings using double quote as a separator and backslash as escape character
into parts."""
out = ""
i = 0
while i < len(s):
c = s[i]
if c == "\\":
# Skip escape character.
i += 1
# This will raise IndexError if there is no character
# after escape character. This is expected
# as this is an invalid input.
out += s[i]
elif c == '"':
# Separator
yield out
out = ""
else:
out += c
i += 1
yield out
def handle_dovecot_request(msg, db, config: Config):
def handle_dovecot_request(msg, db, mail_domain):
short_command = msg[0]
if short_command == "L": # LOOKUP
parts = msg[1:].split("\t")
# Dovecot <2.3.17 has only one part,
# do not attempt to read any other parts for compatibility.
keyname = parts[0]
namespace, type, args = keyname.split("/", 2)
args = list(split_and_unescape(args))
keyname, user = parts[:2]
namespace, type, *args = keyname.split("/")
reply_command = "F"
res = ""
if namespace == "shared":
if type == "userdb":
user = args[0]
if user.endswith(f"@{config.mail_domain}"):
res = lookup_userdb(db, config, user)
if user.endswith(f"@{mail_domain}"):
res = lookup_userdb(db, user)
if res:
reply_command = "O"
else:
reply_command = "N"
elif type == "passdb":
user = args[1]
if user.endswith(f"@{config.mail_domain}"):
res = lookup_passdb(db, config, user, cleartext_password=args[0])
if user.endswith(f"@{mail_domain}"):
res = lookup_passdb(db, user, cleartext_password=args[0])
if res:
reply_command = "O"
else:
@@ -162,19 +115,6 @@ def handle_dovecot_request(msg, db, config: Config):
return None
def handle_dovecot_protocol(rfile, wfile, db: Database, config: Config):
while True:
msg = rfile.readline().strip().decode()
if not msg:
break
res = handle_dovecot_request(msg, db, config)
if res:
wfile.write(res.encode("ascii"))
wfile.flush()
else:
logging.warning("request had no answer: %r", msg)
class ThreadedUnixStreamServer(ThreadingMixIn, UnixStreamServer):
request_queue_size = 100
@@ -183,12 +123,22 @@ def main():
socket = sys.argv[1]
passwd_entry = pwd.getpwnam(sys.argv[2])
db = Database(sys.argv[3])
config = read_config(sys.argv[4])
with open("/etc/mailname", "r") as fp:
mail_domain = fp.read().strip()
class Handler(StreamRequestHandler):
def handle(self):
try:
handle_dovecot_protocol(self.rfile, self.wfile, db, config)
while True:
msg = self.rfile.readline().strip().decode()
if not msg:
break
res = handle_dovecot_request(msg, db, mail_domain)
if res:
self.wfile.write(res.encode("ascii"))
self.wfile.flush()
else:
logging.warn("request had no answer: %r", msg)
except Exception:
logging.exception("Exception in the handler")
raise

View File

@@ -2,7 +2,7 @@
Description=Chatmail dict authentication proxy for dovecot
[Service]
ExecStart={execpath} /run/dovecot/doveauth.socket vmail /home/vmail/passdb.sqlite {config_path}
ExecStart={execpath} /run/dovecot/doveauth.socket vmail /home/vmail/passdb.sqlite
Restart=always
RestartSec=30

View File

@@ -1,84 +0,0 @@
#!/usr/bin/env python3
"""Advanced echo bot example.
it will echo back any message that has non-empty text and also supports the /help command.
"""
import logging
import os
import sys
from deltachat_rpc_client import Bot, DeltaChat, EventType, Rpc, events
from chatmaild.newemail import create_newemail_dict
from chatmaild.config import read_config
hooks = events.HookCollection()
@hooks.on(events.RawEvent)
def log_event(event):
if event.kind == EventType.INFO:
logging.info(event.msg)
elif event.kind == EventType.WARNING:
logging.warning(event.msg)
@hooks.on(events.RawEvent(EventType.ERROR))
def log_error(event):
logging.error(event.msg)
@hooks.on(events.MemberListChanged)
def on_memberlist_changed(event):
logging.info(
"member %s was %s", event.member, "added" if event.member_added else "removed"
)
@hooks.on(events.GroupImageChanged)
def on_group_image_changed(event):
logging.info("group image %s", "deleted" if event.image_deleted else "changed")
@hooks.on(events.GroupNameChanged)
def on_group_name_changed(event):
logging.info("group name changed, old name: %s", event.old_name)
@hooks.on(events.NewMessage(func=lambda e: not e.command))
def echo(event):
snapshot = event.message_snapshot
if snapshot.text or snapshot.file:
snapshot.chat.send_message(text=snapshot.text, file=snapshot.file)
@hooks.on(events.NewMessage(command="/help"))
def help_command(event):
snapshot = event.message_snapshot
snapshot.chat.send_text("Send me any message and I will echo it back")
def main():
path = os.environ.get("PATH")
venv_path = sys.argv[0].strip("echobot")
os.environ["PATH"] = path + ":" + venv_path
with Rpc() as rpc:
deltachat = DeltaChat(rpc)
system_info = deltachat.get_system_info()
logging.info("Running deltachat core %s", system_info.deltachat_core_version)
accounts = deltachat.get_all_accounts()
account = accounts[0] if accounts else deltachat.add_account()
bot = Bot(account, hooks)
if not bot.is_configured():
config = read_config(sys.argv[1])
password = create_newemail_dict(config).get("password")
email = "echo@" + config.mail_domain
bot.configure(email, password)
bot.run_forever()
if __name__ == "__main__":
logging.basicConfig(level=logging.INFO)
main()

View File

@@ -1,11 +0,0 @@
[Unit]
Description=Chatmail echo bot for testing it works
[Service]
ExecStart={execpath} {config_path}
Environment="PATH={remote_venv_dir}:$PATH"
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

View File

@@ -111,9 +111,6 @@ class BeforeQueueHandler:
if not mail_encrypted and check_mdn(message, envelope):
return
if envelope.mail_from in self.config.passthrough_senders:
return
passthrough_recipients = self.config.passthrough_recipients
envelope_from_domain = from_addr.split("@").pop()
for recipient in envelope.rcpt_tos:

View File

@@ -7,37 +7,11 @@ mail_domain = {mail_domain}
# If you only do private test deploys, you don't need to modify any settings below
#
#
# Account Restrictions
#
# how many mails a user can send out per minute
max_user_send_per_minute = 60
# maximum mailbox size of a chatmail account
max_mailbox_size = 100M
# days after which mails are unconditionally deleted
delete_mails_after = 40
# minimum length a username must have
username_min_length = 9
# maximum length a username can have
username_max_length = 9
# minimum length a password must have
password_min_length = 9
# list of chatmail accounts which can send outbound un-encrypted mail
passthrough_senders =
# list of e-mail recipients for which to accept outbound un-encrypted mails
passthrough_recipients = xstore@testrun.org groupsbot@hispanilandia.net
#
# Deployment Details
#
passthrough_recipients =
# where the filtermail SMTP service listens
filtermail_smtp_port = 10080
@@ -45,10 +19,6 @@ filtermail_smtp_port = 10080
# postfix accepts on the localhost reinject SMTP port
postfix_reinject_port = 10025
#
# Privacy Policy
#
# postal address of privacy contact
privacy_postal =

View File

@@ -1,7 +1,7 @@
[privacy]
passthrough_recipients = privacy@testrun.org xstore@testrun.org groupsbot@hispanilandia.net
passthrough_recipients = privacy@testrun.org
privacy_postal =
Merlinux GmbH, Represented by the managing director H. Krekel,

View File

@@ -1,149 +0,0 @@
import pwd
from queue import Queue
from threading import Thread
from socketserver import (
UnixStreamServer,
StreamRequestHandler,
ThreadingMixIn,
)
from .config import read_config
import sys
import logging
import os
import requests
DICTPROXY_LOOKUP_CHAR = "L"
DICTPROXY_SET_CHAR = "S"
DICTPROXY_BEGIN_TRANSACTION_CHAR = "B"
DICTPROXY_COMMIT_TRANSACTION_CHAR = "C"
DICTPROXY_TRANSACTION_CHARS = "SBC"
class Notifier:
def __init__(self):
self.guid2token = {}
self.to_notify_queue = Queue()
def set_token(self, guid, token):
self.guid2token[guid] = token
def new_message_for_guid(self, guid):
self.to_notify_queue.put(guid)
def thread_run_loop(self):
requests_session = requests.Session()
while 1:
self.thread_run_one(requests_session)
def thread_run_one(self, requests_session):
guid = self.to_notify_queue.get()
token = self.guid2token.get(guid)
if token:
response = requests_session.post(
"https://notifications.delta.chat/notify",
data=token,
timeout=60,
)
if response.status_code == 410:
# 410 Gone status code
# means the token is no longer valid.
del self.guid2token[guid]
def handle_dovecot_protocol(rfile, wfile, notifier):
# HELLO message, ignored.
msg = rfile.readline().strip().decode()
transactions = {}
while True:
msg = rfile.readline().strip().decode()
if not msg:
break
res = handle_dovecot_request(msg, transactions, notifier)
if res:
wfile.write(res.encode("ascii"))
wfile.flush()
def handle_dovecot_request(msg, transactions, notifier):
# see https://doc.dovecot.org/3.0/developer_manual/design/dict_protocol/
short_command = msg[0]
parts = msg[1:].split("\t")
if short_command == DICTPROXY_LOOKUP_CHAR:
return "N\n"
if short_command not in (DICTPROXY_TRANSACTION_CHARS):
return
transaction_id = parts[0]
if short_command == DICTPROXY_BEGIN_TRANSACTION_CHAR:
transactions[transaction_id] = "O\n"
elif short_command == DICTPROXY_COMMIT_TRANSACTION_CHAR:
# returns whether it failed or succeeded.
return transactions.pop(transaction_id, "N\n")
elif short_command == DICTPROXY_SET_CHAR:
# See header of
# <https://github.com/dovecot/core/blob/5e7965632395793d9355eb906b173bf28d2a10ca/src/lib-storage/mailbox-attribute.h>
# for the documentation on the structure of the key.
# Request GETMETADATA "INBOX" /private/chatmail
# results in a query for
# priv/dd72550f05eadc65542a1200cac67ad7/chatmail
#
# Request GETMETADATA "" /private/chatmail
# results in
# priv/dd72550f05eadc65542a1200cac67ad7/vendor/vendor.dovecot/pvt/server/chatmail
keyname = parts[1].split("/")
value = parts[2] if len(parts) > 2 else ""
if keyname[0] == "priv" and keyname[2] == "devicetoken":
notifier.set_token(keyname[1], value)
elif keyname[0] == "priv" and keyname[2] == "messagenew":
notifier.new_message_for_guid(keyname[1])
else:
# Transaction failed.
transactions[transaction_id] = "F\n"
class ThreadedUnixStreamServer(ThreadingMixIn, UnixStreamServer):
request_queue_size = 100
def main():
socket, username, config = sys.argv[1:]
passwd_entry = pwd.getpwnam(username)
# XXX config is not currently used
config = read_config(config)
notifier = Notifier()
class Handler(StreamRequestHandler):
def handle(self):
try:
handle_dovecot_protocol(self.rfile, self.wfile, notifier)
except Exception:
logging.exception("Exception in the handler")
raise
try:
os.unlink(socket)
except FileNotFoundError:
pass
# start notifier thread for signalling new messages to
# Delta Chat notification server
t = Thread(target=notifier.thread_run_loop)
t.setDaemon(True)
t.start()
with ThreadedUnixStreamServer(socket, Handler) as server:
os.chown(socket, uid=passwd_entry.pw_uid, gid=passwd_entry.pw_gid)
try:
server.serve_forever()
except KeyboardInterrupt:
pass

View File

@@ -1,25 +0,0 @@
#!/usr/bin/env python3
from pathlib import Path
import time
import sys
def main(vmail_dir=None):
if vmail_dir is None:
vmail_dir = sys.argv[1]
accounts = 0
ci_accounts = 0
for path in Path(vmail_dir).iterdir():
accounts += 1
if path.name[:3] in ("ci-", "ac_"):
ci_accounts += 1
timestamp = int(time.time() * 1000)
print(f"accounts {accounts} {timestamp}")
print(f"ci_accounts {ci_accounts} {timestamp}")
if __name__ == "__main__":
main()

View File

@@ -1,31 +1,23 @@
#!/usr/local/lib/chatmaild/venv/bin/python3
#!/usr/bin/python3
""" CGI script for creating new accounts. """
import json
import random
import secrets
import string
from chatmaild.config import read_config, Config
CONFIG_PATH = "/usr/local/lib/chatmaild/chatmail.ini"
ALPHANUMERIC = string.ascii_lowercase + string.digits
ALPHANUMERIC_PUNCT = string.ascii_letters + string.digits + string.punctuation
mail_domain_path = "/etc/mailname"
def create_newemail_dict(config: Config):
user = "".join(random.choices(ALPHANUMERIC, k=config.username_min_length))
password = "".join(
secrets.choice(ALPHANUMERIC_PUNCT)
for _ in range(config.password_min_length + 3)
)
return dict(email=f"{user}@{config.mail_domain}", password=f"{password}")
def create_newemail_dict(domain):
alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
user = "".join(random.choices(alphanumeric, k=9))
password = "".join(random.choices(alphanumeric, k=12))
return dict(email=f"{user}@{domain}", password=f"{password}")
def print_new_account():
config = read_config(CONFIG_PATH)
creds = create_newemail_dict(config)
domain = open(mail_domain_path).read().strip()
creds = create_newemail_dict(domain=domain)
print("Content-Type: application/json")
print("")

View File

@@ -7,7 +7,7 @@ Date: Sun, 15 Oct 2023 16:41:44 +0000
Message-ID: <Mr.3gckbNy5bch.uK3Hd2Ws6-w@c2.testrun.org>
References: <Mr.3gckbNy5bch.uK3Hd2Ws6-w@c2.testrun.org>
Chat-Version: 1.0
Autocrypt: addr={from_addr}; prefer-encrypt=mutual;
Autocrypt: addr=foobar@c2.testrun.org; prefer-encrypt=mutual;
keydata=xjMEZSrw3hYJKwYBBAHaRw8BAQdAiEKNQFU28c6qsx4vo/JHdt73RXdjMOmByf/XsGiJ7m
nNFzxmb29iYXJAYzIudGVzdHJ1bi5vcmc+wosEEBYIADMCGQEFAmUq8N4CGwMECwkIBwYVCAkKCwID
FgIBFiEEGil0OvTIa6RngmCLUYNnEa9leJAACgkQUYNnEa9leJCX3gEAhm0MehE5byBBU1avPczr/I
@@ -20,4 +20,4 @@ Content-Type: text/plain; charset=utf-8; format=flowed; delsp=no
Hi!

View File

@@ -1,6 +1,4 @@
import random
from pathlib import Path
import os
import importlib.resources
import itertools
from email.parser import BytesParser
@@ -59,12 +57,7 @@ def db(tmpdir):
@pytest.fixture
def maildata(request):
try:
datadir = importlib.resources.files(__package__).joinpath("mail-data")
except TypeError:
# in python3.9 or lower, the above doesn't work, so we get datadir this way:
datadir = Path(os.getcwd()).joinpath("chatmaild/src/chatmaild/tests/mail-data")
datadir = importlib.resources.files(__package__).joinpath("mail-data")
assert datadir.exists(), datadir
def maildata(name, from_addr, to_addr):

View File

@@ -1,16 +1,17 @@
from chatmaild.config import read_config
def test_read_config_basic(example_config):
assert example_config.mail_domain == "chat.example.org"
assert not example_config.privacy_supervisor and not example_config.privacy_mail
assert not example_config.privacy_pdo and not example_config.privacy_postal
def test_read_config_basic(make_config):
config = make_config("chat.example.org")
assert config.mail_domain == "chat.example.org"
assert not config.privacy_supervisor and not config.privacy_mail
assert not config.privacy_pdo and not config.privacy_postal
inipath = example_config._inipath
inipath = config._inipath
inipath.write_text(inipath.read_text().replace("60", "37"))
example_config = read_config(inipath)
assert example_config.max_user_send_per_minute == 37
assert example_config.mail_domain == "chat.example.org"
config = read_config(inipath)
assert config.max_user_send_per_minute == 37
assert config.mail_domain == "chat.example.org"
def test_read_config_testrun(make_config):
@@ -23,10 +24,4 @@ def test_read_config_testrun(make_config):
assert config.filtermail_smtp_port == 10080
assert config.postfix_reinject_port == 10025
assert config.max_user_send_per_minute == 60
assert config.max_mailbox_size == "100M"
assert config.delete_mails_after == "40"
assert config.username_min_length == 9
assert config.username_max_length == 9
assert config.password_min_length == 9
assert "privacy@testrun.org" in config.passthrough_recipients
assert config.passthrough_senders == []
assert config.passthrough_recipients

View File

@@ -1,49 +1,37 @@
import io
import json
import pytest
import queue
import threading
import queue
import traceback
import chatmaild.doveauth
from chatmaild.doveauth import (
get_user_data,
lookup_passdb,
handle_dovecot_request,
handle_dovecot_protocol,
)
from chatmaild.doveauth import get_user_data, lookup_passdb, handle_dovecot_request
from chatmaild.database import DBError
def test_basic(db, example_config):
lookup_passdb(db, example_config, "asdf12345@chat.example.org", "q9mr3faue")
data = get_user_data(db, example_config, "asdf12345@chat.example.org")
def test_basic(db):
lookup_passdb(db, "link2xt@c1.testrun.org", "Pieg9aeToe3eghuthe5u")
data = get_user_data(db, "link2xt@c1.testrun.org")
assert data
data2 = lookup_passdb(
db, example_config, "asdf12345@chat.example.org", "q9mr3jewvadsfaue"
)
data2 = lookup_passdb(db, "link2xt@c1.testrun.org", "Pieg9aeToe3eghuthe5u")
assert data == data2
def test_dont_overwrite_password_on_wrong_login(db, example_config):
def test_dont_overwrite_password_on_wrong_login(db):
"""Test that logging in with a different password doesn't create a new user"""
res = lookup_passdb(
db, example_config, "newuser12@chat.example.org", "kajdlkajsldk12l3kj1983"
)
res = lookup_passdb(db, "newuser1@something.org", "kajdlkajsldk12l3kj1983")
assert res["password"]
res2 = lookup_passdb(db, example_config, "newuser12@chat.example.org", "kajdslqwe")
res2 = lookup_passdb(db, "newuser1@something.org", "kajdlqweqwe")
# this function always returns a password hash, which is actually compared by dovecot.
assert res["password"] == res2["password"]
def test_nocreate_file(db, monkeypatch, tmpdir, example_config):
def test_nocreate_file(db, monkeypatch, tmpdir):
p = tmpdir.join("nocreate")
p.write("")
monkeypatch.setattr(chatmaild.doveauth, "NOCREATE_FILE", str(p))
lookup_passdb(
db, example_config, "newuser12@chat.example.org", "zequ0Aimuchoodaechik"
)
assert not get_user_data(db, example_config, "newuser12@chat.example.org")
lookup_passdb(db, "newuser1@something.org", "zequ0Aimuchoodaechik")
assert not get_user_data(db, "newuser1@something.org")
def test_db_version(db):
@@ -57,34 +45,21 @@ def test_too_high_db_version(db):
db.ensure_tables()
def test_handle_dovecot_request(db, example_config):
# Test that password can contain ", ', \ and /
def test_handle_dovecot_request(db):
msg = (
'Lshared/passdb/laksjdlaksjdlak\\\\sjdlk\\"12j\\\'3l1/k2j3123"'
"some42123@chat.example.org\tsome42123@chat.example.org"
"Lshared/passdb/laksjdlaksjdlaksjdlk12j3l1k2j3123/"
"some42@c3.testrun.org\tsome42@c3.testrun.org"
)
res = handle_dovecot_request(msg, db, example_config)
res = handle_dovecot_request(msg, db, "c3.testrun.org")
assert res
assert res[0] == "O" and res.endswith("\n")
userdata = json.loads(res[1:].strip())
assert (
userdata["home"]
== "/home/vmail/mail/chat.example.org/some42123@chat.example.org"
)
assert userdata["home"] == "/home/vmail/some42@c3.testrun.org"
assert userdata["uid"] == userdata["gid"] == "vmail"
assert userdata["password"].startswith("{SHA512-CRYPT}")
def test_handle_dovecot_protocol(db, example_config):
rfile = io.BytesIO(
b"H3\t2\t0\t\tauth\nLshared/userdb/foobar@chat.example.org\tfoobar@chat.example.org\n"
)
wfile = io.BytesIO()
handle_dovecot_protocol(rfile, wfile, db, example_config)
assert wfile.getvalue() == b"N\n"
def test_50_concurrent_lookups_different_accounts(db, gencreds, example_config):
def test_50_concurrent_lookups_different_accounts(db, gencreds):
num_threads = 50
req_per_thread = 5
results = queue.Queue()
@@ -93,7 +68,7 @@ def test_50_concurrent_lookups_different_accounts(db, gencreds, example_config):
for i in range(req_per_thread):
addr, password = gencreds()
try:
lookup_passdb(db, example_config, addr, password)
lookup_passdb(db, addr, password)
except Exception:
results.put(traceback.format_exc())
else:

View File

@@ -127,19 +127,3 @@ def test_excempt_privacy(maildata, gencreds, handler):
content = msg.as_bytes()
assert "500" in handler.check_DATA(envelope=env2)
def test_passthrough_senders(gencreds, handler, maildata):
acc1 = gencreds()[0]
to_addr = "recipient@something.org"
handler.config.passthrough_senders = [acc1]
msg = maildata("plain.eml", acc1, to_addr)
class env:
mail_from = acc1
rcpt_tos = to_addr
content = msg.as_bytes()
# assert that None/no error is returned
assert not handler.check_DATA(envelope=env)

View File

@@ -1,132 +0,0 @@
import io
from chatmaild.metadata import (
handle_dovecot_request,
handle_dovecot_protocol,
Notifier,
)
def test_handle_dovecot_request_lookup_fails():
notifier = Notifier()
res = handle_dovecot_request("Lpriv/123/chatmail", {}, notifier)
assert res == "N\n"
def test_handle_dovecot_request_happy_path():
notifier = Notifier()
transactions = {}
# lookups return the same NOTFOUND result
res = handle_dovecot_request("Lpriv/123/chatmail", transactions, notifier)
assert res == "N\n"
assert not notifier.guid2token and not transactions
# set device token in a transaction
tx = "1111"
msg = f"B{tx}\tuser"
res = handle_dovecot_request(msg, transactions, notifier)
assert not res and not notifier.guid2token
assert transactions == {tx: "O\n"}
msg = f"S{tx}\tpriv/guid00/devicetoken\t01234"
res = handle_dovecot_request(msg, transactions, notifier)
assert not res
assert len(transactions) == 1
assert len(notifier.guid2token) == 1
assert notifier.guid2token["guid00"] == "01234"
msg = f"C{tx}"
res = handle_dovecot_request(msg, transactions, notifier)
assert res == "O\n"
assert len(transactions) == 0
assert notifier.guid2token["guid00"] == "01234"
# trigger notification for incoming message
assert handle_dovecot_request(f"B{tx}\tuser", transactions, notifier) is None
msg = f"S{tx}\tpriv/guid00/messagenew"
assert handle_dovecot_request(msg, transactions, notifier) is None
assert notifier.to_notify_queue.get() == "guid00"
assert notifier.to_notify_queue.qsize() == 0
assert handle_dovecot_request(f"C{tx}\tuser", transactions, notifier) == "O\n"
assert not transactions
def test_handle_dovecot_protocol_set_devicetoken():
rfile = io.BytesIO(
b"\n".join(
[
b"HELLO",
b"Btx00\tuser",
b"Stx00\tpriv/guid00/devicetoken\t01234",
b"Ctx00",
]
)
)
wfile = io.BytesIO()
notifier = Notifier()
handle_dovecot_protocol(rfile, wfile, notifier)
assert notifier.guid2token["guid00"] == "01234"
assert wfile.getvalue() == b"O\n"
def test_handle_dovecot_protocol_messagenew():
rfile = io.BytesIO(
b"\n".join(
[
b"HELLO",
b"Btx01\tuser",
b"Stx01\tpriv/guid00/messagenew",
b"Ctx01",
]
)
)
wfile = io.BytesIO()
notifier = Notifier()
handle_dovecot_protocol(rfile, wfile, notifier)
assert wfile.getvalue() == b"O\n"
assert notifier.to_notify_queue.get() == "guid00"
assert notifier.to_notify_queue.qsize() == 0
def test_notifier_thread_run():
requests = []
class ReqMock:
def post(self, url, data, timeout):
requests.append((url, data, timeout))
class Result:
status_code = 200
return Result()
notifier = Notifier()
notifier.set_token("guid00", "01234")
notifier.new_message_for_guid("guid00")
notifier.thread_run_one(ReqMock())
url, data, timeout = requests[0]
assert data == "01234"
assert len(notifier.guid2token) == 1
def test_notifier_thread_run_gone_removes_token():
requests = []
class ReqMock:
def post(self, url, data, timeout):
requests.append((url, data, timeout))
class Result:
status_code = 410
return Result()
notifier = Notifier()
notifier.set_token("guid00", "01234")
notifier.new_message_for_guid("guid00")
assert notifier.guid2token["guid00"] == "01234"
notifier.thread_run_one(ReqMock())
url, data, timeout = requests[0]
assert data == "01234"
assert len(notifier.guid2token) == 0

View File

@@ -1,16 +0,0 @@
from chatmaild.metrics import main
def test_main(tmp_path, capsys):
for x in ("ci-asllkj", "ac_12l3kj", "qweqwe", "ci-l1k2j31l2k3"):
tmp_path.joinpath(x).mkdir()
main(tmp_path)
out, _ = capsys.readouterr()
d = {}
for line in out.split("\n"):
if line.strip():
name, num, _ = line.split()
d[name] = int(num)
assert d["accounts"] == 4
assert d["ci_accounts"] == 3

View File

@@ -4,24 +4,26 @@ import chatmaild
from chatmaild.newemail import create_newemail_dict, print_new_account
def test_create_newemail_dict(example_config):
ac1 = create_newemail_dict(example_config)
def test_create_newemail_dict():
ac1 = create_newemail_dict(domain="example.org")
assert "@" in ac1["email"]
assert len(ac1["password"]) >= 10
ac2 = create_newemail_dict(example_config)
ac2 = create_newemail_dict(domain="example.org")
assert ac1["email"] != ac2["email"]
assert ac1["password"] != ac2["password"]
def test_print_new_account(capsys, monkeypatch, maildomain, tmpdir, example_config):
monkeypatch.setattr(chatmaild.newemail, "CONFIG_PATH", str(example_config._inipath))
def test_print_new_account(capsys, monkeypatch, maildomain, tmpdir):
p = tmpdir.join("mailname")
p.write(maildomain)
monkeypatch.setattr(chatmaild.newemail, "mail_domain_path", str(p))
print_new_account()
out, err = capsys.readouterr()
lines = out.split("\n")
assert lines[0] == "Content-Type: application/json"
assert not lines[1]
dic = json.loads(lines[2])
assert dic["email"].endswith(f"@{example_config.mail_domain}")
assert dic["email"].endswith(f"@{maildomain}")
assert len(dic["password"]) >= 10

View File

@@ -1,7 +1,6 @@
"""
Chat Mail pyinfra deploy.
"""
import sys
import importlib.resources
import subprocess
@@ -85,29 +84,14 @@ def _install_remote_venv_with_chatmaild(config) -> None:
],
)
files.template(
src=importlib.resources.files(__package__).joinpath("metrics.cron.j2"),
dest="/etc/cron.d/chatmail-metrics",
user="root",
group="root",
mode="644",
config={
"mail_domain": config.mail_domain,
"execpath": f"{remote_venv_dir}/bin/chatmail-metrics",
},
)
# install systemd units
for fn in (
"doveauth",
"filtermail",
"echobot",
"chatmail-metadata",
):
params = dict(
execpath=f"{remote_venv_dir}/bin/{fn}",
config_path=remote_chatmail_inipath,
remote_venv_dir=remote_venv_dir,
)
source_path = importlib.resources.files("chatmaild").joinpath(f"{fn}.service.f")
content = source_path.read_text().format(**params).encode()
@@ -128,24 +112,10 @@ def _install_remote_venv_with_chatmaild(config) -> None:
)
def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
def _configure_opendkim(domain: str, dkim_selector: str) -> bool:
"""Configures OpenDKIM"""
need_restart = False
server.group(name="Create opendkim group", group="opendkim", system=True)
server.user(
name="Create opendkim user",
user="opendkim",
groups=["opendkim"],
system=True,
)
server.user(
name="Add postfix user to opendkim group for socket access",
user="postfix",
groups=["opendkim"],
system=True,
)
main_config = files.template(
src=importlib.resources.files(__package__).joinpath("opendkim/opendkim.conf"),
dest="/etc/opendkim.conf",
@@ -156,24 +126,6 @@ def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
)
need_restart |= main_config.changed
screen_script = files.put(
src=importlib.resources.files(__package__).joinpath("opendkim/screen.lua"),
dest="/etc/opendkim/screen.lua",
user="root",
group="root",
mode="644",
)
need_restart |= screen_script.changed
final_script = files.put(
src=importlib.resources.files(__package__).joinpath("opendkim/final.lua"),
dest="/etc/opendkim/final.lua",
user="root",
group="root",
mode="644",
)
need_restart |= final_script.changed
files.directory(
name="Add opendkim directory to /etc",
path="/etc/opendkim",
@@ -202,6 +154,7 @@ def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
config={"domain_name": domain, "opendkim_selector": dkim_selector},
)
need_restart |= signing_table.changed
files.directory(
name="Add opendkim socket directory to /var/spool/postfix",
path="/var/spool/postfix/opendkim",
@@ -211,11 +164,6 @@ def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
present=True,
)
apt.packages(
name="apt install opendkim opendkim-tools",
packages=["opendkim", "opendkim-tools"],
)
if not host.get_fact(File, f"/etc/dkimkeys/{dkim_selector}.private"):
server.shell(
name="Generate OpenDKIM domain keys",
@@ -292,31 +240,10 @@ def _configure_postfix(config: Config, debug: bool = False) -> bool:
)
need_restart |= master_config.changed
header_cleanup = files.put(
src=importlib.resources.files(__package__).joinpath(
"postfix/submission_header_cleanup"
),
dest="/etc/postfix/submission_header_cleanup",
user="root",
group="root",
mode="644",
)
need_restart |= header_cleanup.changed
# Login map that 1:1 maps email address to login.
login_map = files.put(
src=importlib.resources.files(__package__).joinpath("postfix/login_map"),
dest="/etc/postfix/login_map",
user="root",
group="root",
mode="644",
)
need_restart |= login_map.changed
return need_restart
def _configure_dovecot(config: Config, debug: bool = False) -> bool:
def _configure_dovecot(mail_server: str, debug: bool = False) -> bool:
"""Configures Dovecot IMAP server."""
need_restart = False
@@ -326,7 +253,7 @@ def _configure_dovecot(config: Config, debug: bool = False) -> bool:
user="root",
group="root",
mode="644",
config=config,
config={"hostname": mail_server},
debug=debug,
)
need_restart |= main_config.changed
@@ -338,35 +265,15 @@ def _configure_dovecot(config: Config, debug: bool = False) -> bool:
mode="644",
)
need_restart |= auth_config.changed
lua_push_notification_script = files.put(
src=importlib.resources.files(__package__).joinpath(
"dovecot/push_notification.lua"
),
dest="/etc/dovecot/push_notification.lua",
user="root",
group="root",
mode="644",
)
need_restart |= lua_push_notification_script.changed
sieve_script = files.put(
src=importlib.resources.files(__package__).joinpath(
"dovecot/default.sieve"
),
dest="/etc/dovecot/default.sieve",
user="root",
group="root",
mode="644",
)
need_restart |= sieve_script.changed
files.template(
src=importlib.resources.files(__package__).joinpath("dovecot/expunge.cron.j2"),
files.put(
src=importlib.resources.files(__package__)
.joinpath("dovecot/expunge.cron")
.open("rb"),
dest="/etc/cron.d/expunge",
user="root",
group="root",
mode="644",
config=config,
)
# as per https://doc.dovecot.org/configuration_manual/os/
@@ -439,65 +346,41 @@ def _configure_nginx(domain: str, debug: bool = False) -> bool:
return need_restart
def _remove_rspamd() -> None:
"""Remove rspamd"""
apt.packages(name="Remove rspamd", packages="rspamd", present=False)
def check_config(config):
mail_domain = config.mail_domain
if mail_domain != "testrun.org" and not mail_domain.endswith(".testrun.org"):
blocked_words = "merlinux schmieder testrun.org".split()
for key in config.__dict__:
value = config.__dict__[key]
if key.startswith("privacy") and any(x in str(value) for x in blocked_words):
for value in config.__dict__.values():
if any(x in value for x in blocked_words):
raise ValueError(
f"please set your own privacy contacts/addresses in {config._inipath}"
)
return config
def deploy_chatmail(config_path: Path) -> None:
def deploy_chatmail(mail_domain: str, mail_server: str, dkim_selector: str) -> None:
"""Deploy a chat-mail instance.
:param config_path: path to chatmail.ini
:param mail_domain: domain part of your future email addresses
:param mail_server: the DNS name under which your mail server is reachable
:param dkim_selector:
"""
config = read_config(config_path)
check_config(config)
mail_domain = config.mail_domain
from .www import build_webpages
apt.update(name="apt update", cache_time=24 * 3600)
server.group(name="Create vmail group", group="vmail", system=True)
server.user(name="Create vmail user", user="vmail", group="vmail", system=True)
# Run local DNS resolver `unbound`.
# `resolvconf` takes care of setting up /etc/resolv.conf
# to use 127.0.0.1 as the resolver.
apt.packages(
name="Install unbound",
packages=["unbound", "unbound-anchor", "dnsutils"],
)
server.shell(
name="Generate root keys for validating DNSSEC",
commands=[
"unbound-anchor -a /var/lib/unbound/root.key || true",
"systemctl reset-failed unbound.service",
],
)
systemd.service(
name="Start and enable unbound",
service="unbound.service",
running=True,
enabled=True,
server.group(name="Create opendkim group", group="opendkim", system=True)
server.user(
name="Add postfix user to opendkim group for socket access",
user="postfix",
groups=["opendkim"],
system=True,
)
# Deploy acmetool to have TLS certificates.
deploy_acmetool(
nginx_hook=True,
domains=[mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"],
)
deploy_acmetool(nginx_hook=True, domains=[mail_server, f"mta-sts.{mail_server}"])
apt.packages(
name="Install Postfix",
@@ -506,7 +389,15 @@ def deploy_chatmail(config_path: Path) -> None:
apt.packages(
name="Install Dovecot",
packages=["dovecot-imapd", "dovecot-lmtpd", "dovecot-sieve"],
packages=["dovecot-imapd", "dovecot-lmtpd"],
)
apt.packages(
name="Install OpenDKIM",
packages=[
"opendkim",
"opendkim-tools",
],
)
apt.packages(
@@ -519,7 +410,11 @@ def deploy_chatmail(config_path: Path) -> None:
packages=["fcgiwrap"],
)
www_path = importlib.resources.files(__package__).joinpath("../../../www").resolve()
pkg_root = importlib.resources.files(__package__)
chatmail_ini = pkg_root.joinpath("../../../chatmail.ini").resolve()
config = read_config(chatmail_ini)
check_config(config)
www_path = pkg_root.joinpath("../../../www").resolve()
build_dir = www_path.joinpath("build")
src_dir = www_path.joinpath("src")
@@ -528,14 +423,12 @@ def deploy_chatmail(config_path: Path) -> None:
_install_remote_venv_with_chatmaild(config)
debug = False
dovecot_need_restart = _configure_dovecot(config, debug=debug)
dovecot_need_restart = _configure_dovecot(mail_server, debug=debug)
postfix_need_restart = _configure_postfix(config, debug=debug)
opendkim_need_restart = _configure_opendkim(mail_domain, dkim_selector)
mta_sts_need_restart = _install_mta_sts_daemon()
nginx_need_restart = _configure_nginx(mail_domain)
_remove_rspamd()
opendkim_need_restart = _configure_opendkim(mail_domain, "opendkim")
systemd.service(
name="Start and enable OpenDKIM",
service="opendkim.service",

View File

@@ -1,8 +1,6 @@
import importlib.resources
from pyinfra.operations import apt, files, systemd, server
from pyinfra import host
from pyinfra.facts.systemd import SystemdStatus
from pyinfra.operations import apt, files, server
def deploy_acmetool(nginx_hook=False, email="", domains=[]):
@@ -48,30 +46,6 @@ def deploy_acmetool(nginx_hook=False, email="", domains=[]):
mode="644",
)
service_file = files.put(
src=importlib.resources.files(__package__).joinpath(
"acmetool-redirector.service"
),
dest="/etc/systemd/system/acmetool-redirector.service",
user="root",
group="root",
mode="644",
)
if host.get_fact(SystemdStatus).get("nginx.service"):
systemd.service(
name="Stop nginx service to free port 80",
service="nginx",
running=False,
)
systemd.service(
name="Setup acmetool-redirector service",
service="acmetool-redirector.service",
running=True,
enabled=True,
restarted=service_file.changed,
)
server.shell(
name=f"Request certificate for: { ', '.join(domains) }",
commands=[f"acmetool want { ' '.join(domains)}"],

View File

@@ -1,11 +0,0 @@
[Unit]
Description=acmetool HTTP redirector
[Service]
Type=notify
ExecStart=/usr/bin/acmetool redirector --service.uid=daemon
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target

View File

@@ -1,15 +1,12 @@
{chatmail_domain}. A {ipv4}
{chatmail_domain}. AAAA {ipv6}
{chatmail_domain}. MX 10 {chatmail_domain}.
_submission._tcp.{chatmail_domain}. SRV 0 1 587 {chatmail_domain}.
_submissions._tcp.{chatmail_domain}. SRV 0 1 465 {chatmail_domain}.
_imap._tcp.{chatmail_domain}. SRV 0 1 143 {chatmail_domain}.
_imaps._tcp.{chatmail_domain}. SRV 0 1 993 {chatmail_domain}.
{chatmail_domain}. CAA 128 issue "letsencrypt.org;accounturi={acme_account_url}"
{chatmail_domain}. IN CAA 128 issue "letsencrypt.org;accounturi={acme_account_url}"
{chatmail_domain}. TXT "v=spf1 a:{chatmail_domain} -all"
_dmarc.{chatmail_domain}. TXT "v=DMARC1;p=reject;adkim=s;aspf=s"
_mta-sts.{chatmail_domain}. TXT "v=STSv1; id={sts_id}"
mta-sts.{chatmail_domain}. CNAME {chatmail_domain}.
www.{chatmail_domain}. CNAME {chatmail_domain}.
_dmarc.{chatmail_domain}. TXT "v=DMARC1;p=reject;rua=mailto:{email};ruf=mailto:{email};fo=1;adkim=r;aspf=r"
_mta-sts.{chatmail_domain}. IN TXT "v=STSv1; id={sts_id}"
mta-sts.{chatmail_domain}. IN CNAME {chatmail_domain}.
_smtp._tls.{chatmail_domain}. IN TXT "v=TLSRPTv1;rua=mailto:{email}"
{dkim_entry}
_adsp._domainkey.{chatmail_domain}. TXT "dkim=discardable"

View File

@@ -2,8 +2,8 @@
Provides the `cmdeploy` entry point function,
along with command line option and subcommand parsing.
"""
import argparse
import datetime
import shutil
import subprocess
import importlib.resources
@@ -15,7 +15,6 @@ from pathlib import Path
from termcolor import colored
from chatmaild.config import read_config, write_initial_config
from cmdeploy.dns import show_dns, check_necessary_dns
#
@@ -33,16 +32,11 @@ def init_cmd_options(parser):
def init_cmd(args, out):
"""Initialize chatmail config file."""
mail_domain = args.chatmail_domain
if args.inipath.exists():
print(f"Path exists, not modifying: {args.inipath}")
else:
write_initial_config(args.inipath, mail_domain)
out.green(f"created config file for {mail_domain} in {args.inipath}")
check_necessary_dns(
out,
mail_domain,
)
out.red(f"Path exists, not modifying: {args.inipath}")
raise SystemExit(1)
write_initial_config(args.inipath, args.chatmail_domain)
out.green(f"created config file for {args.chatmail_domain} in {args.inipath}")
def run_cmd_options(parser):
@@ -56,35 +50,47 @@ def run_cmd_options(parser):
def run_cmd(args, out):
"""Deploy chatmail services on the remote server."""
mail_domain = args.config.mail_domain
if not check_necessary_dns(
out,
mail_domain,
):
sys.exit(1)
env = os.environ.copy()
env["CHATMAIL_INI"] = args.inipath
deploy_path = importlib.resources.files(__package__).joinpath("deploy.py").resolve()
env["CHATMAIL_DOMAIN"] = args.config.mail_domain
deploy_path = "cmdeploy/src/cmdeploy/deploy.py"
pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
cmd = f"{pyinf} --ssh-user root {args.config.mail_domain} {deploy_path}"
out.check_call(cmd, env=env)
print("Deploy completed, call `cmdeploy dns` next.")
def dns_cmd_options(parser):
parser.add_argument(
"--zonefile",
dest="zonefile",
help="print the whole zonefile for deploying directly",
)
def dns_cmd(args, out):
"""Generate dns zone file."""
exit_code = show_dns(args, out)
exit(exit_code)
template = importlib.resources.files(__package__).joinpath("chatmail.zone.f")
ssh = f"ssh root@{args.config.mail_domain}"
def read_dkim_entries(entry):
lines = []
for line in entry.split("\n"):
if line.startswith(";") or not line.strip():
continue
line = line.replace("\t", " ")
lines.append(line)
return "\n".join(lines)
acme_account_url = out.shell_output(f"{ssh} -- acmetool account-url")
dkim_entry = read_dkim_entries(out.shell_output(f"{ssh} -- opendkim-genzone -F"))
out(
f"[writing {args.config.mail_domain} zone data (using space as separator) to stdout output]",
green=True,
)
print(
template.read_text()
.format(
acme_account_url=acme_account_url,
email=f"root@{args.config.mail_domain}",
sts_id=datetime.datetime.now().strftime("%Y%m%d%H%M"),
chatmail_domain=args.config.mail_domain,
dkim_entry=dkim_entry,
)
.strip()
)
def status_cmd(args, out):
@@ -104,15 +110,6 @@ def status_cmd(args, out):
print(line)
def test_cmd_options(parser):
parser.add_argument(
"--slow",
dest="slow",
action="store_true",
help="also run slow tests",
)
def test_cmd(args, out):
"""Run local and online tests for chatmail deployment.
@@ -124,18 +121,9 @@ def test_cmd(args, out):
out.check_call(f"{sys.executable} -m pip install deltachat")
pytest_path = shutil.which("pytest")
pytest_args = [
pytest_path,
"cmdeploy/src/",
"-n4",
"-rs",
"-x",
"-vrx",
"--durations=5",
]
if args.slow:
pytest_args.append("--slow")
ret = out.run_ret(pytest_args)
ret = out.run_ret(
[pytest_path, "cmdeploy/src/", "-n4", "-rs", "-x", "-vrx", "--durations=5"]
)
return ret
@@ -157,7 +145,7 @@ def fmt_cmd_options(parser):
def fmt_cmd(args, out):
"""Run formattting fixes (ruff and black) on all chatmail source code."""
"""Run formattting fixes (fuff and black) on all chatmail source code."""
sources = [str(importlib.resources.files(x)) for x in ("chatmaild", "cmdeploy")]
black_args = [shutil.which("black")]
@@ -213,15 +201,9 @@ class Out:
color = "red" if red else ("green" if green else None)
print(colored(msg, color), file=file)
def shell_output(self, arg, no_print=False, timeout=10):
if not no_print:
self(f"[$ {arg}]", file=sys.stderr)
output = subprocess.STDOUT
else:
output = subprocess.DEVNULL
return subprocess.check_output(
arg, shell=True, timeout=timeout, stderr=output
).decode()
def shell_output(self, arg):
self(f"[$ {arg}]", file=sys.stderr)
return subprocess.check_output(arg, shell=True).decode()
def check_call(self, arg, env=None, quiet=False):
if not quiet:

View File

@@ -1,16 +1,18 @@
import os
import importlib.resources
import pyinfra
from cmdeploy import deploy_chatmail
def main():
config_path = os.getenv(
"CHATMAIL_INI",
importlib.resources.files("cmdeploy").joinpath("../../../chatmail.ini"),
)
mail_domain = os.getenv("CHATMAIL_DOMAIN")
mail_server = os.getenv("CHATMAIL_SERVER", mail_domain)
dkim_selector = os.getenv("CHATMAIL_DKIM_SELECTOR", "dkim")
deploy_chatmail(config_path)
assert mail_domain
assert mail_server
assert dkim_selector
deploy_chatmail(mail_domain, mail_server, dkim_selector)
if pyinfra.is_cli:

View File

@@ -1,205 +0,0 @@
import sys
import requests
import importlib
import subprocess
import datetime
from typing import Optional
class DNS:
def __init__(self, out, mail_domain):
self.session = requests.Session()
self.out = out
self.ssh = f"ssh root@{mail_domain} -- "
try:
self.shell(f"unbound-control flush_zone {mail_domain}")
except subprocess.CalledProcessError:
pass
def shell(self, cmd):
try:
return self.out.shell_output(f"{self.ssh}{cmd}", no_print=True)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired) as e:
if "exit status 255" in str(e) or "timed out" in str(e):
self.out.red(f"Error: can't reach the server with: {self.ssh[:-4]}")
sys.exit(1)
else:
raise
def get_ipv4(self):
cmd = "ip a | grep 'inet ' | grep 'scope global' | grep -oE '[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}' | head -1"
return self.shell(cmd).strip()
def get_ipv6(self):
cmd = "ip a | grep inet6 | grep 'scope global' | sed -e 's#/64 scope global##' | sed -e 's#inet6##'"
return self.shell(cmd).strip()
def get(self, typ: str, domain: str) -> str:
"""Get a DNS entry or empty string if there is none."""
dig_result = self.shell(f"dig -r -q {domain} -t {typ} +short")
line = dig_result.partition("\n")[0]
return line
def check_ptr_record(self, ip: str, mail_domain) -> bool:
"""Check the PTR record for an IPv4 or IPv6 address."""
result = self.shell(f"dig -r -x {ip} +short").rstrip()
return result == f"{mail_domain}."
def show_dns(args, out) -> int:
"""Check existing DNS records, optionally write them to zone file, return exit code 0 or 1."""
template = importlib.resources.files(__package__).joinpath("chatmail.zone.f")
mail_domain = args.config.mail_domain
ssh = f"ssh root@{mail_domain}"
dns = DNS(out, mail_domain)
print("Checking your DKIM keys and DNS entries...")
try:
acme_account_url = out.shell_output(f"{ssh} -- acmetool account-url")
except subprocess.CalledProcessError:
print("Please run `cmdeploy run` first.")
return 1
dkim_selector = "opendkim"
dkim_pubkey = out.shell_output(
ssh + f" -- openssl rsa -in /etc/dkimkeys/{dkim_selector}.private"
" -pubout 2>/dev/null | awk '/-/{next}{printf(\"%s\",$0)}'"
)
dkim_entry_value = f"v=DKIM1;k=rsa;p={dkim_pubkey};s=email;t=s"
dkim_entry_str = ""
while len(dkim_entry_value) >= 255:
dkim_entry_str += '"' + dkim_entry_value[:255] + '" '
dkim_entry_value = dkim_entry_value[255:]
dkim_entry_str += '"' + dkim_entry_value + '"'
dkim_entry = f"{dkim_selector}._domainkey.{mail_domain}. TXT {dkim_entry_str}"
ipv6 = dns.get_ipv6()
reverse_ipv6 = dns.check_ptr_record(ipv6, mail_domain)
ipv4 = dns.get_ipv4()
reverse_ipv4 = dns.check_ptr_record(ipv4, mail_domain)
to_print = []
with open(template, "r") as f:
zonefile = (
f.read()
.format(
acme_account_url=acme_account_url,
sts_id=datetime.datetime.now().strftime("%Y%m%d%H%M"),
chatmail_domain=args.config.mail_domain,
dkim_entry=dkim_entry,
ipv6=ipv6,
ipv4=ipv4,
)
.strip()
)
try:
with open(args.zonefile, "w+") as zf:
zf.write(zonefile)
print(f"DNS records successfully written to: {args.zonefile}")
return 0
except TypeError:
pass
for line in zonefile.splitlines():
line = line.format(
acme_account_url=acme_account_url,
sts_id=datetime.datetime.now().strftime("%Y%m%d%H%M"),
chatmail_domain=args.config.mail_domain,
dkim_entry=dkim_entry,
ipv6=ipv6,
).strip()
for typ in ["A", "AAAA", "CNAME", "CAA"]:
if f" {typ} " in line:
domain, value = line.split(f" {typ} ")
current = dns.get(typ, domain.strip()[:-1])
if current != value.strip():
to_print.append(line)
if " MX " in line:
domain, typ, prio, value = line.split()
current = dns.get(typ, domain[:-1])
if not current:
to_print.append(line)
elif current.split()[1] != value:
print(line.replace(prio, str(int(current[0]) + 1)))
if " SRV " in line:
domain, typ, prio, weight, port, value = line.split()
current = dns.get("SRV", domain[:-1])
if current != f"{prio} {weight} {port} {value}":
to_print.append(line)
if " TXT " in line:
domain, value = line.split(" TXT ")
current = dns.get("TXT", domain.strip()[:-1])
if domain.startswith("_mta-sts."):
if current:
if current.split("id=")[0] == value.split("id=")[0]:
continue
# TXT records longer than 255 bytes
# are split into multiple <character-string>s.
# This typically happens with DKIM record
# which contains long RSA key.
#
# Removing `" "` before comparison
# to get back a single string.
if current.replace('" "', "") != value.replace('" "', ""):
to_print.append(line)
exit_code = 0
if to_print:
to_print.insert(
0, "You should configure the following DNS entries at your provider:\n"
)
to_print.append(
"\nIf you already configured the DNS entries, wait a bit until the DNS entries propagate to the Internet."
)
print("\n".join(to_print))
exit_code = 1
else:
out.green("Great! All your DNS entries are correct.")
to_print = []
if not reverse_ipv4:
to_print.append(f"\tIPv4:\t{ipv4}\t{args.config.mail_domain}")
if not reverse_ipv6:
to_print.append(f"\tIPv6:\t{ipv6}\t{args.config.mail_domain}")
if len(to_print) > 0:
if len(to_print) == 1:
warning = "You should add the following PTR/reverse DNS entry:"
else:
warning = "You should add the following PTR/reverse DNS entries:"
out.red(warning)
for entry in to_print:
print(entry)
print(
"You can do so at your hosting provider (maybe this isn't your DNS provider)."
)
exit_code = 1
return exit_code
def check_necessary_dns(out, mail_domain):
"""Check whether $mail_domain and mta-sts.$mail_domain resolve."""
dns = DNS(out, mail_domain)
ipv4 = dns.get("A", mail_domain)
ipv6 = dns.get("AAAA", mail_domain)
mta_entry = dns.get("CNAME", "mta-sts." + mail_domain)
www_entry = dns.get("CNAME", "www." + mail_domain)
to_print = []
if not (ipv4 or ipv6):
to_print.append(f"\t{mail_domain}.\t\t\tA<your server's IPv4 address>")
if mta_entry != mail_domain + ".":
to_print.append(f"\tmta-sts.{mail_domain}.\tCNAME\t{mail_domain}.")
if www_entry != mail_domain + ".":
to_print.append(f"\twww.{mail_domain}.\tCNAME\t{mail_domain}.")
if to_print:
to_print.insert(
0,
"\nFor chatmail to work, you need to configure this at your DNS provider:\n",
)
for line in to_print:
print(line)
print()
else:
dns.out.green("\nAll necessary DNS entries seem to be set.")
return True

View File

@@ -1,10 +1,5 @@
uri = proxy:/run/dovecot/doveauth.socket:auth
iterate_disable = yes
default_pass_scheme = plain
# %E escapes characters " (double quote), ' (single quote) and \ (backslash) with \ (backslash).
# See <https://doc.dovecot.org/configuration_manual/config_file/config_variables/#modifiers>
# for documentation.
#
# We escape user-provided input and use double quote as a separator.
password_key = passdb/%Ew"%Eu
user_key = userdb/%Eu
password_key = passdb/%w/%u
user_key = userdb/%u

View File

@@ -1,5 +0,0 @@
require ["imap4flags"];
if header :is ["Auto-Submitted"] ["auto-replied", "auto-generated"] {
addflag "$Auto";
}

View File

@@ -13,15 +13,13 @@ auth_cache_size = 100M
mail_debug = yes
{% endif %}
mail_server_admin = mailto:root@{{ config.mail_domain }}
mail_server_comment = Chatmail server
mail_plugins = quota
# these are the capabilities Delta Chat cares about actually
# so let's keep the network overhead per login small
# https://github.com/deltachat/deltachat-core-rust/blob/master/src/imap/capabilities.rs
imap_capability = IMAP4rev1 IDLE MOVE QUOTA CONDSTORE NOTIFY METADATA XDELTAPUSH
imap_capability = IMAP4rev1 IDLE MOVE QUOTA CONDSTORE NOTIFY
# Authentication for system users.
@@ -71,32 +69,14 @@ mail_privileged_group = vmail
## Mail processes
##
# Pass all IMAP METADATA requests to the server implementing Dovecot's dict protocol.
mail_attribute_dict = proxy:/run/dovecot/metadata.socket:metadata
# Enable IMAP COMPRESS (RFC 4978).
# <https://datatracker.ietf.org/doc/html/rfc4978.html>
protocol imap {
mail_plugins = $mail_plugins imap_zlib imap_quota
imap_metadata = yes
}
protocol lmtp {
# quota plugin documentation:
# <https://doc.dovecot.org/configuration_manual/quota_plugin/>
#
# notify plugin is a dependency of push_notification plugin:
# <https://doc.dovecot.org/settings/plugin/notify-plugin/>
#
# push_notification plugin documentation:
# <https://doc.dovecot.org/configuration_manual/push_notification/>
#
# mail_lua and push_notification_lua are needed for Lua push notification handler.
# <https://doc.dovecot.org/configuration_manual/push_notification/#configuration>
#
# Sieve to mark messages that should not be notified as \Seen
# <https://doc.dovecot.org/configuration_manual/sieve/configuration/>
mail_plugins = $mail_plugins quota mail_lua notify push_notification push_notification_lua sieve
mail_plugins = $mail_plugins quota
}
plugin {
@@ -106,21 +86,13 @@ plugin {
plugin {
# for now we define static quota-rules for all users
quota = maildir:User quota
quota_rule = *:storage={{ config.max_mailbox_size }}
quota_rule = *:storage=100M
quota_max_mail_size=30M
quota_grace = 0
# quota_over_flag_value = TRUE
}
# push_notification configuration
plugin {
# <https://doc.dovecot.org/configuration_manual/push_notification/#lua-lua>
push_notification_driver = lua:file=/etc/dovecot/push_notification.lua
}
plugin {
sieve_default = file:/etc/dovecot/default.sieve
}
service lmtp {
user=vmail
@@ -165,8 +137,8 @@ service imap-login {
}
ssl = required
ssl_cert = </var/lib/acme/live/{{ config.mail_domain }}/fullchain
ssl_key = </var/lib/acme/live/{{ config.mail_domain }}/privkey
ssl_cert = </var/lib/acme/live/{{ config.hostname }}/fullchain
ssl_key = </var/lib/acme/live/{{ config.hostname }}/privkey
ssl_dh = </usr/share/dovecot/dh.pem
ssl_min_protocol = TLSv1.2
ssl_prefer_server_ciphers = yes

View File

@@ -0,0 +1,4 @@
2 0 * * * dovecot doveadm expunge -A SEEN BEFORE 40d INBOX
2 0 * * * dovecot doveadm expunge -A SEEN BEFORE 40d Deltachat
2 0 * * * dovecot doveadm expunge -A SEEN BEFORE 40d Trash
2 30 * * * dovecot doveadm purge -A

View File

@@ -1,10 +0,0 @@
# delete all mails after {{ config.delete_mails_after }} days, in the Inbox
2 0 * * * vmail find /home/vmail/mail/{{ config.mail_domain }} -path '*/cur/*' -mtime +{{ config.delete_mails_after }} -type f -delete
# or in any IMAP subfolder
2 0 * * * vmail find /home/vmail/mail/{{ config.mail_domain }} -path '*/.*/cur/*' -mtime +{{ config.delete_mails_after }} -type f -delete
# even if they are unseen
2 0 * * * vmail find /home/vmail/mail/{{ config.mail_domain }} -path '*/new/*' -mtime +{{ config.delete_mails_after }} -type f -delete
2 0 * * * vmail find /home/vmail/mail/{{ config.mail_domain }} -path '*/.*/new/*' -mtime +{{ config.delete_mails_after }} -type f -delete
# or only temporary (but then they shouldn't be around after {{ config.delete_mails_after }} days anyway).
2 0 * * * vmail find /home/vmail/mail/{{ config.mail_domain }} -path '*/tmp/*' -mtime +{{ config.delete_mails_after }} -type f -delete
2 0 * * * vmail find /home/vmail/mail/{{ config.mail_domain }} -path '*/.*/tmp/*' -mtime +{{ config.delete_mails_after }} -type f -delete

View File

@@ -1,32 +0,0 @@
function dovecot_lua_notify_begin_txn(user)
return user
end
function contains(v, needle)
for _, keyword in ipairs(v) do
if keyword == needle then
return true
end
end
return false
end
function dovecot_lua_notify_event_message_new(user, event)
local mbox = user:mailbox(event.mailbox)
mbox:sync()
if user.username ~= event.from_address then
-- Incoming message
if not contains(event.keywords, "$Auto") then
-- Not an Auto-Submitted message, notifying.
-- Notify METADATA server about new message.
mbox:metadata_set("/private/messagenew", "")
end
end
mbox:free()
end
function dovecot_lua_notify_end_txn(ctx, success)
end

View File

@@ -6,7 +6,7 @@ import io
def gen_qr_png_data(maildomain):
url = f"DCACCOUNT:https://{maildomain}/new"
url = f"DCACCOUNT:https://{maildomain}/cgi-bin/newemail.py"
image = gen_qr(maildomain, url)
temp = io.BytesIO()
image.save(temp, format="png")

View File

@@ -1 +0,0 @@
*/5 * * * * root {{ config.execpath }} /home/vmail/mail/{{ config.mail_domain }} >/var/www/html/metrics

View File

@@ -41,44 +41,15 @@ http {
try_files $uri $uri/ =404;
}
location /metrics {
default_type text/plain;
}
location /new {
if ($request_method = GET) {
# Redirect to Delta Chat,
# which will in turn do a POST request.
return 301 dcaccount:https://{{ config.domain_name }}/new;
}
fastcgi_pass unix:/run/fcgiwrap.socket;
include /etc/nginx/fastcgi_params;
fastcgi_param SCRIPT_FILENAME /usr/lib/cgi-bin/newemail.py;
}
# Old URL for compatibility with e.g. printed QR codes.
#
# Copy-paste instead of redirect to /new
# because Delta Chat core does not follow redirects.
#
# Redirects are only for browsers.
location /cgi-bin/newemail.py {
if ($request_method = GET) {
return 301 dcaccount:https://{{ config.domain_name }}/new;
}
fastcgi_pass unix:/run/fcgiwrap.socket;
include /etc/nginx/fastcgi_params;
fastcgi_param SCRIPT_FILENAME /usr/lib/cgi-bin/newemail.py;
}
# add cgi-bin support
include /usr/share/doc/fcgiwrap/examples/nginx.conf;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
# Redirect www. to non-www
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name www.{{ config.domain_name }};
return 301 $scheme://{{ config.domain_name }}$request_uri;
}
return 301 https://$host$request_uri;
}
}

View File

@@ -1 +1 @@
{{ config.opendkim_selector }}._domainkey.{{ config.domain_name }} {{ config.domain_name }}:{{ config.opendkim_selector }}:/etc/dkimkeys/{{ config.opendkim_selector }}.private
dkim._domainkey.{{ config.domain_name }} {{ config.domain_name }}:{{ config.opendkim_selector }}:/etc/dkimkeys/dkim.private

View File

@@ -1,28 +0,0 @@
if odkim.internal_ip(ctx) == 1 then
-- Outgoing message will be signed,
-- no need to look for signatures.
return nil
end
nsigs = odkim.get_sigcount(ctx)
if nsigs == nil then
return nil
end
for i = 1, nsigs do
sig = odkim.get_sighandle(ctx, i - 1)
sigres = odkim.sig_result(sig)
-- All signatures that do not correspond to From:
-- were ignored in screen.lua and return sigres -1.
--
-- Any valid signature that was not ignored like this
-- means the message is acceptable.
if sigres == 0 then
return nil
end
end
odkim.set_reply(ctx, "554", "5.7.1", "No valid DKIM signature found")
odkim.set_result(ctx, SMFIS_REJECT)
return nil

View File

@@ -8,12 +8,10 @@ SyslogSuccess yes
# oversigned, because it is often the identity key used by reputation systems
# and thus somewhat security sensitive.
Canonicalization relaxed/simple
#Mode sv
#SubDomains no
OversignHeaders From
On-BadSignature reject
On-KeyNotFound reject
On-NoSignature reject
# Signing domain, selector, and key (required). For example, perform signing
# for domain "example.com" with selector "2020" (2020._domainkey.example.com),
# using the private key stored in /etc/dkimkeys/example.private. More granular
@@ -24,15 +22,6 @@ KeyFile /etc/dkimkeys/{{ config.opendkim_selector }}.private
KeyTable /etc/dkimkeys/KeyTable
SigningTable refile:/etc/dkimkeys/SigningTable
# Sign Autocrypt header in addition to the default specified in RFC 6376.
SignHeaders *,+autocrypt
# Script to ignore signatures that do not correspond to the From: domain.
ScreenPolicyScript /etc/opendkim/screen.lua
# Script to reject mails without a valid DKIM signature.
FinalPolicyScript /etc/opendkim/final.lua
# In Debian, opendkim runs as user "opendkim". A umask of 007 is required when
# using a local socket with MTAs that access the socket as a non-privileged
# user (for example, Postfix). You may need to add user "postfix" to group
@@ -40,10 +29,22 @@ FinalPolicyScript /etc/opendkim/final.lua
UserID opendkim
UMask 007
# Socket for the MTA connection (required). If the MTA is inside a chroot jail,
# it must be ensured that the socket is accessible. In Debian, Postfix runs in
# a chroot in /var/spool/postfix, therefore a Unix socket would have to be
# configured as shown on the last line below.
#Socket local:/run/opendkim/opendkim.sock
#Socket inet:8891@localhost
#Socket inet:8891
Socket local:/var/spool/postfix/opendkim/opendkim.sock
PidFile /run/opendkim/opendkim.pid
# Hosts for which to sign rather than verify, default is 127.0.0.1. See the
# OPERATION section of opendkim(8) for more information.
#InternalHosts 192.168.0.0/16, 10.0.0.0/8, 172.16.0.0/12
# The trust anchor enables DNSSEC. In Debian, the trust anchor file is provided
# by the package dns-root-data.
TrustAnchorFile /usr/share/dns/root.key
#Nameservers 127.0.0.1

View File

@@ -1,21 +0,0 @@
-- Ignore signatures that do not correspond to the From: domain.
from_domain = odkim.get_fromdomain(ctx)
if from_domain == nil then
return nil
end
n = odkim.get_sigcount(ctx)
if n == nil then
return nil
end
for i = 1, n do
sig = odkim.get_sighandle(ctx, i - 1)
sig_domain = odkim.sig_getdomain(sig)
if from_domain ~= sig_domain then
odkim.sig_ignore(sig)
end
end
return nil

View File

@@ -1 +0,0 @@
/^(.*)$/ ${1}

View File

@@ -11,8 +11,9 @@ append_dot_mydomain = no
readme_directory = no
# See http://www.postfix.org/COMPATIBILITY_README.html
compatibility_level = 3.6
# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 2 on
# fresh installs.
compatibility_level = 2
# TLS parameters
smtpd_tls_cert_file=/var/lib/acme/live/{{ config.mail_domain }}/fullchain
@@ -23,31 +24,6 @@ smtp_tls_CApath=/etc/ssl/certs
smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_policy_maps = socketmap:inet:127.0.0.1:8461:postfix
smtpd_tls_protocols = >=TLSv1.2
# Disable anonymous cipher suites
# and known insecure algorithms.
#
# Disabling anonymous ciphers
# does not generally improve security
# because clients that want to verify certificate
# will not select them anyway,
# but makes cipher suite list shorter and security scanners happy.
# See <https://www.postfix.org/TLS_README.html> for discussion.
#
# Only ancient insecure ciphers should be disabled here
# as MTA clients that do not support more secure cipher
# likely do not support MTA-STS either and will
# otherwise fall back to using plaintext connection.
smtpd_tls_exclude_ciphers = aNULL, RC4, MD5, DES
# Override client's preference order.
# <https://www.postfix.org/postconf.5.html#tls_preempt_cipherlist>
#
# This is mostly to ensure cipher suites with forward secrecy
# are preferred over non cipher suites without forward secrecy.
# See <https://www.postfix.org/FORWARD_SECRECY_README.html#server_fs>.
tls_preempt_cipherlist = yes
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = {{ config.mail_domain }}
@@ -71,9 +47,5 @@ inet_protocols = all
virtual_transport = lmtp:unix:private/dovecot-lmtp
virtual_mailbox_domains = {{ config.mail_domain }}
mua_client_restrictions = permit_sasl_authenticated, reject
mua_sender_restrictions = reject_sender_login_mismatch, permit_sasl_authenticated, reject
mua_helo_restrictions = permit_mynetworks, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, permit
# 1:1 map MAIL FROM to SASL login name.
smtpd_sender_login_maps = regexp:/etc/postfix/login_map
smtpd_milters = unix:opendkim/opendkim.sock
non_smtpd_milters = $smtpd_milters

View File

@@ -11,10 +11,13 @@
# ==========================================================================
{% if debug == true %}
smtp inet n - y - - smtpd -v
{%- else %}
{% else %}
smtp inet n - y - - smtpd
{%- endif %}
-o smtpd_milters=unix:opendkim/opendkim.sock
{% endif %}
#smtp inet n - y - 1 postscreen
#smtpd pass - - y - - smtpd
#dnsblog unix - - y - 0 dnsblog
#tlsproxy unix - - y - 0 tlsproxy
submission inet n - y - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
@@ -31,7 +34,6 @@ submission inet n - y - - smtpd
-o milter_macro_daemon_name=ORIGINATING
-o smtpd_client_connection_count_limit=1000
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
-o cleanup_service_name=authclean
smtps inet n - y - - smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
@@ -48,7 +50,6 @@ smtps inet n - y - - smtpd
-o smtpd_client_connection_count_limit=1000
-o milter_macro_daemon_name=ORIGINATING
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
-o cleanup_service_name=authclean
#628 inet n - y - - qmqpd
pickup unix n - y 60 1 pickup
cleanup unix n - y - 0 cleanup
@@ -79,14 +80,3 @@ filter unix - n n - - lmtp
# Local SMTP server for reinjecting filered mail.
localhost:{{ config.postfix_reinject_port }} inet n - n - 10 smtpd
-o syslog_name=postfix/reinject
-o smtpd_milters=unix:opendkim/opendkim.sock
-o cleanup_service_name=authclean
# Cleanup `Received` headers for authenticated mail
# to avoid leaking client IP.
#
# We do not do this for received mails
# as this will break DKIM signatures
# if `Received` header is signed.
authclean unix n - - - 0 cleanup
-o header_checks=regexp:/etc/postfix/submission_header_cleanup

View File

@@ -1,4 +0,0 @@
/^Received:/ IGNORE
/^X-Originating-IP:/ IGNORE
/^X-Mailer:/ IGNORE
/^User-Agent:/ IGNORE

View File

@@ -30,31 +30,31 @@ def test_login_smtp(benchmark, smtp, gencreds):
class TestDC:
def test_autoconfigure(self, benchmark, cmfactory):
def dc_autoconfig_and_idle_ready():
def autoconfig_and_idle_ready():
cmfactory.get_online_accounts(1)
benchmark(dc_autoconfig_and_idle_ready, 5)
benchmark(autoconfig_and_idle_ready, 5)
def test_ping_pong(self, benchmark, cmfactory):
ac1, ac2 = cmfactory.get_online_accounts(2)
chat = cmfactory.get_accepted_chat(ac1, ac2)
def dc_ping_pong():
def ping_pong():
chat.send_text("ping")
msg = ac2.wait_next_incoming_message()
msg.chat.send_text("pong")
ac1.wait_next_incoming_message()
benchmark(dc_ping_pong, 5)
benchmark(ping_pong, 5)
def test_send_10_receive_10(self, benchmark, cmfactory, lp):
ac1, ac2 = cmfactory.get_online_accounts(2)
chat = cmfactory.get_accepted_chat(ac1, ac2)
def dc_send_10_receive_10():
def send_10_receive_10():
for i in range(10):
chat.send_text(f"hello {i}")
for i in range(10):
ac2.wait_next_incoming_message()
benchmark(dc_send_10_receive_10, 5)
benchmark(send_10_receive_10, 5)

View File

@@ -1,17 +1,6 @@
import pytest
import threading
import queue
import socket
from chatmaild.config import read_config
from cmdeploy.cmdeploy import main
def test_init(tmp_path, maildomain):
inipath = tmp_path.joinpath("chatmail.ini")
main(["init", "--config", str(inipath), maildomain])
config = read_config(inipath)
assert config.mail_domain == maildomain
def test_login_basic_functioning(imap_or_smtp, gencreds, lp):
@@ -79,24 +68,3 @@ def test_concurrent_logins_same_account(
for _ in conns:
assert login_results.get()
def test_no_vrfy(chatmail_config):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((chatmail_config.mail_domain, 25))
banner = sock.recv(1024)
print(banner)
sock.send(b"VRFY wrongaddress@%s\r\n" % (chatmail_config.mail_domain.encode(),))
result = sock.recv(1024)
print(result)
sock.send(b"VRFY echo@%s\r\n" % (chatmail_config.mail_domain.encode(),))
result2 = sock.recv(1024)
print(result2)
assert result[0:10] == result2[0:10]
sock.send(b"VRFY wrongaddress\r\n")
result = sock.recv(1024)
print(result)
sock.send(b"VRFY echo\r\n")
result2 = sock.recv(1024)
print(result2)
assert result[0:10] == result2[0:10] == b"252 2.0.0 "

View File

@@ -1,25 +1,6 @@
import requests
from cmdeploy.genqr import gen_qr_png_data
def test_gen_qr_png_data(maildomain):
data = gen_qr_png_data(maildomain)
assert data
def test_fastcgi_working(maildomain, chatmail_config):
url = f"https://{maildomain}/new"
print(url)
res = requests.post(url)
assert maildomain in res.json().get("email")
assert len(res.json().get("password")) > chatmail_config.password_min_length
def test_newemail_configure(maildomain, rpc):
"""Test configuring accounts by scanning a QR code works."""
url = f"DCACCOUNT:https://{maildomain}/new"
for i in range(3):
account_id = rpc.add_account()
rpc.set_config_from_qr(account_id, url)
rpc.configure(account_id)

View File

@@ -42,59 +42,22 @@ def test_reject_forged_from(cmsetup, maildata, gencreds, lp, forgeaddr):
assert "500" in str(e.value)
def test_authenticated_from(cmsetup, maildata):
"""Test that envelope FROM must be the same as login."""
user1, user2, user3 = cmsetup.gen_users(3)
msg = maildata("encrypted.eml", from_addr=user2.addr, to_addr=user3.addr)
with pytest.raises(smtplib.SMTPException) as e:
user1.smtp.sendmail(
from_addr=user2.addr, to_addrs=[user3.addr], msg=msg.as_string()
)
assert e.value.recipients[user3.addr][0] == 553
@pytest.mark.parametrize("from_addr", ["fake@example.org", "fake@testrun.org"])
def test_reject_missing_dkim(cmsetup, maildata, from_addr):
"""Test that emails with missing or wrong DMARC, DKIM, and SPF entries are rejected."""
recipient = cmsetup.gen_users(1)[0]
msg = maildata("plain.eml", from_addr=from_addr, to_addr=recipient.addr).as_string()
with smtplib.SMTP(cmsetup.maildomain, 25) as s:
with pytest.raises(smtplib.SMTPDataError, match="No valid DKIM signature"):
s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg)
@pytest.mark.slow
def test_exceed_rate_limit(cmsetup, gencreds, maildata, chatmail_config):
def test_exceed_rate_limit(cmsetup, gencreds, maildata):
"""Test that the per-account send-mail limit is exceeded."""
user1, user2 = cmsetup.gen_users(2)
mail = maildata(
"encrypted.eml", from_addr=user1.addr, to_addr=user2.addr
).as_string()
for i in range(chatmail_config.max_user_send_per_minute + 5):
for i in range(100):
print("Sending mail", str(i))
try:
user1.smtp.sendmail(user1.addr, [user2.addr], mail)
except smtplib.SMTPException as e:
if i < chatmail_config.max_user_send_per_minute:
if i < 60:
pytest.fail(f"rate limit was exceeded too early with msg {i}")
outcome = e.recipients[user2.addr]
assert outcome[0] == 450
assert b"4.7.1: Too much mail from" in outcome[1]
return
pytest.fail("Rate limit was not exceeded")
def test_expunged(remote, chatmail_config):
outdated_days = int(chatmail_config.delete_mails_after) + 1
find_cmds = [
f"find /home/vmail/mail/{chatmail_config.mail_domain} -path '*/cur/*' -mtime +{outdated_days} -type f",
f"find /home/vmail/mail/{chatmail_config.mail_domain} -path '*/.*/cur/*' -mtime +{outdated_days} -type f",
f"find /home/vmail/mail/{chatmail_config.mail_domain} -path '*/new/*' -mtime +{outdated_days} -type f",
f"find /home/vmail/mail/{chatmail_config.mail_domain} -path '*/.*/new/*' -mtime +{outdated_days} -type f",
f"find /home/vmail/mail/{chatmail_config.mail_domain} -path '*/tmp/*' -mtime +{outdated_days} -type f",
f"find /home/vmail/mail/{chatmail_config.mail_domain} -path '*/.*/tmp/*' -mtime +{outdated_days} -type f",
]
for cmd in find_cmds:
for line in remote.iter_output(cmd):
assert not line

View File

@@ -1,10 +1,6 @@
import time
import re
import random
import pytest
import requests
import ipaddress
class TestEndToEndDeltaChat:
@@ -24,24 +20,14 @@ class TestEndToEndDeltaChat:
assert msg2.text == "message0"
@pytest.mark.slow
def test_exceed_quota(self, cmfactory, lp, tmpdir, remote, chatmail_config):
def test_exceed_quota(self, cmfactory, lp, tmpdir, remote):
"""This is a very slow test as it needs to upload >100MB of mail data
before quota is exceeded, and thus depends on the speed of the upload.
"""
ac1, ac2 = cmfactory.get_online_accounts(2)
chat = cmfactory.get_accepted_chat(ac1, ac2)
def parse_size_limit(limit: str) -> int:
"""Parse a size limit and return the number of bytes as integer.
Example input: 100M, 2.4T, 500 K
"""
units = {"B": 1, "K": 2**10, "M": 2**20, "G": 2**30, "T": 2**40}
size = re.sub(r"([KMGT])", r" \1", limit.upper())
number, unit = [string.strip() for string in size.split()]
return int(float(number) * units[unit])
quota = parse_size_limit(chatmail_config.max_mailbox_size)
quota = 1024 * 1024 * 100
attachsize = 1 * 1024 * 1024
num_to_send = quota // attachsize + 2
lp.sec(f"ac1: send {num_to_send} large files to ac2")
@@ -122,29 +108,3 @@ class TestEndToEndDeltaChat:
for msg in msgs:
assert "error" not in m.get_message_info()
time.sleep(1)
def test_hide_senders_ip_address(cmfactory):
public_ip = requests.get("http://icanhazip.com").content.decode().strip()
assert ipaddress.ip_address(public_ip)
user1, user2 = cmfactory.get_online_accounts(2)
chat = cmfactory.get_accepted_chat(user1, user2)
chat.send_text("testing submission header cleanup")
user2.wait_next_incoming_message()
user2.direct_imap.select_folder("Inbox")
msg = user2.direct_imap.get_all_messages()[0]
assert public_ip not in msg.obj.as_string()
def test_echobot(cmfactory, chatmail_config, lp):
ac = cmfactory.get_online_accounts(1)[0]
lp.sec(f"Send message to echo@{chatmail_config.mail_domain}")
chat = ac.create_chat(f"echo@{chatmail_config.mail_domain}")
text = "hi, I hope you text me back"
chat.send_text(text)
lp.sec("Wait for reply from echobot")
reply = ac.wait_next_incoming_message()
assert reply.text == text

View File

@@ -228,25 +228,18 @@ def imap_or_smtp(request):
@pytest.fixture
def gencreds(chatmail_config):
def gencreds(maildomain):
count = itertools.count()
next(count)
def gen(domain=None):
domain = domain if domain else chatmail_config.mail_domain
domain = domain if domain else maildomain
while 1:
num = next(count)
alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
user = "".join(
random.choices(alphanumeric, k=chatmail_config.username_max_length)
)
if domain == "nine.testrun.org":
user = f"ac{num}_{user}"[:9]
else:
user = f"ac{num}_{user}"[: chatmail_config.username_max_length]
password = "".join(
random.choices(alphanumeric, k=chatmail_config.password_min_length)
)
user = "".join(random.choices(alphanumeric, k=10))
user = f"ac{num}_{user}"[:9]
password = "".join(random.choices(alphanumeric, k=12))
yield f"{user}@{domain}", f"{password}"
return lambda domain=None: next(gen(domain))

View File

@@ -2,6 +2,7 @@ import os
import pytest
from cmdeploy.cmdeploy import get_parser, main
from chatmaild.config import read_config
@pytest.fixture(autouse=True)
@@ -20,7 +21,12 @@ class TestCmdline:
run = parser.parse_args(["run"])
assert init and run
@pytest.mark.xfail(reason="init doesn't exit anymore, check for CLI output instead")
def test_init(self, tmp_path):
main(["init", "chat.example.org"])
inipath = tmp_path.joinpath("chatmail.ini")
config = read_config(inipath)
assert config.mail_domain == "chat.example.org"
def test_init_not_overwrite(self):
main(["init", "chat.example.org"])
with pytest.raises(SystemExit):

View File

@@ -36,30 +36,6 @@ def build_webpages(src_dir, build_dir, config):
print(traceback.format_exc())
def int_to_english(number):
if number >= 0 and number <= 12:
a = [
"zero",
"one",
"two",
"three",
"four",
"five",
"six",
"seven",
"eight",
"nine",
"ten",
"eleven",
"twelve",
]
return a[number]
elif number <= 50:
return str(number)
if number > 50:
return "more"
def _build_webpages(src_dir, build_dir, config):
mail_domain = config.mail_domain
assert src_dir.exists(), src_dir
@@ -72,15 +48,6 @@ def _build_webpages(src_dir, build_dir, config):
for path in src_dir.iterdir():
if path.suffix == ".md":
render_vars, content = prepare_template(path)
render_vars["username_min_length"] = int_to_english(
config.username_min_length
)
render_vars["username_max_length"] = int_to_english(
config.username_max_length
)
render_vars["password_min_length"] = int_to_english(
config.password_min_length
)
target = build_dir.joinpath(path.stem + ".html")
# recursive jinja2 rendering

View File

@@ -1,6 +0,0 @@
#!/usr/bin/env bash
#
# Wrapper for cmdelpoy to run it in activated virtualenv.
set -e
. venv/bin/activate
cmdeploy "$@"

View File

@@ -1,6 +1,9 @@
#!/bin/bash
set -e
python3 -m venv --upgrade-deps venv
python3 -m venv venv
venv/bin/pip install -e chatmaild
venv/bin/pip install -e cmdeploy
source venv/bin/activate
echo activated 'venv' python virtualenv environment containing "cmdeploy" tool

View File

@@ -7,13 +7,14 @@ Welcome to instant, interoperable and [privacy-preserving](privacy.html) messagi
👉 **Tap** or scan this QR code to get a random `@{{config.mail_domain}}` e-mail address
<a href="DCACCOUNT:https://{{ config.mail_domain }}/new">
<a href="DCACCOUNT:https://{{ config.mail_domain }}/cgi-bin/newemail.py">
<img width=300 style="float: none;" src="qr-chatmail-invite-{{config.mail_domain}}.png" /></a>
🐣 **Choose** your Avatar and Name
💬 **Start** chatting with any Delta Chat contacts using [QR invite codes](https://delta.chat/en/help#howtoe2ee)
{% if config.mail_domain != "nine.testrun.org" %}
<div class="experimental">Note: this is only a temporary development chatmail service</div>
{% endif %}
<div class="experimental">Note: this is an experimental service</div>

View File

@@ -3,32 +3,16 @@
## More information
{{ config.mail_domain }} provides a low-maintenance, resource efficient and
interoperable e-mail service for everyone. What's behind a `chatmail` is
effectively a normal e-mail address just like any other but optimized
for the usage in chats, especially DeltaChat.
### Choosing a chatmail address instead of using a random one
In the Delta Chat account setup
you may tap `LOG INTO YOUR E-MAIL ACCOUNT`
and fill the two fields like this:
- `Address`: invent a word with
{% if username_min_length == username_max_length %}
*exactly* {{ username_min_length }}
{% else %}
{{ username_min_length}}
{% if username_max_length == "more" %}
or more
{% else %}
to {{ username_max_length }}
{% endif %}
{% endif %}
characters
and append `@{{config.mail_domain}}` to it.
- `Address`: invent a word with *exactly* nine characters
and append `@{{config.mail_domain}}` to it.
- `Password`: invent at least {{ password_min_length }} characters.
- `Password`: invent at least 9 characters.
If the e-mail address is not yet taken, you'll get that account.
The first login sets your password.
@@ -40,11 +24,11 @@ The first login sets your password.
{{config.mail_domain}} but setting up contact via [QR invite codes](https://delta.chat/en/help#howtoe2ee)
allows your messages to pass freely to any outside recipients.
- You may send up to {{ config.max_user_send_per_minute }} messages per minute.
- You may send up to 60 messages per minute
- Messages are unconditionally removed {{ config.delete_mails_after }} days after arriving on the server.
- Messages are unconditionally removed 40 days after arriving on the server
- You can store up to [{{ config.max_mailbox_size }} messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server).
- You can store up to [100MB messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server)
### Who are the operators? Which software is running?

View File

@@ -1,47 +1,4 @@
body {
font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', 'Segoe UI Emoji', 'Apple Color Emoji', 'Noto Color Emoji', sans-serif;
line-height: 1.4;
font-size: 1.2em;
max-width: 800px;
margin: 20px auto;
padding: 0 10px;
color: #363636;
background: #fff;
}
h1 {
font-size: 2.2em;
margin-top: 0;
}
h1, h2, h3, h4, h5, h6 {
color: #000;
margin-bottom: 12px;
margin-top: 24px;
font-weight: 600;
}
a {
text-decoration: none;
color: #0076d1;
}
a:hover {
text-decoration: underline;
}
img, video {
max-width: 100%;
height: auto;
}
code {
background: #efefef;
padding: 2.5px 5px;
border-radius: 6px;
}
#menu {
display: flex;
flex-wrap: wrap;

View File

@@ -5,8 +5,8 @@
{% if config.webdev %}
<meta http-equiv="refresh" content="3">
{% endif %}
<meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">
<title>{{ config.mail_domain }} {{ pagename }}</title>
<link rel="stylesheet" href="./water.css">
<link rel="stylesheet" href="./main.css">
<link rel="icon" href="/logo.svg">
<link rel=”mask-icon” href=”/logo.svg” color=”#000000">

View File

@@ -62,7 +62,7 @@ we process the following data and details:
Creating an account happens in one of two ways on our mail servers:
- with a QR invitation token
which is scanned using the Delta Chat app
which is scanned using the DeltaChat app
and then the account is created.
- by letting Delta Chat otherwise create an account
@@ -218,16 +218,97 @@ on these or possible objections.
A deletion can be made
directly in the Delta Chat email messenger.
If you have any questions or complaints,
please feel free to contact us by email:
{{ config.privacy_mail }}
a) request information about your personal data processed by us
in accordance with Art. 15 GDPR.
In particular,
you can request information about the processing purposes,
the category of personal data,
the categories of recipients to whom your data have been or will be disclosed,
the planned storage period,
the existence of a right to rectification, erasure, restriction of processing or objection,
the existence of a right of complaint,
the origin of your data if it has not been collected by us,
as well as the existence of automated decision-making including profiling
and, if applicable,
meaningful information about its details;
As a rule, you can contact the supervisory authority of your usual place of residence
b) in accordance with Art. 16 of the GDPR,
immediately request the correction
of inaccurate or incomplete personal data stored by us;
c) pursuant to Article 17 of the GDPR,
to request the erasure of your personal data stored by us,
unless the processing is necessary
for the exercise of the right to freedom of expression and information,
for compliance with a legal obligation,
for reasons of public interest,
or for the establishment, exercise or defence of legal claims;
d) pursuant to Art. 18 GDPR,
to request the restriction of the processing of your personal data,
insofar as the accuracy of the data is disputed by you,
the processing is unlawful,
but you object to its erasure
and we no longer require the data,
but you need it for the assertion, exercise or defence of legal claims
or you have objected to the processing pursuant to Art. 21 GDPR;
e) pursuant to Art. 20 GDPR,
to receive your personal data that you have provided to us
in a structured, common and machine-readable format
or to request that it be transferred to another controller;
f) in accordance with Art. 7 (3) of the GDPR,
to revoke your consent given to us at any time.
This has the consequence that we may no longer continue the data processing
based on this consent in the future; and
g) complain to a supervisory authority
in accordance with Article 77 of the GDPR.
As a rule,
you can contact the supervisory authority of your usual place of residence
or workplace
or our registered office for this purpose.
The supervisory authority responsible for our place of business
is the `{{ config.privacy_supervisor }}`.
If you have any questions or complaints, please feel free to contact us by email:
{{ config.privacy_mail }}
### 5.1 Right to object
If your personal data is processed on the basis of our legitimate interests
in accordance with Art. 6 (1) lit. f GDPR,
you have the right to object to the processing of your personal data
in accordance with Art. 21 GDPR,
provided that there are grounds for this based on your particular situation
or the objection is directed against direct advertising.
In the latter case,
you have a general right of objection,
which will be implemented by us
without specifying a particular situation.
If you wish to exercise your right of objection,
simply send an e-mail to: {{ config.privacy_mail }}
### 5.2 Right to withdraw
If your personal data is processed on the basis of your consent
in accordance with Art. 6 (1) lit. a GDPR
(e.g. via the mailing list),
you can withdraw your consent at any time
and without any disadvantages.
As a result,
we may no longer continue the data processing
that was based on this consent for the future.
However,
the withdrawal of your consent
does not affect the lawfulness of the processing
carried out on the basis of the consent until the withdrawal.
If you wish to make use of your right of withdrawal,
simply send an e-mail to: {{ config.privacy_mail }}
## 6. Validity of this privacy policy

1690
www/src/water.css Normal file

File diff suppressed because it is too large Load Diff