Compare commits

..

21 Commits

Author SHA1 Message Date
holger krekel
4b9480bd29 deploy chatmaild in a virtualenv to make it easier to add dependencies 2023-12-08 20:48:10 +01:00
holger krekel
59c3730d84 fix data fixture access 2023-12-08 20:47:49 +01:00
holger krekel
84db074686 fix README link 2023-12-08 14:59:17 +01:00
holger krekel
7eec0ab301 tweak QR code generation 2023-12-08 14:56:48 +01:00
holger krekel
7cb8f90340 create a wwwdev.sh entry point for helping live web design/development (#92)
* create a wwwdev.sh entry point for developing the web part

* rename script

* fix README

* add a note

* don't depend on deltachat python package

* avoid bailing out on jinja2 errors, and provide file-url for instant clickability

* in webdev mode make page auto-refresh every 3 seconds
2023-12-08 14:32:40 +01:00
missytake
32360061b4 filtermail: address hpk's comments 2023-12-08 12:23:10 +01:00
missytake
2055e9f5b8 filtermail: always allow privacy@testrun.org 2023-12-08 12:23:10 +01:00
holger krekel
8cb77d3b98 be fine with 9 chars for password already 2023-12-07 17:34:19 +01:00
holger krekel
c67fb69af2 Parametrized privacy policy, unified and refined nine/non-nine landing pages (#89)
- move web sources to markdown
- integrate privacy policy template
- create and use chatmail.ini file to driving web-page generation 

Co-authored-by: missytake <missytake@systemli.org>

---------

Co-authored-by: missytake <missytake@systemli.org>
2023-12-07 13:52:00 +01:00
holger krekel
960bc1599b add missing simple test file for generating a qr code 2023-12-06 11:00:07 +01:00
holger krekel
0f05216ea0 mention QRcode in readme and modify nine.testrun.org index page to include it 2023-12-05 14:22:42 +01:00
holger krekel
75551224b3 revert unneccessary reformatting and unused file 2023-12-05 14:22:42 +01:00
holger krekel
0b8de41da2 put index.html into www/ dir, as it's not config 2023-12-05 14:22:42 +01:00
holger krekel
f9b5783296 streamline text to be less redundant 2023-12-05 14:22:42 +01:00
holger krekel
d3281cc746 Update deploy-chatmail/src/deploy_chatmail/nginx/index.html.j2
Co-authored-by: missytake <missytake@systemli.org>

use example config as recommended by fcgiwrap/README.debian
2023-12-05 14:22:42 +01:00
holger krekel
4c7e39c10c add origin of genqr code 2023-12-05 14:22:42 +01:00
holger krekel
7b3c1d5ab9 streamline index.html 2023-12-05 14:22:42 +01:00
holger krekel
2d5eb86776 make QR code clickable, verified it works on android and desktop 2023-12-05 14:22:42 +01:00
holger krekel
5c9d9a98b3 works 2023-12-05 14:22:42 +01:00
missytake
5eb5c09052 redirect HTTPS traffic to HTTPS. fix #81 2023-11-28 16:40:19 +01:00
missytake
a86e135967 opendkim: correctly specify SigningTable in opendkim.conf 2023-11-26 07:40:25 +01:00
39 changed files with 2686 additions and 202 deletions

5
.gitignore vendored
View File

@@ -3,6 +3,11 @@ __pycache__/
*.py[cod]
*$py.class
*.swp
www/privacy.html*
www/index.html*
www/info.html*
*qr-*.png
# C extensions
*.so

View File

@@ -1,3 +1,6 @@
<img width="800px" src="www/src/collage-top.png"/>
# Chatmail instances optimized for Delta Chat apps
This repository helps to setup a ready-to-use chatmail instance
@@ -22,21 +25,53 @@ after which the initially specified password is required for using them.
export CHATMAIL_DOMAIN=c1.testrun.org # replace with your host
4. Deploy the chat mail instance to your chatmail server:
4. Fill in privacy contact data into the `chatmail.ini` file
5. Deploy the chat mail instance to your chatmail server:
scripts/deploy.sh
This script uses `pyinfra` and `ssh` to setup packages and configure
the chatmail instance on your remote server.
This script remotely sets up packages and configures the chatmail provider.
5. Run `scripts/generate-dns-zone.sh` and
6. Run `scripts/generate-dns-zone.sh` and
transfer the generated DNS records at your DNS provider
6. Start a Delta Chat app and create a new account
by typing an e-mail address with an arbitrary username
and `@<your-chatmail-domain>` appended.
Use an at least 10-character random password.
### Home page and getting started for users
The `deploy.sh` script deploys
- a default `index.html` along with a QR code that users can click to
create accounts on your chatmail provider,
- a default `info.html` that is linked from the home page,
- a default `policy.html` that is linked from the home page.
All files are generated by the according markdown `.md` file in the `www` directory.
### Refining the web pages
The `scripts/webdev.sh` script supports live development of the chatmail web presence:
```
scripts/init.sh # to locally initialize python virtual environments etc.
scripts/webdev.sh
```
- uses the `www/src/page-layout.html` file for producing html documents
from `www/src/*.md` files.
- continously builds the web presence reading files from `www/src` directory
and generating html files and copying assets to the `www/build` directory.
- Starts a browser window automatically where you can "refresh" as needed.
Note that this script is not needed for running `scripts/deploy.sh"
which deploys the whole chatmail setup remotely.
The code that generates the web pages is identical
which means that `webdev.sh` gives a pretty good preview.
### Ports

15
chatmail.ini Normal file
View File

@@ -0,0 +1,15 @@
[config]
privacy_postal =
Merlinux GmbH, Represented by the managing director H. Krekel,
Reichgrafen Str. 20, 79102 Freiburg, Germany
privacy_mail = delta-privacy@merlinux.eu
privacy_pdo =
Prof. Dr. Fabian Schmieder, lexICT UG (limited), Ostfeldstr. 49, 30559 Hannover.
You can contact him at *delta-privacy@merlinux.eu* (Keyword: DPO)
privacy_supervisor =
State Commissioner for Data Protection and Freedom of Information of
Baden-Württemberg in 70173 Stuttgart, Germany.

View File

@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
name = "chatmaild"
version = "0.1"
dependencies = [
"aiosmtpd"
"aiosmtpd",
]
[project.scripts]

View File

@@ -28,8 +28,8 @@ def is_allowed_to_create(user, cleartext_password) -> bool:
logging.warning(f"blocked account creation because {NOCREATE_FILE!r} exists.")
return False
if len(cleartext_password) < 10:
logging.warning("Password needs to be at least 10 characters long")
if len(cleartext_password) < 9:
logging.warning("Password needs to be at least 9 characters long")
return False
parts = user.split("@")

View File

@@ -2,7 +2,7 @@
Description=Dict authentication proxy for dovecot
[Service]
ExecStart=/usr/local/bin/doveauth /run/dovecot/doveauth.socket vmail /home/vmail/passdb.sqlite
ExecStart={execpath} /run/dovecot/doveauth.socket vmail /home/vmail/passdb.sqlite
Restart=always
RestartSec=30

View File

@@ -34,6 +34,14 @@ def check_encrypted(message):
return True
def is_passthrough_recipient(recipient):
"""Check whether a recipient is configured as passthrough."""
passthroughlist = ["privacy@testrun.org"]
if recipient in passthroughlist:
return True
return False
def check_mdn(message, envelope):
if len(envelope.rcpt_tos) != 1:
return False
@@ -118,6 +126,9 @@ def check_DATA(envelope):
if envelope.mail_from == recipient:
# Always allow sending emails to self.
continue
if is_passthrough_recipient(recipient):
# Always allow recipients marked as passthrough
continue
res = recipient.split("@")
if len(res) != 2:
return f"500 Invalid address <{recipient}>"

View File

@@ -2,7 +2,7 @@
Description=Chatmail Postfix BeforeQeue filter
[Service]
ExecStart=/usr/local/bin/filtermail 10080
ExecStart={execpath} 10080
Restart=always
RestartSec=30

View File

@@ -0,0 +1,28 @@
#!/usr/bin/python3
""" CGI script for creating new accounts. """
import json
import random
mailname_path = "/etc/mailname"
def create_newemail_dict(domain):
alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
user = "".join(random.choices(alphanumeric, k=9))
password = "".join(random.choices(alphanumeric, k=12))
return dict(email=f"{user}@{domain}", password=f"{password}")
def print_new_account():
domain = open(mailname_path).read().strip()
creds = create_newemail_dict(domain=domain)
print("Content-Type: application/json")
print("")
print(json.dumps(creds))
if __name__ == "__main__":
print_new_account()

View File

@@ -7,6 +7,9 @@ name = "deploy-chatmail"
version = "0.1"
dependencies = [
"pyinfra",
"pillow",
"qrcode",
"markdown",
]
[tool.pytest.ini_options]

View File

@@ -1,72 +1,102 @@
"""
Chat Mail pyinfra deploy.
"""
import sys
import importlib.resources
import subprocess
import shutil
import io
import configparser
from pathlib import Path
from pyinfra import host
from pyinfra.operations import apt, files, server, systemd
from pyinfra.operations import apt, files, server, systemd, pip
from pyinfra.facts.files import File
from pyinfra.facts.systemd import SystemdEnabled
from .acmetool import deploy_acmetool
def _install_chatmaild() -> None:
chatmaild_filename = "chatmaild-0.1.tar.gz"
chatmaild_path = importlib.resources.files(__package__).joinpath(
f"../../../dist/{chatmaild_filename}"
def _build_chatmaild(dist_dir) -> None:
dist_dir = Path(dist_dir).resolve()
if dist_dir.exists():
shutil.rmtree(dist_dir)
dist_dir.mkdir()
subprocess.check_output(
[sys.executable, "-m", "build", "-n"]
+ ["--sdist", "chatmaild", "--outdir", str(dist_dir)]
)
remote_path = f"/tmp/{chatmaild_filename}"
if Path(str(chatmaild_path)).exists():
entries = list(dist_dir.iterdir())
assert len(entries) == 1
return entries[0]
def _install_remote_venv_with_chatmaild() -> None:
dist_file = _build_chatmaild(dist_dir=Path("chatmaild/dist"))
remote_base_dir = "/usr/local/lib/chatmaild"
remote_dist_file = f"{remote_base_dir}/dist/{dist_file.name}"
remote_venv_dir = f"{remote_base_dir}/venv"
root_owned = dict(user="root", group="root", mode="644")
apt.packages(
name="apt install python3-virtualenv",
packages=["python3-virtualenv"],
)
files.put(
name="Upload chatmaild source package",
src=dist_file.open("rb"),
dest=remote_dist_file,
create_remote_dir=True,
**root_owned,
)
pip.virtualenv(
name=f"chatmaild virtualenv {remote_venv_dir}",
path=remote_venv_dir,
always_copy=True,
)
server.shell(
name=f"forced pip-install {dist_file.name}",
commands=[
f"{remote_venv_dir}/bin/pip install --force-reinstall {remote_dist_file}"
],
)
# disable legacy doveauth-dictproxy.service
if host.get_fact(SystemdEnabled).get("doveauth-dictproxy.service"):
systemd.service(
name="Disable legacy doveauth-dictproxy.service",
service="doveauth-dictproxy.service",
running=False,
enabled=False,
)
# install systemd units
for fn in (
"doveauth",
"filtermail",
):
execpath = f"{remote_venv_dir}/bin/{fn}"
source_path = importlib.resources.files("chatmaild").joinpath(f"{fn}.service.f")
content = source_path.read_text().format(execpath=execpath).encode()
files.put(
name="Upload chatmaild source package",
src=chatmaild_path.open("rb"),
dest=remote_path,
name=f"Upload {fn}.service",
src=io.BytesIO(content),
dest=f"/etc/systemd/system/{fn}.service",
**root_owned,
)
apt.packages(
name="apt install python3-aiosmtpd python3-pip python3-venv",
packages=["python3-aiosmtpd", "python3-pip", "python3-venv"],
systemd.service(
name=f"Setup {fn} service",
service=f"{fn}.service",
running=True,
enabled=True,
restarted=True,
daemon_reload=True,
)
# --no-deps because aiosmtplib is installed with `apt`.
server.shell(
name="install chatmaild with pip",
commands=[f"pip install --break-system-packages {remote_path}"],
)
# disable legacy doveauth-dictproxy.service
if host.get_fact(SystemdEnabled).get("doveauth-dictproxy.service"):
systemd.service(
name="Disable legacy doveauth-dictproxy.service",
service="doveauth-dictproxy.service",
running=False,
enabled=False,
)
for fn in (
"doveauth",
"filtermail",
):
files.put(
name=f"Upload {fn}.service",
src=importlib.resources.files("chatmaild")
.joinpath(f"{fn}.service")
.open("rb"),
dest=f"/etc/systemd/system/{fn}.service",
user="root",
group="root",
mode="644",
)
systemd.service(
name=f"Setup {fn} service",
service=f"{fn}.service",
running=True,
enabled=True,
restarted=True,
daemon_reload=True,
)
def _configure_opendkim(domain: str, dkim_selector: str) -> bool:
"""Configures OpenDKIM"""
@@ -245,7 +275,7 @@ def _configure_dovecot(mail_server: str, debug: bool = False) -> bool:
return need_restart
def _configure_nginx(domain: str, mail_server: str) -> bool:
def _configure_nginx(domain: str, debug: bool = False) -> bool:
"""Configures nginx HTTP server."""
need_restart = False
@@ -275,13 +305,47 @@ def _configure_nginx(domain: str, mail_server: str) -> bool:
user="root",
group="root",
mode="644",
config={"mail_server": mail_server},
config={"domain_name": domain},
)
need_restart |= mta_sts_config.changed
# install CGI newemail script
#
cgi_dir = "/usr/lib/cgi-bin"
files.directory(
name=f"Ensure {cgi_dir} exists",
path=cgi_dir,
user="root",
group="root",
)
files.put(
name="Upload cgi newemail.py script",
src=importlib.resources.files("chatmaild").joinpath("newemail.py").open("rb"),
dest=f"{cgi_dir}/newemail.py",
user="root",
group="root",
mode="755",
)
return need_restart
def get_ini_settings(mail_domain, inipath):
parser = configparser.ConfigParser()
parser.read(inipath)
settings = {key: value.strip() for (key, value) in parser["config"].items()}
if mail_domain != "testrun.org" and not mail_domain.endswith(".testrun.org"):
for value in settings.values():
value = value.lower()
if "merlinux" in value or "schmieder" in value or "@testrun.org" in value:
raise ValueError(
f"please set your own privacy contacts/addresses in {inipath}"
)
settings["mail_domain"] = mail_domain
return settings
def deploy_chatmail(mail_domain: str, mail_server: str, dkim_selector: str) -> None:
"""Deploy a chat-mail instance.
@@ -289,6 +353,7 @@ def deploy_chatmail(mail_domain: str, mail_server: str, dkim_selector: str) -> N
:param mail_server: the DNS name under which your mail server is reachable
:param dkim_selector:
"""
from .www import build_webpages
apt.update(name="apt update", cache_time=24 * 3600)
server.group(name="Create vmail group", group="vmail", system=True)
@@ -328,19 +393,28 @@ def deploy_chatmail(mail_domain: str, mail_server: str, dkim_selector: str) -> N
packages=["nginx"],
)
_install_chatmaild()
apt.packages(
name="Install fcgiwrap",
packages=["fcgiwrap"],
)
pkg_root = importlib.resources.files(__package__)
chatmail_ini = pkg_root.joinpath("../../../chatmail.ini").resolve()
config = get_ini_settings(mail_domain, chatmail_ini)
www_path = pkg_root.joinpath("../../../www").resolve()
build_dir = www_path.joinpath("build")
src_dir = www_path.joinpath("src")
build_webpages(src_dir, build_dir, config)
files.rsync(f"{build_dir}/", "/var/www/html", flags=["-avz"])
_install_remote_venv_with_chatmaild()
debug = False
dovecot_need_restart = _configure_dovecot(mail_server, debug=debug)
postfix_need_restart = _configure_postfix(mail_domain, debug=debug)
opendkim_need_restart = _configure_opendkim(mail_domain, dkim_selector)
nginx_need_restart = _configure_nginx(mail_domain, mail_server)
mta_sts_need_restart = _install_mta_sts_daemon()
# deploy web pages and info if we have them
pkg_root = importlib.resources.files(__package__)
www_path = pkg_root.joinpath(f"../../../www/{mail_domain}").resolve()
if www_path.is_dir():
files.rsync(f"{www_path}/", "/var/www/html", flags=["-avz"])
nginx_need_restart = _configure_nginx(mail_domain)
systemd.service(
name="Start and enable OpenDKIM",

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

View File

@@ -0,0 +1,87 @@
import importlib
import qrcode
import os
from PIL import ImageFont, ImageDraw, Image
import io
def gen_qr_png_data(maildomain):
url = f"DCACCOUNT:https://{maildomain}/cgi-bin/newemail.py"
image = gen_qr(maildomain, url)
temp = io.BytesIO()
image.save(temp, format="png")
temp.seek(0)
return temp
def gen_qr(maildomain, url):
# taken and modified from
# https://github.com/deltachat/mailadm/blob/master/src/mailadm/gen_qr.py
# info = f"{maildomain} invite code"
info = ""
# load QR code
qr = qrcode.QRCode(
version=1,
error_correction=qrcode.constants.ERROR_CORRECT_H,
box_size=1,
border=1,
)
qr.add_data(url)
qr.make(fit=True)
qr_img = qr.make_image(fill_color="black", back_color="white")
# paint all elements
ttf_path = str(
importlib.resources.files(__package__).joinpath("data/opensans-regular.ttf")
)
logo_red_path = str(
importlib.resources.files(__package__).joinpath("data/delta-chat-bw.png")
)
assert os.path.exists(ttf_path), ttf_path
font_size = 16
font = ImageFont.truetype(font=ttf_path, size=font_size)
num_lines = ((info).count("\n") + 1) if info else 0
size = width = 384
qr_padding = 6
text_height = font_size * num_lines
height = size + text_height
image = Image.new("RGBA", (width, height), "white")
qr_final_size = width - (qr_padding * 2)
if num_lines:
draw = ImageDraw.Draw(image)
# draw text
if hasattr(font, "getsize"):
info_pos = (width - font.getsize(info.strip())[0]) // 2
else:
info_pos = (width - font.getbbox(info.strip())[3]) // 2
draw.multiline_text(
(info_pos, size - qr_padding // 2),
info,
font=font,
fill="black",
align="right",
)
# paste QR code
image.paste(
qr_img.resize((qr_final_size, qr_final_size), resample=Image.NEAREST),
(qr_padding, qr_padding),
)
# background delta logo
logo2_img = Image.open(logo_red_path)
logo2_width = int(size / 6)
logo2 = logo2_img.resize((logo2_width, logo2_width), resample=Image.NEAREST)
pos = int((size / 2) - (logo2_width / 2))
image.paste(logo2, (pos, pos), mask=logo2)
return image

View File

@@ -1,4 +1,4 @@
version: STSv1
mode: enforce
mx: {{ config.mail_server }}
mx: {{ config.domain_name }}
max_age: 2419200

View File

@@ -20,16 +20,14 @@ http {
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_certificate /var/lib/acme/live/{{ config.domain_name }}/fullchain;
ssl_certificate_key /var/lib/acme/live/{{ config.domain_name }}/privkey;
gzip on;
server {
listen 80 default_server;
listen [::]:80 default_server;
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
ssl_certificate /var/lib/acme/live/{{ config.domain_name }}/fullchain;
ssl_certificate_key /var/lib/acme/live/{{ config.domain_name }}/privkey;
root /var/www/html;
@@ -42,28 +40,16 @@ http {
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
}
# add cgi-bin support
include /usr/share/doc/fcgiwrap/examples/nginx.conf;
}
server {
listen 80;
listen [::]:80;
listen 443 ssl;
listen [::]:443 ssl;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /var/www/html;
index index.html index.htm;
server_name mta-sts.{{ config.domain_name }};
ssl_certificate /var/lib/acme/live/mta-sts.{{ config.domain_name }}/fullchain;
ssl_certificate_key /var/lib/acme/live/mta-sts.{{ config.domain_name }}/privkey;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
}
}
return 301 https://$host$request_uri;
}
}

View File

@@ -20,7 +20,7 @@ Domain {{ config.domain_name }}
Selector {{ config.opendkim_selector }}
KeyFile /etc/dkimkeys/{{ config.opendkim_selector }}.private
KeyTable /etc/dkimkeys/KeyTable
SigningTable /etc/dkimkeys/SigningTable
SigningTable refile:/etc/dkimkeys/SigningTable
# In Debian, opendkim runs as user "opendkim". A umask of 007 is required when
# using a local socket with MTAs that access the socket as a non-privileged

View File

@@ -0,0 +1,110 @@
import importlib.resources
import webbrowser
import hashlib
import time
import traceback
import markdown
from jinja2 import Template
from .genqr import gen_qr_png_data
from deploy_chatmail import get_ini_settings
def snapshot_dir_stats(somedir):
d = {}
for path in somedir.iterdir():
if path.is_file() and path.name[0] != "." and path.suffix != ".swp":
mtime = path.stat().st_mtime
hash = hashlib.md5(path.read_bytes()).hexdigest()
d[path] = (mtime, hash)
return d
def prepare_template(source):
assert source.exists(), source
render_vars = {}
render_vars["pagename"] = "home" if source.stem == "index" else source.stem
render_vars["markdown_html"] = markdown.markdown(source.read_text())
page_layout = source.with_name("page-layout.html").read_text()
return render_vars, page_layout
def build_webpages(src_dir, build_dir, config):
try:
_build_webpages(src_dir, build_dir, config)
except Exception:
print(traceback.format_exc())
def _build_webpages(src_dir, build_dir, config):
mail_domain = config["mail_domain"]
assert src_dir.exists(), src_dir
if not build_dir.exists():
build_dir.mkdir()
qr_path = build_dir.joinpath(f"qr-chatmail-invite-{mail_domain}.png")
qr_path.write_bytes(gen_qr_png_data(mail_domain).read())
for path in src_dir.iterdir():
if path.suffix == ".md":
render_vars, content = prepare_template(path)
target = build_dir.joinpath(path.stem + ".html")
# recursive jinja2 rendering
while 1:
new = Template(content).render(config=config, **render_vars)
if new == content:
break
content = new
with target.open("w") as f:
f.write(content)
elif path.name != "page-layout.html":
target = build_dir.joinpath(path.name)
target.write_bytes(path.read_bytes())
return build_dir
def main():
chatmail_domain = "example.testrun.org"
path = importlib.resources.files(__package__)
reporoot = path.joinpath("../../../").resolve()
inipath = reporoot.joinpath("chatmail.ini")
config = get_ini_settings(chatmail_domain, inipath)
config["webdev"] = True
www_path = reporoot.joinpath("www")
src_path = www_path.joinpath("src")
stats = None
build_dir = www_path.joinpath("build")
src_dir = www_path.joinpath("src")
index_path = build_dir.joinpath("index.html")
# start web page generation, open a browser and wait for changes
build_webpages(src_dir, build_dir, config)
webbrowser.open(str(index_path))
stats = snapshot_dir_stats(src_path)
print(f"\nOpened URL: file://{index_path.resolve()}\n")
print(f"watching {src_path} directory for changes")
changenum = 0
for count in range(0, 1000000):
newstats = snapshot_dir_stats(src_path)
if newstats == stats and count % 60 != 0:
count += 1
time.sleep(1.0)
continue
for key in newstats:
if stats[key] != newstats[key]:
print(f"*** CHANGED: {key}")
changenum += 1
stats = newstats
build_webpages(src_dir, build_dir, config)
print(f"[{changenum}] regenerated web pages at: {index_path}")
print(f"URL: file://{index_path.resolve()}\n\n")
count = 0
if __name__ == "__main__":
main()

View File

@@ -4,12 +4,5 @@ echo -----------------------------------------
echo deploying to $CHATMAIL_DOMAIN
echo -----------------------------------------
echo WARNING: in five seconds deploy to $CHATMAIL_DOMAIN starts
sleep 5
venv/bin/python3 -m build -n --sdist chatmaild --outdir dist
venv/bin/pyinfra --ssh-user root "$CHATMAIL_DOMAIN" \
deploy-chatmail/src/deploy_chatmail/deploy.py
rm -r dist/

View File

@@ -1,6 +1,5 @@
#!/bin/sh
: ${CHATMAIL_DOMAIN:=c1.testrun.org}
: ${CHATMAIL_SERVER:=$CHATMAIL_DOMAIN}
: ${CHATMAIL_SSH:=$CHATMAIL_DOMAIN}
set -e
@@ -9,22 +8,16 @@ EMAIL="root@$CHATMAIL_DOMAIN"
ACME_ACCOUNT_URL="$($SSH -- acmetool account-url)"
cat <<EOF
$CHATMAIL_DOMAIN. MX 10 $CHATMAIL_SERVER.
$CHATMAIL_DOMAIN. TXT "v=spf1 a:$CHATMAIL_SERVER -all"
$CHATMAIL_DOMAIN. MX 10 $CHATMAIL_DOMAIN.
$CHATMAIL_DOMAIN. TXT "v=spf1 a:$CHATMAIL_DOMAIN -all"
_dmarc.$CHATMAIL_DOMAIN. TXT "v=DMARC1;p=reject;rua=mailto:$EMAIL;ruf=mailto:$EMAIL;fo=1;adkim=r;aspf=r"
_submission._tcp.$CHATMAIL_SERVER. SRV 0 1 587 $CHATMAIL_SERVER.
_submissions._tcp.$CHATMAIL_SERVER. SRV 0 1 465 $CHATMAIL_SERVER.
_imap._tcp.$CHATMAIL_SERVER. SRV 0 1 143 $CHATMAIL_SERVER.
_imaps._tcp.$CHATMAIL_SERVER. SRV 0 1 993 $CHATMAIL_SERVER.
_submission._tcp.$CHATMAIL_DOMAIN. SRV 0 1 587 $CHATMAIL_DOMAIN.
_submissions._tcp.$CHATMAIL_DOMAIN. SRV 0 1 465 $CHATMAIL_DOMAIN.
_imap._tcp.$CHATMAIL_DOMAIN. SRV 0 1 143 $CHATMAIL_DOMAIN.
_imaps._tcp.$CHATMAIL_DOMAIN. SRV 0 1 993 $CHATMAIL_DOMAIN.
$CHATMAIL_DOMAIN. IN CAA 128 issue "letsencrypt.org;accounturi=$ACME_ACCOUNT_URL"
_mta-sts.$CHATMAIL_DOMAIN. IN TXT "v=STSv1; id=$(date -u '+%Y%m%d%H%M')"
mta-sts.$CHATMAIL_SERVER. IN CNAME $CHATMAIL_SERVER.
_smtp._tls.$CHATMAIL_SERVER. IN TXT "v=TLSRPTv1;rua=mailto:$EMAIL"
mta-sts.$CHATMAIL_DOMAIN. IN CNAME $CHATMAIL_DOMAIN.
_smtp._tls.$CHATMAIL_DOMAIN. IN TXT "v=TLSRPTv1;rua=mailto:$EMAIL"
EOF
if [ "$CHATMAIL_DOMAIN" != "$CHATMAIL_SERVER" ]; then
cat <<EOF
mta-sts.$CHATMAIL_DOMAIN. IN CNAME mta-sts.$CHATMAIL_SERVER.
_smtp._tls.$CHATMAIL_DOMAIN. IN CNAME _smtp._tls.$CHATMAIL_SERVER.
EOF
fi
$SSH opendkim-genzone -F | sed 's/^;.*$//;/^$/d'

View File

@@ -3,6 +3,6 @@ set -e
python3 -m venv venv
pip=venv/bin/pip
$pip install pyinfra pytest build 'setuptools>=68' tox deltachat
$pip install pyinfra pytest build 'setuptools>=68' tox
$pip install -e deploy-chatmail
$pip install -e chatmaild

9
scripts/webdev.sh Executable file
View File

@@ -0,0 +1,9 @@
#!/usr/bin/env bash
echo -----------------------------------------
echo starting local webdev
echo -----------------------------------------
venv/bin/python3 -m deploy_chatmail.www

View File

@@ -1,4 +1,4 @@
from chatmaild.filtermail import check_encrypted, check_DATA, SendRateLimiter, check_mdn
from chatmaild.filtermail import check_encrypted, check_DATA, SendRateLimiter, check_mdn, is_passthrough_recipient
import pytest
@@ -80,3 +80,28 @@ def test_send_rate_limiter():
else:
assert i == SendRateLimiter.MAX_USER_SEND_PER_MINUTE + 1
break
def test_excempt_privacy(maildata, gencreds):
from_addr = gencreds()[0]
to_addr = "privacy@testrun.org"
false_to = "privacy@tstrn.org"
false_to2 = "prvcy@testrun.org"
assert is_passthrough_recipient(to_addr)
msg = maildata("plain.eml", from_addr, to_addr)
class env:
mail_from = from_addr
rcpt_tos = [to_addr]
content = msg.as_bytes()
# assert that None/no error is returned
assert not check_DATA(envelope=env)
class env2:
mail_from = from_addr
rcpt_tos = [to_addr, false_to, false_to2]
content = msg.as_bytes()
assert "500" in check_DATA(envelope=env2)

View File

@@ -0,0 +1,29 @@
import json
import chatmaild
from chatmaild.newemail import create_newemail_dict, print_new_account
def test_create_newemail_dict():
ac1 = create_newemail_dict(domain="example.org")
assert "@" in ac1["email"]
assert len(ac1["password"]) >= 10
ac2 = create_newemail_dict(domain="example.org")
assert ac1["email"] != ac2["email"]
assert ac1["password"] != ac2["password"]
def test_print_new_account(capsys, monkeypatch, maildomain, tmpdir):
p = tmpdir.join("mailname")
p.write(maildomain)
monkeypatch.setattr(chatmaild.newemail, "mailname_path", str(p))
print_new_account()
out, err = capsys.readouterr()
lines = out.split("\n")
assert lines[0] == "Content-Type: application/json"
assert not lines[1]
dic = json.loads(lines[2])
assert dic["email"].endswith(f"@{maildomain}")
assert len(dic["password"]) >= 10

View File

@@ -278,11 +278,13 @@ class ChatmailTestProcess:
@pytest.fixture
def cmfactory(request, gencreds, tmpdir, data, maildomain):
def cmfactory(request, gencreds, tmpdir, maildomain):
# cloned from deltachat.testplugin.amfactory
pytest.importorskip("deltachat")
from deltachat.testplugin import ACFactory
data = request.getfixturevalue("data")
testproc = ChatmailTestProcess(request.config, maildomain, gencreds)
am = ACFactory(request=request, tmpdir=tmpdir, testprocess=testproc, data=data)
@@ -326,6 +328,18 @@ class Remote:
break
@pytest.fixture
def lp(request):
class LP:
def sec(self, msg):
print(f"---- {msg} ----")
def indent(self, msg):
print(f" {msg}")
return LP()
@pytest.fixture
def maildata(request, gencreds):
datadir = conftestdir.joinpath("mail-data")

View File

@@ -0,0 +1,6 @@
from deploy_chatmail.genqr import gen_qr_png_data
def test_gen_qr_png_data(maildomain):
data = gen_qr_png_data(maildomain)
assert data

47
tests/test_helpers.py Normal file
View File

@@ -0,0 +1,47 @@
import textwrap
import importlib.resources
from deploy_chatmail.www import build_webpages
from deploy_chatmail import get_ini_settings
def create_ini(inipath):
inipath.write_text(
textwrap.dedent(
"""\
[config]
privacy_postal =
address-line1
address-line2
privacy_mail = privacy@example.org
privacy_pdo =
address-line3
"""
)
)
def test_build_webpages(tmp_path):
pkgroot = importlib.resources.files("deploy_chatmail")
src_dir = pkgroot.joinpath("../../../www/src").resolve()
assert src_dir.exists(), src_dir
inipath = tmp_path.joinpath("chatmail.ini")
create_ini(inipath)
config = get_ini_settings("example.org", inipath)
build_dir = tmp_path.joinpath("build")
build_webpages(src_dir, build_dir, config)
def test_get_settings(tmp_path):
inipath = tmp_path.joinpath("chatmail.ini")
create_ini(inipath)
d = get_ini_settings("x.testrun.org", inipath)
assert d["privacy_postal"] == "address-line1\naddress-line2"
assert d["privacy_mail"] == "privacy@example.org"
assert d["privacy_pdo"] == "address-line3"
assert d["mail_domain"] == "x.testrun.org"

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 163 KiB

View File

@@ -1,75 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>nine.testrun.org - Experimenting with the Future of Email</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<style>
.wrapper {
width: 100%;
max-width: 596px;
margin: 0 auto;
}
.section {
width: 100%;
max-width: 596px;
}
.text {
box-sizing: border-box;
padding: 9px;
font-size: 18px;
font-family: "Swansea", "Helvetica", sans-serif;
color: black;
}
a {
color: black;
}
h1, h2, h3 {
font-size: 18px;
font-weight: bold;
}
</style>
</head>
<body>
<div class="wrapper">
<img class="section" src="collage-top.png" />
<div class="section text">
<h1>Dear Delta Chat users and newcomers,</h1>
<p>
welcome to the first public "chat-mail instance",
a small and lean e-mail provider for smooth chatting.
Install Delta Chat or add an account:
<ul>
<li>Tap "LOG INTO YOUR E-MAIL ACCOUNT".</li>
<li>Address: invent a word with <i>exactly</i> nine characters
and append @nine.testrun.org to it.</li>
<li>Password: invent at least 10 characters. The first login sets your password.</li>
</ul>
If the e-mail address is not yet taken, you'll get that account.
</p>
<p>
<img class="section" src="collage-down.png" />
<h2>What's behind it, how does it operate?</h2>
<p>nine.testrun.org is run
by a small group of devs and sysadmins, reachable via root@.
They want to keep this instance running at least until end 2024.
Current limits:
<ul>
<li>Un-encrypted mails can not leave the chat-mail instance.</li>
<li>Use <a href="https://delta.chat/en/help#howtoe2ee">
guaranteed end-to-end encryption via QR code scans</a>
to setup contact with users outside of the chat-mail instance.
</li>
<li>You may send up to 60 messages per minute.</li>
<li>Messages are unconditionally removed 40 days after arrival.</li>
<li>Max storage per user is 100MB.</li>
</ul>
<h2>Why are other email providers 1000 times more complicated?</h2>
<p>¯\_(ツ)_/¯</p>
</div>
</div>
</body>
</html>

BIN
www/src/collage-info.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB

BIN
www/src/collage-privacy.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

BIN
www/src/collage-top.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 60 KiB

19
www/src/index.md Normal file
View File

@@ -0,0 +1,19 @@
<img width="800px" src="collage-top.png"/>
## Dear [Delta Chat](https://get.delta.chat) users and newcomers,
Welcome to instant, interoperable and [privacy-preserving](privacy.html) messaging :)
👉 **Tap** or scan this QR code to get a random `@{{config.mail_domain}}` e-mail address
<a href="DCACCOUNT:https://{{ config.mail_domain }}/cgi-bin/newemail.py">
<img width=300 style="float: none;" src="qr-chatmail-invite-{{config.mail_domain}}.png" /></a>
🐣 **Choose** your Avatar and Name
💬 **Start** chatting with any Delta Chat contacts using [QR invite codes](https://delta.chat/en/help#howtoe2ee)
## ⚡ Note: this is an experimental service ⚡

39
www/src/info.md Normal file
View File

@@ -0,0 +1,39 @@
<img width="800px" src="collage-info.png"/>
## More information
### Choosing a chatmail address instead of using a random one
In the Delta Chat account setup
you may tap `LOG INTO YOUR E-MAIL ACCOUNT`
and fill the two fields like this:
- `Address`: invent a word with *exactly* nine characters
and append `@{{config.mail_domain}}` to it.
- `Password`: invent at least 9 characters.
If the e-mail address is not yet taken, you'll get that account.
The first login sets your password.
### Rate and storage limits
- Un-encrypted messages are blocked to recipients outside
{{config.mail_domain}} but setting up contact via [QR invite codes](https://delta.chat/en/help#howtoe2ee)
allows your messages to pass freely to any outside recipients.
- You may send up to 60 messages per minute
- Messages are unconditionally removed 40 days after arriving on the server
- You can store up to [100MB messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server)
### Who are the operators? Which software is running?
This chatmail provider is run by a small voluntary group of devs and sysadmins,
who [publically develop chatmail provider setups](https://github.com/deltachat/chatmail).
Chatmail setups aim to be very low-maintenance, resource efficient and
interoperable with any other standards-compliant e-mail service.

20
www/src/page-layout.html Normal file
View File

@@ -0,0 +1,20 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
{% if config.webdev %}
<meta http-equiv="refresh" content="3">
{% endif %}
<title>{{ config.mail_domain }} {{ pagename }}</title>
<link rel="stylesheet" href="./water.css">
</head>
<body>
{{ markdown_html }}
<footer>
<a href="index.html">home</a> |
<a href="info.html">more info</a> |
<a href="privacy.html">privacy</a> |
<a href="https://github.com/deltachat/chatmail">-> public development </a>
</footer>
</body>
</html>

321
www/src/privacy.md Normal file
View File

@@ -0,0 +1,321 @@
<img width="800px" src="collage-privacy.png"/>
# Privacy Policy for {{ config.mail_domain }}
We want to show you in a fair and transparent way
what personal data is processed by us.
We follow a strict privacy-by-design approach
and try to avoid processing your data in the first place,
but as you may know,
the internet,
and in particular sending e-mail messages,
does not work without data.
Still,
it's only fair that you know at all times
what personal data is processed
when you use our service.
If you have any remaining questions about data protection, please contact us.
## 1. Name and contact information
Responsible for the processing of your personal data is:
```
{{ config.privacy_postal }}
```
E-mail: {{ config.privacy_mail }}
We have appointed a data protection officer:
```
{{ config.privacy_pdo }}
```
## 2. Processing when using chat e-mail services
We provide e-mail services optimized for the use from [Delta Chat](https://delta.chat) apps
and process only the data necessary
for the setup and technical execution of the e-mail dispatch.
The purpose of the processing is to
read, write, manage, delete, send, and receive emails.
For this purpose,
we operate server-side software
that enables us to send and receive e-mail messages.
Allowing the use of the e-mail service,
we process the following data and details:
- Outgoing and incoming messages (SMTP) are stored for transit
on behalf of their users until the message can be delivered.
- E-Mail-Messages are stored for the recipient and made accessible via IMAP protocols,
until explicitly deleted by the user or until a fixed time period is exceeded,
(*usually 4-8 weeks*).
- IMAP and SMTP protocols are password protected with unique credentials for each account.
- Users can retrieve or delete all stored messages
without intervention from the operators using standard IMAP client tools.
### 3.1 Account setup
Creating an account happens in one of two ways on our mail servers:
- with a QR invitation token
which is scanned using the DeltaChat app
and then the account is created.
- by letting Delta Chat otherwise create an account
and register it with a {{ config.mail_domain }} mail server.
In either case, we process the newly created email address.
No phone numbers,
other email addresses,
or other identifiable data
is currently required.
The legal basis for the processing is
Art. 6 (1) lit. b GDPR,
as you have a usage contract with us
by using our services.
## 3.2 Processing of E-Mail-Messages
In addition,
we will process data
to keep the server infrastructure operational
for purposes of e-mail dispatch
and abuse prevention.
- Therefore,
it is necessary to process the content and/or metadata
(e.g., headers of the email as well as smtp chatter)
of E-Mail-Messages in transit.
- We will keep logs of messages in transit for a limited time.
These logs are used to debug delivery problems and software bugs.
In addition,
we process data to protect the systems from excessive use.
Therefore, limits are enforced:
- rate limits
- storage limits
- message size limits
- any other limit neccessary for the whole server to function in a healthy way
and to prevent abuse.
The processing and use of the above permissions
are performed to provide the service.
The data processing is necessary for the use of our services,
therefore the legal basis of the processing is
Art. 6 (1) lit. b GDPR,
as you have a usage contract with us
by using our services.
The legal basis for the data processing
for the purposes of security and abuse prevention is
Art. 6 (1) lit. f GDPR.
Our legitimate interest results
from the aforementioned purposes.
We will not use the collected data
for the purpose of drawing conclusions
about your person.
## 3. Processing when using our Website
When you visit our website,
the browser used on your end device
automatically sends information to the server of our website.
This information is temporarily stored in a so-called log file.
The following information is collected and stored
until it is automatically deleted
(*usually 7 days*):
- used type of browser,
- used operating system,
- access date and time as well as
- country of origin and IP address,
- the requested file name or HTTP resource,
- the amount of data transferred,
- the access status (file transferred, file not found, etc.) and
- the page from which the file was requested.
This website is hosted by an external service provider (hoster).
The personal data collected on this website is stored
on the hoster's servers.
Our hoster will process your data
only to the extent necessary to fulfill its obligations
to perform under our instructions.
In order to ensure data protection-compliant processing,
we have concluded a data processing agreement with our hoster.
The aforementioned data is processed by us for the following purposes:
- Ensuring a reliable connection setup of the website,
- ensuring a convenient use of our website,
- checking and ensuring system security and stability, and
- for other administrative purposes.
The legal basis for the data processing is
Art. 6 (1) lit. f GDPR.
Our legitimate interest results
from the aforementioned purposes of data collection.
We will not use the collected data
for the purpose of drawing conclusions about your person.
## 4. Transfer of Data
Your personal data
will not be transferred to third parties
for purposes other than those listed below:
a) you have given your express consent
in accordance with Art. 6 para. 1 sentence 1 lit. a GDPR,
b) the disclosure is necessary for the assertion, exercise or defence of legal claims
pursuant to Art. 6 (1) sentence 1 lit. f GDPR
and there is no reason to assume that you have
an overriding interest worthy of protection
in the non-disclosure of your data,
c) in the event that there is a legal obligation to disclose your data
pursuant to Art. 6 para. 1 sentence 1 lit. c GDPR,
as well as
d) this is legally permissible and necessary
in accordance with Art. 6 Para. 1 S. 1 lit. b GDPR
for the processing of contractual relationships with you,
e) this is carried out by a service provider
acting on our behalf and on our exclusive instructions,
whom we have carefully selected (Art. 28 (1) GDPR)
and with whom we have concluded a corresponding contract on commissioned processing (Art. 28 (3) GDPR),
which obliges our contractor,
among other things,
to implement appropriate security measures
and grants us comprehensive control powers.
## 5. Rights of the data subject
The rights arise from Articles 12 to 23 GDPR.
Since no personal data is stored on our servers,
even in encrypted form,
there is no need to provide information
on these or possible objections.
A deletion can be made
directly in the Delta Chat email messenger.
a) request information about your personal data processed by us
in accordance with Art. 15 GDPR.
In particular,
you can request information about the processing purposes,
the category of personal data,
the categories of recipients to whom your data have been or will be disclosed,
the planned storage period,
the existence of a right to rectification, erasure, restriction of processing or objection,
the existence of a right of complaint,
the origin of your data if it has not been collected by us,
as well as the existence of automated decision-making including profiling
and, if applicable,
meaningful information about its details;
b) in accordance with Art. 16 of the GDPR,
immediately request the correction
of inaccurate or incomplete personal data stored by us;
c) pursuant to Article 17 of the GDPR,
to request the erasure of your personal data stored by us,
unless the processing is necessary
for the exercise of the right to freedom of expression and information,
for compliance with a legal obligation,
for reasons of public interest,
or for the establishment, exercise or defence of legal claims;
d) pursuant to Art. 18 GDPR,
to request the restriction of the processing of your personal data,
insofar as the accuracy of the data is disputed by you,
the processing is unlawful,
but you object to its erasure
and we no longer require the data,
but you need it for the assertion, exercise or defence of legal claims
or you have objected to the processing pursuant to Art. 21 GDPR;
e) pursuant to Art. 20 GDPR,
to receive your personal data that you have provided to us
in a structured, common and machine-readable format
or to request that it be transferred to another controller;
f) in accordance with Art. 7 (3) of the GDPR,
to revoke your consent given to us at any time.
This has the consequence that we may no longer continue the data processing
based on this consent in the future; and
g) complain to a supervisory authority
in accordance with Article 77 of the GDPR.
As a rule,
you can contact the supervisory authority of your usual place of residence
or workplace
or our registered office for this purpose.
The supervisory authority responsible for our place of business
is the `{{ config.privacy_supervisor }}`.
If you have any questions or complaints, please feel free to contact us by email:
{{ config.privacy_mail }}
### 5.1 Right to object
If your personal data is processed on the basis of our legitimate interests
in accordance with Art. 6 (1) lit. f GDPR,
you have the right to object to the processing of your personal data
in accordance with Art. 21 GDPR,
provided that there are grounds for this based on your particular situation
or the objection is directed against direct advertising.
In the latter case,
you have a general right of objection,
which will be implemented by us
without specifying a particular situation.
If you wish to exercise your right of objection,
simply send an e-mail to: {{ config.privacy_mail }}
### 5.2 Right to withdraw
If your personal data is processed on the basis of your consent
in accordance with Art. 6 (1) lit. a GDPR
(e.g. via the mailing list),
you can withdraw your consent at any time
and without any disadvantages.
As a result,
we may no longer continue the data processing
that was based on this consent for the future.
However,
the withdrawal of your consent
does not affect the lawfulness of the processing
carried out on the basis of the consent until the withdrawal.
If you wish to make use of your right of withdrawal,
simply send an e-mail to: {{ config.privacy_mail }}
## 6. Validity of this privacy policy
This data protection declaration is valid
as of *December 2023*.
Due to the further development of our service and offers
or due to changed legal or official requirements,
it may become necessary to revise this data protection declaration from time to time.

1690
www/src/water.css Normal file

File diff suppressed because it is too large Load Diff