Compare commits

..

1 Commits

Author SHA1 Message Date
Keonik1
62f028bc67 www: add markdown language switcher 2025-10-07 22:16:06 +02:00
29 changed files with 243 additions and 707 deletions

View File

@@ -77,7 +77,7 @@ jobs:
cmdeploy init staging-ipv4.testrun.org cmdeploy init staging-ipv4.testrun.org
sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' chatmail.ini sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' chatmail.ini
- run: cmdeploy run --verbose --skip-dns-check - run: cmdeploy run
- name: set DNS entries - name: set DNS entries
run: | run: |

View File

@@ -75,7 +75,7 @@ jobs:
- run: cmdeploy init staging2.testrun.org - run: cmdeploy init staging2.testrun.org
- run: cmdeploy run --verbose --skip-dns-check - run: cmdeploy run --verbose
- name: set DNS entries - name: set DNS entries
run: | run: |

View File

@@ -2,54 +2,25 @@
## untagged ## untagged
- filtermail: run CPU-intensive handle_DATA in a thread pool executor
([#676](https://github.com/chatmail/relay/pull/676))
- don't use the complicated logging module in filtermail to exclude a potential source of errors.
([#674](https://github.com/chatmail/relay/pull/674))
- Specify nginx.conf to only handle `mail_domain`, www, and mta-sts domains
([#636](https://github.com/chatmail/relay/pull/636))
- Setup TURN server
([#621](https://github.com/chatmail/relay/pull/621))
- cmdeploy: make --ssh-host work with localhost
([#659](https://github.com/chatmail/relay/pull/659))
- Update iroh-relay to 0.35.0 - Update iroh-relay to 0.35.0
([#650](https://github.com/chatmail/relay/pull/650)) ([#650](https://github.com/chatmail/relay/pull/650))
- filtermail: accept mails from Protonmail
([#616](https://github.com/chatmail/relay/pull/655))
- Ignore all RCPT TO: parameters - Ignore all RCPT TO: parameters
([#651](https://github.com/chatmail/relay/pull/651)) ([#651](https://github.com/chatmail/relay/pull/651))
- Increase opendkim DNS Timeout from 5 to 60 seconds
([#672](https://github.com/chatmail/relay/pull/672))
- Add config parameter for Let's Encrypt ACME email
([#663](https://github.com/chatmail/relay/pull/663))
- Use max username length in newemail.py, not min - Use max username length in newemail.py, not min
([#648](https://github.com/chatmail/relay/pull/648)) ([#648](https://github.com/chatmail/relay/pull/648))
- Add startup for `fcgiwrap.service` because sometimes it did not start automatically.
([#657](https://github.com/chatmail/relay/pull/657))
- Add `cmdeploy init --force` command for recreating chatmail.ini
([#656](https://github.com/chatmail/relay/pull/656))
- Increase maxproc for reinjecting ports from 10 to 100 - Increase maxproc for reinjecting ports from 10 to 100
([#646](https://github.com/chatmail/relay/pull/646)) ([#646](https://github.com/chatmail/relay/pull/646))
- Add markdown tabs blocks for rendering multilingual pages.
Add russian language support to `index.md`, `privacy.md`, and `info.md`.
([#658](https://github.com/chatmail/relay/pull/658))
- Allow ports 143 and 993 to be used by `dovecot` process - Allow ports 143 and 993 to be used by `dovecot` process
([#639](https://github.com/chatmail/relay/pull/639)) ([#639](https://github.com/chatmail/relay/pull/639))
- Add `--skip-dns-check` argument to `cmdeploy run` command, which disables DNS record checking before installation.
([#661](https://github.com/chatmail/relay/pull/661))
## 1.7.0 2025-09-11 ## 1.7.0 2025-09-11
- Make www upload path configurable - Make www upload path configurable

View File

@@ -29,7 +29,6 @@ echobot = "chatmaild.echo:main"
chatmail-metrics = "chatmaild.metrics:main" chatmail-metrics = "chatmaild.metrics:main"
delete_inactive_users = "chatmaild.delete_inactive_users:main" delete_inactive_users = "chatmaild.delete_inactive_users:main"
lastlogin = "chatmaild.lastlogin:main" lastlogin = "chatmaild.lastlogin:main"
turnserver = "chatmaild.turnserver:main"
[project.entry-points.pytest11] [project.entry-points.pytest11]
"chatmaild.testplugin" = "chatmaild.tests.plugin" "chatmaild.testplugin" = "chatmaild.tests.plugin"

View File

@@ -33,6 +33,10 @@ class Config:
self.password_min_length = int(params["password_min_length"]) self.password_min_length = int(params["password_min_length"])
self.passthrough_senders = params["passthrough_senders"].split() self.passthrough_senders = params["passthrough_senders"].split()
self.passthrough_recipients = params["passthrough_recipients"].split() self.passthrough_recipients = params["passthrough_recipients"].split()
self.is_development_instance = (
params.get("is_development_instance", "true").lower() == "true"
)
self.languages = (params.get("languages", "EN").split())
self.www_folder = params.get("www_folder", "") self.www_folder = params.get("www_folder", "")
self.filtermail_smtp_port = int(params["filtermail_smtp_port"]) self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
self.filtermail_smtp_port_incoming = int( self.filtermail_smtp_port_incoming = int(
@@ -44,7 +48,6 @@ class Config:
) )
self.mtail_address = params.get("mtail_address") self.mtail_address = params.get("mtail_address")
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true" self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
self.acme_email = params.get("acme_email", "")
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true" self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
if "iroh_relay" not in params: if "iroh_relay" not in params:
self.iroh_relay = "https://" + params["mail_domain"] self.iroh_relay = "https://" + params["mail_domain"]

View File

@@ -2,6 +2,7 @@
import asyncio import asyncio
import base64 import base64
import binascii import binascii
import logging
import sys import sys
import time import time
from email import policy from email import policy
@@ -82,14 +83,8 @@ def check_openpgp_payload(payload: bytes):
return False return False
def check_armored_payload(payload: str, outgoing: bool): def check_armored_payload(payload: str):
"""Check the armored PGP message for invalid content. prefix = "-----BEGIN PGP MESSAGE-----\r\n\r\n"
:param payload: the armored PGP message
:param outgoing: whether the message is outgoing or incoming
:return: whether the message is a valid PGP message
"""
prefix = "-----BEGIN PGP MESSAGE-----\r\n"
if not payload.startswith(prefix): if not payload.startswith(prefix):
return False return False
payload = payload.removeprefix(prefix) payload = payload.removeprefix(prefix)
@@ -101,17 +96,6 @@ def check_armored_payload(payload: str, outgoing: bool):
return False return False
payload = payload.removesuffix(suffix) payload = payload.removesuffix(suffix)
# Disallow comments in outgoing messages
version_comment = "Version: "
if payload.startswith(version_comment):
splitindex = payload.find("\r\n") + 2
payload = payload[splitindex:]
if outgoing:
return False
while payload.startswith("\r\n"):
payload = payload.removeprefix("\r\n")
# Remove CRC24. # Remove CRC24.
payload = payload.rpartition("=")[0] payload = payload.rpartition("=")[0]
@@ -147,7 +131,7 @@ def is_securejoin(message):
return True return True
def check_encrypted(message, outgoing=True): def check_encrypted(message):
"""Check that the message is an OpenPGP-encrypted message. """Check that the message is an OpenPGP-encrypted message.
MIME structure of the message must correspond to <https://www.rfc-editor.org/rfc/rfc3156>. MIME structure of the message must correspond to <https://www.rfc-editor.org/rfc/rfc3156>.
@@ -174,7 +158,7 @@ def check_encrypted(message, outgoing=True):
if part.get_content_type() != "application/octet-stream": if part.get_content_type() != "application/octet-stream":
return False return False
if not check_armored_payload(part.get_payload(), outgoing=outgoing): if not check_armored_payload(part.get_payload()):
return False return False
else: else:
return False return False
@@ -228,7 +212,7 @@ class OutgoingBeforeQueueHandler:
self.send_rate_limiter = SendRateLimiter() self.send_rate_limiter = SendRateLimiter()
async def handle_MAIL(self, server, session, envelope, address, mail_options): async def handle_MAIL(self, server, session, envelope, address, mail_options):
log_info(f"handle_MAIL from {address}") logging.info(f"handle_MAIL from {address}")
envelope.mail_from = address envelope.mail_from = address
max_sent = self.config.max_user_send_per_minute max_sent = self.config.max_user_send_per_minute
if not self.send_rate_limiter.is_sending_allowed(address, max_sent): if not self.send_rate_limiter.is_sending_allowed(address, max_sent):
@@ -241,15 +225,11 @@ class OutgoingBeforeQueueHandler:
return "250 OK" return "250 OK"
async def handle_DATA(self, server, session, envelope): async def handle_DATA(self, server, session, envelope):
loop = asyncio.get_running_loop() logging.info("handle_DATA before-queue")
return await loop.run_in_executor(None, self.sync_handle_DATA, envelope)
def sync_handle_DATA(self, envelope):
log_info("handle_DATA before-queue")
error = self.check_DATA(envelope) error = self.check_DATA(envelope)
if error: if error:
return error return error
log_info("re-injecting the mail that passed checks") logging.info("re-injecting the mail that passed checks")
client = SMTPClient("localhost", self.config.postfix_reinject_port) client = SMTPClient("localhost", self.config.postfix_reinject_port)
client.sendmail( client.sendmail(
envelope.mail_from, envelope.rcpt_tos, envelope.original_content envelope.mail_from, envelope.rcpt_tos, envelope.original_content
@@ -258,10 +238,10 @@ class OutgoingBeforeQueueHandler:
def check_DATA(self, envelope): def check_DATA(self, envelope):
"""the central filtering function for e-mails.""" """the central filtering function for e-mails."""
log_info(f"Processing DATA message from {envelope.mail_from}") logging.info(f"Processing DATA message from {envelope.mail_from}")
message = BytesParser(policy=policy.default).parsebytes(envelope.content) message = BytesParser(policy=policy.default).parsebytes(envelope.content)
mail_encrypted = check_encrypted(message, outgoing=True) mail_encrypted = check_encrypted(message)
_, from_addr = parseaddr(message.get("from").strip()) _, from_addr = parseaddr(message.get("from").strip())
@@ -298,15 +278,11 @@ class IncomingBeforeQueueHandler:
self.config = config self.config = config
async def handle_DATA(self, server, session, envelope): async def handle_DATA(self, server, session, envelope):
loop = asyncio.get_running_loop() logging.info("handle_DATA before-queue")
return await loop.run_in_executor(None, self.sync_handle_DATA, envelope)
def sync_handle_DATA(self, envelope):
log_info("handle_DATA before-queue")
error = self.check_DATA(envelope) error = self.check_DATA(envelope)
if error: if error:
return error return error
log_info("re-injecting the mail that passed checks") logging.info("re-injecting the mail that passed checks")
# the smtp daemon on reinject_port_incoming gives it to dkim milter # the smtp daemon on reinject_port_incoming gives it to dkim milter
# which looks at source address to determine whether to verify or sign # which looks at source address to determine whether to verify or sign
@@ -322,10 +298,10 @@ class IncomingBeforeQueueHandler:
def check_DATA(self, envelope): def check_DATA(self, envelope):
"""the central filtering function for e-mails.""" """the central filtering function for e-mails."""
log_info(f"Processing DATA message from {envelope.mail_from}") logging.info(f"Processing DATA message from {envelope.mail_from}")
message = BytesParser(policy=policy.default).parsebytes(envelope.content) message = BytesParser(policy=policy.default).parsebytes(envelope.content)
mail_encrypted = check_encrypted(message, outgoing=False) mail_encrypted = check_encrypted(message)
if mail_encrypted or is_securejoin(message): if mail_encrypted or is_securejoin(message):
print("Incoming: Filtering encrypted mail.", file=sys.stderr) print("Incoming: Filtering encrypted mail.", file=sys.stderr)
@@ -364,19 +340,16 @@ class SendRateLimiter:
return False return False
def log_info(msg):
print(msg, file=sys.stderr)
def main(): def main():
args = sys.argv[1:] args = sys.argv[1:]
assert len(args) == 2 assert len(args) == 2
config = read_config(args[0]) config = read_config(args[0])
mode = args[1] mode = args[1]
logging.basicConfig(level=logging.WARN)
loop = asyncio.new_event_loop() loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop) asyncio.set_event_loop(loop)
assert mode in ["incoming", "outgoing"] assert mode in ["incoming", "outgoing"]
task = asyncmain_beforequeue(config, mode) task = asyncmain_beforequeue(config, mode)
loop.create_task(task) loop.create_task(task)
log_info("entering serving loop") logging.info("entering serving loop")
loop.run_forever() loop.run_forever()

View File

@@ -45,13 +45,16 @@ passthrough_senders =
# (space-separated, item may start with "@" to whitelist whole recipient domains) # (space-separated, item may start with "@" to whitelist whole recipient domains)
passthrough_recipients = xstore@testrun.org echo@{mail_domain} passthrough_recipients = xstore@testrun.org echo@{mail_domain}
# path to www directory - documented here: https://github.com/chatmail/relay/#custom-web-pages
#www_folder = www
# #
# Deployment Details # Deployment Details
# #
# A space-separated list of languages to be displayed on the site.
# Now available languages: EN RU
# You can also use the keyword "ALL"
# NOTE: The order of languages affects their order on the page
languages = EN
# SMTP outgoing filtermail and reinjection # SMTP outgoing filtermail and reinjection
filtermail_smtp_port = 10080 filtermail_smtp_port = 10080
postfix_reinject_port = 10025 postfix_reinject_port = 10025
@@ -63,9 +66,6 @@ postfix_reinject_port_incoming = 10026
# if set to "True" IPv6 is disabled # if set to "True" IPv6 is disabled
disable_ipv6 = False disable_ipv6 = False
# Your email adress, which will be used in acmetool to manage Let's Encrypt SSL certificates
acme_email =
# Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail # Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail
# service. # service.
# If you set it to anything else, the service will be disabled # If you set it to anything else, the service will be disabled

View File

@@ -7,7 +7,6 @@ from .config import read_config
from .dictproxy import DictProxy from .dictproxy import DictProxy
from .filedict import FileDict from .filedict import FileDict
from .notifier import Notifier from .notifier import Notifier
from .turnserver import turn_credentials
def _is_valid_token_timestamp(timestamp, now): def _is_valid_token_timestamp(timestamp, now):
@@ -76,12 +75,11 @@ class Metadata:
class MetadataDictProxy(DictProxy): class MetadataDictProxy(DictProxy):
def __init__(self, notifier, metadata, iroh_relay=None, turn_hostname=None): def __init__(self, notifier, metadata, iroh_relay=None):
super().__init__() super().__init__()
self.notifier = notifier self.notifier = notifier
self.metadata = metadata self.metadata = metadata
self.iroh_relay = iroh_relay self.iroh_relay = iroh_relay
self.turn_hostname = turn_hostname
def handle_lookup(self, parts): def handle_lookup(self, parts):
# Lpriv/43f5f508a7ea0366dff30200c15250e3/devicetoken\tlkj123poi@c2.testrun.org # Lpriv/43f5f508a7ea0366dff30200c15250e3/devicetoken\tlkj123poi@c2.testrun.org
@@ -100,11 +98,6 @@ class MetadataDictProxy(DictProxy):
): ):
# Handle `GETMETADATA "" /shared/vendor/deltachat/irohrelay` # Handle `GETMETADATA "" /shared/vendor/deltachat/irohrelay`
return f"O{self.iroh_relay}\n" return f"O{self.iroh_relay}\n"
elif keyname == "vendor/vendor.dovecot/pvt/server/vendor/deltachat/turn":
res = turn_credentials()
port = 3478
return f"O{self.turn_hostname}:{port}:{res}\n"
logging.warning(f"lookup ignored: {parts!r}") logging.warning(f"lookup ignored: {parts!r}")
return "N\n" return "N\n"
@@ -128,7 +121,6 @@ def main():
config = read_config(config_path) config = read_config(config_path)
iroh_relay = config.iroh_relay iroh_relay = config.iroh_relay
mail_domain = config.mail_domain
vmail_dir = config.mailboxes_dir vmail_dir = config.mailboxes_dir
if not vmail_dir.exists(): if not vmail_dir.exists():
@@ -142,10 +134,7 @@ def main():
notifier.start_notification_threads(metadata.remove_token_from_addr) notifier.start_notification_threads(metadata.remove_token_from_addr)
dictproxy = MetadataDictProxy( dictproxy = MetadataDictProxy(
notifier=notifier, notifier=notifier, metadata=metadata, iroh_relay=iroh_relay
metadata=metadata,
iroh_relay=iroh_relay,
turn_hostname=mail_domain,
) )
dictproxy.serve_forever_from_socket(socket) dictproxy.serve_forever_from_socket(socket)

View File

@@ -241,9 +241,8 @@ def test_cleartext_passthrough_senders(gencreds, handler, maildata):
def test_check_armored_payload(): def test_check_armored_payload():
prefix = "-----BEGIN PGP MESSAGE-----\r\n" payload = """-----BEGIN PGP MESSAGE-----\r
comment = "Version: ProtonMail\r\n" \r
payload = """\r
wU4DSqFx0d1yqAoSAQdAYkX/ZN/Az4B0k7X47zKyWrXxlDEdS3WOy0Yf2+GJTFgg\r wU4DSqFx0d1yqAoSAQdAYkX/ZN/Az4B0k7X47zKyWrXxlDEdS3WOy0Yf2+GJTFgg\r
Zk5ql0mLG8Ze+ZifCS0XMO4otlemSyJ0K1ZPdFMGzUDBTgNqzkFabxXoXRIBB0AM\r Zk5ql0mLG8Ze+ZifCS0XMO4otlemSyJ0K1ZPdFMGzUDBTgNqzkFabxXoXRIBB0AM\r
755wlX41X6Ay3KhnwBq7yEqSykVH6F3x11iHPKraLCAGZoaS8bKKNy/zg5slda1X\r 755wlX41X6Ay3KhnwBq7yEqSykVH6F3x11iHPKraLCAGZoaS8bKKNy/zg5slda1X\r
@@ -279,25 +278,16 @@ UN4fiB0KR9JyG2ayUdNJVkXZSZLnHyRgiaadlpUo16LVvw==\r
\r \r
""" """
commented_payload = prefix + comment + payload assert check_armored_payload(payload) == True
assert check_armored_payload(commented_payload, outgoing=False) == True
assert check_armored_payload(commented_payload, outgoing=True) == False
payload = prefix + payload
assert check_armored_payload(payload, outgoing=False) == True
assert check_armored_payload(payload, outgoing=True) == True
payload = payload.removesuffix("\r\n") payload = payload.removesuffix("\r\n")
assert check_armored_payload(payload, outgoing=False) == True assert check_armored_payload(payload) == True
assert check_armored_payload(payload, outgoing=True) == True
payload = payload.removesuffix("\r\n") payload = payload.removesuffix("\r\n")
assert check_armored_payload(payload, outgoing=False) == True assert check_armored_payload(payload) == True
assert check_armored_payload(payload, outgoing=True) == True
payload = payload.removesuffix("\r\n") payload = payload.removesuffix("\r\n")
assert check_armored_payload(payload, outgoing=False) == True assert check_armored_payload(payload) == True
assert check_armored_payload(payload, outgoing=True) == True
payload = """-----BEGIN PGP MESSAGE-----\r payload = """-----BEGIN PGP MESSAGE-----\r
\r \r
@@ -305,8 +295,7 @@ HELLOWORLD
-----END PGP MESSAGE-----\r -----END PGP MESSAGE-----\r
\r \r
""" """
assert check_armored_payload(payload, outgoing=False) == False assert check_armored_payload(payload) == False
assert check_armored_payload(payload, outgoing=True) == False
payload = """-----BEGIN PGP MESSAGE-----\r payload = """-----BEGIN PGP MESSAGE-----\r
\r \r
@@ -314,8 +303,7 @@ HELLOWORLD
-----END PGP MESSAGE-----\r -----END PGP MESSAGE-----\r
\r \r
""" """
assert check_armored_payload(payload, outgoing=False) == False assert check_armored_payload(payload) == False
assert check_armored_payload(payload, outgoing=True) == False
# Test payload using partial body length # Test payload using partial body length
# as generated by GopenPGP. # as generated by GopenPGP.
@@ -357,5 +345,4 @@ myLbG7cJB787QjplEyVe2P/JBO6xYvbkJLf9Q+HaviTO25rugRSrYsoKMDfO8VlQ\r
=6iHb\r =6iHb\r
-----END PGP MESSAGE-----\r -----END PGP MESSAGE-----\r
""" """
assert check_armored_payload(payload, outgoing=False) == True assert check_armored_payload(payload) == True
assert check_armored_payload(payload, outgoing=True) == True

View File

@@ -1,9 +0,0 @@
#!/usr/bin/env python3
import socket
def turn_credentials() -> str:
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client_socket:
client_socket.connect("/run/chatmail-turn/turn.socket")
with client_socket.makefile("rb") as file:
return file.readline().decode("utf-8")

View File

@@ -20,6 +20,7 @@ dependencies = [
"pytest-xdist", "pytest-xdist",
"execnet", "execnet",
"imap_tools", "imap_tools",
"pymdown-extensions",
] ]
[project.scripts] [project.scripts]

View File

@@ -128,7 +128,6 @@ def _install_remote_venv_with_chatmaild(config) -> None:
"echobot", "echobot",
"chatmail-metadata", "chatmail-metadata",
"lastlogin", "lastlogin",
"turnserver",
): ):
execpath = fn if fn != "filtermail-incoming" else "filtermail" execpath = fn if fn != "filtermail-incoming" else "filtermail"
params = dict( params = dict(
@@ -498,56 +497,6 @@ def check_config(config):
return config return config
def deploy_turn_server(config):
(url, sha256sum) = {
"x86_64": (
"https://github.com/chatmail/chatmail-turn/releases/download/v0.3/chatmail-turn-x86_64-linux",
"841e527c15fdc2940b0469e206188ea8f0af48533be12ecb8098520f813d41e4",
),
"aarch64": (
"https://github.com/chatmail/chatmail-turn/releases/download/v0.3/chatmail-turn-aarch64-linux",
"a5fc2d06d937b56a34e098d2cd72a82d3e89967518d159bf246dc69b65e81b42",
),
}[host.get_fact(facts.server.Arch)]
need_restart = False
existing_sha256sum = host.get_fact(Sha256File, "/usr/local/bin/chatmail-turn")
if existing_sha256sum != sha256sum:
server.shell(
name="Download chatmail-turn",
commands=[
f"(curl -L {url} >/usr/local/bin/chatmail-turn.new && (echo '{sha256sum} /usr/local/bin/chatmail-turn.new' | sha256sum -c) && mv /usr/local/bin/chatmail-turn.new /usr/local/bin/chatmail-turn)",
"chmod 755 /usr/local/bin/chatmail-turn",
],
)
need_restart = True
source_path = importlib.resources.files(__package__).joinpath(
"service", "turnserver.service.f"
)
content = source_path.read_text().format(mail_domain=config.mail_domain).encode()
systemd_unit = files.put(
name="Upload turnserver.service",
src=io.BytesIO(content),
dest="/etc/systemd/system/turnserver.service",
user="root",
group="root",
mode="644",
)
need_restart |= systemd_unit.changed
systemd.service(
name="Setup turnserver service",
service="turnserver.service",
running=True,
enabled=True,
restarted=need_restart,
daemon_reload=systemd_unit.changed,
)
def deploy_mtail(config): def deploy_mtail(config):
# Uninstall mtail package, we are going to install a static binary. # Uninstall mtail package, we are going to install a static binary.
apt.packages(name="Uninstall mtail", packages=["mtail"], present=False) apt.packages(name="Uninstall mtail", packages=["mtail"], present=False)
@@ -724,8 +673,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
packages=["rsync"], packages=["rsync"],
) )
deploy_turn_server(config)
# Run local DNS resolver `unbound`. # Run local DNS resolver `unbound`.
# `resolvconf` takes care of setting up /etc/resolv.conf # `resolvconf` takes care of setting up /etc/resolv.conf
# to use 127.0.0.1 as the resolver. # to use 127.0.0.1 as the resolver.
@@ -780,7 +727,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
# Deploy acmetool to have TLS certificates. # Deploy acmetool to have TLS certificates.
tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"] tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"]
deploy_acmetool( deploy_acmetool(
email=config.acme_email,
domains=tls_domains, domains=tls_domains,
) )
@@ -819,7 +765,7 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
if build_dir: if build_dir:
www_path = build_webpages(src_dir, build_dir, config) www_path = build_webpages(src_dir, build_dir, config)
# if it is not a hugo page, upload it as is # if it is not a hugo page, upload it as is
files.rsync(f"{www_path}/", "/var/www/html", flags=["-avz", "--chown=www-data"]) files.rsync(f"{www_path}/", "/var/www/html", flags=["-avz"])
_install_remote_venv_with_chatmaild(config) _install_remote_venv_with_chatmaild(config)
debug = False debug = False
@@ -867,13 +813,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
restarted=nginx_need_restart, restarted=nginx_need_restart,
) )
systemd.service(
name="Start and enable fcgiwrap",
service="fcgiwrap.service",
running=True,
enabled=True,
)
systemd.service( systemd.service(
name="Restart echobot if postfix and dovecot were just started", name="Restart echobot if postfix and dovecot were just started",
service="echobot.service", service="echobot.service",

View File

@@ -19,7 +19,7 @@ from packaging import version
from termcolor import colored from termcolor import colored
from . import dns, remote from . import dns, remote
from .sshexec import SSHExec, LocalExec from .sshexec import SSHExec
# #
# cmdeploy sub commands and options # cmdeploy sub commands and options
@@ -32,30 +32,17 @@ def init_cmd_options(parser):
action="store", action="store",
help="fully qualified DNS domain name for your chatmail instance", help="fully qualified DNS domain name for your chatmail instance",
) )
parser.add_argument(
"--force",
dest="recreate_ini",
action="store_true",
help="force reacreate ini file",
)
def init_cmd(args, out): def init_cmd(args, out):
"""Initialize chatmail config file.""" """Initialize chatmail config file."""
mail_domain = args.chatmail_domain mail_domain = args.chatmail_domain
inipath = args.inipath
if args.inipath.exists(): if args.inipath.exists():
if not args.recreate_ini: print(f"Path exists, not modifying: {args.inipath}")
print(f"[WARNING] Path exists, not modifying: {inipath}")
return 1 return 1
else: else:
print( write_initial_config(args.inipath, mail_domain, overrides={})
f"[WARNING] Force argument was provided, deleting config file: {inipath}" out.green(f"created config file for {mail_domain} in {args.inipath}")
)
inipath.unlink()
write_initial_config(inipath, mail_domain, overrides={})
out.green(f"created config file for {mail_domain} in {inipath}")
def run_cmd_options(parser): def run_cmd_options(parser):
@@ -72,21 +59,17 @@ def run_cmd_options(parser):
help="install/upgrade the server, but disable postfix & dovecot for now", help="install/upgrade the server, but disable postfix & dovecot for now",
) )
parser.add_argument( parser.add_argument(
"--skip-dns-check", "--ssh-host",
dest="dns_check_disabled", dest="ssh_host",
action="store_true", help="specify an SSH host to deploy to; uses mail_domain from chatmail.ini by default",
help="disable checks nslookup for dns",
) )
add_ssh_host_option(parser)
def run_cmd(args, out): def run_cmd(args, out):
"""Deploy chatmail services on the remote server.""" """Deploy chatmail services on the remote server."""
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain sshexec = args.get_sshexec()
sshexec = get_sshexec(ssh_host)
require_iroh = args.config.enable_iroh_relay require_iroh = args.config.enable_iroh_relay
if not args.dns_check_disabled:
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain) remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
if not dns.check_initial_remote_data(remote_data, print=out.red): if not dns.check_initial_remote_data(remote_data, print=out.red):
return 1 return 1
@@ -97,11 +80,8 @@ def run_cmd(args, out):
env["CHATMAIL_REQUIRE_IROH"] = "True" if require_iroh else "" env["CHATMAIL_REQUIRE_IROH"] = "True" if require_iroh else ""
deploy_path = importlib.resources.files(__package__).joinpath("deploy.py").resolve() deploy_path = importlib.resources.files(__package__).joinpath("deploy.py").resolve()
pyinf = "pyinfra --dry" if args.dry_run else "pyinfra" pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
ssh_host = args.config.mail_domain if not args.ssh_host else args.ssh_host
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y" cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
if ssh_host in ["localhost", "@docker"]:
cmd = f"{pyinf} @local {deploy_path} -y"
if version.parse(pyinfra.__version__) < version.parse("3"): if version.parse(pyinfra.__version__) < version.parse("3"):
out.red("Please re-run scripts/initenv.sh to update pyinfra to version 3.") out.red("Please re-run scripts/initenv.sh to update pyinfra to version 3.")
return 1 return 1
@@ -109,7 +89,6 @@ def run_cmd(args, out):
try: try:
retcode = out.check_call(cmd, env=env) retcode = out.check_call(cmd, env=env)
if retcode == 0: if retcode == 0:
if not args.disable_mail:
print("\nYou can try out the relay by talking to this echo bot: ") print("\nYou can try out the relay by talking to this echo bot: ")
sshexec = SSHExec(args.config.mail_domain, verbose=args.verbose) sshexec = SSHExec(args.config.mail_domain, verbose=args.verbose)
print( print(
@@ -139,13 +118,11 @@ def dns_cmd_options(parser):
default=None, default=None,
help="write out a zonefile", help="write out a zonefile",
) )
add_ssh_host_option(parser)
def dns_cmd(args, out): def dns_cmd(args, out):
"""Check DNS entries and optionally generate dns zone file.""" """Check DNS entries and optionally generate dns zone file."""
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain sshexec = args.get_sshexec()
sshexec = get_sshexec(ssh_host, verbose=args.verbose)
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain) remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
if not remote_data: if not remote_data:
return 1 return 1
@@ -299,15 +276,6 @@ class Out:
return proc.returncode return proc.returncode
def add_ssh_host_option(parser):
parser.add_argument(
"--ssh-host",
dest="ssh_host",
help="Run commands on 'localhost', via '@docker', or on a specific SSH host "
"instead of chatmail.ini's mail_domain.",
)
def add_config_option(parser): def add_config_option(parser):
parser.add_argument( parser.add_argument(
"--config", "--config",
@@ -363,16 +331,6 @@ def get_parser():
return parser return parser
def get_sshexec(ssh_host: str, verbose=True):
if ssh_host in ["localhost", "@local"]:
return LocalExec(verbose, docker=False)
elif ssh_host == "@docker":
return LocalExec(verbose, docker=True)
if verbose:
print(f"[ssh] login to {ssh_host}")
return SSHExec(ssh_host, verbose=verbose)
def main(args=None): def main(args=None):
"""Provide main entry point for 'cmdeploy' CLI invocation.""" """Provide main entry point for 'cmdeploy' CLI invocation."""
parser = get_parser() parser = get_parser()
@@ -380,6 +338,12 @@ def main(args=None):
if not hasattr(args, "func"): if not hasattr(args, "func"):
return parser.parse_args(["-h"]) return parser.parse_args(["-h"])
def get_sshexec():
print(f"[ssh] login to {args.config.mail_domain}")
return SSHExec(args.config.mail_domain, verbose=args.verbose)
args.get_sshexec = get_sshexec
out = Out() out = Out()
kwargs = {} kwargs = {}
if args.func.__name__ not in ("init_cmd", "fmt_cmd"): if args.func.__name__ not in ("init_cmd", "fmt_cmd"):

View File

@@ -45,7 +45,8 @@ def check_full_zone(sshexec, remote_data, out, zonefile) -> int:
and return (exitcode, remote_data) tuple.""" and return (exitcode, remote_data) tuple."""
required_diff, recommended_diff = sshexec.logged( required_diff, recommended_diff = sshexec.logged(
remote.rdns.check_zonefile, kwargs=dict(zonefile=zonefile, verbose=False), remote.rdns.check_zonefile,
kwargs=dict(zonefile=zonefile, mail_domain=remote_data["mail_domain"]),
) )
returncode = 0 returncode = 0

View File

@@ -1,11 +1,5 @@
enable_relay = true enable_relay = true
http_bind_addr = "[::]:3340" http_bind_addr = "[::]:3340"
enable_stun = true
# Disable built-in STUN server in iroh-relay 0.35
# as we deploy our own TURN server instead.
# STUN server is going to be removed in iroh-relay 1.0
# and this line can be removed after upgrade.
enable_stun = false
enable_metrics = false enable_metrics = false
metrics_bind_addr = "127.0.0.1:9092" metrics_bind_addr = "127.0.0.1:9092"

View File

@@ -66,7 +66,7 @@ http {
index index.html index.htm; index index.html index.htm;
server_name {{ config.domain_name }} www.{{ config.domain_name }} mta-sts.{{ config.domain_name }}; server_name _;
access_log syslog:server=unix:/dev/log,facility=local7; access_log syslog:server=unix:/dev/log,facility=local7;

View File

@@ -13,7 +13,6 @@ OversignHeaders From
On-BadSignature reject On-BadSignature reject
On-KeyNotFound reject On-KeyNotFound reject
On-NoSignature reject On-NoSignature reject
DNSTimeout 60
# Signing domain, selector, and key (required). For example, perform signing # Signing domain, selector, and key (required). For example, perform signing
# for domain "example.com" with selector "2020" (2020._domainkey.example.com), # for domain "example.com" with selector "2020" (2020._domainkey.example.com),

View File

@@ -12,23 +12,23 @@ All functions of this module
import re import re
from .rshell import CalledProcessError, shell, log_progress from .rshell import CalledProcessError, shell
def perform_initial_checks(mail_domain, pre_command=""): def perform_initial_checks(mail_domain):
"""Collecting initial DNS settings.""" """Collecting initial DNS settings."""
assert mail_domain assert mail_domain
if not shell("dig", fail_ok=True, print=log_progress): if not shell("dig", fail_ok=True):
shell("apt-get update && apt-get install -y dnsutils", print=log_progress) shell("apt-get update && apt-get install -y dnsutils")
A = query_dns("A", mail_domain) A = query_dns("A", mail_domain)
AAAA = query_dns("AAAA", mail_domain) AAAA = query_dns("AAAA", mail_domain)
MTA_STS = query_dns("CNAME", f"mta-sts.{mail_domain}") MTA_STS = query_dns("CNAME", f"mta-sts.{mail_domain}")
WWW = query_dns("CNAME", f"www.{mail_domain}") WWW = query_dns("CNAME", f"www.{mail_domain}")
res = dict(mail_domain=mail_domain, A=A, AAAA=AAAA, MTA_STS=MTA_STS, WWW=WWW) res = dict(mail_domain=mail_domain, A=A, AAAA=AAAA, MTA_STS=MTA_STS, WWW=WWW)
res["acme_account_url"] = shell(pre_command + "acmetool account-url", fail_ok=True, print=log_progress) res["acme_account_url"] = shell("acmetool account-url", fail_ok=True)
res["dkim_entry"], res["web_dkim_entry"] = get_dkim_entry( res["dkim_entry"], res["web_dkim_entry"] = get_dkim_entry(
mail_domain, pre_command, dkim_selector="opendkim" mail_domain, dkim_selector="opendkim"
) )
if not MTA_STS or not WWW or (not A and not AAAA): if not MTA_STS or not WWW or (not A and not AAAA):
@@ -40,12 +40,11 @@ def perform_initial_checks(mail_domain, pre_command=""):
return res return res
def get_dkim_entry(mail_domain, pre_command, dkim_selector): def get_dkim_entry(mail_domain, dkim_selector):
try: try:
dkim_pubkey = shell( dkim_pubkey = shell(
f"{pre_command}openssl rsa -in /etc/dkimkeys/{dkim_selector}.private " f"openssl rsa -in /etc/dkimkeys/{dkim_selector}.private "
"-pubout 2>/dev/null | awk '/-/{next}{printf(\"%s\",$0)}'", "-pubout 2>/dev/null | awk '/-/{next}{printf(\"%s\",$0)}'"
print=log_progress
) )
except CalledProcessError: except CalledProcessError:
return return
@@ -62,7 +61,7 @@ def query_dns(typ, domain):
# Get autoritative nameserver from the SOA record. # Get autoritative nameserver from the SOA record.
soa_answers = [ soa_answers = [
x.split() x.split()
for x in shell(f"dig -r -q {domain} -t SOA +noall +authority +answer", print=log_progress).split( for x in shell(f"dig -r -q {domain} -t SOA +noall +authority +answer").split(
"\n" "\n"
) )
] ]
@@ -72,13 +71,13 @@ def query_dns(typ, domain):
ns = soa[0][4] ns = soa[0][4]
# Query authoritative nameserver directly to bypass DNS cache. # Query authoritative nameserver directly to bypass DNS cache.
res = shell(f"dig @{ns} -r -q {domain} -t {typ} +short", print=log_progress) res = shell(f"dig @{ns} -r -q {domain} -t {typ} +short")
if res: if res:
return res.split("\n")[0] return res.split("\n")[0]
return "" return ""
def check_zonefile(zonefile, verbose=True): def check_zonefile(zonefile, mail_domain):
"""Check expected zone file entries.""" """Check expected zone file entries."""
required = True required = True
required_diff = [] required_diff = []
@@ -90,7 +89,7 @@ def check_zonefile(zonefile, verbose=True):
continue continue
if not zf_line.strip() or zf_line.startswith(";"): if not zf_line.strip() or zf_line.startswith(";"):
continue continue
print(f"dns-checking {zf_line!r}") if verbose else log_progress("") print(f"dns-checking {zf_line!r}")
zf_domain, zf_typ, zf_value = zf_line.split(maxsplit=2) zf_domain, zf_typ, zf_value = zf_line.split(maxsplit=2)
zf_domain = zf_domain.rstrip(".") zf_domain = zf_domain.rstrip(".")
zf_value = zf_value.strip() zf_value = zf_value.strip()

View File

@@ -1,14 +1,7 @@
import sys
from subprocess import DEVNULL, CalledProcessError, check_output from subprocess import DEVNULL, CalledProcessError, check_output
def log_progress(data): def shell(command, fail_ok=False):
sys.stderr.write(".")
sys.stderr.flush()
def shell(command, fail_ok=False, print=print):
print(f"$ {command}") print(f"$ {command}")
args = dict(shell=True) args = dict(shell=True)
if fail_ok: if fail_ok:

View File

@@ -1,16 +0,0 @@
[Unit]
Description=A wrapper for the TURN server
After=network.target
[Service]
Type=simple
Restart=always
ExecStart=/usr/local/bin/chatmail-turn --realm {mail_domain} --socket /run/chatmail-turn/turn.socket
# Create /run/chatmail-turn
RuntimeDirectory=chatmail-turn
User=vmail
Group=vmail
[Install]
WantedBy=multi-user.target

View File

@@ -42,7 +42,6 @@ def bootstrap_remote(gateway, remote=remote):
def print_stderr(item="", end="\n"): def print_stderr(item="", end="\n"):
print(item, file=sys.stderr, end=end) print(item, file=sys.stderr, end=end)
sys.stderr.flush()
class SSHExec: class SSHExec:
@@ -71,6 +70,10 @@ class SSHExec:
raise self.FuncError(data) raise self.FuncError(data)
def logged(self, call, kwargs): def logged(self, call, kwargs):
def log_progress(data):
sys.stderr.write(".")
sys.stderr.flush()
title = call.__doc__ title = call.__doc__
if not title: if not title:
title = call.__name__ title = call.__name__
@@ -79,22 +82,6 @@ class SSHExec:
return self(call, kwargs, log_callback=print_stderr) return self(call, kwargs, log_callback=print_stderr)
else: else:
print_stderr(title, end="") print_stderr(title, end="")
res = self(call, kwargs, log_callback=remote.rshell.log_progress) res = self(call, kwargs, log_callback=log_progress)
print_stderr() print_stderr()
return res return res
class LocalExec:
def __init__(self, verbose=False, docker=False):
self.verbose = verbose
self.docker = docker
def logged(self, call, kwargs: dict):
where = "locally"
if self.docker:
if call == remote.rdns.perform_initial_checks:
kwargs['pre_command'] = "docker exec chatmail "
where = "in docker"
if self.verbose:
print(f"Running {where}: {call.__name__}(**{kwargs})")
return call(**kwargs)

View File

@@ -2,7 +2,6 @@ import datetime
import smtplib import smtplib
import socket import socket
import subprocess import subprocess
import time
import pytest import pytest
@@ -32,8 +31,7 @@ class TestSSHExecutor:
) )
out, err = capsys.readouterr() out, err = capsys.readouterr()
assert err.startswith("Collecting") assert err.startswith("Collecting")
# XXX could not figure out how capturing can be made to work properly assert err.endswith("....\n")
#assert err.endswith("....\n")
assert err.count("\n") == 1 assert err.count("\n") == 1
sshexec.verbose = True sshexec.verbose = True
@@ -42,8 +40,7 @@ class TestSSHExecutor:
) )
out, err = capsys.readouterr() out, err = capsys.readouterr()
lines = err.split("\n") lines = err.split("\n")
# XXX could not figure out how capturing can be made to work properly assert len(lines) > 4
#assert len(lines) > 4
assert remote.rdns.perform_initial_checks.__doc__ in lines[0] assert remote.rdns.perform_initial_checks.__doc__ in lines[0]
def test_exception(self, sshexec, capsys): def test_exception(self, sshexec, capsys):
@@ -72,7 +69,7 @@ def test_timezone_env(remote):
for line in remote.iter_output("env"): for line in remote.iter_output("env"):
print(line) print(line)
if line == "tz=:/etc/localtime": if line == "tz=:/etc/localtime":
return return True
pytest.fail("TZ is not set") pytest.fail("TZ is not set")
@@ -149,16 +146,6 @@ def test_reject_missing_dkim(cmsetup, maildata, from_addr):
s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg) s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg)
def try_n_times(n, f):
for _ in range(n - 1):
try:
return f()
except Exception:
time.sleep(1)
return f()
def test_rewrite_subject(cmsetup, maildata): def test_rewrite_subject(cmsetup, maildata):
"""Test that subject gets replaced with [...].""" """Test that subject gets replaced with [...]."""
user1, user2 = cmsetup.gen_users(2) user1, user2 = cmsetup.gen_users(2)
@@ -171,8 +158,7 @@ def test_rewrite_subject(cmsetup, maildata):
).as_string() ).as_string()
user1.smtp.sendmail(from_addr=user1.addr, to_addrs=[user2.addr], msg=sent_msg) user1.smtp.sendmail(from_addr=user1.addr, to_addrs=[user2.addr], msg=sent_msg)
# The message may need some time to get delivered by postfix. messages = user2.imap.fetch_all_messages()
messages = try_n_times(5, user2.imap.fetch_all_messages)
assert len(messages) == 1 assert len(messages) == 1
rcvd_msg = messages[0] rcvd_msg = messages[0]
assert "Subject: [...]" not in sent_msg assert "Subject: [...]" not in sent_msg
@@ -223,14 +209,8 @@ def test_expunged(remote, chatmail_config):
def test_deployed_state(remote): def test_deployed_state(remote):
try:
git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode() git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode()
except Exception:
git_hash = "unknown\n"
try:
git_diff = subprocess.check_output(["git", "diff"]).decode() git_diff = subprocess.check_output(["git", "diff"]).decode()
except Exception:
git_diff = ""
git_status = [git_hash.strip()] git_status = [git_hash.strip()]
for line in git_diff.splitlines(): for line in git_diff.splitlines():
git_status.append(line.strip().lower()) git_status.append(line.strip().lower())

View File

@@ -26,15 +26,10 @@ class TestCmdline:
def test_init_not_overwrite(self, capsys): def test_init_not_overwrite(self, capsys):
assert main(["init", "chat.example.org"]) == 0 assert main(["init", "chat.example.org"]) == 0
capsys.readouterr() capsys.readouterr()
assert main(["init", "chat.example.org"]) == 1 assert main(["init", "chat.example.org"]) == 1
out, err = capsys.readouterr() out, err = capsys.readouterr()
assert "path exists" in out.lower() assert "path exists" in out.lower()
assert main(["init", "chat.example.org", "--force"]) == 0
out, err = capsys.readouterr()
assert "deleting config file" in out.lower()
def test_www_folder(example_config, tmp_path): def test_www_folder(example_config, tmp_path):
reporoot = importlib.resources.files(__package__).joinpath("../../../../").resolve() reporoot = importlib.resources.files(__package__).joinpath("../../../../").resolve()

View File

@@ -89,14 +89,18 @@ class TestZonefileChecks:
def test_check_zonefile_all_ok(self, cm_data, mockdns_base): def test_check_zonefile_all_ok(self, cm_data, mockdns_base):
zonefile = cm_data.get("zftest.zone") zonefile = cm_data.get("zftest.zone")
parse_zonefile_into_dict(zonefile, mockdns_base) parse_zonefile_into_dict(zonefile, mockdns_base)
required_diff, recommended_diff = remote.rdns.check_zonefile(zonefile) required_diff, recommended_diff = remote.rdns.check_zonefile(
zonefile, "some.domain"
)
assert not required_diff and not recommended_diff assert not required_diff and not recommended_diff
def test_check_zonefile_recommended_not_set(self, cm_data, mockdns_base): def test_check_zonefile_recommended_not_set(self, cm_data, mockdns_base):
zonefile = cm_data.get("zftest.zone") zonefile = cm_data.get("zftest.zone")
zonefile_mocked = zonefile.split("; Recommended")[0] zonefile_mocked = zonefile.split("; Recommended")[0]
parse_zonefile_into_dict(zonefile_mocked, mockdns_base) parse_zonefile_into_dict(zonefile_mocked, mockdns_base)
required_diff, recommended_diff = remote.rdns.check_zonefile(zonefile) required_diff, recommended_diff = remote.rdns.check_zonefile(
zonefile, "some.domain"
)
assert not required_diff assert not required_diff
assert len(recommended_diff) == 8 assert len(recommended_diff) == 8

View File

@@ -11,6 +11,13 @@ from jinja2 import Template
from .genqr import gen_qr_png_data from .genqr import gen_qr_png_data
LANGUAGE_NAMES = {
"EN": " 🇬🇧 English",
"RU": " 🇷🇺 Русский",
# "UA": "Українська",
# "FR": "Français",
# "DE": "Deutsch",
}
def snapshot_dir_stats(somedir): def snapshot_dir_stats(somedir):
d = {} d = {}
@@ -22,12 +29,59 @@ def snapshot_dir_stats(somedir):
return d return d
def prepare_template(source): def prepare_template(source, locales_dir, languages=["EN"]):
assert source.exists(), source assert source.exists(), f"Template {source} not found."
render_vars = {} assert locales_dir.exists(), f"Locales directory {locales_dir} not found."
render_vars["pagename"] = "home" if source.stem == "index" else source.stem base_name = source.stem
render_vars["markdown_html"] = markdown.markdown(source.read_text()) render_vars = {
page_layout = source.with_name("page-layout.html").read_text() "pagename": "home" if base_name == "index" else base_name
}
selected_langs = (
sorted([d.name.upper() for d in locales_dir.iterdir() if d.is_dir()])
if "ALL" in [l.upper() for l in languages]
else [l.upper() for l in languages]
)
markdown_blocks = []
tabs_enabled = False
if len(selected_langs) > 1:
tabs_enabled = True
for lang_code in selected_langs:
lang_folder = locales_dir / lang_code
lang_file = lang_folder / f"{base_name}.md"
lang_name = LANGUAGE_NAMES.get(lang_code, lang_code)
if lang_file.exists():
content = lang_file.read_text().strip()
else:
print(f"[WARNING]: Missing file {lang_file}. Inserting fallback message.")
content = "Content for this language is not available, please contact your server administrator."
if tabs_enabled:
markdown_blocks.append(f"/// tab | {lang_name}\n{content}\n///")
continue
markdown_blocks.append(content)
if not markdown_blocks:
print("[WARNING] No valid language content found. Skipping file.")
return None, None
original_markdown = source.read_text()
combined_markdown = original_markdown.replace("%content placeholder%", "\n\n".join(markdown_blocks))
render_vars["markdown_html"] = markdown.markdown(
combined_markdown,
extensions=["pymdownx.blocks.tab"]
)
page_layout_path = source.with_name("page-layout.html")
assert page_layout_path.exists(), f"Missing template: {page_layout_path}"
page_layout = page_layout_path.read_text()
return render_vars, page_layout return render_vars, page_layout
@@ -80,6 +134,7 @@ def int_to_english(number):
def _build_webpages(src_dir, build_dir, config): def _build_webpages(src_dir, build_dir, config):
mail_domain = config.mail_domain mail_domain = config.mail_domain
languages = config.languages
assert src_dir.exists(), src_dir assert src_dir.exists(), src_dir
if not build_dir.exists(): if not build_dir.exists():
build_dir.mkdir() build_dir.mkdir()
@@ -87,18 +142,19 @@ def _build_webpages(src_dir, build_dir, config):
qr_path = build_dir.joinpath(f"qr-chatmail-invite-{mail_domain}.png") qr_path = build_dir.joinpath(f"qr-chatmail-invite-{mail_domain}.png")
qr_path.write_bytes(gen_qr_png_data(mail_domain).read()) qr_path.write_bytes(gen_qr_png_data(mail_domain).read())
locales_dir = src_dir / "locales"
for path in src_dir.iterdir(): for path in src_dir.iterdir():
if path.suffix == ".md": if path.suffix == ".md":
render_vars, content = prepare_template(path) render_vars, content = prepare_template(path, locales_dir, languages)
render_vars["username_min_length"] = int_to_english(
config.username_min_length if render_vars is None:
) continue
render_vars["username_max_length"] = int_to_english(
config.username_max_length render_vars["username_min_length"] = int_to_english(config.username_min_length)
) render_vars["username_max_length"] = int_to_english(config.username_max_length)
render_vars["password_min_length"] = int_to_english( render_vars["password_min_length"] = int_to_english(config.password_min_length)
config.password_min_length
)
target = build_dir.joinpath(path.stem + ".html") target = build_dir.joinpath(path.stem + ".html")
# recursive jinja2 rendering # recursive jinja2 rendering
@@ -110,9 +166,11 @@ def _build_webpages(src_dir, build_dir, config):
with target.open("w") as f: with target.open("w") as f:
f.write(content) f.write(content)
elif path.name != "page-layout.html":
elif path.name != "page-layout.html" and path.name != "locales":
target = build_dir.joinpath(path.name) target = build_dir.joinpath(path.name)
target.write_bytes(path.read_bytes()) target.write_bytes(path.read_bytes())
return build_dir return build_dir

View File

@@ -1,29 +1,8 @@
<img class="banner" src="collage-top.png"/> <img class="banner" src="collage-top.png"/>
## Dear [Delta Chat](https://get.delta.chat) users and newcomers ... %content placeholder%
{% if config.mail_domain != "nine.testrun.org" %} {% if config.is_development_instance == True %}
Welcome to instant, interoperable and [privacy-preserving](privacy.html) messaging :)
{% else %}
Welcome to the default onboarding server ({{ config.mail_domain }})
for Delta Chat users. For details how it avoids storing personal information
please see our [privacy policy](privacy.html).
{% endif %}
<a class="cta-button" href="DCACCOUNT:https://{{ config.mail_domain }}/new">Get a {{config.mail_domain}} chat profile</a>
If you are viewing this page on a different device
without a Delta Chat app,
you can also **scan this QR code** with Delta Chat:
<a href="DCACCOUNT:https://{{ config.mail_domain }}/new">
<img width=300 style="float: none;" src="qr-chatmail-invite-{{config.mail_domain}}.png" /></a>
🐣 **Choose** your Avatar and Name
💬 **Start** chatting with any Delta Chat contacts using [QR invite codes](https://delta.chat/en/help#howtoe2ee)
{% if config.mail_domain != "nine.testrun.org" %}
<div class="experimental">Note: this is only a temporary development chatmail service</div> <div class="experimental">Note: this is only a temporary development chatmail service</div>
{% endif %} {% endif %}

View File

@@ -1,43 +1,3 @@
<img class="banner" src="collage-info.png"/>
## More information %content placeholder%
{{ config.mail_domain }} provides a low-maintenance, resource efficient and
interoperable e-mail service for everyone. What's behind a `chatmail` is
effectively a normal e-mail address just like any other but optimized
for the usage in chats, especially DeltaChat.
### Rate and storage limits
- Un-encrypted messages are blocked to recipients outside
{{config.mail_domain}} but setting up contact via [QR invite codes](https://delta.chat/en/help#howtoe2ee)
allows your messages to pass freely to any outside recipients.
- You may send up to {{ config.max_user_send_per_minute }} messages per minute.
- You can store up to [{{ config.max_mailbox_size }} messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server).
- Messages are unconditionally removed latest {{ config.delete_mails_after }} days after arriving on the server.
Earlier, if storage may exceed otherwise.
### <a name="account-deletion"></a> Account deletion
If you remove a {{ config.mail_domain }} profile from within the Delta Chat app,
then the according account on the server, along with all associated data,
is automatically deleted {{ config.delete_inactive_users_after }} days afterwards.
If you use multiple devices
then you need to remove the according chat profile from each device
in order for all account data to be removed on the server side.
If you have any further questions or requests regarding account deletion
please send a message from your account to {{ config.privacy_mail }}.
### Who are the operators? Which software is running?
This chatmail provider is run by a small voluntary group of devs and sysadmins,
who [publically develop chatmail provider setups](https://github.com/deltachat/chatmail).
Chatmail setups aim to be very low-maintenance, resource efficient and
interoperable with any other standards-compliant e-mail service.

View File

@@ -84,3 +84,57 @@ code {
color: white !important; color: white !important;
font-weight: bold; font-weight: bold;
} }
.tabbed-set {
position: relative;
display: flex;
flex-wrap: wrap;
margin: 1em 0;
border-radius: 0.1rem;
}
.tabbed-set > input {
display: none;
}
.tabbed-set label {
width: auto;
padding: 0.9375em 1.25em 0.78125em;
font-weight: 700;
font-size: 0.84em;
white-space: nowrap;
border-bottom: 0.15rem solid transparent;
border-top-left-radius: 0.1rem;
border-top-right-radius: 0.1rem;
cursor: pointer;
transition: background-color 250ms, color 250ms;
}
.tabbed-set .tabbed-content {
width: 100%;
display: none;
box-shadow: 0 -.05rem #ddd;
}
.tabbed-set input {
position: absolute;
opacity: 0;
}
.tabbed-set input:checked:nth-child(n+1) + label {
color: red;
border-color: red;
}
@media screen {
.tabbed-set input:nth-child(n+1):checked + label + .tabbed-content {
order: 99;
display: block;
}
}
@media print {
.tabbed-content {
display: contents;
}
}

View File

@@ -1,271 +1,3 @@
<img class="banner" src="collage-privacy.png"/>
# Privacy Policy for {{ config.mail_domain }} %content placeholder%
{% if config.mail_domain == "nine.testrun.org" %}
Welcome to `{{config.mail_domain}}`, the default chatmail onboarding server for Delta Chat users.
It is operated on the side by a small sysops team
on a voluntary basis.
See [other chatmail servers](https://delta.chat/en/chatmail) for alternative server operators.
{% endif %}
## Summary: No personal data asked or collected
This chatmail server neither asks for nor retains personal information.
Chatmail servers exist to reliably transmit (store and deliver) end-to-end encrypted messages
between user's devices running the Delta Chat messenger app.
Technically, you may think of a Chatmail server as
an end-to-end encrypted "messaging router" at Internet-scale.
A chatmail server is very unlike classic e-mail servers (for example Google Mail servers)
that ask for personal data and permanently store messages.
A chatmail server behaves more like the Signal messaging server
but does not know about phone numbers and securely and automatically interoperates
with other chatmail and classic e-mail servers.
Unlike classic e-mail servers, this chatmail server
- unconditionally removes messages after {{ config.delete_mails_after }} days,
- prohibits sending out un-encrypted messages,
- does not store Internet addresses ("IP addresses"),
- does not process IP addresses in relation to email addresses.
Due to the resulting lack of personal data processing
this chatmail server may not require a privacy policy.
Nevertheless, we provide legal details below to make life easier
for data protection specialists and lawyers scrutinizing chatmail operations.
## 1. Name and contact information
Responsible for the processing of your personal data is:
```
{{ config.privacy_postal }}
```
E-mail: {{ config.privacy_mail }}
We have appointed a data protection officer:
```
{{ config.privacy_pdo }}
```
## 2. Processing when using chat e-mail services
We provide services optimized for the use from [Delta Chat](https://delta.chat) apps
and process only the data necessary
for the setup and technical execution of message delivery.
The purpose of the processing is that users can
read, write, manage, delete, send, and receive chat messages.
For this purpose,
we operate server-side software
that enables us to send and receive messages.
We process the following data and details:
- Outgoing and incoming messages (SMTP) are stored for transit
on behalf of their users until the message can be delivered.
- E-Mail-Messages are stored for the recipient and made accessible via IMAP protocols,
until explicitly deleted by the user or until a fixed time period is exceeded,
(*usually 4-8 weeks*).
- IMAP and SMTP protocols are password protected with unique credentials for each account.
- Users can retrieve or delete all stored messages
without intervention from the operators using standard IMAP client tools.
- Users can connect to a "realtime relay service"
to establish Peer-to-Peer connection between user devices,
allowing them to send and retrieve ephemeral messages
which are never stored on the chatmail server, also not in encrypted form.
### 2.1 Account setup
Creating an account happens in one of two ways on our mail servers:
- with a QR invitation token
which is scanned using the Delta Chat app
and then the account is created.
- by letting Delta Chat otherwise create an account
and register it with a {{ config.mail_domain }} mail server.
In either case, we process the newly created email address.
No phone numbers,
other email addresses,
or other identifiable data
is currently required.
The legal basis for the processing is
Art. 6 (1) lit. b GDPR,
as you have a usage contract with us
by using our services.
### 2.2 Processing of E-Mail-Messages
In addition,
we will process data
to keep the server infrastructure operational
for purposes of e-mail dispatch
and abuse prevention.
- Therefore,
it is necessary to process the content and/or metadata
(e.g., headers of the email as well as smtp chatter)
of E-Mail-Messages in transit.
- We will keep logs of messages in transit for a limited time.
These logs are used to debug delivery problems and software bugs.
In addition,
we process data to protect the systems from excessive use.
Therefore, limits are enforced:
- rate limits
- storage limits
- message size limits
- any other limit necessary for the whole server to function in a healthy way
and to prevent abuse.
The processing and use of the above permissions
are performed to provide the service.
The data processing is necessary for the use of our services,
therefore the legal basis of the processing is
Art. 6 (1) lit. b GDPR,
as you have a usage contract with us
by using our services.
The legal basis for the data processing
for the purposes of security and abuse prevention is
Art. 6 (1) lit. f GDPR.
Our legitimate interest results
from the aforementioned purposes.
We will not use the collected data
for the purpose of drawing conclusions
about your person.
## 3. Processing when using our Website
When you visit our website,
the browser used on your end device
automatically sends information to the server of our website.
This information is temporarily stored in a so-called log file.
The following information is collected and stored
until it is automatically deleted
(*usually 7 days*):
- used type of browser,
- used operating system,
- access date and time as well as
- country of origin and IP address,
- the requested file name or HTTP resource,
- the amount of data transferred,
- the access status (file transferred, file not found, etc.) and
- the page from which the file was requested.
This website is hosted by an external service provider (hoster).
The personal data collected on this website is stored
on the hoster's servers.
Our hoster will process your data
only to the extent necessary to fulfill its obligations
to perform under our instructions.
In order to ensure data protection-compliant processing,
we have concluded a data processing agreement with our hoster.
The aforementioned data is processed by us for the following purposes:
- Ensuring a reliable connection setup of the website,
- ensuring a convenient use of our website,
- checking and ensuring system security and stability, and
- for other administrative purposes.
The legal basis for the data processing is
Art. 6 (1) lit. f GDPR.
Our legitimate interest results
from the aforementioned purposes of data collection.
We will not use the collected data
for the purpose of drawing conclusions about your person.
## 4. Transfer of Data
We do not retain any personal data but e-mail messages waiting to be delivered
may contain personal data.
Any such residual personal data will not be transferred to third parties
for purposes other than those listed below:
a) you have given your express consent
in accordance with Art. 6 para. 1 sentence 1 lit. a GDPR,
b) the disclosure is necessary for the assertion, exercise or defence of legal claims
pursuant to Art. 6 (1) sentence 1 lit. f GDPR
and there is no reason to assume that you have
an overriding interest worthy of protection
in the non-disclosure of your data,
c) in the event that there is a legal obligation to disclose your data
pursuant to Art. 6 para. 1 sentence 1 lit. c GDPR,
as well as
d) this is legally permissible and necessary
in accordance with Art. 6 Para. 1 S. 1 lit. b GDPR
for the processing of contractual relationships with you,
e) this is carried out by a service provider
acting on our behalf and on our exclusive instructions,
whom we have carefully selected (Art. 28 (1) GDPR)
and with whom we have concluded a corresponding contract on commissioned processing (Art. 28 (3) GDPR),
which obliges our contractor,
among other things,
to implement appropriate security measures
and grants us comprehensive control powers.
## 5. Rights of the data subject
The rights arise from Articles 12 to 23 GDPR.
Since no personal data is stored on our servers,
even in encrypted form,
there is no need to provide information
on these or possible objections.
A deletion can be made
directly in the Delta Chat email messenger.
If you have any questions or complaints,
please feel free to contact us by email:
{{ config.privacy_mail }}
As a rule, you can contact the supervisory authority of your usual place of residence
or workplace
or our registered office for this purpose.
The supervisory authority responsible for our place of business
is the `{{ config.privacy_supervisor }}`.
## 6. Validity of this privacy policy
This data protection declaration is valid
as of *October 2024*.
Due to the further development of our service and offers
or due to changed legal or official requirements,
it may become necessary to revise this data protection declaration from time to time.