Compare commits

..

25 Commits

Author SHA1 Message Date
missytake b8dcdcba91 docs: add paragraph on how to migrate your local repository to the server 2026-06-03 17:09:54 +02:00
missytake 6a0bf5abe7 docs: webdev needs to be exposed via nginx if run on the relay 2026-06-03 17:09:54 +02:00
missytake e7829672f8 docs: use ssh_host = localhost in getting started docs
remove mentions of the build machine / deployment server separation
2026-06-03 17:09:54 +02:00
missytake 62d19fd910 docs: cmdeploy dns + test are kind of necessary 2026-06-03 17:09:54 +02:00
missytake e1a8d798d3 cmdeploy: add ssh_host chatmail.ini option to deploy remotely 2026-06-03 17:09:54 +02:00
Jagoda Estera Ślązak a9dd9fe3e0 docs: Update overview diagrams (#995)
Adds a detailed diagram describing
all paths a message can take,
that takes into account postfix services.

Additionally, adds OpenDKIM to dependency
diagram.

Fixes: #771
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
Co-authored-by: missytake <missytake@systemli.org>
2026-06-03 12:24:32 +02:00
missytake aa846c3478 fix: expire empty directories (#994)
* fix: respect --dry when expiring empty directories

Co-authored-by: j4n <j4n@systemli.org>
2026-06-03 10:42:28 +02:00
feld 921080125f Aggressive LMTP header cleanup (#816)
This will remove all headers possible during LMTP delivery, except:

- From: required or core does not process the message correctly.
  Also required for cleartext compatibility.
- Message-Id: required for clients to know which messages have been
  downloaded
- Chat-Is-Post-Message: is required for our attachment previews
- Content-Type: required
- For Cleartext compability: To, CC, In-Reply-To, References, Subject,
  and Date
- For Chatmail future expansion, allow Chat-*
- Permit the entire Secure-Join* namespace

Co-authored-by: holger krekel  <holger@merlinux.eu>
2026-06-03 08:43:36 +02:00
Jagoda Estera Ślązak d898f41064 fix: Always deploy unbound.conf.d/chatmail.conf (#993)
This fixes issue with negative cache
only disabled in ipv4-only mode.

Follow up to #992

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-02 12:02:07 +02:00
Jagoda Estera Ślązak e9e012234b feat: Disable negative cache in unbound (#992)
Related:
- https://github.com/chatmail/relay/issues/543
- https://github.com/chatmail/filtermail/pull/170

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-02 10:48:28 +02:00
Jagoda Estera Ślązak bb40c5bb21 fix: Check if all required ports are available for filtermail (#983)
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-01 11:10:25 +02:00
Jagoda Estera Ślązak a229f1bc45 chore(deps): Upgrade filtermail to v0.7 (#982)
## 0.7.0 - 2026-05-26

### Bug Fixes

- Do not crash if accepting new connection fails

### Documentation

- *(readme)* Remove docs for options removed in da9a116

### Features

- [**breaking**] Remove passthrough options that allowed unencrypted mail to pass

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-01 09:55:50 +02:00
link2xt 4ba19b0031 test: set socket security for IMAP and SMTP to "TLS" in "dclogin"
With "default" (like it was for SMTP) or not set (like it was for IMAP),
both TLS and STARTTLS are tried.
Trying STARTTLS against TLS port is going to timeout
because in STARTTLS server talks first,
but when connected to TLS port the server
waits for TLS client hello and does not send anything.

Should not actually matter in tests which connect successfully
on the first try because implicit TLS is tried first.
2026-05-28 22:29:08 +00:00
holger krekel 5eab3a5a25 try using cmlxc main branch fix for delete-server issue 2026-05-28 21:40:05 +02:00
holger krekel 30729d9be0 fix: core 2.50.0 does not have delete_server_after config anymore. 2026-05-28 21:40:05 +02:00
link2xt 4b04aae83b feat: reduce maximal_queue_lifetime from 5d to 2d
If the message is not delivered within 2 days,
it is unlikely to be delivered in 5 days either.
2026-05-20 19:27:58 +00:00
link2xt 0eed92171c fix: reduce maxproc for filtermail-transport LMTP client to 500
This further reduces it from 1000.
For small servers this may be needed if they have low memory.
For large servers may be increased manually for now.
2026-05-20 15:39:11 +00:00
link2xt a5b9a98baa fix: limit the number of LMTP clients for filtermail-transport to 1000
Postfix does not have jitter for deferred mails
and scans the queue periodically every
queue_run_delay (<https://www.postfix.org/postconf.5.html#queue_run_delay>).
As a result it is likely
to try delivering many deferred messages
at the same time.

Normally the number of outgoing connections
should be low even with unreachable destinations,
but after the server downtime
or if admin flushes the queue manually
it is possible that a lot of messages
to the same unreachable destination
expire at once and are moved
from "deferred" into the "active" queue.

Trying to deliver them all at once
may make the server run out of memory
by starting many LMTP clients.
Limiting the number of LMTP processes
turns OOM problem into head of line blocking problem.
Messages sent to reachable destinations
will be delayed as well,
but at least deferred messages will
get distributed over time.

In this case "active" queue may grow
(up to qmgr_message_active_limit defaulting to 20000),
but then admin may notice the problem
and solve it e.g. by making the destinations reachable
or setting up a transport map to route
messages for known dead servers into discard transport.

Eventually the problem should be solved
by filtermail-transport quickly returning temporary errors
for destinations which already have many messages queued,
then we can reduce "maxproc" further.
2026-05-19 22:17:04 +00:00
link2xt ab2d807084 fix: set relay restrictions per smtpd service with default reject
We never want to defer email with a tepporary error when it has destination
that we cannot deliver locally and don't want to relay.
To avoid doing this accidentally, set default action to "reject"
and then override it with the minimal restrictions per smtpd.

Submission ports already had smtpd_relay_restrictions=permit_sasl_authenticated,reject override.

Each smtpd port must have at least one of
reject, reject_unauth_destination, defer, defer_if_permit, defer_unauth_destination
according to <https://www.postfix.org/postconf.5.html#smtpd_relay_restrictions>.

I have set smtpd_relay_restrictions=reject_unauth_destination for port 25 and incoming reinject port,
and smtpd_relay_restrictions=permit_mynetworks,reject for outgoing reinject port.
2026-05-19 15:54:15 +00:00
j4n ce05b26c77 ci: auto-trigger docker build on release tag push
docker-dispatch.yaml previously only fired on push to main and manual
workflow_dispatch, so tagging 1.11.0 did not build the release image.
This change adds matching of X.Y.Z tag.
2026-05-19 14:58:05 +02:00
missytake 77ed93fb7a docs: add scripts/initenv.sh to upgrade instructions 2026-05-18 10:35:25 +02:00
missytake 39d1ecaa03 chore(release): prepare for 1.11.0 2026-05-15 17:13:58 +02:00
holger krekel a266ffd060 fix: fix #972 by increasing file descriptors for filtermail 2026-05-14 22:40:25 +02:00
holger krekel a47bb94143 feat: warn about any unused chatmail.ini parameter at the end of "cmdeploy run" 2026-05-14 20:58:47 +02:00
holger krekel 43ae9fee5c feat!: ignore passthrough_sender and passthrough_recipients to eliminate one more source of unencrypted messages
When running "cmdeploy run" operators will see a warning if their chatmail.ini contains these unused options.
2026-05-14 20:58:47 +02:00
23 changed files with 342 additions and 150 deletions
+2 -2
View File
@@ -20,9 +20,9 @@ concurrency:
jobs:
no-dns:
name: LXC deploy and test
uses: chatmail/cmlxc/.github/workflows/lxc-test.yml@v0.14.6
uses: chatmail/cmlxc/.github/workflows/lxc-test.yml@main
with:
cmlxc_version: v0.14.6
cmlxc_version: main
cmlxc_commands: |
cmlxc init
# single cmdeploy relay test
+3 -3
View File
@@ -29,7 +29,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha }}
persist-credentials: false
- name: download filtermail
run: curl -L https://github.com/chatmail/filtermail/releases/download/v0.6.6/filtermail-x86_64 -o /usr/local/bin/filtermail && chmod +x /usr/local/bin/filtermail
run: curl -L https://github.com/chatmail/filtermail/releases/download/v0.7.0/filtermail-x86_64 -o /usr/local/bin/filtermail && chmod +x /usr/local/bin/filtermail
- name: run chatmaild tests
working-directory: chatmaild
run: pipx run tox
@@ -57,9 +57,9 @@ jobs:
lxc-test:
name: LXC deploy and test
uses: chatmail/cmlxc/.github/workflows/lxc-test.yml@v0.14.6
uses: chatmail/cmlxc/.github/workflows/lxc-test.yml@main
with:
cmlxc_version: v0.14.6
cmlxc_version: main
cmlxc_commands: |
cmlxc init
# single cmdeploy relay test
+1
View File
@@ -9,6 +9,7 @@ name: Trigger Docker build
on:
push:
branches: [main]
tags: ['[0-9]+.[0-9]+.[0-9]+']
workflow_dispatch:
permissions: {}
+43 -1
View File
@@ -1,4 +1,46 @@
# Changelog for chatmail deployment
# Changelog for chatmail deployment
## [1.11.0] - 2026-05-15
### Breaking Changes
- [**breaking**] Drop passthrough_sender and passthrough_recipients chatmail.ini options to eliminate one more source of unencrypted messages
### Features
- Use filtermail for delivery to remote MTAs
- Expose metadata "maxsmtprecipients" value
- Support setup without domain, with only an IPv4 address (#963)
- *(doc/docker)* Introduce docker images in documentation
- DKIM-sign bounce messages (mainly "user does not exist")
- *(config)* Load default values from Config(), not chatmail.ini.f (#853)
- Make turn_socket_path configurable, and cleanup tests and turnserver code.
- Warn about any unused chatmail.ini parameter at the end of "cmdeploy run"
### Bug Fixes
- Make www tests work with editable instead of just plain installs
- Use path with no leading slash for mxdeliv
- Increase filtermail-transport concurrency limit
- Fix #972 by increasing file descriptors for filtermail
- *(mtail)* Correct boot ordering and deploy restart logic
- *(cmdeploy)* Stop and disable unbound-resolvconf
- *(nginx)* Properly redirect www to mail_domain
- *(dns)* Query correct NS if MNAME server is hidden (#954)
- Legacy token metadata storage used list type, but if no new setmetadata happened, the user would not be notified at all.
- *(logging)* Log all http requests to syslog
### Documentation
- Document how to upgrade to new version (#965)
### Other
- *(deps)* Upgrade to filtermail v0.6.4
### Refactor
- Introduce automated change-tracking across deployers
## 1.10.0 2026-04-30
+37 -35
View File
@@ -16,8 +16,10 @@ def read_config(inipath):
class Config:
def __init__(self, inipath, params):
self._inipath = inipath
raw_domain = params["mail_domain"]
params = dict(params)
raw_domain = params.pop("mail_domain")
self.mail_domain_bare = raw_domain
self.ssh_host = params.pop("ssh_host", raw_domain)
if is_valid_ipv4(raw_domain):
self.ipv4_relay = raw_domain
@@ -29,60 +31,59 @@ class Config:
self.mail_domain = raw_domain
self.postfix_myhostname = raw_domain
self.max_user_send_per_minute = int(params.get("max_user_send_per_minute", 60))
self.max_user_send_burst_size = int(params.get("max_user_send_burst_size", 10))
self.max_mailbox_size = params.get("max_mailbox_size", "500M")
self.max_message_size = int(params.get("max_message_size", 31457280))
self.delete_mails_after = params.get("delete_mails_after", "20")
self.delete_large_after = params.get("delete_large_after", "7")
self.max_user_send_per_minute = int(params.pop("max_user_send_per_minute", 60))
self.max_user_send_burst_size = int(params.pop("max_user_send_burst_size", 10))
self.max_mailbox_size = params.pop("max_mailbox_size", "500M")
self.max_message_size = int(params.pop("max_message_size", 31457280))
self.delete_mails_after = params.pop("delete_mails_after", "20")
self.delete_large_after = params.pop("delete_large_after", "7")
self.delete_inactive_users_after = int(
params.get("delete_inactive_users_after", 90)
params.pop("delete_inactive_users_after", 90)
)
self.username_min_length = int(params.get("username_min_length", 9))
self.username_max_length = int(params.get("username_max_length", 9))
self.password_min_length = int(params.get("password_min_length", 9))
self.passthrough_senders = params.get("passthrough_senders", "").split()
self.passthrough_recipients = params.get("passthrough_recipients", "").split()
self.www_folder = params.get("www_folder", "")
self.filtermail_smtp_port = int(params.get("filtermail_smtp_port", "10080"))
self.username_min_length = int(params.pop("username_min_length", 9))
self.username_max_length = int(params.pop("username_max_length", 9))
self.password_min_length = int(params.pop("password_min_length", 9))
self.www_folder = params.pop("www_folder", "")
self.filtermail_smtp_port = int(params.pop("filtermail_smtp_port", "10080"))
self.filtermail_smtp_port_incoming = int(
params.get("filtermail_smtp_port_incoming", "10081")
params.pop("filtermail_smtp_port_incoming", "10081")
)
self.filtermail_http_port_incoming = int(
params.get("filtermail_http_port_incoming", "10082")
params.pop("filtermail_http_port_incoming", "10082")
)
self.filtermail_lmtp_port_transport = int(
params.get("filtermail_lmtp_port_transport", "10083")
params.pop("filtermail_lmtp_port_transport", "10083")
)
self.postfix_reinject_port = int(params.get("postfix_reinject_port", "10025"))
self.postfix_reinject_port = int(params.pop("postfix_reinject_port", "10025"))
self.postfix_reinject_port_incoming = int(
params.get("postfix_reinject_port_incoming", "10026")
params.pop("postfix_reinject_port_incoming", "10026")
)
self.mtail_address = params.get("mtail_address")
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
self.acme_email = params.get("acme_email", "")
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
self.imap_compress = params.get("imap_compress", "false").lower() == "true"
self.turn_socket_path = params.get(
self.mtail_address = params.pop("mtail_address", None)
self.disable_ipv6 = params.pop("disable_ipv6", "false").lower() == "true"
self.acme_email = params.pop("acme_email", "")
self.imap_rawlog = params.pop("imap_rawlog", "false").lower() == "true"
self.imap_compress = params.pop("imap_compress", "false").lower() == "true"
self.turn_socket_path = params.pop(
"turn_socket_path", "/run/chatmail-turn/turn.socket"
)
if "iroh_relay" not in params:
iroh_relay = params.pop("iroh_relay", None)
if iroh_relay is None:
self.iroh_relay = "https://" + raw_domain
self.enable_iroh_relay = True
else:
self.iroh_relay = params["iroh_relay"].strip()
self.iroh_relay = iroh_relay.strip()
self.enable_iroh_relay = False
self.privacy_postal = params.get("privacy_postal")
self.privacy_mail = params.get("privacy_mail")
self.privacy_pdo = params.get("privacy_pdo")
self.privacy_supervisor = params.get("privacy_supervisor")
self.privacy_postal = params.pop("privacy_postal", None)
self.privacy_mail = params.pop("privacy_mail", None)
self.privacy_pdo = params.pop("privacy_pdo", None)
self.privacy_supervisor = params.pop("privacy_supervisor", None)
# TLS certificate management.
# If tls_external_cert_and_key is set, use externally managed certs.
# Otherwise derived from the domain name:
# - Domains starting with "_" use self-signed certificates
# - All other domains use ACME.
external = params.get("tls_external_cert_and_key", "").strip()
external = params.pop("tls_external_cert_and_key", "").strip()
if external:
parts = external.split()
@@ -103,11 +104,12 @@ class Config:
self.tls_key_path = f"/var/lib/acme/live/{raw_domain}/privkey"
# deprecated option
mbdir = params.get("mailboxes_dir", f"/home/vmail/mail/{raw_domain}")
mbdir = params.pop("mailboxes_dir", f"/home/vmail/mail/{raw_domain}")
self.mailboxes_dir = Path(mbdir.strip())
# old unused option (except for first migration from sqlite to maildir store)
self.passdb_path = Path(params.get("passdb_path", "/home/vmail/passdb.sqlite"))
self.passdb_path = Path(params.pop("passdb_path", "/home/vmail/passdb.sqlite"))
self._unused_keys = list(params)
@property
def max_mailbox_size_mb(self):
+10
View File
@@ -168,6 +168,16 @@ class Expiry:
if mbox.last_login and mbox.last_login < cutoff_without_login:
self.remove_mailbox(mbox.basedir)
return
elif mbox.last_login is None:
try:
if not self.dry:
os.rmdir(mbox.basedir)
self.del_mboxes += 1
except OSError:
print_info(
f"Skipped deleting {mbox.basedir}, doesn't have last_login but isn't empty"
)
return
mboxname = os.path.basename(mbox.basedir)
if self.verbose:
+3 -7
View File
@@ -3,6 +3,9 @@
# mail domain (MUST be set to fully qualified chat mail domain)
mail_domain = {mail_domain}
# Where to deploy the relay - if unspecified, mail_domain will be used.
ssh_host = localhost
#
# If you only do private test deploys, you don't need to modify any settings below
#
@@ -42,13 +45,6 @@ mail_domain = {mail_domain}
# minimum length a password must have
#password_min_length = 9
# list of chatmail addresses which can send outbound un-encrypted mail
#passthrough_senders =
# list of e-mail recipients for which to accept outbound un-encrypted mails
# (space-separated, item may start with "@" to whitelist whole recipient domains)
#passthrough_recipients =
# Use externally managed TLS certificates instead of built-in acmetool.
# Paths refer to files on the deployment server (not the build machine).
# Both files must already exist before running cmdeploy.
+6 -2
View File
@@ -45,8 +45,12 @@ def test_read_config_basic_using_defaults(tmp_path, maildomain):
assert example_config.username_min_length == 9
assert example_config.username_max_length == 9
assert example_config.password_min_length == 9
assert example_config.passthrough_recipients == []
assert example_config.passthrough_senders == []
assert example_config._unused_keys == []
def test_config_unused_keys(make_config):
config = make_config("chat.example.org", {"passthrough_senders": "x@y.org"})
assert config._unused_keys == ["passthrough_senders"]
def test_config_userstate_paths(make_config, tmp_path):
@@ -1,6 +1,7 @@
import itertools
import os
import random
import shutil
import time
from datetime import datetime
from fnmatch import fnmatch
@@ -9,6 +10,7 @@ from pathlib import Path
import pytest
from chatmaild.expire import (
Expiry,
FileEntry,
MailboxStat,
expire_to_target,
@@ -104,6 +106,32 @@ def test_stats_mailbox(mbox1):
assert mbox3.last_login is None
def test_mbox_without_password(mbox1, example_config, capsys):
password = Path(mbox1.basedir).joinpath("password")
os.remove(password)
mbox_rescan = MailboxStat(mbox1.basedir)
assert mbox_rescan.last_login is None
exp = Expiry(
example_config, dry=False, now=datetime.now().timestamp(), verbose=False
)
exp.process_mailbox_stat(mbox_rescan)
out, err = capsys.readouterr()
assert "doesn't have last_login but isn't empty" in err
assert os.path.isdir(mbox_rescan.basedir)
for entry in os.scandir(mbox_rescan.basedir):
if os.path.isdir(entry):
shutil.rmtree(entry)
else:
os.remove(entry)
exp.process_mailbox_stat(mbox_rescan)
out, err = capsys.readouterr()
assert "doesn't have last_login but isn't empty" not in err
assert not os.path.isdir(mbox_rescan.basedir)
def test_report_no_mailboxes(example_config):
args = (str(example_config._inipath),)
report_main(args)
+14 -4
View File
@@ -84,10 +84,19 @@ def run_cmd_options(parser):
add_ssh_host_option(parser)
def _warn_unused_settings(unused_keys, out):
if unused_keys:
names = ", ".join(unused_keys)
out.red(
f"WARNING: chatmail.ini contains settings that have no effect: {names}\n"
"Please remove them from chatmail.ini."
)
def run_cmd(args, out):
"""Deploy chatmail services on the remote server."""
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain_bare
ssh_host = args.ssh_host if args.ssh_host else args.config.ssh_host
sshexec = get_sshexec(ssh_host)
require_iroh = args.config.enable_iroh_relay
strict_tls = args.config.tls_cert_mode == "acme"
@@ -107,7 +116,7 @@ def run_cmd(args, out):
pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
if ssh_host == "localhost":
if ssh_host in ["localhost", "@local"]:
cmd = f"{pyinf} @local {deploy_path} -y"
if version.parse(pyinfra.__version__) < version.parse("3"):
@@ -125,6 +134,7 @@ def run_cmd(args, out):
out.green("Deploy completed.")
else:
out.green("Deploy completed, call `cmdeploy dns` next.")
_warn_unused_settings(args.config._unused_keys, out)
return 0
except subprocess.CalledProcessError:
out.red("Deploy failed")
@@ -148,7 +158,7 @@ def dns_cmd(args, out):
ipv4 = args.config.ipv4_relay
print(f"[WARNING] {ipv4} is not a domain, skipping DNS checks.")
return 0
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
ssh_host = args.ssh_host if args.ssh_host else args.config.ssh_host
sshexec = get_sshexec(ssh_host, verbose=args.verbose)
tls_cert_mode = args.config.tls_cert_mode
strict_tls = tls_cert_mode == "acme"
@@ -185,7 +195,7 @@ def status_cmd_options(parser):
def status_cmd(args, out):
"""Display status for online chatmail instance."""
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain_bare
ssh_host = args.ssh_host if args.ssh_host else args.config.ssh_host
sshexec = get_sshexec(ssh_host, verbose=args.verbose)
out.green(f"chatmail domain: {args.config.mail_domain}")
+10 -10
View File
@@ -171,16 +171,14 @@ class UnboundDeployer(Deployer):
"unbound-anchor -a /var/lib/unbound/root.key || true",
],
)
if self.config.disable_ipv6:
self.ensure_directory(
path="/etc/unbound/unbound.conf.d",
)
self.put_template(
"unbound/unbound.conf.j2",
"/etc/unbound/unbound.conf.d/chatmail.conf",
)
else:
self.remove_file("/etc/unbound/unbound.conf.d/chatmail.conf")
self.ensure_directory(
path="/etc/unbound/unbound.conf.d",
)
self.put_template(
"unbound/unbound.conf.j2",
"/etc/unbound/unbound.conf.d/chatmail.conf",
disable_ipv6=self.config.disable_ipv6,
)
def activate(self):
server.shell(
@@ -514,6 +512,8 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
(["master", "smtpd"], config.postfix_reinject_port_incoming),
("filtermail", config.filtermail_smtp_port),
("filtermail", config.filtermail_smtp_port_incoming),
("filtermail", config.filtermail_http_port_incoming),
("filtermail", config.filtermail_lmtp_port_transport),
]
for service, port in port_services:
print(f"Checking if port {port} is available for {service}...")
+3 -3
View File
@@ -20,10 +20,10 @@ class FiltermailDeployer(Deployer):
return
arch = host.get_fact(facts.server.Arch)
url = f"https://github.com/chatmail/filtermail/releases/download/v0.6.6/filtermail-{arch}"
url = f"https://github.com/chatmail/filtermail/releases/download/v0.7.0/filtermail-{arch}"
sha256sum = {
"x86_64": "05c7e7ac244606c2eeb275f2d282ffdbc2403e0169f1cdd3110ffcebdb994a92",
"aarch64": "8cf8bbda4d907beca547b365cc7e6753532a74b1712492d0d2f3d2d8a553fb3d",
"x86_64": "451f295a85b3b12dbb0f89e18ec319f742ee46dec218f20f7923bfb017a248bd",
"aarch64": "6833061b2a2028264fdeb32f0a6123e1ff73de57dace125364016300b748452e",
}[arch]
self.download_executable(url, self.bin_path, sha256sum)
@@ -6,6 +6,7 @@ ExecStart={{ bin_path }} {{ config_path }} transport
Restart=always
RestartSec=30
User=vmail
LimitNOFILE=524288
[Install]
WantedBy=multi-user.target
@@ -1,5 +1,23 @@
# List of headers for incoming messages
# that must be retained for functionality and compatibility reasons
/^From:/ DUNNO
/^Message-Id:/ DUNNO
/^Chat-Is-Post-Message:/ DUNNO
/^Chat-/ DUNNO
/^Content-Type:/ DUNNO
# For receiving clear-text messages (still supported in May 2026)
/^Subject:/ DUNNO
/^Date:/ DUNNO
/^To:/ DUNNO
/^CC:/ DUNNO
/^References:/ DUNNO
/^In-Reply-To:/ DUNNO
# Senders might support Autocrypt 1 but not RFC9788 (Header Protection)
/^Autocrypt:/ DUNNO
# SecureJoin V2 protocol headers (for backward compatibility)
/^Secure-Join/ DUNNO
# Ignore all other headers
/.*/ IGNORE
+17 -1
View File
@@ -53,7 +53,8 @@ smtpd_tls_exclude_ciphers = aNULL, RC4, MD5, DES
# See <https://www.postfix.org/FORWARD_SECRECY_README.html#server_fs>.
tls_preempt_cipherlist = yes
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
# Reject by default, override per smtpd in master.cf
smtpd_relay_restrictions = reject
myhostname = {{ config.postfix_myhostname }}
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
@@ -101,9 +102,24 @@ smtpd_peername_lookup = no
# so instead this is handled in filtermail.
# We use LMTP instead SMTP so we can communicate per-recipient errors back to postfix.
default_transport = lmtp-filtermail:inet:[127.0.0.1]:{{ config.filtermail_lmtp_port_transport }}
# All deliveries over lmtp-filtermail are treated
# as having the same destination [127.0.0.1],
# so it is not possible to limit per-destination concurrency here,
# it is a job for filtermail-transport.
# Total number of parallel deliveries is limited
# by "maxproc" column in /etc/postfix/master.cf for lmtp-filtermail.
# Settings below are to prevent Postfix queue manager
# from limiting the number of LMTP connections to filtermail-transport.
# Read <https://www.postfix.org/TUNING_README.html#rope> and
# <https://www.postfix.org/SCHEDULER_README.html> for the details
# of the Postfix algorithm that we effectively disable here.
lmtp-filtermail_initial_destination_concurrency=10000
lmtp-filtermail_destination_concurrency_limit=10000
# Do not try to deliver messages for more than 2 days.
maximal_queue_lifetime = 2d
{% if not config.ipv4_relay %}
# DKIM-sign locally generated mail (bounces, DSNs).
# These bypass smtpd, so they need explicit milter configuration.
+12 -1
View File
@@ -17,6 +17,7 @@ smtp inet n - y - - smtpd
-o smtpd_tls_security_level=encrypt
-o smtpd_tls_mandatory_protocols=>=TLSv1.2
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port_incoming }}
-o smtpd_relay_restrictions=reject_unauth_destination
submission inet n - y - 5000 smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
@@ -81,12 +82,14 @@ filter unix - n n - - lmtp
-o syslog_name=postfix/reinject
-o milter_macro_daemon_name=ORIGINATING
-o cleanup_service_name=authclean
-o smtpd_relay_restrictions=permit_mynetworks,reject
{% if not config.ipv4_relay %} -o smtpd_milters=unix:opendkim/opendkim.sock
{% endif %}
# Local SMTP server for reinjecting incoming filtered mail
127.0.0.1:{{ config.postfix_reinject_port_incoming }} inet n - n - 100 smtpd
-o syslog_name=postfix/reinject_incoming
-o smtpd_relay_restrictions=reject_unauth_destination
# Cleanup `Received` headers for authenticated mail
# to avoid leaking client IP.
@@ -102,7 +105,15 @@ filter unix - n n - - lmtp
authclean unix n - - - 0 cleanup
-o header_checks=regexp:/etc/postfix/submission_header_cleanup
lmtp-filtermail unix - - y - 10000 lmtp
# Reducing `maxproc` here may result in a head of line blocking
# when there are many messages sent to unreachable destinations
# at the same time.
# LMTP clients here talk to filtermail-transport.
# LMTP has no pipelining,
# so while filtermail-transport tries to deliver the message,
# possibly waiting for a long connection timeout
# or talking to a slow server, LMTP client cannot be reused.
lmtp-filtermail unix - - y - 500 lmtp
-o syslog_name=postfix/lmtp-filtermail
-o lmtp_header_checks=
-o lmtp_tls_security_level=none
@@ -232,6 +232,26 @@ def try_n_times(n, f):
return f()
def test_rewrite_subject(cmsetup, maildata):
"""Test that subject gets replaced with [...]."""
user1, user2 = cmsetup.gen_users(2)
sent_msg = maildata(
"encrypted.eml",
from_addr=user1.addr,
to_addr=user2.addr,
subject="Unencrypted subject",
).as_string()
user1.smtp.sendmail(from_addr=user1.addr, to_addrs=[user2.addr], msg=sent_msg)
# The message may need some time to get delivered by postfix.
messages = try_n_times(5, user2.imap.fetch_all_messages)
assert len(messages) == 1
rcvd_msg = messages[0]
assert "Subject: [...]" not in sent_msg
assert "Subject: [...]" in rcvd_msg
assert "Subject: Unencrypted subject" in sent_msg
assert "Subject: Unencrypted subject" not in rcvd_msg
def test_exceed_rate_limit(cmsetup, gencreds, maildata, chatmail_config):
+6 -6
View File
@@ -62,8 +62,8 @@ def maildomain(chatmail_config):
@pytest.fixture(scope="session")
def sshdomain(maildomain):
return os.environ.get("CHATMAIL_SSH", maildomain)
def sshdomain(chatmail_config):
return os.environ.get("CHATMAIL_SSH", chatmail_config.ssh_host)
@pytest.fixture
@@ -349,9 +349,9 @@ class ChatmailACFactory:
qr = (
f"dclogin:{addr}"
f"?p={password}&v=1"
f"&ih={domain}&ip=993"
f"&sh={domain}&sp=465"
f"&ic=3&ss=default"
f"&ih={domain}&ip=993&is=ssl"
f"&sh={domain}&sp=465&ss=ssl"
f"&ic=3"
)
future = account.add_transport_from_qr.future(qr)
else:
@@ -362,7 +362,7 @@ class ChatmailACFactory:
# ensure messages stay in INBOX so that they can be
# concurrently fetched via extra IMAP connections during tests
account.set_config("delete_server_after", "10")
account.set_config("bcc_self", "1")
accounts.append(account)
for future in futures:
@@ -1,4 +1,7 @@
# Managed by cmdeploy: disable IPv6 in unbound.
# Managed by cmdeploy
server:
{% if disable_ipv6 %}
interface: 127.0.0.1
do-ip6: no
{% endif %}
cache-max-negative-ttl: 0
+1
View File
@@ -60,6 +60,7 @@ and run the following commands:
::
git pull origin main --rebase --autostash
scripts/initenv.sh
scripts/cmdeploy run
If you don't want the latest development version,
+30 -56
View File
@@ -14,21 +14,14 @@ Minimal requirements and prerequisites
You will need the following:
- A Debian 12 **deployment server** with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
- Control over a domain through a DNS provider of your choice.
(there is experimental support for :ref:`IP-only relays <iponly>`).
- A Debian 12 server with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
IPv6 is encouraged if available. Chatmail relay servers only require
1GB RAM, one CPU, and perhaps 10GB storage for a few thousand active
chatmail addresses.
- A Linux or Unix **build machine** with key-based SSH access to the root
user of the deployment server.
You must add a passphrase-protected private key to your local ssh-agent because you
cant type in your passphrase during deployment.
(An ed25519 private key is required due to an `upstream bug in
paramiko <https://github.com/paramiko/paramiko/issues/2191>`_)
- Control over a domain through a DNS provider of your choice
(there is experimental support for :ref:`IP-only relays <iponly>`).
.. _setup:
@@ -38,7 +31,7 @@ Setup with ``scripts/cmdeploy``
We use ``chat.example.org`` as the chatmail domain in the following
steps. Please substitute it with your own domain.
1. Setup the initial DNS records for your deployment server.
1. Setup the initial DNS records for your relay.
The following is an example in the
familiar BIND zone file format with a TTL of 1 hour (3600 seconds).
Please substitute your domain and IP addresses.
@@ -58,22 +51,25 @@ steps. Please substitute it with your own domain.
The ``mta-sts`` CNAME and ``_mta-sts`` TXT records
are not needed for such domains.
2. On your local PC, clone the repository and bootstrap the Python
2. Login to the server with SSH, clone the repository and bootstrap the Python
virtualenv.
::
ssh root@chat.example.org
git clone https://github.com/chatmail/relay
cd relay
scripts/initenv.sh
3. On your local build machine (PC), create a chatmail configuration file
3. Then, create a chatmail configuration file
``chatmail.ini``:
::
scripts/cmdeploy init chat.example.org # <-- use your domain
.. note::
To use self-signed TLS certificates
instead of Let's Encrypt,
use a domain name starting with ``_``
@@ -84,13 +80,7 @@ steps. Please substitute it with your own domain.
See the :doc:`overview`
for details on certificate provisioning.
4. Verify that SSH root login to the deployment server server works:
::
ssh root@chat.example.org # <-- use your domain
5. From your local build machine, setup and configure the remote deployment server:
4. Now run the deployment script to install the relay to the server:
::
@@ -102,7 +92,6 @@ steps. Please substitute it with your own domain.
public).
Docker installation
-------------------
@@ -110,27 +99,33 @@ There is experimental support for running chatmail via Docker.
A monolithic image based on the above cmdeploy method is available `through a separate repository <https://github.com/chatmail/docker/pkgs/container/docker>`_.
See the `chatmail/docker README <https://github.com/chatmail/docker>`_ for full setup instructions.
Other helpful commands
----------------------
To check the status of your deployment server running the chatmail service:
Next Steps
----------
::
scripts/cmdeploy status
To display and check all recommended DNS records:
Now you should display and check all recommended DNS records
to enable federation with other relays:
::
scripts/cmdeploy dns
To test whether your chatmail service is working correctly:
You should also test whether your chatmail service is working correctly:
::
scripts/cmdeploy test
Other Helpful Commands
----------------------
To check the status of your chatmail relay:
::
scripts/cmdeploy status
To measure the performance of your chatmail service:
::
@@ -171,8 +166,9 @@ This starts a local live development cycle for chatmail web pages:
directory and generating HTML files and copying assets to the
``www/build`` directory.
- Starts a browser window automatically where you can “refresh” as
needed.
- if you are running scripts/cmdeploy webdev on the relay itself,
you need to configure a route in /etc/nginx/nginx.conf
to expose the build directory.
Custom web pages
----------------
@@ -190,7 +186,7 @@ Disable automatic address creation
--------------------------------------------------------
If you need to stop address creation, e.g. because some script is wildly
creating addresses, login with ssh to the deployment machine and run:
creating addresses, login with ssh to the relay and run:
::
@@ -246,25 +242,3 @@ The deploy will verify that both files exist on the server.
If you use such a setup, you must trigger the reload explicitly after renewal::
systemctl start tls-cert-reload.service
Migrating to a new build machine
----------------------------------
To move or add a build machine,
clone the relay repository on the new build machine, and copy the ``chatmail.ini`` file from the old build machine.
Make sure ``rsync`` is installed, then initialize the environment:
::
./scripts/initenv.sh
Run safety checks before a new deployment:
::
./scripts/cmdeploy dns
./scripts/cmdeploy status
If you keep multiple build machines (ie laptop and desktop), keep ``chatmail.ini`` in sync between
them.
+16 -2
View File
@@ -1,6 +1,6 @@
Migrating to a new machine
===========================
Migrating the relay to a new server
===================================
This migration tutorial provides a step-wise approach
to safely migrate a chatmail relay from one remote machine to another.
@@ -96,3 +96,17 @@ in this case, just run ``ssh-keygen -R "mail.example.org"`` as recommended.
If you have lowered the Time-to-Live for DNS records in step 1,
better use a higher value again (between 14400 and 86400 seconds) once you are sure everything works.
Migrating a local chatmail/relay repository to the server
=========================================================
To move the directory with your local chatmail/relay repository and ``chatmail.ini`` file,
clone the `relay repository <https://github.com/chatmail/relay/>` to the server where the relay is running,
and copy the ``chatmail.ini`` file from your local chatmail/relay repository there.
If you made local changes to the repository,
you can store them in a file with ``git diff origin/main > local-changes.patch``,
copy the file to the repository on the server,
and run ``git apply local-changes.patch``.
Then you can proceed with the `installation steps <setup>`.
+56 -15
View File
@@ -156,6 +156,7 @@ Chatmail relay dependency diagram
postfix --- |10083|filtermail-transport;
filtermail-outgoing --- |10025 reinject|postfix;
filtermail-incoming --- |10026 reinject|postfix;
postfix --- |milter opendkim.sock|OpenDKIM
dovecot --- |doveauth.socket|doveauth;
dovecot --- |message delivery|maildir["maildir
/home/vmail/.../user"];
@@ -179,26 +180,66 @@ Chatmail relay dependency diagram
style nginx-right fill:#f66;
style postfix fill:#f66;
style dovecot fill:#f66;
style OpenDKIM fill:#f66;
style notification-proxy fill:#f66;
Message between users on the same relay
---------------------------------------
Accepting and delivering mail
-----------------------------
.. mermaid::
:caption: This diagram shows the path a non-federated message takes.
:caption: This diagram shows all the paths a message can take.
graph LR;
sender --> |465|smtps/smtpd;
sender --> |587|submission/smtpd;
smtps/smtpd --> |10080|filtermail;
submission/smtpd --> |10080|filtermail;
filtermail --> |10025|smtpd_reinject;
smtpd_reinject --> cleanup;
cleanup --> qmgr;
qmgr --> smtpd_accepts_message;
qmgr --> |lmtp|dovecot;
dovecot --> recipient;
dovecot --> sender's_other_devices;
flowchart LR
subgraph chatmail relay
subgraph postfix
qmgr .-> lmtp-filtermail["lmtp/lmtp-filtermail (default_transport)"]
qmgr .-> lmtp["lmtp (local_transport)"]
lmtp --> cleanup["cleanup (lmtp_header_cleanup)"]
bounce
smtpd-submission["smtpd/submission"]
smtpd-smtps["smtpd/smtps"]
smtpd-reinject-outgoing["smtpd/reinject-outgoing"] --> authclean["cleanup/authclean (submission_header_cleanup)"]
authclean --> qmgr
smtpd-smtp["smtpd/smtp"]
smtpd-reinject-incoming["smtpd/reinject-incoming"] --> qmgr
end
lmtp-filtermail --LMTP inet:10083--> filtermail-transport
cleanup --LMTP unix:private/dovecot-lmtp --> dovecot
dovecot --> maildir
smtpd-submission --SMTP inet:10080--> filtermail-outgoing
smtpd-smtps --SMTP inet:10080--> filtermail-outgoing
filtermail-outgoing --SMTP inet:10025--> smtpd-reinject-outgoing
open-dkim["OpenDKIM (signing only)"] <--milter unix:opendkim/opendkim.sock--> smtpd-reinject-outgoing
bounce <--milter unix:opendkim/opendkim.sock--> open-dkim
bounce --> qmgr
nginx
smtpd-smtp -.SMTP inet:10081.-> filtermail-incoming
nginx -.HTTP inet:10082.-> filtermail-incoming
filtermail-incoming --SMTP inet:10026--> smtpd-reinject-incoming
end
filtermail-transport -.SMTP inet:25.-> mta1[Remote relay]
filtermail-transport -.HTTPS /mxdeliv.-> mta1
client[Client] -.SMTP inet:587.-> smtpd-submission
client -.SMTP inet:465.-> smtpd-smtps
client -.SMTP inet:443.-> nginx
nginx -.SMTP inet:465.-> smtpd-smtps
mta2[Remote relay] -.SMTP inet:25.-> smtpd-smtp
mta2 -.HTTPS /mxdeliv.-> nginx
style postfix fill:#363
style qmgr fill:#252
style authclean fill:#252
style cleanup fill:#252
style lmtp-filtermail fill:#252
style lmtp fill:#252
style bounce fill:#252
style smtpd-submission fill:#252
style smtpd-smtps fill:#252
style smtpd-reinject-outgoing fill:#252
style smtpd-reinject-incoming fill:#252
style smtpd-smtp fill:#252
style filtermail-outgoing fill:#225
style filtermail-incoming fill:#225
style filtermail-transport fill:#225
Operational details of a chatmail relay
----------------------------------------