Compare commits

..

27 Commits

Author SHA1 Message Date
j4n
645b60d293 docker: make compose work with cgroups (v2), conversion scripts/docs 2026-02-16 14:41:48 +01:00
j4n
f939c307f6 docker: don't overwrite existing DKIM keys on container start
opendkim-genkey was running unconditionally on every startup,
check if file exists and skip.
2026-02-16 14:41:48 +01:00
j4n
ae0b2345de docker: run install stage at build time, configure+activate at startup
Move the CMDEPLOY_STAGES=install execution into the Dockerfile these
operations baked into the image layer. On container start, only
configure and activate stages run by default. Users can override with
CMDEPLOY_STAGES="install,configure,activate" to force a full reinstall
without rebuilding the image.

Also fixes CERTS_MONITORING_TIMEOUT typo in docker-compose.yaml (was
"$CERTS MONITORING TIMEOUT"), and replaces the docker-commit workaround
in docs with CMDEPLOY_STAGES documentation.
2026-02-16 14:41:48 +01:00
j4n
e5ba9f9d03 docker: widen build context to repo root for build-time install stage
The Dockerfile will need access to chatmaild/ and cmdeploy/ source
trees to run CMDEPLOY_STAGES=install via pyinfra during image build,
moving install-time work out of container startup. The previous context
(./docker) only included helper scripts.

Also adds .dockerignore to exclude .git, data/, venv/ etc. from the
build context, and updates COPY paths accordingly.
2026-02-16 14:41:48 +01:00
j4n
e20256c484 feat(cmdeploy): guard against non-running systemd
This enables docker image building without systemd running, which would
make pyinfra SystemdEnabled fail.
2026-02-16 14:41:48 +01:00
j4n
1889f554a3 docker: remove echobot parts that were lingering in the feature branch 2026-02-16 14:41:48 +01:00
Keonik1
f26cb08500 cmdeploy: Add config parameters change_kernel_settings and fs_inotify_max_user_instances_and_watchers 2026-02-16 14:41:48 +01:00
missytake
60ff9821b1 cmdeploy: add config (, ) 2026-02-16 14:41:48 +01:00
missytake
f9fad1fd03 docker: use --network=host so chatmail-turn can use any port 2026-02-16 14:41:48 +01:00
missytake
8be7082d21 docker: open ports for TURN + STUN 2026-02-16 14:41:48 +01:00
missytake
6e5004dc9f docker: move all configuration to example.env 2026-02-16 14:41:48 +01:00
missytake
92b6825b5b doc: fix linebreak 2026-02-16 14:41:48 +01:00
missytake
8bba78ebaf docker: disable port check if docker is running. fix #694 2026-02-16 14:41:48 +01:00
missytake
615613bd66 Suggestions from @Keonik1
Co-authored-by: Keonik <57857901+Keonik1@users.noreply.github.com>
2026-02-16 14:41:48 +01:00
missytake
c5a8d00558 docker: enable DNS checks before cmdeploy run again 2026-02-16 14:41:48 +01:00
Keonik1
38fb191c86 fix unlink if default nginx conf is not exist
- https://github.com/chatmail/relay/pull/614#discussion_r2297828830
2026-02-16 14:41:48 +01:00
Keonik1
dbc386bd00 Fix issue with acmetool
- https://github.com/chatmail/relay/pull/614#discussion_r2279630626
2026-02-16 14:41:48 +01:00
Keonik1
1e617041bd Delete ssh connection from docker installation
- https://github.com/chatmail/relay/pull/614#discussion_r2269986372
- https://github.com/chatmail/relay/pull/614#discussion_r2269991175
- https://github.com/chatmail/relay/pull/614#discussion_r2269995037
- https://github.com/chatmail/relay/pull/614#discussion_r2270004922
2026-02-16 14:41:48 +01:00
Keonik1
959afe6f14 fix docs - nginx "restart" to "reload"
https://github.com/chatmail/relay/pull/614#discussion_r2269896158
2026-02-16 14:41:48 +01:00
Keonik1
c605d1a465 Fix bug with attaching certs 2026-02-16 14:41:48 +01:00
Keonik1
72ae869eab pass values to MAIL_DOMAIN and ACME_EMAIL from vars for docker-compose-default
https://github.com/chatmail/relay/pull/614#discussion_r2279591922
2026-02-16 14:41:48 +01:00
Keonik1
e1be8a24a1 change "restart nginx" to "reload nginx"
https://github.com/chatmail/relay/pull/614#discussion_r2269896158
2026-02-16 14:41:48 +01:00
Keonik1
3896071921 add RECREATE_VENV var
https://github.com/chatmail/relay/pull/614#discussion_r2279742769
2026-02-16 14:41:48 +01:00
Keonik1
0d5e544291 add 465 port
https://github.com/chatmail/relay/pull/614#discussion_r2279707059
2026-02-16 14:41:48 +01:00
Keonik1
31fc856993 add port 80 to docker-compose-default
https://github.com/chatmail/relay/pull/614#discussion_r2279656441
2026-02-16 14:41:48 +01:00
Keonik1
fb798bb6a3 rename dockerfile
https://github.com/chatmail/relay/pull/614#discussion_r2270031966
2026-02-16 14:41:48 +01:00
Keonik1
985e98ccb7 Add installation via docker compose (MVP 1) 2026-02-16 14:41:48 +01:00
29 changed files with 993 additions and 158 deletions

7
.dockerignore Normal file
View File

@@ -0,0 +1,7 @@
.git
data/
venv/
__pycache__
*.pyc
*.orig
.pytest_cache

View File

@@ -71,35 +71,26 @@ jobs:
- name: run deploy-chatmail offline tests - name: run deploy-chatmail offline tests
run: pytest --pyargs cmdeploy run: pytest --pyargs cmdeploy
- name: setup dependencies - run: |
run: | cmdeploy init staging-ipv4.testrun.org
ssh root@staging-ipv4.testrun.org apt update sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' chatmail.ini
ssh root@staging-ipv4.testrun.org apt install -y git python3.11-venv python3-dev gcc sed -i 's/#\s*mtail_address/mtail_address/' chatmail.ini
ssh root@staging-ipv4.testrun.org git clone https://github.com/chatmail/relay
ssh root@staging-ipv4.testrun.org "cd relay && git checkout " ${{ github.head_ref }}
ssh root@staging-ipv4.testrun.org "cd relay && scripts/initenv.sh"
- name: initialize config - run: cmdeploy run --verbose --skip-dns-check
run: |
ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy init staging-ipv4.testrun.org"
ssh root@staging-ipv4.testrun.org "sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' relay/chatmail.ini"
ssh root@staging-ipv4.testrun.org "sed -i 's/#\s*mtail_address/mtail_address/' relay/chatmail.ini"
- run: ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy run --verbose --skip-dns-check"
- name: set DNS entries - name: set DNS entries
run: | run: |
ssh root@staging-ipv4.testrun.org chown opendkim:opendkim -R /etc/dkimkeys ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org chown opendkim:opendkim -R /etc/dkimkeys
ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy dns --zonefile staging-generated.zone" cmdeploy dns --zonefile staging-generated.zone
ssh root@staging-ipv4.testrun.org cat relay/staging-generated.zone >> .github/workflows/staging-ipv4.testrun.org-default.zone cat staging-generated.zone >> .github/workflows/staging-ipv4.testrun.org-default.zone
cat .github/workflows/staging-ipv4.testrun.org-default.zone cat .github/workflows/staging-ipv4.testrun.org-default.zone
scp .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone scp .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone
ssh root@ns.testrun.org nsd-checkzone staging-ipv4.testrun.org /etc/nsd/staging-ipv4.testrun.org.zone ssh root@ns.testrun.org nsd-checkzone staging-ipv4.testrun.org /etc/nsd/staging-ipv4.testrun.org.zone
ssh root@ns.testrun.org systemctl reload nsd ssh root@ns.testrun.org systemctl reload nsd
- name: cmdeploy test - name: cmdeploy test
run: ssh root@staging-ipv4.testrun.org "cd relay && CHATMAIL_DOMAIN2=ci-chatmail.testrun.org scripts/cmdeploy test --slow" run: CHATMAIL_DOMAIN2=ci-chatmail.testrun.org cmdeploy test --slow
- name: cmdeploy dns - name: cmdeploy dns
run: ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy dns -v" run: cmdeploy dns -v

View File

@@ -76,7 +76,6 @@ jobs:
- run: | - run: |
cmdeploy init staging2.testrun.org cmdeploy init staging2.testrun.org
sed -i 's/^ssh_host/#ssh_host/' chatmail.ini
sed -i 's/#\s*mtail_address/mtail_address/' chatmail.ini sed -i 's/#\s*mtail_address/mtail_address/' chatmail.ini
- run: cmdeploy run --verbose --skip-dns-check - run: cmdeploy run --verbose --skip-dns-check

5
.gitignore vendored
View File

@@ -164,3 +164,8 @@ cython_debug/
#.idea/ #.idea/
chatmail.zone chatmail.zone
# docker
/data/
/custom/
.env

View File

@@ -121,6 +121,13 @@
Provide an "fsreport" CLI for more fine grained analysis of message files. Provide an "fsreport" CLI for more fine grained analysis of message files.
([#637](https://github.com/chatmail/relay/pull/637)) ([#637](https://github.com/chatmail/relay/pull/637))
- Add installation via docker compose (MVP 1). The instructions, known issues and limitations are located in `/docs`
([#614](https://github.com/chatmail/relay/pull/614))
- Add configuration parameters
([#614](https://github.com/chatmail/relay/pull/614)):
- `change_kernel_settings` - Whether to change kernel parameters during installation (default: `True`)
- `fs_inotify_max_user_instances_and_watchers` - Value for kernel parameters `fs.inotify.max_user_instances` and `fs.inotify.max_user_watches` (default: `65535`)
## 1.7.0 2025-09-11 ## 1.7.0 2025-09-11

View File

@@ -9,28 +9,30 @@ from chatmaild.user import User
def read_config(inipath): def read_config(inipath):
assert Path(inipath).exists(), inipath assert Path(inipath).exists(), inipath
cfg = iniconfig.IniConfig(inipath) cfg = iniconfig.IniConfig(inipath)
return Config(inipath, params=cfg.sections["params"]) params = cfg.sections["params"]
default_config_content = get_default_config_content(params["mail_domain"])
df_params = iniconfig.IniConfig("ini", data=default_config_content)["params"]
new_params = dict(df_params.items())
new_params.update(params)
return Config(inipath, params=new_params)
class Config: class Config:
def __init__(self, inipath, params): def __init__(self, inipath, params):
self._inipath = inipath self._inipath = inipath
self.mail_domain = params["mail_domain"] self.mail_domain = params["mail_domain"]
self.ssh_host = params.get("ssh_host", self.mail_domain)
self.max_user_send_per_minute = int(params.get("max_user_send_per_minute", 60)) self.max_user_send_per_minute = int(params.get("max_user_send_per_minute", 60))
self.max_user_send_burst_size = int(params.get("max_user_send_burst_size", 10)) self.max_user_send_burst_size = int(params.get("max_user_send_burst_size", 10))
self.max_mailbox_size = params.get("max_mailbox_size", "500M") self.max_mailbox_size = params["max_mailbox_size"]
self.max_message_size = int(params.get("max_message_size", 31457280)) self.max_message_size = int(params.get("max_message_size", "31457280"))
self.delete_mails_after = params.get("delete_mails_after", "20") self.delete_mails_after = params["delete_mails_after"]
self.delete_large_after = params.get("delete_large_after", "7") self.delete_large_after = params["delete_large_after"]
self.delete_inactive_users_after = int( self.delete_inactive_users_after = int(params["delete_inactive_users_after"])
params.get("delete_inactive_users_after", 100) self.username_min_length = int(params["username_min_length"])
) self.username_max_length = int(params["username_max_length"])
self.username_min_length = int(params.get("username_min_length", 9)) self.password_min_length = int(params["password_min_length"])
self.username_max_length = int(params.get("username_max_length", 9)) self.passthrough_senders = params["passthrough_senders"].split()
self.password_min_length = int(params.get("password_min_length", 9)) self.passthrough_recipients = params["passthrough_recipients"].split()
self.passthrough_senders = params.get("passthrough_senders", "").split()
self.passthrough_recipients = params.get("passthrough_recipients", "").split()
self.www_folder = params.get("www_folder", "") self.www_folder = params.get("www_folder", "")
self.filtermail_smtp_port = int(params.get("filtermail_smtp_port", "10080")) self.filtermail_smtp_port = int(params.get("filtermail_smtp_port", "10080"))
self.filtermail_smtp_port_incoming = int( self.filtermail_smtp_port_incoming = int(
@@ -45,6 +47,12 @@ class Config:
self.addr_v4 = os.environ.get("CHATMAIL_ADDR_V4", "") self.addr_v4 = os.environ.get("CHATMAIL_ADDR_V4", "")
self.addr_v6 = os.environ.get("CHATMAIL_ADDR_V6", "") self.addr_v6 = os.environ.get("CHATMAIL_ADDR_V6", "")
self.acme_email = params.get("acme_email", "") self.acme_email = params.get("acme_email", "")
self.change_kernel_settings = (
params.get("change_kernel_settings", "true").lower() == "true"
)
self.fs_inotify_max_user_instances_and_watchers = int(
params["fs_inotify_max_user_instances_and_watchers"]
)
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true" self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
self.imap_compress = params.get("imap_compress", "false").lower() == "true" self.imap_compress = params.get("imap_compress", "false").lower() == "true"
if "iroh_relay" not in params: if "iroh_relay" not in params:

View File

@@ -3,9 +3,6 @@
# mail domain (MUST be set to fully qualified chat mail domain) # mail domain (MUST be set to fully qualified chat mail domain)
mail_domain = {mail_domain} mail_domain = {mail_domain}
# Where to deploy the relay - if unspecified, mail_domain will be used.
ssh_host = localhost
# #
# If you only do private test deploys, you don't need to modify any settings below # If you only do private test deploys, you don't need to modify any settings below
# #
@@ -72,6 +69,16 @@ disable_ipv6 = False
# Your email adress, which will be used in acmetool to manage Let's Encrypt SSL certificates # Your email adress, which will be used in acmetool to manage Let's Encrypt SSL certificates
acme_email = acme_email =
#
# Kernel settings
#
# if you set "True", the kernel settings will be configured according to the values below
change_kernel_settings = True
# change fs.inotify.max_user_instances and fs.inotify.max_user_watches kernel settings
fs_inotify_max_user_instances_and_watchers = 65535
# Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail # Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail
# service. # service.
# If you set it to anything else, the service will be disabled # If you set it to anything else, the service will be disabled

View File

@@ -5,6 +5,11 @@ import os
from pyinfra.operations import files, server, systemd from pyinfra.operations import files, server, systemd
def has_systemd():
"""Returns False during Docker image builds or any other non-systemd environment."""
return os.path.isdir("/run/systemd/system")
def get_resource(arg, pkg=__package__): def get_resource(arg, pkg=__package__):
return importlib.resources.files(pkg).joinpath(arg) return importlib.resources.files(pkg).joinpath(arg)

View File

@@ -88,7 +88,7 @@ def run_cmd_options(parser):
def run_cmd(args, out): def run_cmd(args, out):
"""Deploy chatmail services on the remote server.""" """Deploy chatmail services on the remote server."""
ssh_host = args.ssh_host if args.ssh_host else args.config.ssh_host ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
sshexec = get_sshexec(ssh_host) sshexec = get_sshexec(ssh_host)
require_iroh = args.config.enable_iroh_relay require_iroh = args.config.enable_iroh_relay
if not args.dns_check_disabled: if not args.dns_check_disabled:
@@ -108,7 +108,9 @@ def run_cmd(args, out):
pyinf = "pyinfra --dry" if args.dry_run else "pyinfra" pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y" cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
if ssh_host in ["localhost", "@local", "@docker"]: if ssh_host in ["localhost", "@docker"]:
if ssh_host == "@docker":
env["CHATMAIL_DOCKER"] = "True"
cmd = f"{pyinf} @local {deploy_path} -y" cmd = f"{pyinf} @local {deploy_path} -y"
if version.parse(pyinfra.__version__) < version.parse("3"): if version.parse(pyinfra.__version__) < version.parse("3"):
@@ -149,7 +151,7 @@ def dns_cmd_options(parser):
def dns_cmd(args, out): def dns_cmd(args, out):
"""Check DNS entries and optionally generate dns zone file.""" """Check DNS entries and optionally generate dns zone file."""
ssh_host = args.ssh_host if args.ssh_host else args.config.ssh_host ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
sshexec = get_sshexec(ssh_host, verbose=args.verbose) sshexec = get_sshexec(ssh_host, verbose=args.verbose)
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain) remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
if not remote_data: if not remote_data:
@@ -183,7 +185,7 @@ def status_cmd_options(parser):
def status_cmd(args, out): def status_cmd(args, out):
"""Display status for online chatmail instance.""" """Display status for online chatmail instance."""
ssh_host = args.ssh_host if args.ssh_host else args.config.ssh_host ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
sshexec = get_sshexec(ssh_host, verbose=args.verbose) sshexec = get_sshexec(ssh_host, verbose=args.verbose)
out.green(f"chatmail domain: {args.config.mail_domain}") out.green(f"chatmail domain: {args.config.mail_domain}")
@@ -203,7 +205,6 @@ def test_cmd_options(parser):
action="store_true", action="store_true",
help="also run slow tests", help="also run slow tests",
) )
add_ssh_host_option(parser)
def test_cmd(args, out): def test_cmd(args, out):
@@ -215,9 +216,6 @@ def test_cmd(args, out):
x = importlib.util.find_spec("deltachat") x = importlib.util.find_spec("deltachat")
if x is None: if x is None:
out.check_call(f"{sys.executable} -m pip install deltachat") out.check_call(f"{sys.executable} -m pip install deltachat")
env = os.environ.copy()
if args.ssh_host:
env["CHATMAIL_SSH"] = args.ssh_host
pytest_path = shutil.which("pytest") pytest_path = shutil.which("pytest")
pytest_args = [ pytest_args = [
@@ -231,7 +229,7 @@ def test_cmd(args, out):
] ]
if args.slow: if args.slow:
pytest_args.append("--slow") pytest_args.append("--slow")
ret = out.run_ret(pytest_args, env=env) ret = out.run_ret(pytest_args)
return ret return ret

View File

@@ -25,6 +25,7 @@ from .basedeploy import (
activate_remote_units, activate_remote_units,
configure_remote_units, configure_remote_units,
get_resource, get_resource,
has_systemd,
) )
from .dovecot.deployer import DovecotDeployer from .dovecot.deployer import DovecotDeployer
from .filtermail.deployer import FiltermailDeployer from .filtermail.deployer import FiltermailDeployer
@@ -65,6 +66,8 @@ def _build_chatmaild(dist_dir) -> None:
def remove_legacy_artifacts(): def remove_legacy_artifacts():
if not has_systemd():
return
# disable legacy doveauth-dictproxy.service # disable legacy doveauth-dictproxy.service
if host.get_fact(SystemdEnabled).get("doveauth-dictproxy.service"): if host.get_fact(SystemdEnabled).get("doveauth-dictproxy.service"):
systemd.service( systemd.service(
@@ -299,7 +302,7 @@ class LegacyRemoveDeployer(Deployer):
present=False, present=False,
) )
# remove echobot if it is still running # remove echobot if it is still running
if host.get_fact(SystemdEnabled).get("echobot.service"): if has_systemd() and host.get_fact(SystemdEnabled).get("echobot.service"):
systemd.service( systemd.service(
name="Disable echobot.service", name="Disable echobot.service",
service="echobot.service", service="echobot.service",
@@ -535,12 +538,13 @@ class GithashDeployer(Deployer):
) )
def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -> None: def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool, docker: bool) -> None:
"""Deploy a chat-mail instance. """Deploy a chat-mail instance.
:param config_path: path to chatmail.ini :param config_path: path to chatmail.ini
:param disable_mail: whether to disable postfix & dovecot :param disable_mail: whether to disable postfix & dovecot
:param website_only: if True, only deploy the website :param website_only: if True, only deploy the website
:param docker: whether it is running in a docker container
""" """
config = read_config(config_path) config = read_config(config_path)
check_config(config) check_config(config)
@@ -566,34 +570,35 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
Out().red(f"Deploy failed: mtail_address {config.mtail_address} is not available (VPN up?).\n") Out().red(f"Deploy failed: mtail_address {config.mtail_address} is not available (VPN up?).\n")
exit(1) exit(1)
port_services = [ if not docker:
(["master", "smtpd"], 25), port_services = [
("unbound", 53), (["master", "smtpd"], 25),
("acmetool", 80), ("unbound", 53),
(["imap-login", "dovecot"], 143), ("acmetool", 80),
("nginx", 443), (["imap-login", "dovecot"], 143),
(["master", "smtpd"], 465), ("nginx", 443),
(["master", "smtpd"], 587), (["master", "smtpd"], 465),
(["imap-login", "dovecot"], 993), (["master", "smtpd"], 587),
("iroh-relay", 3340), (["imap-login", "dovecot"], 993),
("mtail", 3903), ("iroh-relay", 3340),
("stats", 3904), ("mtail", 3903),
("nginx", 8443), ("stats", 3904),
(["master", "smtpd"], config.postfix_reinject_port), ("nginx", 8443),
(["master", "smtpd"], config.postfix_reinject_port_incoming), (["master", "smtpd"], config.postfix_reinject_port),
("filtermail", config.filtermail_smtp_port), (["master", "smtpd"], config.postfix_reinject_port_incoming),
("filtermail", config.filtermail_smtp_port_incoming), ("filtermail", config.filtermail_smtp_port),
] ("filtermail", config.filtermail_smtp_port_incoming),
for service, port in port_services: ]
print(f"Checking if port {port} is available for {service}...") for service, port in port_services:
running_service = host.get_fact(Port, port=port) print(f"Checking if port {port} is available for {service}...")
services = [service] if isinstance(service, str) else service running_service = host.get_fact(Port, port=port)
if running_service: services = [service] if isinstance(service, str) else service
if running_service not in services: if running_service:
Out().red( if running_service not in services:
f"Deploy failed: port {port} is occupied by: {running_service}" Out().red(
) f"Deploy failed: port {port} is occupied by: {running_service}"
exit(1) )
exit(1)
tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"] tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"]

View File

@@ -9,6 +9,7 @@ from cmdeploy.basedeploy import (
activate_remote_units, activate_remote_units,
configure_remote_units, configure_remote_units,
get_resource, get_resource,
has_systemd,
) )
@@ -22,10 +23,11 @@ class DovecotDeployer(Deployer):
def install(self): def install(self):
arch = host.get_fact(Arch) arch = host.get_fact(Arch)
if not "dovecot.service" in host.get_fact(SystemdEnabled): if has_systemd() and "dovecot.service" in host.get_fact(SystemdEnabled):
_install_dovecot_package("core", arch) return # already installed and running
_install_dovecot_package("imapd", arch) _install_dovecot_package("core", arch)
_install_dovecot_package("lmtpd", arch) _install_dovecot_package("imapd", arch)
_install_dovecot_package("lmtpd", arch)
def configure(self): def configure(self):
configure_remote_units(self.config.mail_domain, self.units) configure_remote_units(self.config.mail_domain, self.units)
@@ -116,18 +118,19 @@ def _configure_dovecot(config: Config, debug: bool = False) -> (bool, bool):
# as per https://doc.dovecot.org/2.3/configuration_manual/os/ # as per https://doc.dovecot.org/2.3/configuration_manual/os/
# it is recommended to set the following inotify limits # it is recommended to set the following inotify limits
for name in ("max_user_instances", "max_user_watches"): if config.change_kernel_settings:
key = f"fs.inotify.{name}" for name in ("max_user_instances", "max_user_watches"):
if host.get_fact(Sysctl)[key] > 65535: key = f"fs.inotify.{name}"
# Skip updating limits if already sufficient if host.get_fact(Sysctl)[key] > 65535:
# (enables running in incus containers where sysctl readonly) # Skip updating limits if already sufficient
continue # (enables running in incus containers where sysctl readonly)
server.sysctl( continue
name=f"Change {key}", server.sysctl(
key=key, name=f"Change {key}",
value=65535, key=key,
persist=True, value=65535,
) persist=True,
)
timezone_env = files.line( timezone_env = files.line(
name="Set TZ environment variable", name="Set TZ environment variable",

View File

@@ -1,3 +1,2 @@
/^DKIM-Signature:/ IGNORE /^DKIM-Signature:/ IGNORE
/^Authentication-Results:/ IGNORE /^Authentication-Results:/ IGNORE
/^Received:/ IGNORE

View File

@@ -15,8 +15,9 @@ def main():
) )
disable_mail = bool(os.environ.get("CHATMAIL_DISABLE_MAIL")) disable_mail = bool(os.environ.get("CHATMAIL_DISABLE_MAIL"))
website_only = bool(os.environ.get("CHATMAIL_WEBSITE_ONLY")) website_only = bool(os.environ.get("CHATMAIL_WEBSITE_ONLY"))
docker = bool(os.environ.get("CHATMAIL_DOCKER"))
deploy_chatmail(config_path, disable_mail, website_only) deploy_chatmail(config_path, disable_mail, website_only, docker)
if pyinfra.is_cli: if pyinfra.is_cli:

View File

@@ -85,31 +85,16 @@ class SSHExec:
class LocalExec: class LocalExec:
FuncError = FuncError
def __init__(self, verbose=False, docker=False): def __init__(self, verbose=False, docker=False):
self.verbose = verbose self.verbose = verbose
self.docker = docker self.docker = docker
def __call__(self, call, kwargs=None, log_callback=None):
if kwargs is None:
kwargs = {}
return call(**kwargs)
def logged(self, call, kwargs: dict): def logged(self, call, kwargs: dict):
title = call.__doc__
if not title:
title = call.__name__
where = "locally" where = "locally"
if self.docker: if self.docker:
if call == remote.rdns.perform_initial_checks: if call == remote.rdns.perform_initial_checks:
kwargs["pre_command"] = "docker exec chatmail " kwargs["pre_command"] = "docker exec chatmail "
where = "in docker" where = "in docker"
if self.verbose: if self.verbose:
print_stderr(f"Running {where}: {title}(**{kwargs})") print(f"Running {where}: {call.__name__}(**{kwargs})")
return self(call, kwargs, log_callback=print_stderr) return call(**kwargs)
else:
print_stderr(title, end="")
res = self(call, kwargs, log_callback=remote.rshell.log_progress)
print_stderr()
return res

View File

@@ -7,13 +7,13 @@ import time
import pytest import pytest
from cmdeploy import remote from cmdeploy import remote
from cmdeploy.cmdeploy import get_sshexec from cmdeploy.sshexec import SSHExec
class TestSSHExecutor: class TestSSHExecutor:
@pytest.fixture(scope="class") @pytest.fixture(scope="class")
def sshexec(self, sshdomain): def sshexec(self, sshdomain):
return get_sshexec(sshdomain) return SSHExec(sshdomain)
def test_ls(self, sshexec): def test_ls(self, sshexec):
out = sshexec(call=remote.rdns.shell, kwargs=dict(command="ls")) out = sshexec(call=remote.rdns.shell, kwargs=dict(command="ls"))
@@ -27,7 +27,6 @@ class TestSSHExecutor:
assert res["A"] or res["AAAA"] assert res["A"] or res["AAAA"]
def test_logged(self, sshexec, maildomain, capsys): def test_logged(self, sshexec, maildomain, capsys):
sshexec.verbose = False
sshexec.logged( sshexec.logged(
remote.rdns.perform_initial_checks, kwargs=dict(mail_domain=maildomain) remote.rdns.perform_initial_checks, kwargs=dict(mail_domain=maildomain)
) )
@@ -53,8 +52,6 @@ class TestSSHExecutor:
remote.rdns.perform_initial_checks, remote.rdns.perform_initial_checks,
kwargs=dict(mail_domain=None), kwargs=dict(mail_domain=None),
) )
except AssertionError:
pass
except sshexec.FuncError as e: except sshexec.FuncError as e:
assert "rdns.py" in str(e) assert "rdns.py" in str(e)
assert "AssertionError" in str(e) assert "AssertionError" in str(e)

View File

@@ -7,7 +7,7 @@ import pytest
import requests import requests
from cmdeploy.remote import rshell from cmdeploy.remote import rshell
from cmdeploy.cmdeploy import get_sshexec from cmdeploy.sshexec import SSHExec
@pytest.fixture @pytest.fixture
@@ -90,7 +90,7 @@ class TestEndToEndDeltaChat:
lp.sec(f"filling remote inbox for {user}") lp.sec(f"filling remote inbox for {user}")
fn = f"7743102289.M843172P2484002.c20,S={quota},W=2398:2," fn = f"7743102289.M843172P2484002.c20,S={quota},W=2398:2,"
path = chatmail_config.mailboxes_dir.joinpath(user, "cur", fn) path = chatmail_config.mailboxes_dir.joinpath(user, "cur", fn)
sshexec = get_sshexec(sshdomain) sshexec = SSHExec(sshdomain)
sshexec(call=rshell.write_numbytes, kwargs=dict(path=str(path), num=120)) sshexec(call=rshell.write_numbytes, kwargs=dict(path=str(path), num=120))
res = sshexec(call=rshell.dovecot_recalc_quota, kwargs=dict(user=user)) res = sshexec(call=rshell.dovecot_recalc_quota, kwargs=dict(user=user))
assert res["percent"] >= 100 assert res["percent"] >= 100

View File

@@ -5,11 +5,7 @@ from cmdeploy.cmdeploy import main
def test_status_cmd(chatmail_config, capsys, request): def test_status_cmd(chatmail_config, capsys, request):
os.chdir(request.config.invocation_params.dir) os.chdir(request.config.invocation_params.dir)
command = ["status"] assert main(["status"]) == 0
if os.getenv("CHATMAIL_SSH"):
command.append("--ssh-host")
command.append(os.getenv("CHATMAIL_SSH"))
assert main(command) == 0
status_out = capsys.readouterr() status_out = capsys.readouterr()
print(status_out.out) print(status_out.out)

View File

@@ -54,8 +54,8 @@ def maildomain(chatmail_config):
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def sshdomain(chatmail_config): def sshdomain(maildomain):
return os.environ.get("CHATMAIL_SSH", chatmail_config.ssh_host) return os.environ.get("CHATMAIL_SSH", maildomain)
@pytest.fixture @pytest.fixture
@@ -337,14 +337,8 @@ class Remote:
def iter_output(self, logcmd=""): def iter_output(self, logcmd=""):
getjournal = "journalctl -f" if not logcmd else logcmd getjournal = "journalctl -f" if not logcmd else logcmd
print(self.sshdomain)
match self.sshdomain:
case "@local": command = []
case "localhost": command = []
case _: command = ["ssh", f"root@{self.sshdomain}"]
[command.append(arg) for arg in getjournal.split()]
self.popen = subprocess.Popen( self.popen = subprocess.Popen(
command, ["ssh", f"root@{self.sshdomain}", getjournal],
stdout=subprocess.PIPE, stdout=subprocess.PIPE,
) )
while 1: while 1:

View File

@@ -16,11 +16,18 @@ You will need the following:
- Control over a domain through a DNS provider of your choice. - Control over a domain through a DNS provider of your choice.
- A Debian 12 server with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports. - A Debian 12 **deployment server** with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
IPv6 is encouraged if available. Chatmail relay servers only require IPv6 is encouraged if available. Chatmail relay servers only require
1GB RAM, one CPU, and perhaps 10GB storage for a few thousand active 1GB RAM, one CPU, and perhaps 10GB storage for a few thousand active
chatmail addresses. chatmail addresses.
- A Linux or Unix **build machine** with key-based SSH access to the root
user of the deployment server.
You must add a passphrase-protected private key to your local ssh-agent because you
cant type in your passphrase during deployment.
(An ed25519 private key is required due to an `upstream bug in
paramiko <https://github.com/paramiko/paramiko/issues/2191>`_)
Setup with ``scripts/cmdeploy`` Setup with ``scripts/cmdeploy``
------------------------------------- -------------------------------------
@@ -28,7 +35,7 @@ Setup with ``scripts/cmdeploy``
We use ``chat.example.org`` as the chatmail domain in the following We use ``chat.example.org`` as the chatmail domain in the following
steps. Please substitute it with your own domain. steps. Please substitute it with your own domain.
1. Setup the initial DNS records for your relay. 1. Setup the initial DNS records for your deployment server.
The following is an example in the The following is an example in the
familiar BIND zone file format with a TTL of 1 hour (3600 seconds). familiar BIND zone file format with a TTL of 1 hour (3600 seconds).
Please substitute your domain and IP addresses. Please substitute your domain and IP addresses.
@@ -40,24 +47,29 @@ steps. Please substitute it with your own domain.
www.chat.example.org. 3600 IN CNAME chat.example.org. www.chat.example.org. 3600 IN CNAME chat.example.org.
mta-sts.chat.example.org. 3600 IN CNAME chat.example.org. mta-sts.chat.example.org. 3600 IN CNAME chat.example.org.
2. Login to the server with SSH, clone the repository and bootstrap the Python 2. On your local PC, clone the repository and bootstrap the Python
virtualenv. virtualenv.
:: ::
ssh root@chat.example.org
git clone https://github.com/chatmail/relay git clone https://github.com/chatmail/relay
cd relay cd relay
scripts/initenv.sh scripts/initenv.sh
3. Then, create a chatmail configuration file 3. On your local build machine (PC), create a chatmail configuration file
``chatmail.ini``: ``chatmail.ini``:
:: ::
scripts/cmdeploy init chat.example.org # <-- use your domain scripts/cmdeploy init chat.example.org # <-- use your domain
4. Now run the deployment script to install the relay to the server: 4. Verify that SSH root login to the deployment server server works:
::
ssh root@chat.example.org # <-- use your domain
5. From your local build machine, setup and configure the remote deployment server:
:: ::
@@ -68,31 +80,33 @@ steps. Please substitute it with your own domain.
configure at your DNS provider (it can take some time until they are configure at your DNS provider (it can take some time until they are
public). public).
Next Steps Docker installation
---------- -------------------
Now you should display and check all recommended DNS records We have experimental support for `docker compose <https://github.com/chatmail/relay/blob/docker-rebase/docs/DOCKER_INSTALLATION_EN.md>`_,
to enable federation with other relays: but it is not covered by automated tests yet,
so don't expect everything to work.
:: Other helpful commands
scripts/cmdeploy dns
You should also test whether your chatmail service is working correctly:
::
scripts/cmdeploy test
Other Helpful Commands
---------------------- ----------------------
To check the status of your chatmail relay: To check the status of your deployment server running the chatmail service:
:: ::
scripts/cmdeploy status scripts/cmdeploy status
To display and check all recommended DNS records:
::
scripts/cmdeploy dns
To test whether your chatmail service is working correctly:
::
scripts/cmdeploy test
To measure the performance of your chatmail service: To measure the performance of your chatmail service:
@@ -134,9 +148,8 @@ This starts a local live development cycle for chatmail web pages:
directory and generating HTML files and copying assets to the directory and generating HTML files and copying assets to the
``www/build`` directory. ``www/build`` directory.
- if you are running scripts/cmdeploy webdev on the relay itself, - Starts a browser window automatically where you can “refresh” as
you need to configure a route in /etc/nginx/nginx.conf needed.
to expose the build directory.
Custom web pages Custom web pages
---------------- ----------------
@@ -154,7 +167,7 @@ Disable automatic address creation
-------------------------------------------------------- --------------------------------------------------------
If you need to stop address creation, e.g. because some script is wildly If you need to stop address creation, e.g. because some script is wildly
creating addresses, login with ssh to the relay and run: creating addresses, login with ssh to the deployment machine and run:
:: ::
@@ -162,3 +175,24 @@ creating addresses, login with ssh to the relay and run:
Chatmail address creation will be denied while this file is present. Chatmail address creation will be denied while this file is present.
Migrating to a new build machine
----------------------------------
To move or add a build machine,
clone the relay repository on the new build machine, and copy the ``chatmail.ini`` file from the old build machine.
Make sure ``rsync`` is installed, then initialize the environment:
::
./scripts/initenv.sh
Run safety checks before a new deployment:
::
./scripts/cmdeploy dns
./scripts/cmdeploy status
If you keep multiple build machines (ie laptop and desktop), keep ``chatmail.ini`` in sync between
them.

52
docker-compose.yaml Normal file
View File

@@ -0,0 +1,52 @@
services:
chatmail:
build:
context: ./
dockerfile: docker/chatmail_relay.dockerfile
image: chatmail-relay:latest
restart: unless-stopped
container_name: chatmail
# Required for systemd — use only one of the following:
cgroup: host # compose v2 only
# privileged: true # compose v1 (not tested)
tty: true # required for logs
tmpfs: # required for systemd
- /tmp
- /run
- /run/lock
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
environment:
CHANGE_KERNEL_SETTINGS: "False"
MAIL_DOMAIN: $MAIL_DOMAIN
ACME_EMAIL: $ACME_EMAIL
RECREATE_VENV: $RECREATE_VENV
MAX_MESSAGE_SIZE: $MAX_MESSAGE_SIZE
DEBUG_COMMANDS_ENABLED: $DEBUG_COMMANDS_ENABLED
FORCE_REINIT_INI_FILE: $FORCE_REINIT_INI_FILE
USE_FOREIGN_CERT_MANAGER: $USE_FOREIGN_CERT_MANAGER
ENABLE_CERTS_MONITORING: $ENABLE_CERTS_MONITORING
CERTS_MONITORING_TIMEOUT: $CERTS_MONITORING_TIMEOUT
IS_DEVELOPMENT_INSTANCE: $IS_DEVELOPMENT_INSTANCE
CMDEPLOY_STAGES: ${CMDEPLOY_STAGES:-}
network_mode: "host"
volumes:
## system
- /sys/fs/cgroup:/sys/fs/cgroup:rw # required for systemd
- ./:/opt/chatmail
## data
- ./data/chatmail:/home
- ./data/chatmail-dkimkeys:/etc/dkimkeys
- ./data/chatmail-acme:/var/lib/acme
## custom resources
# - ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
## debug
# - ./docker/files/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
# - ./docker/files/entrypoint.sh:/entrypoint.sh
# - ./docker/files/update_ini.sh:/update_ini.sh

View File

@@ -0,0 +1,100 @@
FROM jrei/systemd-debian:12 AS base
ENV LANG=en_US.UTF-8
RUN echo 'APT::Install-Recommends "0";' > /etc/apt/apt.conf.d/01norecommend && \
echo 'APT::Install-Suggests "0";' >> /etc/apt/apt.conf.d/01norecommend && \
apt-get update && \
apt-get install -y \
ca-certificates && \
DEBIAN_FRONTEND=noninteractive \
TZ=Europe/London \
apt-get install -y tzdata && \
apt-get install -y locales && \
sed -i -e "s/# $LANG.*/$LANG UTF-8/" /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=$LANG \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && \
apt-get install -y \
git \
python3 \
python3-venv \
python3-virtualenv \
gcc \
python3-dev \
opendkim \
opendkim-tools \
curl \
rsync \
unbound \
unbound-anchor \
dnsutils \
postfix \
acl \
nginx \
libnginx-mod-stream \
fcgiwrap \
cron \
&& for pkg in core imapd lmtpd; do \
case "$pkg" in \
core) sha256="43f593332e22ac7701c62d58b575d2ca409e0f64857a2803be886c22860f5587" ;; \
imapd) sha256="8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86" ;; \
lmtpd) sha256="2f69ba5e35363de50962d42cccbfe4ed8495265044e244007d7ccddad77513ab" ;; \
esac; \
url="https://download.delta.chat/dovecot/dovecot-${pkg}_2.3.21%2Bdfsg1-3_amd64.deb"; \
file="/tmp/$(basename "$url")"; \
curl -fsSL "$url" -o "$file"; \
echo "$sha256 $file" | sha256sum -c -; \
apt-get install -y "$file"; \
rm -f "$file"; \
done \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /opt/chatmail
# --- Build-time install stage ---
# Bake the "install" deployer stage into the image; we can't use
# scripts/initenv.sh because /opt/chatmail is empty at build time as
# source arrives at runtime via volume mount., so we use a throwaway venv.
# On container start only "configure,activate" stages run.
COPY . /tmp/chatmail-src/
WORKDIR /tmp/chatmail-src
# Dummy config — deploy_chatmail() needs a parseable ini to instantiate deployers
RUN printf '[params]\nmail_domain = build.local\n' > /tmp/chatmail.ini
# Do what initenv.sh would do without the docs
RUN python3 -m venv /tmp/build-venv && \
/tmp/build-venv/bin/pip install --no-cache-dir \
-e chatmaild -e cmdeploy
RUN CMDEPLOY_STAGES=install \
CHATMAIL_INI=/tmp/chatmail.ini \
CHATMAIL_DOCKER=True \
/tmp/build-venv/bin/pyinfra @local \
/tmp/chatmail-src/cmdeploy/src/cmdeploy/run.py -y
RUN rm -rf /tmp/chatmail-src /tmp/build-venv /tmp/chatmail.ini
WORKDIR /opt/chatmail
# --- End build-time install stage ---
ARG SETUP_CHATMAIL_SERVICE_PATH=/lib/systemd/system/setup_chatmail.service
COPY ./docker/files/setup_chatmail.service "$SETUP_CHATMAIL_SERVICE_PATH"
RUN ln -sf "$SETUP_CHATMAIL_SERVICE_PATH" "/etc/systemd/system/multi-user.target.wants/setup_chatmail.service"
COPY --chmod=555 ./docker/files/setup_chatmail_docker.sh /setup_chatmail_docker.sh
COPY --chmod=555 ./docker/files/update_ini.sh /update_ini.sh
COPY --chmod=555 ./docker/files/entrypoint.sh /entrypoint.sh
VOLUME ["/sys/fs/cgroup", "/home"]
STOPSIGNAL SIGRTMIN+3
ENTRYPOINT ["/entrypoint.sh"]
CMD [ "--default-standard-output=journal+console", \
"--default-standard-error=journal+console" ]

84
docker/cm_ini_to_env.py Executable file
View File

@@ -0,0 +1,84 @@
#!/usr/bin/env python3
"""Convert a chatmail.ini to a Docker .env file.
Usage: python docker/cm_ini_to_env.py [chatmail.ini] [.env]
Reads the ini file, extracts all non-default key=value pairs,
and writes them as UPPER_CASE env vars suitable for docker-compose.
"""
import configparser
import sys
from pathlib import Path
# Keys that only make sense for bare-metal deploys or are handled
# separately by the Docker setup and should not appear in .env.
SKIP_KEYS = set()
# Keys that exist in .env but have a different name than the ini key.
# ini_key -> env_key
RENAMES = {}
def read_ini(path):
"""Return dict of key=value from [params] section."""
cp = configparser.ConfigParser()
cp.read(path)
if not cp.has_section("params"):
sys.exit(f"Error: {path} has no [params] section")
return dict(cp.items("params"))
def read_defaults():
"""Return dict of default values from the ini template."""
template = Path(__file__).resolve().parent.parent / "chatmaild/src/chatmaild/ini/chatmail.ini.f"
if not template.exists():
return {}
cp = configparser.ConfigParser()
cp.read(template)
if not cp.has_section("params"):
return {}
defaults = {}
for key, value in cp.items("params"):
# Template placeholders like {mail_domain} aren't real defaults.
if "{" not in value:
defaults[key] = value
return defaults
def ini_to_env(ini_path, only_non_default=True):
"""Yield (ENV_KEY, value) pairs from an ini file."""
params = read_ini(ini_path)
defaults = read_defaults() if only_non_default else {}
for key, value in sorted(params.items()):
if key in SKIP_KEYS:
continue
if only_non_default and key in defaults and value.strip() == defaults[key].strip():
continue
env_key = RENAMES.get(key, key.upper())
yield env_key, value.strip()
def main():
ini_path = sys.argv[1] if len(sys.argv) > 1 else "chatmail.ini"
env_path = sys.argv[2] if len(sys.argv) > 2 else None
if not Path(ini_path).exists():
sys.exit(f"Error: {ini_path} not found")
lines = []
for env_key, value in ini_to_env(ini_path):
lines.append(f'{env_key}="{value}"')
output = "\n".join(lines) + "\n"
if env_path:
Path(env_path).write_text(output)
print(f"Wrote {len(lines)} variables to {env_path}")
else:
print(output, end="")
if __name__ == "__main__":
main()

11
docker/example.env Normal file
View File

@@ -0,0 +1,11 @@
MAIL_DOMAIN="chat.example.com"
# ACME_EMAIL=""
# RECREATE_VENV="false"
# MAX_MESSAGE_SIZE="50M"
# DEBUG_COMMANDS_ENABLED="true"
# FORCE_REINIT_INI_FILE="true"
# USE_FOREIGN_CERT_MANAGER="True"
# ENABLE_CERTS_MONITORING="true"
# CERTS_MONITORING_TIMEOUT=10
# IS_DEVELOPMENT_INSTANCE="True"
# CMDEPLOY_STAGES - default: "configure,activate". Set to "install,configure,activate" to force full reinstall.

11
docker/files/entrypoint.sh Executable file
View File

@@ -0,0 +1,11 @@
#!/bin/bash
set -eo pipefail
unlink /etc/nginx/sites-enabled/default || true
SETUP_CHATMAIL_SERVICE_PATH="${SETUP_CHATMAIL_SERVICE_PATH:-/lib/systemd/system/setup_chatmail.service}"
env_vars=$(printenv | cut -d= -f1 | xargs)
sed -i "s|<envs_list>|$env_vars|g" $SETUP_CHATMAIL_SERVICE_PATH
exec /lib/systemd/systemd $@

View File

@@ -0,0 +1,14 @@
[Unit]
Description=Run container setup commands
After=multi-user.target
ConditionPathExists=/setup_chatmail_docker.sh
[Service]
Type=oneshot
ExecStart=/bin/bash /setup_chatmail_docker.sh
RemainAfterExit=true
WorkingDirectory=/opt/chatmail
PassEnvironment=<envs_list>
[Install]
WantedBy=multi-user.target

View File

@@ -0,0 +1,84 @@
#!/bin/bash
set -eo pipefail
export INI_FILE="${INI_FILE:-chatmail.ini}"
export ENABLE_CERTS_MONITORING="${ENABLE_CERTS_MONITORING:-true}"
export CERTS_MONITORING_TIMEOUT="${CERTS_MONITORING_TIMEOUT:-60}"
export PATH_TO_SSL="${PATH_TO_SSL:-/var/lib/acme/live/${MAIL_DOMAIN}}"
export CHANGE_KERNEL_SETTINGS=${CHANGE_KERNEL_SETTINGS:-"False"}
export RECREATE_VENV=${RECREATE_VENV:-"false"}
if [ -z "$MAIL_DOMAIN" ]; then
echo "ERROR: Environment variable 'MAIL_DOMAIN' must be set!" >&2
exit 1
fi
debug_commands() {
echo "Executing debug commands"
# git config --global --add safe.directory /opt/chatmail
# ./scripts/initenv.sh
}
calculate_hash() {
find "$PATH_TO_SSL" -type f -exec sha1sum {} \; | sort | sha1sum | awk '{print $1}'
}
monitor_certificates() {
if [ "$ENABLE_CERTS_MONITORING" != "true" ]; then
echo "Certs monitoring disabled."
exit 0
fi
current_hash=$(calculate_hash)
previous_hash=$current_hash
while true; do
current_hash=$(calculate_hash)
if [[ "$current_hash" != "$previous_hash" ]]; then
# TODO: add an option to restart at a specific time interval
echo "[INFO] Certificate's folder hash was changed, reloading nginx, dovecot and postfix services."
systemctl reload nginx.service
systemctl reload dovecot.service
systemctl reload postfix.service
previous_hash=$current_hash
fi
sleep $CERTS_MONITORING_TIMEOUT
done
}
### MAIN
if [ "$DEBUG_COMMANDS_ENABLED" = true ]; then
debug_commands
fi
if [ "$FORCE_REINIT_INI_FILE" = true ]; then
INI_CMD_ARGS=--force
fi
if [ ! -f /etc/dkimkeys/opendkim.private ]; then
/usr/sbin/opendkim-genkey -D /etc/dkimkeys -d $MAIL_DOMAIN -s opendkim
fi
chown opendkim:opendkim /etc/dkimkeys/opendkim.private
chown opendkim:opendkim /etc/dkimkeys/opendkim.txt
# TODO: Move to debug_commands after git clone is moved to dockerfile.
git config --global --add safe.directory /opt/chatmail
if [ "$RECREATE_VENV" = true ]; then
rm -rf venv
fi
# Skip venv creation if it already exists
if [ ! -x venv/bin/python ] || [ ! -x venv/bin/cmdeploy ]; then
./scripts/initenv.sh
fi
./scripts/cmdeploy init --config "${INI_FILE}" $INI_CMD_ARGS $MAIL_DOMAIN || true
bash /update_ini.sh
export CMDEPLOY_STAGES="${CMDEPLOY_STAGES:-configure,activate}"
./scripts/cmdeploy run --ssh-host @docker
echo "ForwardToConsole=yes" >> /etc/systemd/journald.conf
systemctl restart systemd-journald
monitor_certificates &

View File

@@ -0,0 +1,79 @@
#!/bin/bash
set -eo pipefail
INI_FILE="${INI_FILE:-chatmail.ini}"
if [ ! -f "$INI_FILE" ]; then
echo "Error: file $INI_FILE not found." >&2
exit 1
fi
TMP_FILE="$(mktemp)"
convert_to_bytes() {
local value="$1"
if [[ "$value" =~ ^([0-9]+)([KkMmGgTt])$ ]]; then
local num="${BASH_REMATCH[1]}"
local unit="${BASH_REMATCH[2]}"
case "$unit" in
[Kk]) echo $((num * 1024)) ;;
[Mm]) echo $((num * 1024 * 1024)) ;;
[Gg]) echo $((num * 1024 * 1024 * 1024)) ;;
[Tt]) echo $((num * 1024 * 1024 * 1024 * 1024)) ;;
esac
elif [[ "$value" =~ ^[0-9]+$ ]]; then
echo "$value"
else
echo "Error: incorrect size format: $value." >&2
return 1
fi
}
process_specific_params() {
local key=$1
local value=$2
local destination_file=$3
if [[ "$key" == "max_message_size" ]]; then
converted=$(convert_to_bytes "$value") || exit 1
if grep -q -e "## .* = .* bytes" "$destination_file"; then
sed "s|## .* = .* bytes|## $value = $converted bytes|g" "$destination_file";
else
echo "## $value = $converted bytes" >> "$destination_file"
fi
echo "$key = $converted" >> "$destination_file"
else
echo "$key = $value" >> "$destination_file"
fi
}
while IFS= read -r line; do
if [[ "$line" =~ ^[[:space:]]*#.* || "$line" =~ ^[[:space:]]*$ ]]; then
echo "$line" >> "$TMP_FILE"
continue
fi
if [[ "$line" =~ ^([a-z0-9_]+)[[:space:]]*=[[:space:]]*(.*)$ ]]; then
key="${BASH_REMATCH[1]}"
current_value="${BASH_REMATCH[2]}"
env_var_name=$(echo "$key" | tr 'a-z' 'A-Z')
env_value="${!env_var_name}"
if [[ -n "$env_value" ]]; then
process_specific_params "$key" "$env_value" "$TMP_FILE"
else
echo "$line" >> "$TMP_FILE"
fi
else
echo "$line" >> "$TMP_FILE"
fi
done < "$INI_FILE"
PERMS=$(stat -c %a "$INI_FILE")
OWNER=$(stat -c %u "$INI_FILE")
GROUP=$(stat -c %g "$INI_FILE")
chmod "$PERMS" "$TMP_FILE"
chown "$OWNER":"$GROUP" "$TMP_FILE"
mv "$TMP_FILE" "$INI_FILE"

View File

@@ -0,0 +1,185 @@
# Known issues and limitations
- Requires cgroups v2 configured in the system. Operation with cgroups v1 has not been tested.
- Yes, of course, using systemd inside a container is a hack, and it would be better to split it into several services, but since this is an MVP, it turned out to be easier to do it this way initially than to rewrite the entire deployment system.
- The Docker image is only suitable for amd64. If you need to run it on a different architecture, try modifying the Dockerfile (specifically the part responsible for installing dovecot).
# Docker installation
This section provides instructions for installing Chatmail using Docker Compose.
**Note:** Docker Compose v2 is required (`docker compose`, not `docker-compose`) for its support of the `cgroup: host` option in `docker-compose.yaml` is only supported by Compose v2.
[see documentation](https://docs.docker.com/engine/install/debian/#install-using-the-repository)
```shell
apt install docker-ce docker-compose-plugin docker.io- docker-compose-
```
## Preliminary setup
We use `chat.example.org` as the Chatmail domain in the following steps.
Please substitute it with your own domain.
1. Setup the initial DNS records.
The following is an example in the familiar BIND zone file format with
a TTL of 1 hour (3600 seconds).
Please substitute your domain and IP addresses.
```
chat.example.com. 3600 IN A 198.51.100.5
chat.example.com. 3600 IN AAAA 2001:db8::5
www.chat.example.com. 3600 IN CNAME chat.example.com.
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
```
2. clone the repository on your server.
```shell
git clone https://github.com/chatmail/relay
cd relay
```
## Installation
1. Configure kernel parameters because they cannot be changed inside the container, specifically `fs.inotify.max_user_instances` and `fs.inotify.max_user_watches`. Run the following:
```shell
echo "fs.inotify.max_user_instances=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
echo "fs.inotify.max_user_watches=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
sudo sysctl --system
```
2. Copy `./docker/example.env` and rename it to `.env`. This file stores variables used in `docker-compose.yaml`.
```shell
cp ./docker/example.env .env
```
3. Configure environment variables in the `.env` file. These variables are used in the `docker-compose.yaml` file to pass repeated values.
Below is the list of variables used during deployment:
- `MAIL_DOMAIN` The domain name of the future server. (required)
- `DEBUG_COMMANDS_ENABLED` Run debug commands before installation. (default: `false`)
- `FORCE_REINIT_INI_FILE` Recreate the ini configuration file on startup. (default: `false`)
- `USE_FOREIGN_CERT_MANAGER` Use a third-party certificate manager. (default: `false`)
- `RECREATE_VENV` - Recreate the virtual environment (venv). If set to `true`, the environment will be recreated when the container starts, which will increase the startup time of the service but can help avoid certain errors. (default: `false`)
- `INI_FILE` Path to the ini configuration file. (default: `./chatmail.ini`)
- `PATH_TO_SSL` Path to where the certificates are stored. (default: `/var/lib/acme/live/${MAIL_DOMAIN}`)
- `ENABLE_CERTS_MONITORING` Enable certificate monitoring if `USE_FOREIGN_CERT_MANAGER=true`. If certificates change, services will be automatically restarted. (default: `false`)
- `CERTS_MONITORING_TIMEOUT` Interval in seconds to check if certificates have changed. (default: `'60'`)
- `CMDEPLOY_STAGES` Deployment stages to run on container start. (default: `"configure,activate"`). Set to `"install,configure,activate"` to force a full reinstall.
You can also use any variables from the [ini configuration file](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/ini/chatmail.ini.f); they must be in uppercase.
4. Build the Docker image:
```shell
docker compose build chatmail
```
5. Start docker compose and wait for the installation to finish:
```shell
docker compose up -d # start service
docker compose logs -f chatmail # view container logs, press CTRL+C to exit
```
### venv creation
The first container start takes longer because it creates the cmdeploy Python virtualenv at `/opt/chatmail/venv` (persisted on the host via volume mount). Subsequent starts reuse the existing venv. Set `RECREATE_VENV=true` in `.env` to force a rebuild if needed.
6. After installation is complete, you can open `https://<your_domain_name>` in your browser.
## Using custom files
When using Docker, you can apply modified configuration files to make the installation more personalized. This is usually needed for the `www/src` section so that the Chatmail landing page is customized to your taste, but it can be used for any other cases as well.
To replace files correctly:
1. Create the `./custom` directory. It is in `.gitignore`, so it wont cause conflicts when updating.
```shell
mkdir -p ./custom
```
2. Modify the required file. For example, `index.md`:
```shell
mkdir -p ./custom/www/src
nano ./custom/www/src/index.md
```
3. In `docker-compose.yaml`, add the file mount in the `volumes` section:
```yaml
services:
chatmail:
volumes:
...
## custom resources
- ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
```
4. Restart the service:
```shell
docker compose down
docker compose up -d
```
## Migrating from a bare-metal install
If you have an existing bare-metal Chatmail installation and want to switch to Docker:
1. Stop all existing services:
```shell
systemctl stop postfix dovecot doveauth nginx opendkim unbound acmetool-redirector \
filtermail filtermail-incoming chatmail-turn iroh-relay chatmail-metadata \
lastlogin mtail
systemctl disable postfix dovecot doveauth nginx opendkim unbound acmetool-redirector \
filtermail filtermail-incoming chatmail-turn iroh-relay chatmail-metadata \
lastlogin mtail
```
2. Convert your existing `chatmail.ini` to the Docker `.env` format:
```shell
python3 docker/cm_ini_to_env.py /usr/local/lib/chatmaild/chatmail.ini .env
```
3. Copy persistent data into the `./data/` subdirectories:
```shell
mkdir -p data/chatmail-dkimkeys data/chatmail-acme data/chatmail
# DKIM keys
cp -a /etc/dkimkeys/* data/chatmail-dkimkeys/
# ACME certificates and account
rsync -a /var/lib/acme/ data/chatmail-acme/
# Mail data
rsync -a /home/ data/chatmail/
```
Alternatively, you can mount `/home/vmail` directly by changing the volume in `docker-compose.yaml`:
```yaml
- /home/vmail:/home/vmail
```
The three `./data/` subdirectories cover all persistent state. Everything else is regenerated by the `configure` and `activate` stages on container start.
## Forcing a full reinstall
The Docker image bakes the install stage (binary downloads, package setup, chatmaild venv) into the image at build time. On container start, only the `configure` and `activate` stages run by default.
To force a full reinstall (e.g., after updating the source), either rebuild the image:
```shell
docker compose build chatmail
docker compose up -d
```
Or override the stages at runtime without rebuilding:
```shell
CMDEPLOY_STAGES="install,configure,activate" docker compose up -d
```

View File

@@ -0,0 +1,174 @@
# Известные проблемы и ограничения
- Chatmail будет переустановлен при каждом запуске контейнера (при первом - долго, при последующих быстрее). Так устроен изначальный установщик, потому что он не был заточен под docker. В конце документации [представлено](#фиксирование-версии-chatmail) возможное решение
- Требуется настроенный в системе cgroups v2. Работа с cgroups v1 не тестировалась.
- Да, понятно дело что systemd использовать в контейнере костыль и надо это всё разнести на несколько сервисов, но это MVP и в первом приближении оказалось сделать проще так, чем переписывать всю систему развертывания.
- docker образ подходит только для amd64, если нужно запустить на другой архитектуре, попробуйте изменить dockerfile (конкретно ту часть что ответсвенна за установку dovecot)
# Docker installation
Здесь представлена инструкция по установке chatmail с помощью docker-compose.
## Предварительная настройка
We use `chat.example.org` as the chatmail domain in the following steps.
Please substitute it with your own domain.
1. Настройте начальные записи DNS.Ниже приведен пример в привычном формате файла зоны BIND сTTL 1 час (3600 секунд).
Замените домен и IP-адреса на свои.
```
chat.example.com. 3600 IN A 198.51.100.5
chat.example.com. 3600 IN AAAA 2001:db8::5
www.chat.example.com. 3600 IN CNAME chat.example.com.
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
```
2. Склонируйте репозиторий на свой сервер.
```shell
git clone https://github.com/chatmail/relay
cd relay
```
## Installation
1. Настроить параметры ядра, потому что внутри контейнера их нельзя изменить, а конкретно `fs.inotify.max_user_instances` и `fs.inotify.max_user_watches`. Для этого выполнить следующее:
```shell
echo "fs.inotify.max_user_instances=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
echo "fs.inotify.max_user_watches=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
sudo sysctl --system
```
2. Скопировать `./docker/example.env` и переименовать в `.env`. Здесь хранятся переменные, которые используются в `docker-compose.yaml`.
```shell
cp ./docker/example.env .env
```
3. Настроить переменные окружения в `.env` файле. Эти переменные используются в `docker-compose.yaml` файле, чтобы передавать повторяющиеся значения.
Ниже перечислен список переменных учавствующих при развертывании:
- `MAIL_DOMAIN` - Доменное имя будущего сервера. (required)
- `DEBUG_COMMANDS_ENABLED` - Выполнить debug команды перед установкой. (default: `false`)
- `FORCE_REINIT_INI_FILE` - Пересоздавать ini файл конфигурации при запуске. (default: `false`)
- `USE_FOREIGN_CERT_MANAGER` - Использовать сторонний менеджер сертификатов. (default: `false`)
- `RECREATE_VENV` - Пересоздать виртуальное окружение (venv). Если выставлено `true`, то окружение будет пересоздано при запуске контейнера, из-за чего включение сервиса займет больше времени, но поможет избежать ряда ошибок. (default: `false`)
- `INI_FILE` - путь к ini файлу конфигурации. (default: `./chatmail.ini`)
- `PATH_TO_SSL` - Путь где располагаются сертификаты. (default: `/var/lib/acme/live/${MAIL_DOMAIN}`)
- `ENABLE_CERTS_MONITORING` - Включить мониторинг сертификатов, если `USE_FOREIGN_CERT_MANAGER=true`. Если сертфикаты изменятся сервисы будут автоматически перезапущены. (default: `false`)
- `CERTS_MONITORING_TIMEOUT` - Раз во сколько секунд проверять что изменились сертификаты. (default: `'60'`)
Также могут быть использованы все переменные из [ini файла конфигурации](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/ini/chatmail.ini.f), они обязаны быть в uppercase формате.
4. Собрать docker образ
```shell
docker compose build chatmail
```
5. Запустить docker compose и дождаться завершения установки
```shell
docker compose up -d # запуск сервиса
docker compose logs -f chatmail # просмотр логов контейнера. Для выхода нажать CTRL+C
```
6. По окончанию установки можно открыть в браузер `https://<your_domain_name>`
## Использование кастомных файлов
При использовании docker есть возможность использовать измененые файлы конфигурации, чтобы сделать установку более персонализированной. Обычно это требуется для секции `www/src`, чтобы ознакомительная страница Chatmail была сделана на ваш вкус. Но также это можно использовать и для любых других случаев.
Для того чтобы корректно выполнить подмену файлов необходимо
1. создать каталог `./custom`, он находится в `.gitignore`, поэтому при обновлении не вызовет конфликтов.
```shell
mkdir -p ./custom
```
2. Изменить нужный файл. Для примера возьмем `index.md`
```shell
mkdir -p ./custom/www/src
nano ./custom/www/src/index.md
```
3. В `docker-compose.yaml` добавить монтирование файла с помощью секции `volumes`
```yaml
services:
chatmail:
volumes:
...
## custom resources
- ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
```
4. Перезапустить сервис
```shell
docker compose down
docker compose up -d
```
## Фиксирование версии Chatmail
> [!note]
> Это опциональные шаги, их делать требуется только если вас не устраивает что сервис устанавливается каждый раз при запуске
Поскольку в текущей версии docker chatmail сервис устанавливается каждый раз запуске контейнера, чтобы этого не происходило можно зафиксировать версию контейнера после установки. Делается это следующим образом:
1. Зафиксировать текущее состояние сконфигурированного контейнера
```shell
docker container commit chatmail configured-chatmail:$(date +'%Y-%m-%d')
docker image ls | grep configured-chatmail
```
2. Изменить entrypoint для контейнера в `docker-compose.yaml` на
```yaml
services:
chatmail:
image: <image name from step 1>
volumes:
...
## custom resources
- ./custom/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
```
3. Создать файл `./custom/setup_chatmail_docker.sh` с новым файлом конфигурации
```shell
mkdir -p ./custom
cat > ./custom/setup_chatmail_docker.sh << 'EOF'
#!/bin/bash
set -eo pipefail
export ENABLE_CERTS_MONITORING="${ENABLE_CERTS_MONITORING:-true}"
export CERTS_MONITORING_TIMEOUT="${CERTS_MONITORING_TIMEOUT:-60}"
export PATH_TO_SSL="${PATH_TO_SSL:-/var/lib/acme/live/${MAIL_DOMAIN}}"
calculate_hash() {
find "$PATH_TO_SSL" -type f -exec sha1sum {} \; | sort | sha1sum | awk '{print $1}'
}
monitor_certificates() {
if [ "$ENABLE_CERTS_MONITORING" != "true" ]; then
echo "Certs monitoring disabled."
exit 0
fi
current_hash=$(calculate_hash)
previous_hash=$current_hash
while true; do
current_hash=$(calculate_hash)
if [[ "$current_hash" != "$previous_hash" ]]; then
# TODO: add an option to restart at a specific time interval
echo "[INFO] Certificate's folder hash was changed, reloading nginx, dovecot and postfix services."
systemctl reload nginx.service
systemctl reload dovecot.service
systemctl reload postfix.service
previous_hash=$current_hash
fi
sleep $CERTS_MONITORING_TIMEOUT
done
}
monitor_certificates &
EOF
```
4. Перезапустить сервис
```shell
docker compose down
docker compose up -d
```