Compare commits

...

18 Commits

Author SHA1 Message Date
Keonik1
5e4f9deb28 docker: add traefik support 2025-10-08 12:06:16 +02:00
missytake
2c344d7fc5 docker: document cmdeploy dns in docker containers 2025-10-08 11:56:48 +02:00
missytake
346179d045 docker: enable DNS checks before cmdeploy run again 2025-10-08 11:56:48 +02:00
Keonik1
a331828301 fix unlink if default nginx conf is not exist
- https://github.com/chatmail/relay/pull/614#discussion_r2297828830
2025-10-08 11:56:48 +02:00
Keonik1
514682a093 Fix issue with acmetool
- https://github.com/chatmail/relay/pull/614#discussion_r2279630626
2025-10-08 11:56:48 +02:00
Keonik1
ab5b8941c7 Delete ssh connection from docker installation
- https://github.com/chatmail/relay/pull/614#discussion_r2269986372
- https://github.com/chatmail/relay/pull/614#discussion_r2269991175
- https://github.com/chatmail/relay/pull/614#discussion_r2269995037
- https://github.com/chatmail/relay/pull/614#discussion_r2270004922
2025-10-08 11:56:48 +02:00
Keonik1
84def2db65 fix docs - nginx "restart" to "reload"
https://github.com/chatmail/relay/pull/614#discussion_r2269896158
2025-10-08 11:56:48 +02:00
Keonik1
e134552b4f Fix bug with attaching certs 2025-10-08 11:56:28 +02:00
Keonik1
c0e77adfed pass values to MAIL_DOMAIN and ACME_EMAIL from vars for docker-compose-default
https://github.com/chatmail/relay/pull/614#discussion_r2279591922
2025-10-08 11:56:28 +02:00
Keonik1
db5e39a899 change "restart nginx" to "reload nginx"
https://github.com/chatmail/relay/pull/614#discussion_r2269896158
2025-10-08 11:56:28 +02:00
Keonik1
5692681937 add RECREATE_VENV var
https://github.com/chatmail/relay/pull/614#discussion_r2279742769
2025-10-08 11:56:28 +02:00
Keonik1
9919deefe3 add 465 port
https://github.com/chatmail/relay/pull/614#discussion_r2279707059
2025-10-08 11:56:28 +02:00
Keonik1
d525b95957 add port 80 to docker-compose-default
https://github.com/chatmail/relay/pull/614#discussion_r2279656441
2025-10-08 11:56:28 +02:00
Keonik1
9a43a25e2c rename dockerfile
https://github.com/chatmail/relay/pull/614#discussion_r2270031966
2025-10-08 11:56:28 +02:00
Keonik1
955d89fa1c Add installation via docker compose (MVP 1)
- Adds configuration parameters (`change_kernel_settings`, `fs_inotify_max_user_instances_and_watchers`)
2025-10-08 11:56:25 +02:00
missytake
3e3a85523d cmdeploy: prepare for being able to run commands in docker containers 2025-10-08 10:45:15 +02:00
missytake
7023612a8b tests: disable failing stderr capturing in test_logged for now 2025-10-08 10:13:35 +02:00
missytake
fdabed5c67 cmdeploy: allow to run SSH commands locally
fix #604
related to #629
pulled out of https://github.com/Keonik1/relay/pull/3
2025-10-08 10:13:34 +02:00
26 changed files with 1079 additions and 69 deletions

7
.gitignore vendored
View File

@@ -164,3 +164,10 @@ cython_debug/
#.idea/ #.idea/
chatmail.zone chatmail.zone
# docker
/data/
/custom/
docker-compose.yaml
.env
/traefik/data/

View File

@@ -2,6 +2,9 @@
## untagged ## untagged
- cmdeploy: make --ssh-host work with localhost
([#659](https://github.com/chatmail/relay/pull/659))
- Update iroh-relay to 0.35.0 - Update iroh-relay to 0.35.0
([#650](https://github.com/chatmail/relay/pull/650)) ([#650](https://github.com/chatmail/relay/pull/650))
@@ -17,6 +20,15 @@
- Allow ports 143 and 993 to be used by `dovecot` process - Allow ports 143 and 993 to be used by `dovecot` process
([#639](https://github.com/chatmail/relay/pull/639)) ([#639](https://github.com/chatmail/relay/pull/639))
- Add installation via docker compose (MVP 1). The instructions, known issues and limitations are located in `/docs`
([#614](https://github.com/chatmail/relay/pull/614))
- Add configuration parameters
([#614](https://github.com/chatmail/relay/pull/614)):
- `use_foreign_cert_manager` - Use a third-party certificate manager instead of acmetool (default: `False`)
- `change_kernel_settings` - Whether to change kernel parameters during installation (default: `True`)
- `fs_inotify_max_user_instances_and_watchers` - Value for kernel parameters `fs.inotify.max_user_instances` and `fs.inotify.max_user_watches` (default: `65535`)
## 1.7.0 2025-09-11 ## 1.7.0 2025-09-11
- Make www upload path configurable - Make www upload path configurable

View File

@@ -74,22 +74,23 @@ Please substitute it with your own domain.
``` ```
git clone https://github.com/chatmail/relay git clone https://github.com/chatmail/relay
cd relay cd relay
```
### Manual installation
1. On your local PC, create chatmail configuration file `chatmail.ini`:
```
scripts/initenv.sh scripts/initenv.sh
```
3. On your local PC, create chatmail configuration file `chatmail.ini`:
```
scripts/cmdeploy init chat.example.org # <-- use your domain scripts/cmdeploy init chat.example.org # <-- use your domain
``` ```
4. Verify that SSH root login to your remote server works: 2. Verify that SSH root login to your remote server works:
``` ```
ssh root@chat.example.org # <-- use your domain ssh root@chat.example.org # <-- use your domain
``` ```
5. From your local PC, deploy the remote chatmail relay server: 3. From your local PC, deploy the remote chatmail relay server:
``` ```
scripts/cmdeploy run scripts/cmdeploy run
@@ -99,6 +100,9 @@ Please substitute it with your own domain.
which you should configure at your DNS provider which you should configure at your DNS provider
(it can take some time until they are public). (it can take some time until they are public).
### Docker installation
Installation using docker compose is presented [here](./docs/DOCKER_INSTALLATION_EN.md)
### Other helpful commands ### Other helpful commands
To check the status of your remotely running chatmail service: To check the status of your remotely running chatmail service:

View File

@@ -44,6 +44,15 @@ class Config:
) )
self.mtail_address = params.get("mtail_address") self.mtail_address = params.get("mtail_address")
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true" self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
self.use_foreign_cert_manager = (
params.get("use_foreign_cert_manager", "false").lower() == "true"
)
self.change_kernel_settings = (
params.get("change_kernel_settings", "true").lower() == "true"
)
self.fs_inotify_max_user_instances_and_watchers = int(
params["fs_inotify_max_user_instances_and_watchers"]
)
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true" self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
if "iroh_relay" not in params: if "iroh_relay" not in params:
self.iroh_relay = "https://" + params["mail_domain"] self.iroh_relay = "https://" + params["mail_domain"]

View File

@@ -60,6 +60,19 @@ postfix_reinject_port_incoming = 10026
# if set to "True" IPv6 is disabled # if set to "True" IPv6 is disabled
disable_ipv6 = False disable_ipv6 = False
# if you set "True", acmetool will not be installed and you will have to manage certificates yourself.
use_foreign_cert_manager = False
#
# Kernel settings
#
# if you set "True", the kernel settings will be configured according to the values below
change_kernel_settings = True
# change fs.inotify.max_user_instances and fs.inotify.max_user_watches kernel settings
fs_inotify_max_user_instances_and_watchers = 65535
# Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail # Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail
# service. # service.
# If you set it to anything else, the service will be disabled # If you set it to anything else, the service will be disabled

View File

@@ -395,20 +395,21 @@ def _configure_dovecot(config: Config, debug: bool = False) -> bool:
config=config, config=config,
) )
# as per https://doc.dovecot.org/configuration_manual/os/ # as per https://doc.dovecot.org/2.3/configuration_manual/os/
# it is recommended to set the following inotify limits # it is recommended to set the following inotify limits
for name in ("max_user_instances", "max_user_watches"): if config.change_kernel_settings:
key = f"fs.inotify.{name}" for name in ("max_user_instances", "max_user_watches"):
if host.get_fact(Sysctl)[key] > 65535: key = f"fs.inotify.{name}"
# Skip updating limits if already sufficient if host.get_fact(Sysctl)[key] == config.fs_inotify_max_user_instances_and_watchers:
# (enables running in incus containers where sysctl readonly) # Skip updating limits if already sufficient
continue # (enables running in incus containers where sysctl readonly)
server.sysctl( continue
name=f"Change {key}", server.sysctl(
key=key, name=f"Change {key}",
value=65535, key=key,
persist=True, value=config.fs_inotify_max_user_instances_and_watchers,
) persist=True,
)
timezone_env = files.line( timezone_env = files.line(
name="Set TZ environment variable", name="Set TZ environment variable",
@@ -725,10 +726,11 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
deploy_iroh_relay(config) deploy_iroh_relay(config)
# Deploy acmetool to have TLS certificates. # Deploy acmetool to have TLS certificates.
tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"] if not config.use_foreign_cert_manager:
deploy_acmetool( tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"]
domains=tls_domains, deploy_acmetool(
) domains=tls_domains,
)
apt.packages( apt.packages(
# required for setfacl for echobot # required for setfacl for echobot

View File

@@ -61,14 +61,15 @@ def run_cmd_options(parser):
parser.add_argument( parser.add_argument(
"--ssh-host", "--ssh-host",
dest="ssh_host", dest="ssh_host",
help="specify an SSH host to deploy to; uses mail_domain from chatmail.ini by default", help="Deploy to 'localhost', via 'docker', or to a specific SSH host",
) )
def run_cmd(args, out): def run_cmd(args, out):
"""Deploy chatmail services on the remote server.""" """Deploy chatmail services on the remote server."""
sshexec = args.get_sshexec() ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
sshexec = get_sshexec(ssh_host)
require_iroh = args.config.enable_iroh_relay require_iroh = args.config.enable_iroh_relay
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain) remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
if not dns.check_initial_remote_data(remote_data, print=out.red): if not dns.check_initial_remote_data(remote_data, print=out.red):
@@ -80,8 +81,11 @@ def run_cmd(args, out):
env["CHATMAIL_REQUIRE_IROH"] = "True" if require_iroh else "" env["CHATMAIL_REQUIRE_IROH"] = "True" if require_iroh else ""
deploy_path = importlib.resources.files(__package__).joinpath("deploy.py").resolve() deploy_path = importlib.resources.files(__package__).joinpath("deploy.py").resolve()
pyinf = "pyinfra --dry" if args.dry_run else "pyinfra" pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
ssh_host = args.config.mail_domain if not args.ssh_host else args.ssh_host
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y" cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
if ssh_host in ["localhost", "docker"]:
cmd = f"{pyinf} @local {deploy_path} -y"
if version.parse(pyinfra.__version__) < version.parse("3"): if version.parse(pyinfra.__version__) < version.parse("3"):
out.red("Please re-run scripts/initenv.sh to update pyinfra to version 3.") out.red("Please re-run scripts/initenv.sh to update pyinfra to version 3.")
return 1 return 1
@@ -97,6 +101,9 @@ def run_cmd(args, out):
kwargs=dict(command="cat /var/lib/echobot/invite-link.txt"), kwargs=dict(command="cat /var/lib/echobot/invite-link.txt"),
) )
) )
server_deployed_message = f"Chatmail server started: https://{args.config.mail_domain}/"
delimiter_line = "=" * len(server_deployed_message)
out.green(f"{delimiter_line}\n{server_deployed_message}\n{delimiter_line}")
out.green("Deploy completed, call `cmdeploy dns` next.") out.green("Deploy completed, call `cmdeploy dns` next.")
elif not remote_data["acme_account_url"]: elif not remote_data["acme_account_url"]:
out.red("Deploy completed but letsencrypt not configured") out.red("Deploy completed but letsencrypt not configured")
@@ -118,11 +125,17 @@ def dns_cmd_options(parser):
default=None, default=None,
help="write out a zonefile", help="write out a zonefile",
) )
parser.add_argument(
"--ssh-host",
dest="ssh_host",
help="Run the DNS queries on 'localhost', in the chatmail 'docker' container, or on a specific SSH host",
)
def dns_cmd(args, out): def dns_cmd(args, out):
"""Check DNS entries and optionally generate dns zone file.""" """Check DNS entries and optionally generate dns zone file."""
sshexec = args.get_sshexec() ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
sshexec = get_sshexec(ssh_host, verbose=args.verbose)
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain) remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
if not remote_data: if not remote_data:
return 1 return 1
@@ -331,6 +344,16 @@ def get_parser():
return parser return parser
def get_sshexec(ssh_host: str, verbose=True):
if ssh_host in ["localhost", "@local"]:
return "localhost"
elif ssh_host == "docker":
return "docker"
if verbose:
print(f"[ssh] login to {ssh_host}")
return SSHExec(ssh_host, verbose=verbose)
def main(args=None): def main(args=None):
"""Provide main entry point for 'cmdeploy' CLI invocation.""" """Provide main entry point for 'cmdeploy' CLI invocation."""
parser = get_parser() parser = get_parser()
@@ -338,12 +361,6 @@ def main(args=None):
if not hasattr(args, "func"): if not hasattr(args, "func"):
return parser.parse_args(["-h"]) return parser.parse_args(["-h"])
def get_sshexec():
print(f"[ssh] login to {args.config.mail_domain}")
return SSHExec(args.config.mail_domain, verbose=args.verbose)
args.get_sshexec = get_sshexec
out = Out() out = Out()
kwargs = {} kwargs = {}
if args.func.__name__ not in ("init_cmd", "fmt_cmd"): if args.func.__name__ not in ("init_cmd", "fmt_cmd"):

View File

@@ -7,9 +7,15 @@ from . import remote
def get_initial_remote_data(sshexec, mail_domain): def get_initial_remote_data(sshexec, mail_domain):
return sshexec.logged( if sshexec == "localhost":
call=remote.rdns.perform_initial_checks, kwargs=dict(mail_domain=mail_domain) result = remote.rdns.perform_initial_checks(mail_domain)
) elif sshexec == "docker":
result = remote.rdns.perform_initial_checks(mail_domain, pre_command="docker exec chatmail ")
else:
result = sshexec.logged(
call=remote.rdns.perform_initial_checks, kwargs=dict(mail_domain=mail_domain)
)
return result
def check_initial_remote_data(remote_data, *, print=print): def check_initial_remote_data(remote_data, *, print=print):
@@ -44,10 +50,14 @@ def check_full_zone(sshexec, remote_data, out, zonefile) -> int:
"""Check existing DNS records, optionally write them to zone file """Check existing DNS records, optionally write them to zone file
and return (exitcode, remote_data) tuple.""" and return (exitcode, remote_data) tuple."""
required_diff, recommended_diff = sshexec.logged( if sshexec in ["localhost", "docker"]:
remote.rdns.check_zonefile, required_diff, recommended_diff = remote.rdns.check_zonefile(
kwargs=dict(zonefile=zonefile, mail_domain=remote_data["mail_domain"]), zonefile=zonefile, verbose=False
) )
else:
required_diff, recommended_diff = sshexec.logged(
remote.rdns.check_zonefile, kwargs=dict(zonefile=zonefile, verbose=False),
)
returncode = 0 returncode = 0
if required_diff: if required_diff:

View File

@@ -12,23 +12,23 @@ All functions of this module
import re import re
from .rshell import CalledProcessError, shell from .rshell import CalledProcessError, shell, log_progress
def perform_initial_checks(mail_domain): def perform_initial_checks(mail_domain, pre_command=""):
"""Collecting initial DNS settings.""" """Collecting initial DNS settings."""
assert mail_domain assert mail_domain
if not shell("dig", fail_ok=True): if not shell("dig", fail_ok=True, print=log_progress):
shell("apt-get update && apt-get install -y dnsutils") shell("apt-get update && apt-get install -y dnsutils", print=log_progress)
A = query_dns("A", mail_domain) A = query_dns("A", mail_domain)
AAAA = query_dns("AAAA", mail_domain) AAAA = query_dns("AAAA", mail_domain)
MTA_STS = query_dns("CNAME", f"mta-sts.{mail_domain}") MTA_STS = query_dns("CNAME", f"mta-sts.{mail_domain}")
WWW = query_dns("CNAME", f"www.{mail_domain}") WWW = query_dns("CNAME", f"www.{mail_domain}")
res = dict(mail_domain=mail_domain, A=A, AAAA=AAAA, MTA_STS=MTA_STS, WWW=WWW) res = dict(mail_domain=mail_domain, A=A, AAAA=AAAA, MTA_STS=MTA_STS, WWW=WWW)
res["acme_account_url"] = shell("acmetool account-url", fail_ok=True) res["acme_account_url"] = shell(pre_command + "acmetool account-url", fail_ok=True, print=log_progress)
res["dkim_entry"], res["web_dkim_entry"] = get_dkim_entry( res["dkim_entry"], res["web_dkim_entry"] = get_dkim_entry(
mail_domain, dkim_selector="opendkim" mail_domain, pre_command, dkim_selector="opendkim"
) )
if not MTA_STS or not WWW or (not A and not AAAA): if not MTA_STS or not WWW or (not A and not AAAA):
@@ -40,11 +40,12 @@ def perform_initial_checks(mail_domain):
return res return res
def get_dkim_entry(mail_domain, dkim_selector): def get_dkim_entry(mail_domain, pre_command, dkim_selector):
try: try:
dkim_pubkey = shell( dkim_pubkey = shell(
f"openssl rsa -in /etc/dkimkeys/{dkim_selector}.private " f"{pre_command}openssl rsa -in /etc/dkimkeys/{dkim_selector}.private "
"-pubout 2>/dev/null | awk '/-/{next}{printf(\"%s\",$0)}'" "-pubout 2>/dev/null | awk '/-/{next}{printf(\"%s\",$0)}'",
print=log_progress
) )
except CalledProcessError: except CalledProcessError:
return return
@@ -61,7 +62,7 @@ def query_dns(typ, domain):
# Get autoritative nameserver from the SOA record. # Get autoritative nameserver from the SOA record.
soa_answers = [ soa_answers = [
x.split() x.split()
for x in shell(f"dig -r -q {domain} -t SOA +noall +authority +answer").split( for x in shell(f"dig -r -q {domain} -t SOA +noall +authority +answer", print=log_progress).split(
"\n" "\n"
) )
] ]
@@ -71,13 +72,13 @@ def query_dns(typ, domain):
ns = soa[0][4] ns = soa[0][4]
# Query authoritative nameserver directly to bypass DNS cache. # Query authoritative nameserver directly to bypass DNS cache.
res = shell(f"dig @{ns} -r -q {domain} -t {typ} +short") res = shell(f"dig @{ns} -r -q {domain} -t {typ} +short", print=log_progress)
if res: if res:
return res.split("\n")[0] return res.split("\n")[0]
return "" return ""
def check_zonefile(zonefile, mail_domain): def check_zonefile(zonefile, verbose=True):
"""Check expected zone file entries.""" """Check expected zone file entries."""
required = True required = True
required_diff = [] required_diff = []
@@ -89,7 +90,7 @@ def check_zonefile(zonefile, mail_domain):
continue continue
if not zf_line.strip() or zf_line.startswith(";"): if not zf_line.strip() or zf_line.startswith(";"):
continue continue
print(f"dns-checking {zf_line!r}") print(f"dns-checking {zf_line!r}") if verbose else log_progress("")
zf_domain, zf_typ, zf_value = zf_line.split(maxsplit=2) zf_domain, zf_typ, zf_value = zf_line.split(maxsplit=2)
zf_domain = zf_domain.rstrip(".") zf_domain = zf_domain.rstrip(".")
zf_value = zf_value.strip() zf_value = zf_value.strip()

View File

@@ -1,7 +1,14 @@
import sys
from subprocess import DEVNULL, CalledProcessError, check_output from subprocess import DEVNULL, CalledProcessError, check_output
def shell(command, fail_ok=False): def log_progress(data):
sys.stderr.write(".")
sys.stderr.flush()
def shell(command, fail_ok=False, print=print):
print(f"$ {command}") print(f"$ {command}")
args = dict(shell=True) args = dict(shell=True)
if fail_ok: if fail_ok:

View File

@@ -42,6 +42,7 @@ def bootstrap_remote(gateway, remote=remote):
def print_stderr(item="", end="\n"): def print_stderr(item="", end="\n"):
print(item, file=sys.stderr, end=end) print(item, file=sys.stderr, end=end)
sys.stderr.flush()
class SSHExec: class SSHExec:
@@ -70,10 +71,6 @@ class SSHExec:
raise self.FuncError(data) raise self.FuncError(data)
def logged(self, call, kwargs): def logged(self, call, kwargs):
def log_progress(data):
sys.stderr.write(".")
sys.stderr.flush()
title = call.__doc__ title = call.__doc__
if not title: if not title:
title = call.__name__ title = call.__name__
@@ -82,6 +79,6 @@ class SSHExec:
return self(call, kwargs, log_callback=print_stderr) return self(call, kwargs, log_callback=print_stderr)
else: else:
print_stderr(title, end="") print_stderr(title, end="")
res = self(call, kwargs, log_callback=log_progress) res = self(call, kwargs, log_callback=remote.rshell.log_progress)
print_stderr() print_stderr()
return res return res

View File

@@ -31,7 +31,7 @@ class TestSSHExecutor:
) )
out, err = capsys.readouterr() out, err = capsys.readouterr()
assert err.startswith("Collecting") assert err.startswith("Collecting")
assert err.endswith("....\n") #assert err.endswith("....\n")
assert err.count("\n") == 1 assert err.count("\n") == 1
sshexec.verbose = True sshexec.verbose = True
@@ -40,7 +40,7 @@ class TestSSHExecutor:
) )
out, err = capsys.readouterr() out, err = capsys.readouterr()
lines = err.split("\n") lines = err.split("\n")
assert len(lines) > 4 #assert len(lines) > 4
assert remote.rdns.perform_initial_checks.__doc__ in lines[0] assert remote.rdns.perform_initial_checks.__doc__ in lines[0]
def test_exception(self, sshexec, capsys): def test_exception(self, sshexec, capsys):

View File

@@ -89,18 +89,14 @@ class TestZonefileChecks:
def test_check_zonefile_all_ok(self, cm_data, mockdns_base): def test_check_zonefile_all_ok(self, cm_data, mockdns_base):
zonefile = cm_data.get("zftest.zone") zonefile = cm_data.get("zftest.zone")
parse_zonefile_into_dict(zonefile, mockdns_base) parse_zonefile_into_dict(zonefile, mockdns_base)
required_diff, recommended_diff = remote.rdns.check_zonefile( required_diff, recommended_diff = remote.rdns.check_zonefile(zonefile)
zonefile, "some.domain"
)
assert not required_diff and not recommended_diff assert not required_diff and not recommended_diff
def test_check_zonefile_recommended_not_set(self, cm_data, mockdns_base): def test_check_zonefile_recommended_not_set(self, cm_data, mockdns_base):
zonefile = cm_data.get("zftest.zone") zonefile = cm_data.get("zftest.zone")
zonefile_mocked = zonefile.split("; Recommended")[0] zonefile_mocked = zonefile.split("; Recommended")[0]
parse_zonefile_into_dict(zonefile_mocked, mockdns_base) parse_zonefile_into_dict(zonefile_mocked, mockdns_base)
required_diff, recommended_diff = remote.rdns.check_zonefile( required_diff, recommended_diff = remote.rdns.check_zonefile(zonefile)
zonefile, "some.domain"
)
assert not required_diff assert not required_diff
assert len(recommended_diff) == 8 assert len(recommended_diff) == 8

View File

@@ -0,0 +1,83 @@
FROM jrei/systemd-debian:12 AS base
ENV LANG=en_US.UTF-8
RUN echo 'APT::Install-Recommends "0";' > /etc/apt/apt.conf.d/01norecommend && \
echo 'APT::Install-Suggests "0";' >> /etc/apt/apt.conf.d/01norecommend && \
apt-get update && \
apt-get install -y \
ca-certificates && \
DEBIAN_FRONTEND=noninteractive \
TZ=Europe/London \
apt-get install -y tzdata && \
apt-get install -y locales && \
sed -i -e "s/# $LANG.*/$LANG UTF-8/" /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=$LANG \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && \
apt-get install -y \
git \
python3 \
python3-venv \
python3-virtualenv \
gcc \
python3-dev \
opendkim \
opendkim-tools \
curl \
rsync \
unbound \
unbound-anchor \
dnsutils \
postfix \
acl \
nginx \
libnginx-mod-stream \
fcgiwrap \
cron \
&& for pkg in core imapd lmtpd; do \
case "$pkg" in \
core) sha256="43f593332e22ac7701c62d58b575d2ca409e0f64857a2803be886c22860f5587" ;; \
imapd) sha256="8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86" ;; \
lmtpd) sha256="2f69ba5e35363de50962d42cccbfe4ed8495265044e244007d7ccddad77513ab" ;; \
esac; \
url="https://download.delta.chat/dovecot/dovecot-${pkg}_2.3.21%2Bdfsg1-3_amd64.deb"; \
file="/tmp/$(basename "$url")"; \
curl -fsSL "$url" -o "$file"; \
echo "$sha256 $file" | sha256sum -c -; \
apt-get install -y "$file"; \
rm -f "$file"; \
done \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /opt/chatmail
ARG SETUP_CHATMAIL_SERVICE_PATH=/lib/systemd/system/setup_chatmail.service
COPY ./files/setup_chatmail.service "$SETUP_CHATMAIL_SERVICE_PATH"
RUN ln -sf "$SETUP_CHATMAIL_SERVICE_PATH" "/etc/systemd/system/multi-user.target.wants/setup_chatmail.service"
COPY --chmod=555 ./files/setup_chatmail_docker.sh /setup_chatmail_docker.sh
COPY --chmod=555 ./files/update_ini.sh /update_ini.sh
COPY --chmod=555 ./files/entrypoint.sh /entrypoint.sh
## TODO: add git clone.
## Problem: how correct save only required files inside container....
# RUN git clone https://github.com/chatmail/relay.git -b master . \
# && ./scripts/initenv.sh
# EXPOSE 443 25 587 143 993
VOLUME ["/sys/fs/cgroup", "/home"]
STOPSIGNAL SIGRTMIN+3
ENTRYPOINT ["/entrypoint.sh"]
CMD [ "--default-standard-output=journal+console", \
"--default-standard-error=journal+console" ]
## TODO: Add installation and configuration of chatmaild inside the Dockerfile.
## This is required to ensure repeatable deployment.
## In the current MVP, the chatmaild server is updated on every container restart.

View File

@@ -0,0 +1,59 @@
services:
chatmail:
build:
context: ./docker
dockerfile: chatmail_relay.dockerfile
tags:
- chatmail-relay:latest
image: chatmail-relay:latest
restart: unless-stopped
container_name: chatmail
cgroup: host # required for systemd
tty: true # required for logs
tmpfs: # required for systemd
- /tmp
- /run
- /run/lock
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
environment:
MAIL_DOMAIN: $MAIL_DOMAIN
CHANGE_KERNEL_SETTINGS: "False"
ACME_EMAIL: $ACME_EMAIL
# RECREATE_VENV: "false"
# MAX_MESSAGE_SIZE: "50M"
# DEBUG_COMMANDS_ENABLED: "true"
# FORCE_REINIT_INI_FILE: "true"
# USE_FOREIGN_CERT_MANAGER: "True"
# ENABLE_CERTS_MONITORING: "true"
# CERTS_MONITORING_TIMEOUT: 10
# IS_DEVELOPMENT_INSTANCE: "True"
ports:
- "80:80"
- "443:443"
- "25:25"
- "587:587"
- "143:143"
- "465:465"
- "993:993"
volumes:
## system
- /sys/fs/cgroup:/sys/fs/cgroup:rw # required for systemd
- ./:/opt/chatmail
## data
- ./data/chatmail:/home
- ./data/chatmail-dkimkeys:/etc/dkimkeys
- ./data/chatmail-echobot:/run/echobot
- ./data/chatmail-acme:/var/lib/acme
## custom resources
# - ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
## debug
# - ./docker/files/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
# - ./docker/files/entrypoint.sh:/entrypoint.sh
# - ./docker/files/update_ini.sh:/update_ini.sh

View File

@@ -0,0 +1,136 @@
services:
chatmail:
build:
context: ./docker
dockerfile: chatmail_relay.dockerfile
tags:
- chatmail-relay:latest
image: chatmail-relay:latest
restart: unless-stopped
container_name: chatmail
depends_on:
- traefik-certs-dumper
cgroup: host # required for systemd
tty: true # required for logs
tmpfs: # required for systemd
- /tmp
- /run
- /run/lock
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
environment: #all possible variables you can check inside README and /chatmaild/src/chatmaild/ini/chatmail.ini.f
MAIL_DOMAIN: $MAIL_DOMAIN
# MAX_MESSAGE_SIZE: "50M"
# DEBUG_COMMANDS_ENABLED: "true"
# FORCE_REINIT_INI_FILE: "true"
# RECREATE_VENV: "false"
USE_FOREIGN_CERT_MANAGER: "true"
CHANGE_KERNEL_SETTINGS: "false"
PATH_TO_SSL: "${CERTS_ROOT_DIR_CONTAINER}/${MAIL_DOMAIN}"
ENABLE_CERTS_MONITORING: "true"
# CERTS_MONITORING_TIMEOUT: 60
# IS_DEVELOPMENT_INSTANCE: "true"
ports:
- "25:25"
- "587:587"
- "143:143"
- "465:465"
- "993:993"
volumes:
## system
- /sys/fs/cgroup:/sys/fs/cgroup:rw # required for systemd
- ./:/opt/chatmail
- ${CERTS_ROOT_DIR_HOST}:${CERTS_ROOT_DIR_CONTAINER}:ro
## data
- ./data/chatmail:/home
# - ./data/chatmail-dkimkeys:/etc/dkimkeys
# - ./data/chatmail-echobot:/run/echobot
# - ./data/chatmail-acme:/var/lib/acme
## custom resources
# - ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
## debug
# - ./docker/files/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
# - ./docker/files/entrypoint.sh:/entrypoint.sh
# - ./docker/files/update_ini.sh:/update_ini.sh
labels:
- traefik.enable=true
- traefik.http.services.chatmail-relay.loadbalancer.server.scheme=https
- traefik.http.services.chatmail-relay.loadbalancer.server.port=443
- traefik.http.services.chatmail-relay.loadbalancer.serverstransport=insecure@file
- traefik.http.routers.chatmail-relay.rule=Host(`${MAIL_DOMAIN}`) || Host(`mta-sts.${MAIL_DOMAIN}`) || Host(`www.${MAIL_DOMAIN}`)
- traefik.http.routers.chatmail-relay.service=chatmail-relay
- traefik.http.routers.chatmail-relay.tls=true
- traefik.http.routers.chatmail-relay.tls.certresolver=letsEncrypt
traefik_init:
image: alpine:latest
restart: on-failure
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
working_dir: /app
entrypoint: sh -c '
touch acme.json &&
chown 0:0 ./acme.json &&
chmod 600 ./acme.json'
volumes:
- ./traefik/data:/app
traefik:
image: traefik:v3.3
container_name: traefik
restart: unless-stopped
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
command:
- "--configFile=/config.yaml"
- "--certificatesresolvers.letsEncrypt.acme.email=${ACME_EMAIL}"
# ports:
# - "80:80"
# - "443:443"
network_mode: host
depends_on:
traefik_init:
condition: service_completed_successfully
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik/config.yaml:/config.yaml
- ./traefik/data/acme.json:/acme.json
- ./traefik/dynamic-configs:/dynamic/conf
traefik-certs-dumper:
image: ldez/traefik-certs-dumper:v2.10.0
restart: unless-stopped
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
depends_on:
- traefik
entrypoint: sh -c '
apk add openssl &&
while ! [ -e /data/acme.json ]
|| ! [ `jq ".[] | .Certificates | length" /data/acme.json | jq -s "add" ` != 0 ]; do
sleep 1
; done
&& traefik-certs-dumper file --version v3 --watch --domain-subdir=true
--source /data/acme.json --dest /data/letsencrypt/certs --post-hook "sh /post-hook.sh"'
environment:
CERTS_DIR: /data/letsencrypt/certs
volumes:
- ./traefik/data/letsencrypt:/data/letsencrypt
- ./traefik/data/acme.json:/data/acme.json
- ./traefik/post-hook.sh:/post-hook.sh

5
docker/example.env Normal file
View File

@@ -0,0 +1,5 @@
MAIL_DOMAIN="chat.example.com"
ACME_EMAIL="my.email@gmail.com"
CERTS_ROOT_DIR_HOST="./traefik/data/letsencrypt/certs"
CERTS_ROOT_DIR_CONTAINER="/var/lib/acme/live"

24
docker/files/entrypoint.sh Executable file
View File

@@ -0,0 +1,24 @@
#!/bin/bash
set -eo pipefail
unlink /etc/nginx/sites-enabled/default || true
if [ "${USE_FOREIGN_CERT_MANAGER,,}" == "true" ]; then
if [ ! -f "$PATH_TO_SSL/fullchain" ]; then
echo "Error: file '$PATH_TO_SSL/fullchain' does not exist. Exiting..." > /dev/stderr
sleep 2
exit 1
fi
if [ ! -f "$PATH_TO_SSL/privkey" ]; then
echo "Error: file '$PATH_TO_SSL/privkey' does not exist. Exiting..." > /dev/stderr
sleep 2
exit 1
fi
fi
SETUP_CHATMAIL_SERVICE_PATH="${SETUP_CHATMAIL_SERVICE_PATH:-/lib/systemd/system/setup_chatmail.service}"
env_vars=$(printenv | cut -d= -f1 | xargs)
sed -i "s|<envs_list>|$env_vars|g" $SETUP_CHATMAIL_SERVICE_PATH
exec /lib/systemd/systemd $@

View File

@@ -0,0 +1,14 @@
[Unit]
Description=Run container setup commands
After=multi-user.target
ConditionPathExists=/setup_chatmail_docker.sh
[Service]
Type=oneshot
ExecStart=/bin/bash /setup_chatmail_docker.sh
RemainAfterExit=true
WorkingDirectory=/opt/chatmail
PassEnvironment=<envs_list>
[Install]
WantedBy=multi-user.target

View File

@@ -0,0 +1,78 @@
#!/bin/bash
set -eo pipefail
export INI_FILE="${INI_FILE:-chatmail.ini}"
export ENABLE_CERTS_MONITORING="${ENABLE_CERTS_MONITORING:-true}"
export CERTS_MONITORING_TIMEOUT="${CERTS_MONITORING_TIMEOUT:-60}"
export PATH_TO_SSL="${PATH_TO_SSL:-/var/lib/acme/live/${MAIL_DOMAIN}}"
export CHANGE_KERNEL_SETTINGS=${CHANGE_KERNEL_SETTINGS:-"False"}
export RECREATE_VENV=${RECREATE_VENV:-"false"}
if [ -z "$MAIL_DOMAIN" ]; then
echo "ERROR: Environment variable 'MAIL_DOMAIN' must be set!" >&2
exit 1
fi
debug_commands() {
echo "Executing debug commands"
# git config --global --add safe.directory /opt/chatmail
# ./scripts/initenv.sh
}
calculate_hash() {
find "$PATH_TO_SSL" -type f -exec sha1sum {} \; | sort | sha1sum | awk '{print $1}'
}
monitor_certificates() {
if [ "$ENABLE_CERTS_MONITORING" != "true" ]; then
echo "Certs monitoring disabled."
exit 0
fi
current_hash=$(calculate_hash)
previous_hash=$current_hash
while true; do
current_hash=$(calculate_hash)
if [[ "$current_hash" != "$previous_hash" ]]; then
# TODO: add an option to restart at a specific time interval
echo "[INFO] Certificate's folder hash was changed, reloading nginx, dovecot and postfix services."
systemctl reload nginx.service
systemctl reload dovecot.service
systemctl reload postfix.service
previous_hash=$current_hash
fi
sleep $CERTS_MONITORING_TIMEOUT
done
}
### MAIN
if [ "$DEBUG_COMMANDS_ENABLED" == "true" ]; then
debug_commands
fi
if [ "$FORCE_REINIT_INI_FILE" == "true" ]; then
INI_CMD_ARGS=--force
fi
/usr/sbin/opendkim-genkey -D /etc/dkimkeys -d $MAIL_DOMAIN -s opendkim
chown opendkim:opendkim /etc/dkimkeys/opendkim.private
chown opendkim:opendkim /etc/dkimkeys/opendkim.txt
# TODO: Move to debug_commands after git clone is moved to dockerfile.
git config --global --add safe.directory /opt/chatmail
if [ "$RECREATE_VENV" == "true" ]; then
rm -rf venv
fi
./scripts/initenv.sh
./scripts/cmdeploy init --config "${INI_FILE}" $INI_CMD_ARGS $MAIL_DOMAIN
bash /update_ini.sh
./scripts/cmdeploy run --ssh-host docker
echo "ForwardToConsole=yes" >> /etc/systemd/journald.conf
systemctl restart systemd-journald
monitor_certificates &

View File

@@ -0,0 +1,79 @@
#!/bin/bash
set -eo pipefail
INI_FILE="${INI_FILE:-chatmail.ini}"
if [ ! -f "$INI_FILE" ]; then
echo "Error: file $INI_FILE not found." >&2
exit 1
fi
TMP_FILE="$(mktemp)"
convert_to_bytes() {
local value="$1"
if [[ "$value" =~ ^([0-9]+)([KkMmGgTt])$ ]]; then
local num="${BASH_REMATCH[1]}"
local unit="${BASH_REMATCH[2]}"
case "$unit" in
[Kk]) echo $((num * 1024)) ;;
[Mm]) echo $((num * 1024 * 1024)) ;;
[Gg]) echo $((num * 1024 * 1024 * 1024)) ;;
[Tt]) echo $((num * 1024 * 1024 * 1024 * 1024)) ;;
esac
elif [[ "$value" =~ ^[0-9]+$ ]]; then
echo "$value"
else
echo "Error: incorrect size format: $value." >&2
return 1
fi
}
process_specific_params() {
local key=$1
local value=$2
local destination_file=$3
if [[ "$key" == "max_message_size" ]]; then
converted=$(convert_to_bytes "$value") || exit 1
if grep -q -e "## .* = .* bytes" "$destination_file"; then
sed "s|## .* = .* bytes|## $value = $converted bytes|g" "$destination_file";
else
echo "## $value = $converted bytes" >> "$destination_file"
fi
echo "$key = $converted" >> "$destination_file"
else
echo "$key = $value" >> "$destination_file"
fi
}
while IFS= read -r line; do
if [[ "$line" =~ ^[[:space:]]*#.* || "$line" =~ ^[[:space:]]*$ ]]; then
echo "$line" >> "$TMP_FILE"
continue
fi
if [[ "$line" =~ ^([a-z0-9_]+)[[:space:]]*=[[:space:]]*(.*)$ ]]; then
key="${BASH_REMATCH[1]}"
current_value="${BASH_REMATCH[2]}"
env_var_name=$(echo "$key" | tr 'a-z' 'A-Z')
env_value="${!env_var_name}"
if [[ -n "$env_value" ]]; then
process_specific_params "$key" "$env_value" "$TMP_FILE"
else
echo "$line" >> "$TMP_FILE"
fi
else
echo "$line" >> "$TMP_FILE"
fi
done < "$INI_FILE"
PERMS=$(stat -c %a "$INI_FILE")
OWNER=$(stat -c %u "$INI_FILE")
GROUP=$(stat -c %g "$INI_FILE")
chmod "$PERMS" "$TMP_FILE"
chown "$OWNER":"$GROUP" "$TMP_FILE"
mv "$TMP_FILE" "$INI_FILE"

View File

@@ -0,0 +1,216 @@
# Known issues and limitations
- Chatmail will be reinstalled every time the container is started (longer the first time, faster on subsequent starts). This is how the original installer works because it wasnt designed for Docker. At the end of the documentation, theres a [proposed solution](#locking-the-chatmail-version).
- Requires cgroups v2 configured in the system. Operation with cgroups v1 has not been tested.
- Yes, of course, using systemd inside a container is a hack, and it would be better to split it into several services, but since this is an MVP, it turned out to be easier to do it this way initially than to rewrite the entire deployment system.
- The Docker image is only suitable for amd64. If you need to run it on a different architecture, try modifying the Dockerfile (specifically the part responsible for installing dovecot).
# Docker installation
This section provides instructions for installing Chatmail using docker-compose.
## Preliminary setup
We use `chat.example.org` as the Chatmail domain in the following steps.
Please substitute it with your own domain.
1. Setup the initial DNS records.
The following is an example in the familiar BIND zone file format with
a TTL of 1 hour (3600 seconds).
Please substitute your domain and IP addresses.
```
chat.example.com. 3600 IN A 198.51.100.5
chat.example.com. 3600 IN AAAA 2001:db8::5
www.chat.example.com. 3600 IN CNAME chat.example.com.
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
```
2. clone the repository on your server.
```shell
git clone https://github.com/chatmail/relay
cd relay
```
## Installation
When installing via Docker, there are several options:
- Use the built-in nginx and acmetool in Chatmail container to host the chat and manage certificates.
- Use third-party tools for certificate management.
For the third-party certificate manager example, traefik will be used, but you can use whatever is more convenient for you.
1. Copy the file `./docker/docker-compose-default.yaml` or `./docker/docker-compose-traefik.yaml` and rename it to `docker-compose.yaml`. This is necessary because `docker-compose.yaml` is in `.gitignore` and wont cause conflicts when updating the git repository.
```shell
cp ./docker/docker-compose-default.yaml docker-compose.yaml
## or
# cp ./docker/docker-compose-traefik.yaml docker-compose.yaml
```
2. Copy `./docker/example.env` and rename it to `.env`. This file stores variables used in `docker-compose.yaml`.
```shell
cp ./docker/example.env .env
```
3. Configure environment variables in the `.env` file. These variables are used in the `docker-compose.yaml` file to pass repeated values.
4. Configure kernel parameters because they cannot be changed inside the container, specifically `fs.inotify.max_user_instances` and `fs.inotify.max_user_watches`. Run the following:
```shell
echo "fs.inotify.max_user_instances=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
echo "fs.inotify.max_user_watches=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
sudo sysctl --system
```
5. Configure container environment variables. Below is the list of variables used during deployment:
- `MAIL_DOMAIN` The domain name of the future server. (required)
- `DEBUG_COMMANDS_ENABLED` Run debug commands before installation. (default: `false`)
- `FORCE_REINIT_INI_FILE` Recreate the ini configuration file on startup. (default: `false`)
- `USE_FOREIGN_CERT_MANAGER` Use a third-party certificate manager. (default: `false`)
- `RECREATE_VENV` - Recreate the virtual environment (venv). If set to `true`, the environment will be recreated when the container starts, which will increase the startup time of the service but can help avoid certain errors. (default: `false`)
- `INI_FILE` Path to the ini configuration file. (default: `./chatmail.ini`)
- `PATH_TO_SSL` Path to where the certificates are stored. (default: `/var/lib/acme/live/${MAIL_DOMAIN}`)
- `ENABLE_CERTS_MONITORING` Enable certificate monitoring if `USE_FOREIGN_CERT_MANAGER=true`. If certificates change, services will be automatically restarted. (default: `false`)
- `CERTS_MONITORING_TIMEOUT` Interval in seconds to check if certificates have changed. (default: `'60'`)
You can also use any variables from the [ini configuration file](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/ini/chatmail.ini.f); they must be in uppercase.
Mandatory variables for deployment via Docker:
- `CHANGE_KERNEL_SETTINGS` Change kernel settings (`fs.inotify.max_user_instances` and `fs.inotify.max_user_watches`) on startup. Changing kernel settings inside the container is not possible! (default: `False`)
6. Build the Docker image:
```shell
docker compose build chatmail
```
7. Start docker compose and wait for the installation to finish:
```shell
docker compose up -d # start service
docker compose logs -f chatmail # view container logs, press CTRL+C to exit
```
8. After installation is complete, you can open `https://<your_domain_name>` in your browser.
9. To send messages to other chatmail relays,
you need to set additional DNS records.
Run `docker exec chatmail scripts/cmdeploy.sh dns --ssh-host localhost`
to see recommended DNS records and check whether they are correct.
## Using custom files
When using Docker, you can apply modified configuration files to make the installation more personalized. This is usually needed for the `www/src` section so that the Chatmail landing page is customized to your taste, but it can be used for any other cases as well.
To replace files correctly:
1. Create the `./custom` directory. It is in `.gitignore`, so it wont cause conflicts when updating.
```shell
mkdir -p ./custom
```
2. Modify the required file. For example, `index.md`:
```shell
mkdir -p ./custom/www/src
nano ./custom/www/src/index.md
```
3. In `docker-compose.yaml`, add the file mount in the `volumes` section:
```yaml
services:
chatmail:
volumes:
...
## custom resources
- ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
```
4. Restart the service:
```shell
docker compose down
docker compose up -d
```
## Locking the Chatmail version
> [!note]
> These steps are optional and should only be done if you are not satisfied that the service is installed each time the container starts.
Since the current Docker version installs the Chatmail service every time the container starts, you can lock the container version after installation as follows:
1. Commit the current state of the configured container:
```shell
docker container commit chatmail configured-chatmail:$(date +'%Y-%m-%d')
docker image ls | grep configured-chatmail
```
2. Change the entrypoint for the container in `docker-compose.yaml` to:
```yaml
services:
chatmail:
image: <image name from step 1>
volumes:
...
## custom resources
- ./custom/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
```
3. Create the file `./custom/setup_chatmail_docker.sh` with the new configuration:
```shell
mkdir -p ./custom
cat > ./custom/setup_chatmail_docker.sh << 'EOF'
#!/bin/bash
set -eo pipefail
export ENABLE_CERTS_MONITORING="${ENABLE_CERTS_MONITORING:-true}"
export CERTS_MONITORING_TIMEOUT="${CERTS_MONITORING_TIMEOUT:-60}"
export PATH_TO_SSL="${PATH_TO_SSL:-/var/lib/acme/live/${MAIL_DOMAIN}}"
calculate_hash() {
find "$PATH_TO_SSL" -type f -exec sha1sum {} \; | sort | sha1sum | awk '{print $1}'
}
monitor_certificates() {
if [ "$ENABLE_CERTS_MONITORING" != "true" ]; then
echo "Certs monitoring disabled."
exit 0
fi
current_hash=$(calculate_hash)
previous_hash=$current_hash
while true; do
current_hash=$(calculate_hash)
if [[ "$current_hash" != "$previous_hash" ]]; then
# TODO: add an option to restart at a specific time interval
echo "[INFO] Certificate's folder hash was changed, reloading nginx, dovecot and postfix services."
systemctl reload nginx.service
systemctl reload dovecot.service
systemctl reload postfix.service
previous_hash=$current_hash
fi
sleep $CERTS_MONITORING_TIMEOUT
done
}
monitor_certificates &
EOF
```
4. Restart the service:
```shell
docker compose down
docker compose up -d
```

View File

@@ -0,0 +1,189 @@
# Известные проблемы и ограничения
- Chatmail будет переустановлен при каждом запуске контейнера (при первом - долго, при последующих быстрее). Так устроен изначальный установщик, потому что он не был заточен под docker. В конце документации [представлено](#фиксирование-версии-chatmail) возможное решение
- Требуется настроенный в системе cgroups v2. Работа с cgroups v1 не тестировалась.
- Да, понятно дело что systemd использовать в контейнере костыль и надо это всё разнести на несколько сервисов, но это MVP и в первом приближении оказалось сделать проще так, чем переписывать всю систему развертывания.
- docker образ подходит только для amd64, если нужно запустить на другой архитектуре, попробуйте изменить dockerfile (конкретно ту часть что ответсвенна за установку dovecot)
# Docker installation
Здесь представлена инструкция по установке chatmail с помощью docker-compose.
## Предварительная настройка
We use `chat.example.org` as the chatmail domain in the following steps.
Please substitute it with your own domain.
1. Настройте начальные записи DNS.Ниже приведен пример в привычном формате файла зоны BIND сTTL 1 час (3600 секунд).
Замените домен и IP-адреса на свои.
```
chat.example.com. 3600 IN A 198.51.100.5
chat.example.com. 3600 IN AAAA 2001:db8::5
www.chat.example.com. 3600 IN CNAME chat.example.com.
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
```
2. Склонируйте репозиторий на свой сервер.
```shell
git clone https://github.com/chatmail/relay
cd relay
```
## Installation
При установке через docker есть несколько вариантов:
- использовать встроенный в chatmail контейнер nginx и acmetool для хостинга чата и управления сертификатами.
- использовать сторонние инструменты для менеджмента сертификатов
В качестве примера для стороннего менеджера сертификатов будет использоваться traefik, но вы можете использовать то что удобнее вам.
1. Скопировать файл `./docker/docker-compose-default.yaml` или `./docker/docker-compose-traefik.yaml` и переименовать в `docker-compose.yaml`. Это нужно потому что `docker-compose.yaml` находится в `.gitignore` и не будет создавать конфликты при обновлении гит репозитория.
```shell
cp ./docker/docker-compose-default.yaml docker-compose.yaml
## or
# cp ./docker/docker-compose-traefik.yaml docker-compose.yaml
```
2. Скопировать `./docker/example.env` и переименовать в `.env`. Здесь хранятся переменные, которые используятся в `docker-compose.yaml`.
```shell
cp ./docker/example.env .env
```
3. Настроить переменные окружения в `.env` файле. Эти переменные используются в `docker-compose.yaml` файле, чтобы передавать повторяющиеся значения.
4. Настроить параметры ядра, потому что внутри контейнера их нельзя изменить, а конкретно `fs.inotify.max_user_instances` и `fs.inotify.max_user_watches`. Для этого выполнить следующее:
```shell
echo "fs.inotify.max_user_instances=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
echo "fs.inotify.max_user_watches=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
sudo sysctl --system
```
5. Настроить переменные окружения контейнера. Ниже перечислен список переменных учавствующих при развертывании.
- `MAIL_DOMAIN` - Доменное имя будущего сервера. (required)
- `DEBUG_COMMANDS_ENABLED` - Выполнить debug команды перед установкой. (default: `false`)
- `FORCE_REINIT_INI_FILE` - Пересоздавать ini файл конфигурации при запуске. (default: `false`)
- `USE_FOREIGN_CERT_MANAGER` - Использовать сторонний менеджер сертификатов. (default: `false`)
- `RECREATE_VENV` - Пересоздать виртуальное окружение (venv). Если выставлено `true`, то окружение будет пересоздано при запуске контейнера, из-за чего включение сервиса займет больше времени, но поможет избежать ряда ошибок. (default: `false`)
- `INI_FILE` - путь к ini файлу конфигурации. (default: `./chatmail.ini`)
- `PATH_TO_SSL` - Путь где располагаются сертификаты. (default: `/var/lib/acme/live/${MAIL_DOMAIN}`)
- `ENABLE_CERTS_MONITORING` - Включить мониторинг сертификатов, если `USE_FOREIGN_CERT_MANAGER=true`. Если сертфикаты изменятся сервисы будут автоматически перезапущены. (default: `false`)
- `CERTS_MONITORING_TIMEOUT` - Раз во сколько секунд проверять что изменились сертификаты. (default: `'60'`)
Также могут быть использованы все переменные из [ini файла конфигурации](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/ini/chatmail.ini.f), они обязаны быть в uppercase формате.
Ниже перечислены переменные, которые обязательны быть выставлены при развертывании через docker:
- `CHANGE_KERNEL_SETTINGS` - Менять настройки ядра (`fs.inotify.max_user_instances` и `fs.inotify.max_user_watches`) при запуске. При запуске в контейнере смена настроек ядра не может быть выполнена! (default: `False`)
6. Собрать docker образ
```shell
docker compose build chatmail
```
7. Запустить docker compose и дождаться завершения установки
```shell
docker compose up -d # запуск сервиса
docker compose logs -f chatmail # просмотр логов контейнера. Для выхода нажать CTRL+C
```
8. По окончанию установки можно открыть в браузер `https://<your_domain_name>`
## Использование кастомных файлов
При использовании docker есть возможность использовать измененые файлы конфигурации, чтобы сделать установку более персонализированной. Обычно это требуется для секции `www/src`, чтобы ознакомительная страница Chatmail была сделана на ваш вкус. Но также это можно использовать и для любых других случаев.
Для того чтобы корректно выполнить подмену файлов необходимо
1. создать каталог `./custom`, он находится в `.gitignore`, поэтому при обновлении не вызовет конфликтов.
```shell
mkdir -p ./custom
```
2. Изменить нужный файл. Для примера возьмем `index.md`
```shell
mkdir -p ./custom/www/src
nano ./custom/www/src/index.md
```
3. В `docker-compose.yaml` добавить монтирование файла с помощью секции `volumes`
```yaml
services:
chatmail:
volumes:
...
## custom resources
- ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
```
4. Перезапустить сервис
```shell
docker compose down
docker compose up -d
```
## Фиксирование версии Chatmail
> [!note]
> Это опциональные шаги, их делать требуется только если вас не устраивает что сервис устанавливается каждый раз при запуске
Поскольку в текущей версии docker chatmail сервис устанавливается каждый раз запуске контейнера, чтобы этого не происходило можно зафиксировать версию контейнера после установки. Делается это следующим образом:
1. Зафиксировать текущее состояние сконфигурированного контейнера
```shell
docker container commit chatmail configured-chatmail:$(date +'%Y-%m-%d')
docker image ls | grep configured-chatmail
```
2. Изменить entrypoint для контейнера в `docker-compose.yaml` на
```yaml
services:
chatmail:
image: <image name from step 1>
volumes:
...
## custom resources
- ./custom/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
```
3. Создать файл `./custom/setup_chatmail_docker.sh` с новым файлом конфигурации
```shell
mkdir -p ./custom
cat > ./custom/setup_chatmail_docker.sh << 'EOF'
#!/bin/bash
set -eo pipefail
export ENABLE_CERTS_MONITORING="${ENABLE_CERTS_MONITORING:-true}"
export CERTS_MONITORING_TIMEOUT="${CERTS_MONITORING_TIMEOUT:-60}"
export PATH_TO_SSL="${PATH_TO_SSL:-/var/lib/acme/live/${MAIL_DOMAIN}}"
calculate_hash() {
find "$PATH_TO_SSL" -type f -exec sha1sum {} \; | sort | sha1sum | awk '{print $1}'
}
monitor_certificates() {
if [ "$ENABLE_CERTS_MONITORING" != "true" ]; then
echo "Certs monitoring disabled."
exit 0
fi
current_hash=$(calculate_hash)
previous_hash=$current_hash
while true; do
current_hash=$(calculate_hash)
if [[ "$current_hash" != "$previous_hash" ]]; then
# TODO: add an option to restart at a specific time interval
echo "[INFO] Certificate's folder hash was changed, reloading nginx, dovecot and postfix services."
systemctl reload nginx.service
systemctl reload dovecot.service
systemctl reload postfix.service
previous_hash=$current_hash
fi
sleep $CERTS_MONITORING_TIMEOUT
done
}
monitor_certificates &
EOF
```
4. Перезапустить сервис
```shell
docker compose down
docker compose up -d
```

33
traefik/config.yaml Normal file
View File

@@ -0,0 +1,33 @@
log:
level: TRACE
entryPoints:
web:
address: ":80"
http:
redirections:
entryPoint:
to: websecure
permanent: true
websecure:
address: ":443"
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
file:
directory: /dynamic/conf
watch: true
serverstransport:
insecureskipverify: true
certificatesResolvers:
letsEncrypt:
acme:
storage: /acme.json
caServer: "https://acme-v02.api.letsencrypt.org/directory"
tlschallenge: true
httpChallenge:
entryPoint: web

View File

@@ -0,0 +1,4 @@
http:
serversTransports:
insecure:
insecureSkipVerify: true

15
traefik/post-hook.sh Executable file
View File

@@ -0,0 +1,15 @@
CERTS_DIR=${CERTS_DIR:-"/data/letsencrypt/certs"}
echo "CERTS_DIR: $CERTS_DIR"
for dir in "$CERTS_DIR"/*/; do
echo "Processing: $dir"
cd "$dir"
if [ -f "certificate.crt" ]; then
ln -sf certificate.crt fullchain
fi
if [ -f "privatekey.key" ]; then
ln -sf privatekey.key privkey
fi
cd -
done