Update the quick-start and CLI reference sections to reflect
per-relay cached images (localchat-test0, localchat-test1),
parallel deploy, DNS readiness checks, and revised memory
limits (256 MiB for DNS container). Add mention of section
timing summary printed at the end of lxc-test.
Extend the Out class with section(), section_line(), and print()
methods that replace the standalone _section()/_section_line()
helpers. Section timings are recorded and printed as a summary
at the end of lxc-test.
Add RelayContainer.check_dns() which retries 'getent hosts
pypi.org' to verify external DNS resolution works inside the
container, called right after configure_dns() in lxc-start.
Add DIFF_EXCLUDES to get_version_string() so that diffs limited
to test directories do not cause a version mismatch and
unnecessary redeployment.
Update test_lxc.py to use QuietOut for the new Out API.
Switch from a single localchat-relay image to per-relay cached
images (localchat-test0, localchat-test1) and add a DNS image
(localchat-ns). Assign static IPs via a fixed incusbr0 bridge
subnet (10.200.200.0/24) so containers always get deterministic
addresses.
Container launch is split into 'incus init' + device-override +
'incus start' to set the static IP before boot.
Deploy runs in parallel via _run_cmdeploy_parallel(), which
captures output per-relay and shows progress lines. Tests now
run in both directions (test0↔test1, test1↔test0).
publish_image() returns bool (True if published, False if cached)
so lxc-test can report cache hits.
Move _build_chatmaild from deployers.py into util.py as
build_chatmaild_sdist() with fcntl-based file locking so
parallel deploys do not race on the sdist. The build is
called once from run_cmd() before pyinfra starts; deployers.py
now only calls get_chatmaild_sdist() to locate the pre-built
archive.
Add test_build_chatmaild_sdist and test_get_chatmaild_sdist_errors.
Replace the two staging-server CI workflows and their zone-file
helpers with a single lxc-test job in ci.yaml that runs
'cmdeploy lxc-test' inside an ubuntu-24.04 runner.
The new workflow installs Incus from the Zabbly apt repository,
initialises it, bootstraps the venv, caches the base LXC image
together with SSH keys, and runs the full LXC pipeline
(container creation, deploy, DNS zones, tests).
Add cmdeploy "lxc-test" command to run cmdeploy against local containers,
with supplementary lxc-start, lxc-stop and lxc-status subcommands.
See doc/source/lxc.rst for full documentation including prerequisites,
DNS setup, TLS handling, DNS-free testing, and known limitations.
Apart from adding lxc-specific docs, tests, and implementation files in the cmdeploy/lxc directory,
this PR adds the --ssh-config option to cmdeploy run/dns/status/test commands and pyinfra invocations,
and also to sshexec (Execnet) handling. This allows for the host to need no DNS entries for a relay,
and route all resolution through ssh-config. This is used by the "lxc-test" command, which performs
a completely local setup -- again, see docs for more details.
While working on DNS/SSH things i also unified all zone-file handling
to use actual BIND format as it is easy enough to parse back.
ConnectionRefusedError/FileNotFoundError/TimeoutError from
turn_credentials() would kill the dict proxy connection.
Return N (not found) response instead and log the error.
- Reject localparts with chars outside [a-z0-9._-] to prevent
filesystem issues from crafted usernames via IMAP/SMTP auth
- Use filelock to serialize concurrent account creation for same
address, preventing TOCTOU race where two threads both create
an account and last writer wins
Per Python docs, secrets module should be used for security-sensitive
data. random.choices uses Mersenne Twister PRNG which is predictable.
secrets.choice was already used for password generation in the same file.
Similar data is already generated by fsreport
available for the relay operator
and metrics for prometheus are generated by mtail.
Closes <https://github.com/chatmail/relay/issues/431>
Upgrade to filtermail v0.5, which has a built-in DKIM verifier
and disable OpenDKIM on reinject_incoming.
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
Adds a new tls_external_cert_and_key config option for chatmail servers
that manage their own TLS certificates (e.g. via an external ACME client
or a load balancer).
A systemd path unit (tls-cert-reload.path) watches the certificate file
via inotify and automatically reloads dovecot and nginx when it changes.
Postfix reads certs per TLS handshake so needs no reload.
Also extracts openssl_selfsigned_args() so cert generation parameters
are shared between SelfSignedTlsDeployer and the e2e test.
* cleanup: remove CFFI deltachat bindings usage, and consolidate test support with rpc-bindings
major simplification: all chatmail fixtures used in the test are now created inside the cmdeploy plugin,
and do not inherit anything from other fixture machineries, let alone the legacy deltachat CFFI ones.
also fix that pytest report headers show correct chatmail domains under test
- chatmaild:
- basedeploy.py: Add has_systemd() guard. During Docker image builds
there's no running systemd, so deployers that query SystemdEnabled
facts would crash; this change might also be helpful for non-systemd
platforms.
- cmdeploy:
- cmdeploy.py:
- when deploying to @docker, auto-set CHATMAIL_NOPORTCHECK and
CHATMAIL_NOSYSCTL since neither makes sense inside a container
- --config default now reads CHATMAIL_INI env var, so Docker
entrypoints can point to a mounted ini without CLI flags.
- deployers.py:
- skip port check / CHATMAIL_NOPORTCHECK
- skip echobot systemd cleanup w/ has_systemd
- dovecot/deployer.py:
- Guard sysctl writes behind CHATMAIL_NOSYSCTL
- invert dovecot install check so it works without systemd
- sshexec.py: Add __call__ to LocalExec so cmdeploy status works with
@local target. Without it, cmdeploy status tried to call the
executor directly and got TypeError.
Consolidated from j4n/docker branch commits (selection):
- 8953fde feat(cmdeploy): read CHATMAIL_INI env var for default --config path
- 81d7782 fix(cmdeploy): add __call__ to LocalExec so status works with @local
- 8bba78e docker: disable port check if docker is running. fix#694
- 865b514 docker: replace config flags with env vars, drop docker param (instead of f26cb08)
Files: cmdeploy/src/cmdeploy/{basedeploy,cmdeploy,deployers,sshexec,dovecot/deployer}.py
Co-authored-by: Keonik1 <keonik.dev@gmail.com>
Co-authored-by: missytake <missytake@systemli.org>
This elimitates the problem of acmetool failing
to start when nginx is installed already and uses port 80.
This also makes nginx redirect HTTP requests to HTTPS
for setups that don't have acmetool.
feat: support self-signed TLS via underscore domain convention
Domains starting with "_" (e.g. _chat.example.org) automatically use
self-signed TLS certificates instead of ACME/Let's Encrypt. The TLS
mode is derived from the domain name — no separate config option needed.
Internally, when config.tls_cert_mode is "self" (underscore domain):
- Generate self-signed certificates via openssl
- Set Postfix smtp_tls_security_level to "encrypt" (opportunistic TLS)
- Add smtp_tls_policy_map entry for underscore domains
- Skip ACME, MTA-STS and www CNAME checks in `cmdeploy dns`
- Serve /new via GET (not redirect to dcaccount:) with rate-limiting
(nginx limit_req, 2r/s burst=5)
- Return dclogin: URLs with ic=3 (AcceptInvalidCertificates) from /new
- Render QR codes client-side via JavaScript and qrcode-svg
- Use config.tls_cert_path/tls_key_path in Postfix, Dovecot and nginx
templates instead of hardcoded ACME paths
