Replace CHATMAIL_SSH env var with --ssh-host / --ssh-config pytest options.
Monkey-patch socket.getaddrinfo to resolve .localchat domains via
ssh-config mappings. Add process cleanup to Remote, fix subprocess
stdin inheritance, and pass ssh_config through online tests.
The get_parser() loop that scanned globals() for *_cmd names was fragile
and forced # noqa: F401 on all lxc imports (ruff couldn't see they were
used dynamically).
Replace it with an explicit SUBCOMMANDS list of
(cmd_func, options_func, needs_config) tuples. This makes the full set
of subcommands visible at a glance, their registration order defined,
and the imports unconditionally used (no more noqa suppressions).
Also add --ssh-config option to run/dns/status/test so all SSH
resolution can go through a config file, used by lxc-test for
completely local setups.
SSHExec accepts ssh_config parameter, passed through to execnet.
New resolve_host_from_ssh_config() / resolve_key_from_ssh_config()
work around paramiko's silent failures with custom ssh configs.
Add cmdeploy "lxc-test" command to run cmdeploy against local containers,
with supplementary lxc-start, lxc-stop and lxc-status subcommands.
See doc/source/lxc.rst for full documentation including prerequisites,
DNS setup, TLS handling, DNS-free testing, and known limitations.
Remove chatmail.zone.j2 and build DNS records directly in dns.py
using standard BIND format (name TTL IN type rdata) as it is easy
enough to parse back. Add parse_zone_records() for consuming the
new format, update rdns.py check_zonefile() and test data accordingly.
Move the Out output-printer class to cmdeploy/util.py so it is shared
across CLI modules. All print/shell calls in lxc/cli.py, lxc/incus.py,
and dns.py now route through Out instead of bare print().
Key additions:
- Out.section() / Out.section_line(): coloured section headers scaled
to the current terminal width (or $_CMDEPLOY_WIDTH for sub-processes).
- Out.shell(): merges stdout/stderr, prefixes each output line, and
prints a red error line with the exit code on failure.
- Out.new_prefixed_out(): indented sub-printer that shares section_timings.
- 'cmdeploy -v / -vv' exposes the verbosity levels.
- shell(), collapse(), get_git_hash() and get_version_string() helpers.
- Tests for Out added to test_util.py.
Also update chatmaild MockOut fixture to match the new Out API.
This change was accidentally added in cf96be2cbb
Relay should not stop validating TLS certificates of other relays
just because it has a self-signed or externally managed certificate.
Externally managed certificate is likely to even be valid.
Since change 635ac7 we try to install Dovecot, even if it is already
running, which fails Dovecot upgrades fail when the installed version
differs from the target because dovecot-imapd/lmtpd dependencies
on dovecot-core: packages are installed one at a time via apt.deb(),
i.e. `dpkg -i`, and dpkg cannot satisfy them dependencies:
```
dpkg: dependency problems prevent configuration of dovecot-imapd:
dovecot-imapd depends on dovecot-core (= 1:2.3.21+dfsg1-3); however:
Version of dovecot-core on system is 1:2.3.21.1+dfsg1-1~bpo12+1.
```
Split _install_dovecot_package into _download_dovecot_package (download
only, return path) and a single server.shell call that passes all .deb
files to dpkg -i together. Uses the same 3-step pattern as pyinfra's
apt.deb: tolerant first dpkg -i, apt-get --fix-broken, then final
dpkg -i to fail if there are still errors.
Prevent services from auto-starting during package installation by
installing a policy-rc.d that exits 101. This avoids dovecot startup
failures when no TLS cert exists yet (e.g. acmetool failed on first run).
Picked out of 62fe113b from hpk/lxcdeploy branch.
The old code did not install updates when the service was running; check
installed version instead of systemd status. Also, rewrite install logic
to extract dovecot version and hashes as module-level constants.
Use blocked_service_startup from lxcdeploy branch as it solves our
problem here too.
Disables IP verification by upgrading filtermail to v0.6,
changelog: <https://github.com/chatmail/filtermail/releases/tag/v0.6.0>
Messages using domain-literal addresses no longer require
to match the origin SMTP connection IP anymore.
This allows for example a relay using IPv4 email addresses
to send messages to other relays over IPv6.
This is not considering a breaking change as IP-address-only
relays are not considered a stable feature.
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
ConnectionRefusedError/FileNotFoundError/TimeoutError from
turn_credentials() would kill the dict proxy connection.
Return N (not found) response instead and log the error.
- Reject localparts with chars outside [a-z0-9._-] to prevent
filesystem issues from crafted usernames via IMAP/SMTP auth
- Use filelock to serialize concurrent account creation for same
address, preventing TOCTOU race where two threads both create
an account and last writer wins
Per Python docs, secrets module should be used for security-sensitive
data. random.choices uses Mersenne Twister PRNG which is predictable.
secrets.choice was already used for password generation in the same file.
Similar data is already generated by fsreport
available for the relay operator
and metrics for prometheus are generated by mtail.
Closes <https://github.com/chatmail/relay/issues/431>
Upgrade to filtermail v0.5, which has a built-in DKIM verifier
and disable OpenDKIM on reinject_incoming.
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
Adds a new tls_external_cert_and_key config option for chatmail servers
that manage their own TLS certificates (e.g. via an external ACME client
or a load balancer).
A systemd path unit (tls-cert-reload.path) watches the certificate file
via inotify and automatically reloads dovecot and nginx when it changes.
Postfix reads certs per TLS handshake so needs no reload.
Also extracts openssl_selfsigned_args() so cert generation parameters
are shared between SelfSignedTlsDeployer and the e2e test.
* cleanup: remove CFFI deltachat bindings usage, and consolidate test support with rpc-bindings
major simplification: all chatmail fixtures used in the test are now created inside the cmdeploy plugin,
and do not inherit anything from other fixture machineries, let alone the legacy deltachat CFFI ones.
also fix that pytest report headers show correct chatmail domains under test
- chatmaild:
- basedeploy.py: Add has_systemd() guard. During Docker image builds
there's no running systemd, so deployers that query SystemdEnabled
facts would crash; this change might also be helpful for non-systemd
platforms.
- cmdeploy:
- cmdeploy.py:
- when deploying to @docker, auto-set CHATMAIL_NOPORTCHECK and
CHATMAIL_NOSYSCTL since neither makes sense inside a container
- --config default now reads CHATMAIL_INI env var, so Docker
entrypoints can point to a mounted ini without CLI flags.
- deployers.py:
- skip port check / CHATMAIL_NOPORTCHECK
- skip echobot systemd cleanup w/ has_systemd
- dovecot/deployer.py:
- Guard sysctl writes behind CHATMAIL_NOSYSCTL
- invert dovecot install check so it works without systemd
- sshexec.py: Add __call__ to LocalExec so cmdeploy status works with
@local target. Without it, cmdeploy status tried to call the
executor directly and got TypeError.
Consolidated from j4n/docker branch commits (selection):
- 8953fde feat(cmdeploy): read CHATMAIL_INI env var for default --config path
- 81d7782 fix(cmdeploy): add __call__ to LocalExec so status works with @local
- 8bba78e docker: disable port check if docker is running. fix#694
- 865b514 docker: replace config flags with env vars, drop docker param (instead of f26cb08)
Files: cmdeploy/src/cmdeploy/{basedeploy,cmdeploy,deployers,sshexec,dovecot/deployer}.py
Co-authored-by: Keonik1 <keonik.dev@gmail.com>
Co-authored-by: missytake <missytake@systemli.org>
