mirror of
https://github.com/chatmail/relay.git
synced 2026-05-14 09:54:38 +00:00
feat: add LXC container support for local chatmail development
Add cmdeploy "lxc-test" command to run cmdeploy against local containers, with supplementary lxc-start, lxc-stop and lxc-status subcommands. See doc/source/lxc.rst for full documentation including prerequisites, DNS setup, TLS handling, DNS-free testing, and known limitations. Apart from adding lxc-specific docs, tests, and implementation files in the cmdeploy/lxc directory, this PR adds the --ssh-config option to cmdeploy run/dns/status/test commands and pyinfra invocations, and also to sshexec (Execnet) handling. This allows for the host to need no DNS entries for a relay, and route all resolution through ssh-config. This is used by the "lxc-test" command, which performs a completely local setup -- again, see docs for more details. While working on DNS/SSH things i also unified all zone-file handling to use actual BIND format as it is easy enough to parse back.
This commit is contained in:
holger krekel
@@ -15,7 +15,7 @@ author = 'chatmail collective'
|
||||
|
||||
extensions = [
|
||||
#'sphinx.ext.autodoc',
|
||||
#'sphinx.ext.viewdoc',
|
||||
#'sphinx.ext.viewcode',
|
||||
'sphinxcontrib.mermaid',
|
||||
]
|
||||
|
||||
|
||||
@@ -16,5 +16,6 @@ Contributions and feedback welcome through the https://github.com/chatmail/relay
|
||||
proxy
|
||||
migrate
|
||||
overview
|
||||
lxc
|
||||
related
|
||||
faq
|
||||
|
||||
312
doc/source/lxc.rst
Normal file
312
doc/source/lxc.rst
Normal file
@@ -0,0 +1,312 @@
|
||||
Local testing with LXC/Incus
|
||||
============================
|
||||
|
||||
.. warning::
|
||||
|
||||
cmdeploy LXC support is geared towards local testing and CI, only.
|
||||
Do not base production setups on it.
|
||||
|
||||
|
||||
The ``cmdeploy`` tool includes support for running
|
||||
chatmail relays inside local
|
||||
`Incus <https://linuxcontainers.org/incus/>`_ LXC containers.
|
||||
This is useful for development, testing, and CI
|
||||
without requiring a remote server.
|
||||
LXC system containers behave like lightweight virtual machines.
|
||||
They share the host's kernel but run their own init system
|
||||
(systemd), package manager, and network stack,
|
||||
so the cmdeploy deployment scripts work exactly
|
||||
as they would on a real Debian server or cloud VPS.
|
||||
Incus is
|
||||
`well supported
|
||||
<https://linuxcontainers.org/incus/docs/main/installing/>`_
|
||||
on Debian, Ubuntu, Arch, Fedora,
|
||||
and other major distributions.
|
||||
|
||||
|
||||
Prerequisites
|
||||
-------------
|
||||
|
||||
Install `Incus <https://linuxcontainers.org/incus/>`_
|
||||
(LXC container manager).
|
||||
See the `official installation guide
|
||||
<https://linuxcontainers.org/incus/docs/main/installing/>`_
|
||||
for full details, or use one of the shortcuts below:
|
||||
|
||||
**Debian 12+ / Ubuntu 24.04+**
|
||||
Incus is in the default repositories::
|
||||
|
||||
sudo apt install incus
|
||||
|
||||
**Older Debian / Ubuntu**: Use the
|
||||
`Zabbly package repository
|
||||
<https://github.com/zabbly/incus>`_::
|
||||
|
||||
curl -fsSL https://pkgs.zabbly.com/get/incus-stable | sudo bash
|
||||
|
||||
**Arch Linux**::
|
||||
|
||||
sudo pacman -S incus
|
||||
|
||||
After installing, initialise and grant yourself access::
|
||||
|
||||
sudo incus admin init --minimal
|
||||
sudo usermod -aG incus-admin $USER
|
||||
|
||||
.. warning::
|
||||
|
||||
You **must log out and back in** (or run ``newgrp incus-admin``)
|
||||
after adding yourself to the group.
|
||||
Without this, all ``cmdeploy lxc-*`` commands
|
||||
will fail with permission errors.
|
||||
|
||||
Verify the installation works by running ``incus list``,
|
||||
which should print an empty table without errors.
|
||||
|
||||
|
||||
Quick start
|
||||
-----------
|
||||
|
||||
::
|
||||
|
||||
cd relay
|
||||
scripts/initenv.sh # bootstrap venv
|
||||
cmdeploy lxc-test # create containers, deploy, test
|
||||
|
||||
|
||||
The ``lxc-test`` command executes each ``cmdeploy`` subprocess command
|
||||
so you can copy-paste and run them individually.
|
||||
No host DNS delegation or ``~/.ssh/config`` changes are needed
|
||||
because lxc-test passes ssh-related CLI options to "cmdeploy run" and "cmdeploy test" commands.
|
||||
|
||||
|
||||
CLI reference
|
||||
--------------
|
||||
|
||||
``lxc-start [--ipv4-only] [--run] [NAME ...]``
|
||||
Create and start containers.
|
||||
Without arguments, creates ``test0-localchat`` and ``ns-localchat`` (DNS).
|
||||
Pass one or more ``NAME`` arguments to create user relay containers instead
|
||||
(e.g. ``cmdeploy lxc-start myrelay``).
|
||||
Use ``--ipv4-only`` to set ``disable_ipv6 = True`` in the generated ``chatmail.ini``,
|
||||
producing an IPv4-only relay.
|
||||
Use ``--run`` to automatically run ``cmdeploy run`` on each container after starting it.
|
||||
Generates ``lxconfigs/ssh-config``.
|
||||
It reuses existing containers and resets DNS zones to minimal records.
|
||||
|
||||
``lxc-stop [--destroy] [--destroy-all] [NAME ...]``
|
||||
Stop relay containers.
|
||||
Without arguments, stops ``test0-localchat``
|
||||
and ``test1-localchat``.
|
||||
Pass ``NAME`` to stop specific containers.
|
||||
Use ``--destroy`` to also delete the containers
|
||||
and their config files.
|
||||
Use ``--destroy-all`` to additionally destroy
|
||||
the ``ns-localchat`` DNS container **and** remove
|
||||
the cached ``localchat-base`` and ``localchat-relay``
|
||||
images, giving a fully clean slate
|
||||
for the next ``lxc-test``.
|
||||
User containers are **never** destroyed
|
||||
unless named explicitly.
|
||||
|
||||
``lxc-test [--one]``
|
||||
Idempotent full pipeline:
|
||||
|
||||
1. ``lxc-start``: create ``test0`` + ``test1``
|
||||
containers, minimal DNS
|
||||
|
||||
2. ``cmdeploy run``: deploy chatmail services on each relay
|
||||
|
||||
3. publish ``localchat-relay`` image after first successful deploy
|
||||
|
||||
4. ``cmdeploy dns --zonefile``: generate standard
|
||||
BIND-format zone files, load full DNS records
|
||||
|
||||
5. ``cmdeploy test``: run full test suite
|
||||
with ``-n4 -x``
|
||||
|
||||
By default creates, deploys, and tests both ``test0`` and ``test1``
|
||||
for dual-domain federation testing (sets ``CHATMAIL_DOMAIN2=_test1.localchat``).
|
||||
test0 runs dual-stack (IPv4 + IPv6) while test1 runs IPv4-only (``disable_ipv6 = True``).
|
||||
Pass ``--one`` to only deploy and test against ``test0``
|
||||
(skips ``test1``, does not set ``CHATMAIL_DOMAIN2``).
|
||||
|
||||
``lxc-status``
|
||||
Show live status of all LXC containers (including the DNS container),
|
||||
deploy freshness (comparing ``/etc/chatmail-version``
|
||||
against local ``git rev-parse HEAD`` and ``git diff``),
|
||||
SSH config inclusion, and host DNS forwarding for ``.localchat``.
|
||||
Reports **IN-SYNC**, **DIRTY** (hash matches but uncommitted changes exist),
|
||||
**STALE** (different commit), or **NOT DEPLOYED**.
|
||||
|
||||
|
||||
Container types
|
||||
-----------------
|
||||
|
||||
**Test relay containers** (``test0-localchat``, ``test1-localchat``)
|
||||
Created automatically by ``lxc-test``.
|
||||
**test0** has IPv4 and IPv6 configured,
|
||||
**test1** is IPv4-only (``disable_ipv6 = True``).
|
||||
|
||||
**User relay containers** (``<name>-localchat``)
|
||||
Created by ``cmdeploy lxc-start <name>``
|
||||
where ``<name>`` does not start with ``test``.
|
||||
These are personal development instances,
|
||||
never touched by ``lxc-stop --destroy`` unless named explicitly.
|
||||
|
||||
**DNS container** (``ns-localchat``)
|
||||
Singleton container running PowerDNS.
|
||||
Created automatically when any relay is started.
|
||||
|
||||
|
||||
.. _lxc-ssh-config:
|
||||
|
||||
SSH configuration
|
||||
-----------------
|
||||
|
||||
``cmdeploy lxc-start`` generates ``lxconfigs/ssh-config``,
|
||||
a standard OpenSSH config file mapping every container name,
|
||||
its domain, and a short alias to the container's IP address::
|
||||
|
||||
Host test0-localchat _test0.localchat _test0
|
||||
Hostname 10.204.0.42
|
||||
User root
|
||||
IdentityFile /path/to/relay/lxconfigs/id_localchat
|
||||
IdentitiesOnly yes
|
||||
StrictHostKeyChecking accept-new
|
||||
UserKnownHostsFile /dev/null
|
||||
LogLevel ERROR
|
||||
|
||||
All ``cmdeploy`` commands (``run``, ``dns``, ``status``, ``test``)
|
||||
accept ``--ssh-config lxconfigs/ssh-config`` to use this file.
|
||||
``lxc-test`` passes it automatically.
|
||||
|
||||
**Using containers from the host shell:**
|
||||
|
||||
To make ``ssh _test0`` work from any terminal, add one line to ``~/.ssh/config``::
|
||||
|
||||
Include /absolute/path/to/relay/lxconfigs/ssh-config
|
||||
|
||||
|
||||
.. _lxc-dns-setup:
|
||||
.. _localchat-tld:
|
||||
|
||||
``.localchat`` DNS and name resolution
|
||||
---------------------------------------
|
||||
|
||||
All LXC-managed chatmail domains use the ``.localchat`` pseudo-TLD
|
||||
(e.g. ``_test0.localchat``, ``_test1.localchat``),
|
||||
a non-delegated suffix that exists only within the local PowerDNS infrastructure.
|
||||
A dedicated DNS container (``ns-localchat``)
|
||||
is created so that local test relays interact
|
||||
with DNS similar to a regular public Internet setup.
|
||||
On first start, ``cmdeploy lxc-start`` creates this container
|
||||
running two `PowerDNS <https://www.powerdns.com/>`_ services:
|
||||
|
||||
* **pdns-server** (authoritative) serves ``.localchat``
|
||||
zones from a local SQLite database.
|
||||
|
||||
* **pdns-recursor** (recursive) listens on the Incus
|
||||
bridge so all containers can use it.
|
||||
Forwards ``.localchat`` queries to the local
|
||||
authoritative server and everything else to Quad9 (``9.9.9.9``).
|
||||
|
||||
After the DNS container is up, ``lxc-start`` configures the Incus bridge
|
||||
to advertise its IP via DHCP and disables Incus's own DNS.
|
||||
DNS records are then created in two phases matching the "cmdeploy run" deployment flow:
|
||||
|
||||
1. **``lxc-start``** resets each relay zone to
|
||||
**SOA, NS, and A** records (plus **AAAA** for dual-stack containers).
|
||||
If host DNS resolution is configured, users can
|
||||
afterwards run ``cmdeploy run --config lxconfigs/chatmail-test0.ini
|
||||
--ssh-config lxconfigs/ssh-config --ssh-host _test0.localchat``.
|
||||
LXC subcommands do not depend on host DNS resolution
|
||||
and resolve addresses via ``lxconfigs/ssh-config``.
|
||||
|
||||
2. **``cmdeploy dns --zonefile``** generates a standard
|
||||
BIND-format zone file (MX, TXT/SPF, TXT/DMARC,
|
||||
TXT/MTA-STS, SRV, CNAME, DKIM) and loads it
|
||||
into PowerDNS.
|
||||
|
||||
This two-phase approach prevents premature configuration of mail records
|
||||
before the relay is actually deployed and running.
|
||||
Once ``cmdeploy run`` deploys `Unbound <https://nlnetlabs.nl/projects/unbound/>`_
|
||||
inside a relay container, Unbound has a configuration plugin snippet
|
||||
that forwards all ``.localchat`` queries to the PowerDNS recursor,
|
||||
and lets all other queries go through normal recursive resolution.
|
||||
|
||||
|
||||
State outside the repository
|
||||
-----------------------------
|
||||
|
||||
All generated configuration by lxc subcommands live in ``lxconfigs/``
|
||||
(git-ignored), including the SSH key pair (``id_localchat``),
|
||||
per-container ``chatmail-*.ini`` files, zone files, and ``ssh-config``.
|
||||
|
||||
The only state *outside* the repository is the Incus containers and images themselves
|
||||
(managed via the ``incus`` CLI, labelled with ``user.localchat-managed=true``).
|
||||
Two cached images are published to the local Incus image store:
|
||||
|
||||
* ``localchat-base``: Debian 12 with openssh-server and Python (built on first run)
|
||||
|
||||
* ``localchat-relay``: fully deployed relay snapshot,
|
||||
published after the first successful ``cmdeploy run``.
|
||||
Subsequent relay containers launch from this image
|
||||
so the deploy step is mostly no-ops (roughly 3× faster than a fresh deploy).
|
||||
Relay containers are limited to **500 MiB RAM** and the DNS container to **100 MiB**.
|
||||
|
||||
|
||||
.. _lxc-tls:
|
||||
|
||||
TLS handling and underscore domains
|
||||
------------------------------------
|
||||
|
||||
Container domains start with ``_`` (e.g. ``_test0.localchat``).
|
||||
As described in :doc:`getting_started` ("Running a relay with self-signed certificates"),
|
||||
underscore domains automatically use self-signed TLS
|
||||
and ``smtp_tls_security_level = encrypt``.
|
||||
This permits cross-relay federation between LXC containers
|
||||
without any external certificate authority.
|
||||
Delta Chat clients connecting to these relays
|
||||
must be configured with
|
||||
``certificateChecks = acceptInvalidCertificates``
|
||||
(the test fixtures handle this automatically).
|
||||
`PR #7926 on chatmail-core <https://github.com/chatmail/core/pull/7926>`_
|
||||
is meant to make this special setting unnecessary for chatmail clients
|
||||
that are connecting to underscore domains.
|
||||
|
||||
|
||||
Known limitations
|
||||
------------------
|
||||
|
||||
The LXC environment differs from a production
|
||||
deployment in several ways:
|
||||
|
||||
**No ACME / Let's Encrypt**:
|
||||
Self-signed TLS only (see :ref:`lxc-tls`);
|
||||
ACME code paths are never exercised locally.
|
||||
|
||||
**No inbound connections from the internet**:
|
||||
Containers sit on a private Incus bridge and are not port-forwarded.
|
||||
Only the host and other containers on the same bridge can reach them.
|
||||
|
||||
**Local federation only**:
|
||||
Cross-relay mail delivery (e.g. test0 → test1) works between containers on the same host,
|
||||
but these relays are invisible to any external mail server.
|
||||
|
||||
**DNS is local only**:
|
||||
The ``.localchat`` pseudo-TLD is not resolvable from the wider internet
|
||||
(see :ref:`lxc-dns-setup`).
|
||||
|
||||
**IPv6 is ULA-only**:
|
||||
Containers receive IPv6 addresses from the ``fd42:...`` ULA range on the Incus bridge.
|
||||
These are not globally routable, but are sufficient for testing IPv6 service binding
|
||||
(Postfix, Dovecot, Nginx) and DNS AAAA records inside the local environment.
|
||||
test1 runs with ``disable_ipv6 = True`` to exercise the IPv4-only deployment path.
|
||||
|
||||
**TURN server does not start**:
|
||||
``chatmail-turn`` discovers its listen addresses by enumerating globally routable IPs but
|
||||
LXC containers only have private RFC 1918 addresses (``10.x.x.x``),
|
||||
so the address list is empty and the server exits immediately.
|
||||
`PR #11 on chatmail-turn <https://github.com/chatmail/chatmail-turn/pull/11>`_
|
||||
is meant to fix this.
|
||||
Reference in New Issue
Block a user