Compare commits

..

19 Commits

Author SHA1 Message Date
Jagoda Ślązak
5cd887f6aa feat: Sign bounce messages
Closes #873

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-04-14 13:06:41 +02:00
Alexandre Gauthier
54863453c2 fix(cmdeploy): Set permissions on dovecot pin
Ensure the preferences.d snippet that pins dovecot packages to block
Debian dist-upgrades is owned by root:root and has 644 permissions.

Files in this directory are generally expected to be world readable to ensure unprivileged operations such as apt-get in simulation mode. Having them not world readable breaks such usages.
2026-04-10 15:52:49 +02:00
Jagoda Estera Ślązak
74326a8c54 feat(nginx): Route /mxdeliv/ to configurable port (#901) 2026-04-08 19:11:11 +02:00
holger krekel
59e5dea597 fix: make "cmdeploy test --config ..." work, without requiring or implicitely falling back to a "chatmail.ini" in parent dirs 2026-04-08 19:05:51 +02:00
holger krekel
d7d89d66c1 fix: properly terminate and wait on subprocesses on teardown 2026-04-08 19:05:51 +02:00
holger krekel
00d723bd6e refactor: deployer improvements (VM detection, mailboxes dir ensured to be there, proper unbound on ipv4) 2026-04-08 19:05:51 +02:00
holger krekel
c257bfca4b feat: update chatmail-turn to support private addresses 2026-04-08 19:05:51 +02:00
holger krekel
82c9831369 refactor: unify DNS zone-file to standard BIND format 2026-04-08 19:05:51 +02:00
Jagoda Estera Ślązak
b835318ce9 chore(deps): Upgrade to filtermail 0.6.1 (#910)
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-04-07 12:48:40 +02:00
j4n
b4a46d23e6 fix(cmdeploy): pin dovecot packages to prevent apt upgrades
As our .deb packages use Debian's version naming scheme, deploy an apt
preferences file that sets Pin-Priority: -1 for all dovecot-* packages
for every version of dovecot-* from every origin.
2026-03-31 17:12:30 +02:00
DarkCat09
c6d9d27a84 fix(deps): add rpc server to cmdeploy along with client 2026-03-29 16:02:24 +00:00
DarkCat09
4521f03c99 fix: remove duplicate deps from cmdeploy 2026-03-29 13:52:08 +00:00
DarkCat09
c78859aec6 fix(deps): add aiosmtpd to testenv 2026-03-29 13:52:08 +00:00
DarkCat09
98bd5944cc chore(deps): remove unused deps from chatmaild 2026-03-29 13:52:08 +00:00
link2xt
e8933c455f fix: set default smtp_tls_security_level to "verify" unconditionally
This change was accidentally added in cf96be2cbb
Relay should not stop validating TLS certificates of other relays
just because it has a self-signed or externally managed certificate.
Externally managed certificate is likely to even be valid.
2026-03-23 19:52:49 +00:00
link2xt
d3a483c403 feat(postfix): prefer IPv4 in SMTP client 2026-03-22 21:05:02 +00:00
j4n
e687120d96 fix(cmdeploy): Install dovecot .deb packages atomically
Since change 635ac7 we try to install Dovecot, even if it is already
running, which fails Dovecot upgrades fail when the installed version
differs from the target because dovecot-imapd/lmtpd dependencies
on dovecot-core: packages are installed one at a time via apt.deb(),
i.e. `dpkg -i`, and dpkg cannot satisfy them dependencies:
```
  dpkg: dependency problems prevent configuration of dovecot-imapd:
    dovecot-imapd depends on dovecot-core (= 1:2.3.21+dfsg1-3); however:
      Version of dovecot-core on system is 1:2.3.21.1+dfsg1-1~bpo12+1.
```

Split _install_dovecot_package into _download_dovecot_package (download
only, return path) and a single server.shell call that passes all .deb
files to dpkg -i together. Uses the same 3-step pattern as pyinfra's
apt.deb: tolerant first dpkg -i, apt-get --fix-broken, then final
dpkg -i to fail if there are still errors.
2026-03-21 16:17:37 +01:00
373[Ø]™
7409bd3452 Merge pull request #898 from chatmail/373/decom-cron
chore(cmdeploy): stop installing cron package
2026-03-19 10:55:36 +00:00
ccclxxiii
1a34172487 chore(cmdeploy): stop installing cron package 2026-03-18 20:35:27 +00:00
39 changed files with 327 additions and 1254 deletions

View File

@@ -1,18 +0,0 @@
data/
venv/
__pycache__
*.pyc
*.orig
*.ini
.pytest_cache
.env
# Slim build context — .git/ alone can be 100s of MB
.git
.github/
docs/
tests/
# Exclude markdown files but keep www/src/*.md (used by WebsiteDeployer)
*.md
!www/**/*.md

View File

@@ -15,7 +15,7 @@ jobs:
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: download filtermail
run: curl -L https://github.com/chatmail/filtermail/releases/download/v0.6.0/filtermail-x86_64 -o /usr/local/bin/filtermail && chmod +x /usr/local/bin/filtermail
run: curl -L https://github.com/chatmail/filtermail/releases/download/v0.6.1/filtermail-x86_64 -o /usr/local/bin/filtermail && chmod +x /usr/local/bin/filtermail
- name: run chatmaild tests
working-directory: chatmaild
run: pipx run tox

View File

@@ -4,7 +4,6 @@ on:
push:
branches:
- main
- j4n/docker-pr
pull_request:
paths-ignore:
- 'scripts/**'
@@ -12,67 +11,7 @@ on:
- 'CHANGELOG.md'
- 'LICENSE'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-docker:
name: Build Docker image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
outputs:
image: ${{ steps.image-ref.outputs.image }}
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GHCR
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
# Tagged releases: v1.2.3 -> :1.2.3, :1.2, :latest
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
# Branch pushes: foo/docker-pr -> :foo-docker-pr
type=ref,event=branch
# Always: :sha-<hash>
type=sha
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: docker/chatmail_relay.dockerfile
push: ${{ github.event_name == 'push' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
GIT_HASH=${{ github.sha }}
- name: Output image reference
id: image-ref
run: |
SHORT_SHA=$(echo "${{ github.sha }}" | cut -c1-7)
IMAGE="${{ env.REGISTRY }}/$(echo "${{ env.IMAGE_NAME }}" | tr '[:upper:]' '[:lower:]'):sha-${SHORT_SHA}"
echo "image=${IMAGE}" >> "$GITHUB_OUTPUT"
deploy:
name: deploy on staging-ipv4.testrun.org, and run tests
runs-on: ubuntu-latest
@@ -116,7 +55,6 @@ jobs:
run: echo venv/bin >>$GITHUB_PATH
- name: upload TLS cert after rebuilding
id: wait-for-vps
run: |
echo " --- wait until staging-ipv4.testrun.org VPS is rebuilt --- "
rm ~/.ssh/known_hosts
@@ -130,8 +68,8 @@ jobs:
rsync -avz dkimkeys-restore/dkimkeys root@staging-ipv4.testrun.org:/etc/ || true
ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org chown root:root -R /var/lib/acme || true
- name: run deploy-chatmail offline tests
run: pytest --pyargs cmdeploy
- name: run deploy-chatmail offline tests
run: pytest --pyargs cmdeploy
- name: setup dependencies
run: |
@@ -146,6 +84,7 @@ jobs:
ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy init staging-ipv4.testrun.org"
ssh root@staging-ipv4.testrun.org "sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' relay/chatmail.ini"
ssh root@staging-ipv4.testrun.org "sed -i 's/#\s*mtail_address/mtail_address/' relay/chatmail.ini"
ssh root@staging-ipv4.testrun.org "sed -i 's/max_mailbox_size = 500M/max_mailbox_size = 10k/' relay/chatmail.ini"
- run: ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy run --verbose --skip-dns-check --ssh-host localhost"
@@ -164,133 +103,3 @@ jobs:
- name: cmdeploy dns
run: ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy dns -v --ssh-host localhost"
# --- Docker deploy (push only, runs even if bare failed) ---
- name: stop bare services
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
ssh root@staging-ipv4.testrun.org 'systemctl stop postfix dovecot nginx opendkim unbound filtermail doveauth chatmail-metadata iroh-relay mtail fcgiwrap acmetool 2>/dev/null || true'
- name: install Docker on VPS
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
ssh root@staging-ipv4.testrun.org 'apt-get update && apt-get install -y ca-certificates curl'
ssh root@staging-ipv4.testrun.org 'install -m 0755 -d /etc/apt/keyrings'
ssh root@staging-ipv4.testrun.org 'curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc && chmod a+r /etc/apt/keyrings/docker.asc'
ssh root@staging-ipv4.testrun.org 'echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list'
ssh root@staging-ipv4.testrun.org 'apt-get update && apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin'
- name: prepare Docker bind mounts
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
ssh root@staging-ipv4.testrun.org 'mkdir -p /srv/chatmail/certs /srv/chatmail/dkim'
ssh root@staging-ipv4.testrun.org 'cp -a /var/lib/acme/. /srv/chatmail/certs/ && cp -a /etc/dkimkeys/. /srv/chatmail/dkim/' || true
- name: upload chatmail.ini for Docker
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
# Reuse chatmail.ini already created by the bare-metal deploy steps
ssh root@staging-ipv4.testrun.org "cp relay/chatmail.ini /srv/chatmail/chatmail.ini"
- name: deploy with Docker
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
SHORT_SHA=$(echo "${{ github.sha }}" | cut -c1-7)
GHCR_IMAGE="${{ env.REGISTRY }}/$(echo "${{ env.IMAGE_NAME }}" | tr '[:upper:]' '[:lower:]'):sha-${SHORT_SHA}"
rsync -avz --exclude='.git' --exclude='venv' --exclude='__pycache__' ./ root@staging-ipv4.testrun.org:/srv/chatmail/relay/
# Login to GHCR on VPS and pull pre-built image
echo "${{ secrets.GITHUB_TOKEN }}" | ssh root@staging-ipv4.testrun.org 'docker login ghcr.io -u ${{ github.actor }} --password-stdin'
ssh root@staging-ipv4.testrun.org "docker pull ${GHCR_IMAGE}"
ssh root@staging-ipv4.testrun.org "cd /srv/chatmail/relay && CHATMAIL_IMAGE=${GHCR_IMAGE} MAIL_DOMAIN=staging-ipv4.testrun.org docker compose -f docker/docker-compose.yaml -f docker/docker-compose.ci.yaml up -d"
- name: wait for container healthy
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
# Stream journald inside the container
ssh root@staging-ipv4.testrun.org 'docker exec chatmail journalctl -f --no-pager' &
LOG_PID=$!
trap "kill $LOG_PID 2>/dev/null || true" EXIT
for i in $(seq 1 60); do
status=$(ssh root@staging-ipv4.testrun.org 'docker inspect --format={{.State.Health.Status}} chatmail 2>/dev/null' || echo "missing")
echo " [$i/60] status=$status"
if [ "$status" = "healthy" ]; then
echo "Container is healthy."
exit 0
fi
if [ "$status" = "unhealthy" ]; then
echo "Container is unhealthy!"
break
fi
sleep 5
done
echo "Container did not become healthy."
kill $LOG_PID 2>/dev/null || true
echo "--- failed units ---"
ssh root@staging-ipv4.testrun.org 'docker exec chatmail systemctl --failed --no-pager' || true
echo "--- service logs ---"
ssh root@staging-ipv4.testrun.org 'docker exec chatmail journalctl -u dovecot -u postfix -u nginx -u unbound --no-pager -n 50' || true
echo "--- listening ports ---"
ssh root@staging-ipv4.testrun.org 'docker exec chatmail ss -tlnp' || true
echo "--- chatmail.ini ---"
ssh root@staging-ipv4.testrun.org 'docker exec chatmail cat /etc/chatmail/chatmail.ini' || true
exit 1
- name: show container state
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
echo "--- listening ports ---"
ssh root@staging-ipv4.testrun.org 'docker exec chatmail ss -tlnp'
echo "--- chatmail.ini ---"
ssh root@staging-ipv4.testrun.org 'docker exec chatmail cat /etc/chatmail/chatmail.ini'
- name: Docker integration tests
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
ssh root@staging-ipv4.testrun.org 'docker exec chatmail cmdeploy test --slow --ssh-host @local'
- name: Docker DNS
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
# Reset zone file in case bare DNS already appended to it
git checkout .github/workflows/staging-ipv4.testrun.org-default.zone
ssh root@staging-ipv4.testrun.org 'docker exec chatmail chown opendkim:opendkim -R /etc/dkimkeys'
ssh root@staging-ipv4.testrun.org 'docker exec chatmail cmdeploy dns --ssh-host @local --zonefile /opt/chatmail/staging.zone --verbose'
ssh root@staging-ipv4.testrun.org 'docker cp chatmail:/opt/chatmail/staging.zone /tmp/staging.zone'
scp root@staging-ipv4.testrun.org:/tmp/staging.zone staging-generated.zone
cat staging-generated.zone >> .github/workflows/staging-ipv4.testrun.org-default.zone
cat .github/workflows/staging-ipv4.testrun.org-default.zone
scp .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone
ssh root@ns.testrun.org nsd-checkzone staging-ipv4.testrun.org /etc/nsd/staging-ipv4.testrun.org.zone
ssh root@ns.testrun.org systemctl reload nsd
- name: Docker final DNS check
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: ssh root@staging-ipv4.testrun.org 'docker exec chatmail cmdeploy dns -v --ssh-host @local'
# --- Cleanup ---
- name: add SSH keys
if: >-
!cancelled()
&& steps.wait-for-vps.outcome == 'success'
run: ssh root@staging-ipv4.testrun.org 'curl -s https://github.com/hpk42.keys https://github.com/j4n.keys >> .ssh/authorized_keys'

View File

@@ -4,7 +4,6 @@ on:
push:
branches:
- main
- j4n/docker-pr
pull_request:
paths-ignore:
- 'scripts/**'
@@ -12,67 +11,7 @@ on:
- 'CHANGELOG.md'
- 'LICENSE'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-docker:
name: Build Docker image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
outputs:
image: ${{ steps.image-ref.outputs.image }}
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GHCR
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
# Tagged releases: v1.2.3 -> :1.2.3, :1.2, :latest
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
# Branch pushes: foo/docker-pr -> :foo-docker-pr
type=ref,event=branch
# Always: :sha-<hash>
type=sha
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: docker/chatmail_relay.dockerfile
push: ${{ github.event_name == 'push' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
GIT_HASH=${{ github.sha }}
- name: Output image reference
id: image-ref
run: |
SHORT_SHA=$(echo "${{ github.sha }}" | cut -c1-7)
IMAGE="${{ env.REGISTRY }}/$(echo "${{ env.IMAGE_NAME }}" | tr '[:upper:]' '[:lower:]'):sha-${SHORT_SHA}"
echo "image=${IMAGE}" >> "$GITHUB_OUTPUT"
deploy:
name: deploy on staging2.testrun.org, and run tests
runs-on: ubuntu-latest
@@ -116,7 +55,6 @@ jobs:
run: echo venv/bin >>$GITHUB_PATH
- name: upload TLS cert after rebuilding
id: wait-for-vps
run: |
echo " --- wait until staging2.testrun.org VPS is rebuilt --- "
rm ~/.ssh/known_hosts
@@ -133,12 +71,12 @@ jobs:
- name: add hpk42 key to staging server
run: ssh root@staging2.testrun.org 'curl -s https://github.com/hpk42.keys >> .ssh/authorized_keys'
- name: run deploy-chatmail offline tests
run: pytest --pyargs cmdeploy
- name: run deploy-chatmail offline tests
run: pytest --pyargs cmdeploy
- run: |
cmdeploy init staging2.testrun.org
sed -i 's/#\s*mtail_address/mtail_address/' chatmail.ini
sed -i 's/#\s*mtail_address/mtail_address/;s/max_mailbox_size = 500M/max_mailbox_size = 10k/' chatmail.ini
- run: cmdeploy run --verbose --skip-dns-check
@@ -157,133 +95,3 @@ jobs:
- name: cmdeploy dns
run: cmdeploy dns -v
# --- Docker deploy (push only, runs even if bare failed) ---
- name: stop bare services
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
ssh root@staging2.testrun.org 'systemctl stop postfix dovecot nginx opendkim unbound filtermail doveauth chatmail-metadata iroh-relay mtail fcgiwrap acmetool 2>/dev/null || true'
- name: install Docker on VPS
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
ssh root@staging2.testrun.org 'apt-get update && apt-get install -y ca-certificates curl'
ssh root@staging2.testrun.org 'install -m 0755 -d /etc/apt/keyrings'
ssh root@staging2.testrun.org 'curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc && chmod a+r /etc/apt/keyrings/docker.asc'
ssh root@staging2.testrun.org 'echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list'
ssh root@staging2.testrun.org 'apt-get update && apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin'
- name: prepare Docker bind mounts
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
ssh root@staging2.testrun.org 'mkdir -p /srv/chatmail/certs /srv/chatmail/dkim'
ssh root@staging2.testrun.org 'cp -a /var/lib/acme/. /srv/chatmail/certs/ && cp -a /etc/dkimkeys/. /srv/chatmail/dkim/' || true
- name: upload chatmail.ini for Docker
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
# Reuse chatmail.ini already created by the bare-metal deploy steps
scp chatmail.ini root@staging2.testrun.org:/srv/chatmail/chatmail.ini
- name: deploy with Docker
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
SHORT_SHA=$(echo "${{ github.sha }}" | cut -c1-7)
GHCR_IMAGE="${{ env.REGISTRY }}/$(echo "${{ env.IMAGE_NAME }}" | tr '[:upper:]' '[:lower:]'):sha-${SHORT_SHA}"
rsync -avz --exclude='.git' --exclude='venv' --exclude='__pycache__' ./ root@staging2.testrun.org:/srv/chatmail/relay/
# Login to GHCR on VPS and pull pre-built image
echo "${{ secrets.GITHUB_TOKEN }}" | ssh root@staging2.testrun.org 'docker login ghcr.io -u ${{ github.actor }} --password-stdin'
ssh root@staging2.testrun.org "docker pull ${GHCR_IMAGE}"
ssh root@staging2.testrun.org "cd /srv/chatmail/relay && CHATMAIL_IMAGE=${GHCR_IMAGE} MAIL_DOMAIN=staging2.testrun.org docker compose -f docker/docker-compose.yaml -f docker/docker-compose.ci.yaml up -d"
- name: wait for container healthy
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
# Stream journald inside the container
ssh root@staging2.testrun.org 'docker exec chatmail journalctl -f --no-pager' &
LOG_PID=$!
trap "kill $LOG_PID 2>/dev/null || true" EXIT
for i in $(seq 1 60); do
status=$(ssh root@staging2.testrun.org 'docker inspect --format={{.State.Health.Status}} chatmail 2>/dev/null' || echo "missing")
echo " [$i/60] status=$status"
if [ "$status" = "healthy" ]; then
echo "Container is healthy."
exit 0
fi
if [ "$status" = "unhealthy" ]; then
echo "Container is unhealthy!"
break
fi
sleep 5
done
echo "Container did not become healthy."
kill $LOG_PID 2>/dev/null || true
echo "--- failed units ---"
ssh root@staging2.testrun.org 'docker exec chatmail systemctl --failed --no-pager' || true
echo "--- service logs ---"
ssh root@staging2.testrun.org 'docker exec chatmail journalctl -u dovecot -u postfix -u nginx -u unbound --no-pager -n 50' || true
echo "--- listening ports ---"
ssh root@staging2.testrun.org 'docker exec chatmail ss -tlnp' || true
echo "--- chatmail.ini ---"
ssh root@staging2.testrun.org 'docker exec chatmail cat /etc/chatmail/chatmail.ini' || true
exit 1
- name: show container state
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
echo "--- listening ports ---"
ssh root@staging2.testrun.org 'docker exec chatmail ss -tlnp'
echo "--- chatmail.ini ---"
ssh root@staging2.testrun.org 'docker exec chatmail cat /etc/chatmail/chatmail.ini'
- name: Docker integration tests
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
ssh root@staging2.testrun.org 'docker exec chatmail cmdeploy test --slow --ssh-host @local'
- name: Docker DNS
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: |
# Reset zone file in case bare DNS already appended to it
git checkout .github/workflows/staging.testrun.org-default.zone
ssh root@staging2.testrun.org 'docker exec chatmail chown opendkim:opendkim -R /etc/dkimkeys'
ssh root@staging2.testrun.org 'docker exec chatmail cmdeploy dns --ssh-host @local --zonefile /opt/chatmail/staging.zone --verbose'
ssh root@staging2.testrun.org 'docker cp chatmail:/opt/chatmail/staging.zone /tmp/staging.zone'
scp root@staging2.testrun.org:/tmp/staging.zone staging-generated.zone
cat staging-generated.zone >> .github/workflows/staging.testrun.org-default.zone
cat .github/workflows/staging.testrun.org-default.zone
scp .github/workflows/staging.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging2.testrun.org.zone
ssh root@ns.testrun.org nsd-checkzone staging2.testrun.org /etc/nsd/staging2.testrun.org.zone
ssh root@ns.testrun.org systemctl reload nsd
- name: Docker final DNS check
if: >-
!cancelled() && github.event_name == 'push'
&& steps.wait-for-vps.outcome == 'success'
run: ssh root@staging2.testrun.org 'docker exec chatmail cmdeploy dns -v --ssh-host @local'
# --- Cleanup ---
- name: add SSH keys
if: >-
!cancelled()
&& steps.wait-for-vps.outcome == 'success'
run: ssh root@staging2.testrun.org 'curl -s https://github.com/hpk42.keys https://github.com/j4n.keys >> .ssh/authorized_keys'

6
.gitignore vendored
View File

@@ -164,9 +164,3 @@ cython_debug/
#.idea/
chatmail.zone
# docker
/data/
/custom/
docker/docker-compose.override.yaml
docker/.env

View File

@@ -6,10 +6,7 @@ build-backend = "setuptools.build_meta"
name = "chatmaild"
version = "0.3"
dependencies = [
"aiosmtpd",
"iniconfig",
"deltachat-rpc-server",
"deltachat-rpc-client",
"filelock",
"requests",
"crypt-r >= 3.13.1 ; python_version >= '3.11'",
@@ -70,6 +67,7 @@ commands =
deps = pytest
pdbpp
pytest-localserver
aiosmtpd
execnet
commands = pytest -v -rsXx {posargs}
"""

View File

@@ -38,6 +38,7 @@ class Config:
self.filtermail_smtp_port_incoming = int(
params.get("filtermail_smtp_port_incoming", "10081")
)
self.filtermail_http_port = int(params.get("filtermail_http_port", "10082"))
self.postfix_reinject_port = int(params.get("postfix_reinject_port", "10025"))
self.postfix_reinject_port_incoming = int(
params.get("postfix_reinject_port_incoming", "10026")

View File

@@ -10,7 +10,6 @@ dependencies = [
"pillow",
"qrcode",
"markdown",
"pytest",
"setuptools>=68",
"termcolor",
"build",
@@ -21,6 +20,7 @@ dependencies = [
"execnet",
"imap_tools",
"deltachat-rpc-client",
"deltachat-rpc-server",
]
[project.scripts]

View File

@@ -1,32 +0,0 @@
;
; Required DNS entries for chatmail servers
;
{% if A %}
{{ mail_domain }}. A {{ A }}
{% endif %}
{% if AAAA %}
{{ mail_domain }}. AAAA {{ AAAA }}
{% endif %}
{{ mail_domain }}. MX 10 {{ mail_domain }}.
{% if strict_tls %}
_mta-sts.{{ mail_domain }}. TXT "v=STSv1; id={{ sts_id }}"
mta-sts.{{ mail_domain }}. CNAME {{ mail_domain }}.
{% endif %}
www.{{ mail_domain }}. CNAME {{ mail_domain }}.
{{ dkim_entry }}
;
; Recommended DNS entries for interoperability and security-hardening
;
{{ mail_domain }}. TXT "v=spf1 a ~all"
_dmarc.{{ mail_domain }}. TXT "v=DMARC1;p=reject;adkim=s;aspf=s"
{% if acme_account_url %}
{{ mail_domain }}. CAA 0 issue "letsencrypt.org;accounturi={{ acme_account_url }}"
{% endif %}
_adsp._domainkey.{{ mail_domain }}. TXT "dkim=discardable"
_submission._tcp.{{ mail_domain }}. SRV 0 1 587 {{ mail_domain }}.
_submissions._tcp.{{ mail_domain }}. SRV 0 1 465 {{ mail_domain }}.
_imap._tcp.{{ mail_domain }}. SRV 0 1 143 {{ mail_domain }}.
_imaps._tcp.{{ mail_domain }}. SRV 0 1 993 {{ mail_domain }}.

View File

@@ -108,7 +108,9 @@ def run_cmd(args, out):
pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
if ssh_host == "localhost":
if ssh_host in ["localhost", "@docker"]:
if ssh_host == "@docker":
env["CHATMAIL_NOPORTCHECK"] = "True"
cmd = f"{pyinf} @local {deploy_path} -y"
if version.parse(pyinfra.__version__) < version.parse("3"):
@@ -207,6 +209,7 @@ def test_cmd(args, out):
"""Run local and online tests for chatmail deployment."""
env = os.environ.copy()
env["CHATMAIL_INI"] = str(args.inipath.absolute())
if args.ssh_host:
env["CHATMAIL_SSH"] = args.ssh_host
@@ -313,7 +316,7 @@ def add_ssh_host_option(parser):
parser.add_argument(
"--ssh-host",
dest="ssh_host",
help="Run commands on 'localhost' or on a specific SSH host "
help="Run commands on 'localhost', via '@docker', or on a specific SSH host "
"instead of chatmail.ini's mail_domain.",
)
@@ -373,12 +376,14 @@ def get_parser():
return parser
def get_sshexec(ssh_host: str, verbose=True):
def get_sshexec(ssh_host: str, verbose=True, **kwargs):
if ssh_host in ["localhost", "@local"]:
return LocalExec(verbose)
return LocalExec(verbose, docker=False)
elif ssh_host == "@docker":
return LocalExec(verbose, docker=True)
if verbose:
print(f"[ssh] login to {ssh_host}")
return SSHExec(ssh_host, verbose=verbose)
return SSHExec(ssh_host, verbose=verbose, **kwargs)
def main(args=None):

View File

@@ -2,6 +2,7 @@
Chat Mail pyinfra deploy.
"""
import os
import shutil
import subprocess
import sys
@@ -13,7 +14,6 @@ from pyinfra import facts, host, logger
from pyinfra.api import FactBase
from pyinfra.facts import hardware
from pyinfra.facts.files import Sha256File
from pyinfra.facts.server import Command
from pyinfra.facts.systemd import SystemdEnabled
from pyinfra.operations import apt, files, pip, server, systemd
@@ -24,6 +24,7 @@ from .basedeploy import (
Deployer,
Deployment,
activate_remote_units,
blocked_service_startup,
configure_remote_units,
get_resource,
has_systemd,
@@ -149,33 +150,16 @@ class UnboundDeployer(Deployer):
self.need_restart = False
def install(self):
# Run local DNS resolver `unbound`.
# `resolvconf` takes care of setting up /etc/resolv.conf
# to use 127.0.0.1 as the resolver.
# Run local DNS resolver `unbound`. `resolvconf` takes care of
# setting up /etc/resolv.conf to use 127.0.0.1 as the resolver.
#
# On an IPv4-only system, if unbound is started but not
# configured, it causes subsequent steps to fail to resolve hosts.
# Here, we use policy-rc.d to prevent unbound from starting up
# on initial install. Later, we will configure it and start it.
#
# For documentation about policy-rc.d, see:
# https://people.debian.org/~hmh/invokerc.d-policyrc.d-specification.txt
#
files.put(
src=get_resource("policy-rc.d"),
dest="/usr/sbin/policy-rc.d",
user="root",
group="root",
mode="755",
)
apt.packages(
name="Install unbound",
packages=["unbound", "unbound-anchor", "dnsutils"],
)
files.file("/usr/sbin/policy-rc.d", present=False)
# On an IPv4-only system, if unbound is started but not configured,
# it causes subsequent steps to fail to resolve hosts.
with blocked_service_startup():
apt.packages(
name="Install unbound",
packages=["unbound", "unbound-anchor", "dnsutils"],
)
def configure(self):
server.shell(
@@ -336,12 +320,12 @@ class TurnDeployer(Deployer):
def install(self):
(url, sha256sum) = {
"x86_64": (
"https://github.com/chatmail/chatmail-turn/releases/download/v0.3/chatmail-turn-x86_64-linux",
"841e527c15fdc2940b0469e206188ea8f0af48533be12ecb8098520f813d41e4",
"https://github.com/chatmail/chatmail-turn/releases/download/v0.4/chatmail-turn-x86_64-linux",
"1ec1f5c50122165e858a5a91bcba9037a28aa8cb8b64b8db570aa457c6141a8a",
),
"aarch64": (
"https://github.com/chatmail/chatmail-turn/releases/download/v0.3/chatmail-turn-aarch64-linux",
"a5fc2d06d937b56a34e098d2cd72a82d3e89967518d159bf246dc69b65e81b42",
"https://github.com/chatmail/chatmail-turn/releases/download/v0.4/chatmail-turn-aarch64-linux",
"0fb3e792419494e21ecad536464929dba706bb2c88884ed8f1788141d26fc756",
),
}[host.get_fact(facts.server.Arch)]
@@ -474,8 +458,9 @@ class ChatmailDeployer(Deployer):
("iroh", None, None),
]
def __init__(self, mail_domain):
self.mail_domain = mail_domain
def __init__(self, config):
self.config = config
self.mail_domain = config.mail_domain
def install(self):
files.put(
@@ -498,12 +483,18 @@ class ChatmailDeployer(Deployer):
name="Install rsync",
packages=["rsync"],
)
apt.packages(
name="Ensure cron is installed",
packages=["cron"],
)
def configure(self):
# metadata crashes if the mailboxes dir does not exist
files.directory(
name="Ensure vmail mailbox directory exists",
path=str(self.config.mailboxes_dir),
user="vmail",
group="vmail",
mode="700",
present=True,
)
# This file is used by auth proxy.
# https://wiki.debian.org/EtcMailName
server.shell(
@@ -593,7 +584,7 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
Out().red(f"Deploy failed: mtail_address {config.mtail_address} is not available (VPN up?).\n")
exit(1)
if host.get_fact(Command, "systemd-detect-virt -c || true") == "none":
if not os.environ.get("CHATMAIL_NOPORTCHECK"):
port_services = [
(["master", "smtpd"], 25),
("unbound", 53),
@@ -633,7 +624,7 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
tls_deployer = get_tls_deployer(config, mail_domain)
all_deployers = [
ChatmailDeployer(mail_domain),
ChatmailDeployer(config),
LegacyRemoveDeployer(),
FiltermailDeployer(),
JournaldDeployer(),

View File

@@ -1,11 +1,22 @@
import datetime
import importlib
from jinja2 import Template
from . import remote
def parse_zone_records(text):
"""Yield ``(name, ttl, rtype, rdata)`` from standard BIND-format text."""
for raw_line in text.splitlines():
line = raw_line.strip()
if not line or line.startswith(";"):
continue
try:
name, ttl, _in, rtype, rdata = line.split(None, 4)
except ValueError:
raise ValueError(f"Bad zone record line: {line!r}") from None
name = name.rstrip(".")
yield name, ttl, rtype.upper(), rdata
def get_initial_remote_data(sshexec, mail_domain):
return sshexec.logged(
call=remote.rdns.perform_initial_checks, kwargs=dict(mail_domain=mail_domain)
@@ -31,13 +42,39 @@ def get_filled_zone_file(remote_data):
if not sts_id:
remote_data["sts_id"] = datetime.datetime.now().strftime("%Y%m%d%H%M")
template = importlib.resources.files(__package__).joinpath("chatmail.zone.j2")
content = template.read_text()
zonefile = Template(content).render(**remote_data)
lines = [x.strip() for x in zonefile.split("\n") if x.strip()]
d = remote_data["mail_domain"]
def append_record(name, rtype, rdata, ttl=3600):
lines.append(f"{name:<40} {ttl:<6} IN {rtype:<5} {rdata}")
lines = ["; Required DNS entries"]
if remote_data.get("A"):
append_record(f"{d}.", "A", remote_data["A"])
if remote_data.get("AAAA"):
append_record(f"{d}.", "AAAA", remote_data["AAAA"])
append_record(f"{d}.", "MX", f"10 {d}.")
if remote_data.get("strict_tls"):
append_record(f"_mta-sts.{d}.", "TXT", f'"v=STSv1; id={remote_data["sts_id"]}"')
append_record(f"mta-sts.{d}.", "CNAME", f"{d}.")
append_record(f"www.{d}.", "CNAME", f"{d}.")
lines.append(remote_data["dkim_entry"])
lines.append("")
zonefile = "\n".join(lines)
return zonefile
lines.append("; Recommended DNS entries")
append_record(f"{d}.", "TXT", '"v=spf1 a ~all"')
append_record(f"_dmarc.{d}.", "TXT", '"v=DMARC1;p=reject;adkim=s;aspf=s"')
if remote_data.get("acme_account_url"):
append_record(
f"{d}.",
"CAA",
f'0 issue "letsencrypt.org;accounturi={remote_data["acme_account_url"]}"',
)
append_record(f"_adsp._domainkey.{d}.", "TXT", '"dkim=discardable"')
append_record(f"_submission._tcp.{d}.", "SRV", f"0 1 587 {d}.")
append_record(f"_submissions._tcp.{d}.", "SRV", f"0 1 465 {d}.")
append_record(f"_imap._tcp.{d}.", "SRV", f"0 1 143 {d}.")
append_record(f"_imaps._tcp.{d}.", "SRV", f"0 1 993 {d}.")
lines.append("")
return "\n".join(lines)
def check_full_zone(sshexec, remote_data, out, zonefile) -> int:
@@ -58,7 +95,8 @@ def check_full_zone(sshexec, remote_data, out, zonefile) -> int:
returncode = 1
if remote_data.get("dkim_entry") in required_diff:
out(
"If the DKIM entry above does not work with your DNS provider, you can try this one:\n"
"If the DKIM entry above does not work with your DNS provider,"
" you can try this one:\n"
)
out(remote_data.get("web_dkim_entry") + "\n")
if recommended_diff:

View File

@@ -1,3 +1,4 @@
import io
import urllib.request
from chatmaild.config import Config
@@ -37,9 +38,33 @@ class DovecotDeployer(Deployer):
def install(self):
arch = host.get_fact(Arch)
with blocked_service_startup():
_install_dovecot_package("core", arch)
_install_dovecot_package("imapd", arch)
_install_dovecot_package("lmtpd", arch)
debs = []
for pkg in ("core", "imapd", "lmtpd"):
deb = _download_dovecot_package(pkg, arch)
if deb:
debs.append(deb)
if debs:
deb_list = " ".join(debs)
server.shell(
name="Install dovecot packages",
commands=[
f"dpkg --force-confdef --force-confold -i {deb_list} 2> /dev/null || true",
"DEBIAN_FRONTEND=noninteractive apt-get -y --fix-broken install",
f"dpkg --force-confdef --force-confold -i {deb_list}",
],
)
files.put(
name="Pin dovecot packages to block Debian dist-upgrades",
src=io.StringIO(
"Package: dovecot-*\n"
"Pin: version *\n"
"Pin-Priority: -1\n"
),
dest="/etc/apt/preferences.d/pin-dovecot",
user="root",
group="root",
mode="644",
)
def configure(self):
configure_remote_units(self.config.mail_domain, self.units)
@@ -72,7 +97,8 @@ def _pick_url(primary, fallback):
return fallback
def _install_dovecot_package(package: str, arch: str):
def _download_dovecot_package(package: str, arch: str):
"""Download a dovecot .deb if needed, return its path (or None)."""
arch = "amd64" if arch == "x86_64" else arch
arch = "arm64" if arch == "aarch64" else arch
@@ -80,11 +106,11 @@ def _install_dovecot_package(package: str, arch: str):
sha256 = DOVECOT_SHA256.get((package, arch))
if sha256 is None:
apt.packages(packages=[pkg_name])
return
return None
installed_versions = host.get_fact(DebPackages).get(pkg_name, [])
if DOVECOT_VERSION in installed_versions:
return
return None
url_version = DOVECOT_VERSION.replace("+", "%2B")
deb_base = f"{pkg_name}_{url_version}_{arch}.deb"
@@ -101,10 +127,21 @@ def _install_dovecot_package(package: str, arch: str):
cache_time=60 * 60 * 24 * 365 * 10, # never redownload the package
)
apt.deb(name=f"Install {pkg_name}", src=deb_filename)
return deb_filename
def _configure_dovecot(config: Config, debug: bool = False) -> (bool, bool):
def _can_set_inotify_limits() -> bool:
is_container = (
host.get_fact(
Command,
"systemd-detect-virt --container --quiet 2>/dev/null && echo yes || true",
)
== "yes"
)
return not is_container
def _configure_dovecot(config: Config, debug: bool = False) -> tuple[bool, bool]:
"""Configures Dovecot IMAP server."""
need_restart = False
daemon_reload = False
@@ -139,7 +176,7 @@ def _configure_dovecot(config: Config, debug: bool = False) -> (bool, bool):
# as per https://doc.dovecot.org/2.3/configuration_manual/os/
# it is recommended to set the following inotify limits
can_modify = host.get_fact(Command, "systemd-detect-virt -c || true") == "none"
can_modify = _can_set_inotify_limits()
for name in ("max_user_instances", "max_user_watches"):
key = f"fs.inotify.{name}"
value = host.get_fact(Sysctl)[key]
@@ -147,7 +184,7 @@ def _configure_dovecot(config: Config, debug: bool = False) -> (bool, bool):
continue
if not can_modify:
print(
"\n!!!! refusing to attempt sysctl setting in shared-kernel containers\n"
"\n!!!! refusing to attempt sysctl setting in containers\n"
f"!!!! dovecot: sysctl {key!r}={value}, should be >65534 for production setups\n"
"!!!!"
)

View File

@@ -14,10 +14,10 @@ class FiltermailDeployer(Deployer):
def install(self):
arch = host.get_fact(facts.server.Arch)
url = f"https://github.com/chatmail/filtermail/releases/download/v0.6.0/filtermail-{arch}"
url = f"https://github.com/chatmail/filtermail/releases/download/v0.6.1/filtermail-{arch}"
sha256sum = {
"x86_64": "3fd8b18282252c75a5bbfa603d8c1b65f6563e5e920bddf3e64e451b7cdb43ce",
"aarch64": "2bd191de205f7fd60158dd8e3516ab7e3efb14627696f3d7dc186bdcd9e10a43",
"x86_64": "48b3fb80c092d00b9b0a0ef77a8673496da3b9aed5ec1851e1df936d5589d62f",
"aarch64": "c65bd5f45df187d3d65d6965a285583a3be0f44a6916ff12909ff9a8d702c22e",
}[arch]
self.need_restart |= files.download(
name="Download filtermail",

View File

@@ -73,6 +73,10 @@ http {
access_log syslog:server=unix:/dev/log,facility=local7;
location /mxdeliv/ {
proxy_pass http://127.0.0.1:{{ config.filtermail_http_port }};
}
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.

View File

@@ -20,7 +20,7 @@ smtpd_tls_key_file={{ config.tls_key_path }}
smtpd_tls_security_level=may
smtp_tls_CApath=/etc/ssl/certs
smtp_tls_security_level={{ "verify" if config.tls_cert_mode == "acme" else "encrypt" }}
smtp_tls_security_level=verify
# Send SNI extension when connecting to other servers.
# <https://www.postfix.org/postconf.5.html#smtp_tls_servername>
smtp_tls_servername = hostname
@@ -88,6 +88,22 @@ inet_protocols = ipv4
inet_protocols = all
{% endif %}
# Postfix does not try IPv4 and IPv6 connections
# concurrently as of version 3.7.11.
#
# When relay has both A (IPv4) and AAAA (IPv6) records,
# but broken IPv6 connectivity,
# every second message is delayed by the connection timeout
# <https://www.postfix.org/postconf.5.html#smtp_connect_timeout>
# which defaults to 30 seconds. Reducing timeouts is not a solution
# as this will result in a failure to connect to slow servers.
#
# As a workaround we always prefer IPv4 when it is available.
#
# The setting is documented at
# <https://www.postfix.org/postconf.5.html#smtp_address_preference>
smtp_address_preference=ipv4
virtual_transport = lmtp:unix:private/dovecot-lmtp
virtual_mailbox_domains = {{ config.mail_domain }}
lmtp_header_checks = regexp:/etc/postfix/lmtp_header_cleanup

View File

@@ -51,12 +51,15 @@ smtps inet n - y - 5000 smtpd
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
#628 inet n - y - - qmqpd
pickup unix n - y 60 1 pickup
-o cleanup_service_name=signlocal
cleanup unix n - y - 0 cleanup
qmgr unix n - n 300 1 qmgr
#qmgr unix n - n 300 1 oqmgr
tlsmgr unix - - y 1000? 1 tlsmgr
rewrite unix - - y - - trivial-rewrite
bounce unix - - y - 0 bounce
-o internal_mail_filter_classes=bounce
-o cleanup_service_name=signlocal
defer unix - - y - 0 bounce
trace unix - - y - 0 bounce
verify unix - - y - 1 verify
@@ -100,3 +103,9 @@ filter unix - n n - - lmtp
# cannot send unprotected Subject.
authclean unix n - - - 0 cleanup
-o header_checks=regexp:/etc/postfix/submission_header_cleanup
# Signs locally generated bounce messages.
# These can't be signed using smtpd as they are not non-smtpd.
signlocal unix n - - - 0 cleanup
-o milter_macro_daemon_name=ORIGINATING
-o non_smtpd_milters=unix:opendkim/opendkim.sock

View File

@@ -57,9 +57,10 @@ def get_dkim_entry(mail_domain, pre_command, dkim_selector):
dkim_value_raw = f"v=DKIM1;k=rsa;p={dkim_pubkey};s=email;t=s"
dkim_value = '" "'.join(re.findall(".{1,255}", dkim_value_raw))
web_dkim_value = "".join(re.findall(".{1,255}", dkim_value_raw))
name = f"{dkim_selector}._domainkey.{mail_domain}."
return (
f'{dkim_selector}._domainkey.{mail_domain}. TXT "{dkim_value}"',
f'{dkim_selector}._domainkey.{mail_domain}. TXT "{web_dkim_value}"',
f'{name:<40} 3600 IN TXT "{dkim_value}"',
f'{name:<40} 3600 IN TXT "{web_dkim_value}"',
)
@@ -94,7 +95,7 @@ def check_zonefile(zonefile, verbose=True):
if not zf_line.strip() or zf_line.startswith(";"):
continue
print(f"dns-checking {zf_line!r}") if verbose else log_progress("")
zf_domain, zf_typ, zf_value = zf_line.split(maxsplit=2)
zf_domain, _ttl, _in, zf_typ, zf_value = zf_line.split(None, 4)
zf_domain = zf_domain.rstrip(".")
zf_value = zf_value.strip()
query_value = query_dns(zf_typ, zf_domain)

View File

@@ -18,6 +18,8 @@ def openssl_selfsigned_args(domain, cert_path, key_path, days=36500):
"-keyout", str(key_path),
"-out", str(cert_path),
"-subj", f"/CN={domain}",
# Mark as end-entity cert so it cannot be used as a CA to sign others.
"-addext", "basicConstraints=critical,CA:FALSE",
"-addext", "extendedKeyUsage=serverAuth,clientAuth",
"-addext",
f"subjectAltName=DNS:{domain},DNS:www.{domain},DNS:mta-sts.{domain}",

View File

@@ -49,8 +49,9 @@ class SSHExec:
RemoteError = execnet.RemoteError
FuncError = FuncError
def __init__(self, host, verbose=False, python="python3", timeout=60):
self.gateway = execnet.makegateway(f"ssh=root@{host}//python={python}")
def __init__(self, host, verbose=False, python="python3", timeout=60, ssh_options=None):
ssh_options = f"{ssh_options.strip()} " if ssh_options is not None else ""
self.gateway = execnet.makegateway(f"ssh={ssh_options}root@{host}//python={python}")
self._remote_cmdloop_channel = bootstrap_remote(self.gateway, remote)
self.timeout = timeout
self.verbose = verbose
@@ -87,8 +88,9 @@ class SSHExec:
class LocalExec:
FuncError = FuncError
def __init__(self, verbose=False):
def __init__(self, verbose=False, docker=False):
self.verbose = verbose
self.docker = docker
def __call__(self, call, kwargs=None, log_callback=None):
if kwargs is None:
@@ -100,6 +102,10 @@ class LocalExec:
if not title:
title = call.__name__
where = "locally"
if self.docker:
if call == remote.rdns.perform_initial_checks:
kwargs["pre_command"] = "docker exec chatmail "
where = "in docker"
if self.verbose:
print_stderr(f"Running {where}: {title}(**{kwargs})")
return self(call, kwargs, log_callback=print_stderr)

View File

@@ -1,17 +1,18 @@
; Required DNS entries for chatmail servers
zftest.testrun.org. A 135.181.204.127
zftest.testrun.org. AAAA 2a01:4f9:c012:52f4::1
zftest.testrun.org. MX 10 zftest.testrun.org.
_mta-sts.zftest.testrun.org. TXT "v=STSv1; id=202403211706"
mta-sts.zftest.testrun.org. CNAME zftest.testrun.org.
www.zftest.testrun.org. CNAME zftest.testrun.org.
opendkim._domainkey.zftest.testrun.org. TXT "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoYt82CVUyz2ouaqjX2kB+5J80knAyoOU3MGU5aWppmwUwwTvj/oSTSpkc5JMtVTRmKKr8NUDWAL1Yw7dfGqqPHdHfwwjS3BIvDzYx+hzgtz62RnfNgV+/2MAoNpfX7cAFIHdRzEHNtwugc3RDLquqPoupAE3Y2YRw2T5zG5fILh4vwIcJZL5Uq6B92j8wwJqOex" "33n+vm1NKQ9rxo/UsHAmZlJzpooXcG/4igTBxJyJlamVSRR6N7Nul1v//YJb7J6v2o0iPHW6uE0StzKaPPNC2IVosSRFbD9H2oqppltptFSNPlI0E+t0JBWHem6YK7xcugiO3ImMCaaU8g6Jt/wIDAQAB;s=email;t=s"
; Required DNS entries
zftest.testrun.org. 3600 IN A 135.181.204.127
zftest.testrun.org. 3600 IN AAAA 2a01:4f9:c012:52f4::1
zftest.testrun.org. 3600 IN MX 10 zftest.testrun.org.
_mta-sts.zftest.testrun.org. 3600 IN TXT "v=STSv1; id=202403211706"
mta-sts.zftest.testrun.org. 3600 IN CNAME zftest.testrun.org.
www.zftest.testrun.org. 3600 IN CNAME zftest.testrun.org.
opendkim._domainkey.zftest.testrun.org. 3600 IN TXT "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoYt82CVUyz2ouaqjX2kB+5J80knAyoOU3MGU5aWppmwUwwTvj/oSTSpkc5JMtVTRmKKr8NUDWAL1Yw7dfGqqPHdHfwwjS3BIvDzYx+hzgtz62RnfNgV+/2MAoNpfX7cAFIHdRzEHNtwugc3RDLquqPoupAE3Y2YRw2T5zG5fILh4vwIcJZL5Uq6B92j8wwJqOex" "33n+vm1NKQ9rxo/UsHAmZlJzpooXcG/4igTBxJyJlamVSRR6N7Nul1v//YJb7J6v2o0iPHW6uE0StzKaPPNC2IVosSRFbD9H2oqppltptFSNPlI0E+t0JBWHem6YK7xcugiO3ImMCaaU8g6Jt/wIDAQAB;s=email;t=s"
; Recommended DNS entries
_submission._tcp.zftest.testrun.org. SRV 0 1 587 zftest.testrun.org.
_submissions._tcp.zftest.testrun.org. SRV 0 1 465 zftest.testrun.org.
_imap._tcp.zftest.testrun.org. SRV 0 1 143 zftest.testrun.org.
_imaps._tcp.zftest.testrun.org. SRV 0 1 993 zftest.testrun.org.
zftest.testrun.org. CAA 0 issue "letsencrypt.org;accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/1371472956"
zftest.testrun.org. TXT "v=spf1 a:zftest.testrun.org ~all"
_dmarc.zftest.testrun.org. TXT "v=DMARC1;p=reject;adkim=s;aspf=s"
_adsp._domainkey.zftest.testrun.org. TXT "dkim=discardable"
zftest.testrun.org. 3600 IN TXT "v=spf1 a ~all"
_dmarc.zftest.testrun.org. 3600 IN TXT "v=DMARC1;p=reject;adkim=s;aspf=s"
zftest.testrun.org. 3600 IN CAA 0 issue "letsencrypt.org;accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/1371472956"
_adsp._domainkey.zftest.testrun.org. 3600 IN TXT "dkim=discardable"
_submission._tcp.zftest.testrun.org. 3600 IN SRV 0 1 587 zftest.testrun.org.
_submissions._tcp.zftest.testrun.org. 3600 IN SRV 0 1 465 zftest.testrun.org.
_imap._tcp.zftest.testrun.org. 3600 IN SRV 0 1 143 zftest.testrun.org.
_imaps._tcp.zftest.testrun.org. 3600 IN SRV 0 1 993 zftest.testrun.org.

View File

@@ -8,6 +8,7 @@ import pytest
from cmdeploy import remote
from cmdeploy.cmdeploy import get_sshexec
from cmdeploy.sshexec import FuncError
class TestSSHExecutor:
@@ -117,6 +118,33 @@ def test_reject_forged_from(cmsetup, maildata, gencreds, lp, forgeaddr):
assert "500" in str(e.value)
@pytest.mark.slow
def test_bounces_are_signed(cmsetup, cmsetup2, maildata, sshdomain2):
"""Test that bounce messages are dkim signed"""
dkim_rejects_dir = "/tmp/filtermail-rejected/dkim-verify"
sshexec2 = get_sshexec(sshdomain2, ssh_options="-oStrictHostKeyChecking=accept-new")
sshexec2(call=remote.rdns.shell, kwargs=dict(command=f"rm -rf {dkim_rejects_dir}"))
our_user = cmsetup.gen_users(1)[0]
other_user = cmsetup2.gen_users(1)[0]
msg = maildata("encrypted.eml", from_addr=other_user.addr, to_addr=our_user.addr)
# exceed the 10kB quota of our_user mailbox to trigger a bounce message.
def bounce_received():
other_user.smtp.sendmail(
from_addr=other_user.addr, to_addrs=[our_user.addr], msg=msg.as_string()
)
out = sshexec2(call=remote.rdns.shell, kwargs=dict(command=f"journalctl -n 5 -u filtermail-incoming"))
assert "Filtering unencrypted mail." in out
try_n_times(20, bounce_received)
time.sleep(1)
# if bounce was dkim-signed, filtermail shouldn't log the eml.
with pytest.raises(FuncError):
sshexec2(call=remote.rdns.shell, kwargs=dict(command=f"ls {dkim_rejects_dir}"))
def test_authenticated_from(cmsetup, maildata):
"""Test that envelope FROM must be the same as login."""
user1, user2, user3 = cmsetup.gen_users(3)

View File

@@ -35,6 +35,11 @@ def pytest_runtest_setup(item):
def _get_chatmail_config():
inipath = os.environ.get("CHATMAIL_INI")
if inipath:
path = Path(inipath).resolve()
return read_config(path), path
current = Path().resolve()
while 1:
path = current.joinpath("chatmail.ini").resolve()
@@ -388,12 +393,15 @@ def cmfactory(rpc, gencreds, maildomain, chatmail_config):
@pytest.fixture
def remote(sshdomain):
return Remote(sshdomain)
r = Remote(sshdomain)
yield r
r.close()
class Remote:
def __init__(self, sshdomain):
self.sshdomain = sshdomain
self._procs = []
def iter_output(self, logcmd="", ready=None):
getjournal = "journalctl -f" if not logcmd else logcmd
@@ -403,19 +411,32 @@ class Remote:
case "localhost": command = []
case _: command = ["ssh", f"root@{self.sshdomain}"]
[command.append(arg) for arg in getjournal.split()]
self.popen = subprocess.Popen(
popen = subprocess.Popen(
command,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
)
while 1:
line = self.popen.stdout.readline()
res = line.decode().strip().lower()
if not res:
break
if ready is not None:
ready()
ready = None
yield res
self._procs.append(popen)
try:
while 1:
line = popen.stdout.readline()
res = line.decode().strip().lower()
if not res:
break
if ready is not None:
ready()
ready = None
yield res
finally:
popen.terminate()
popen.wait()
def close(self):
while self._procs:
proc = self._procs.pop()
proc.kill()
proc.wait()
@pytest.fixture
@@ -435,6 +456,11 @@ def cmsetup(maildomain, gencreds, ssl_context):
return CMSetup(maildomain, gencreds, ssl_context)
@pytest.fixture
def cmsetup2(maildomain2, gencreds, ssl_context):
return CMSetup(maildomain2, gencreds, ssl_context)
class CMSetup:
def __init__(self, maildomain, gencreds, ssl_context):
self.maildomain = maildomain
@@ -445,7 +471,7 @@ class CMSetup:
print(f"Creating {num} online users")
users = []
for i in range(num):
addr, password = self.gencreds()
addr, password = self.gencreds(self.maildomain)
user = CMUser(self.maildomain, addr, password, self.ssl_context)
assert user.smtp
users.append(user)

View File

@@ -23,15 +23,19 @@ class TestCmdline:
run = parser.parse_args(["run"])
assert init and run
def test_init_not_overwrite(self, capsys):
assert main(["init", "chat.example.org"]) == 0
def test_init_not_overwrite(self, capsys, tmp_path, monkeypatch):
monkeypatch.delenv("CHATMAIL_INI", raising=False)
inipath = tmp_path / "chatmail.ini"
args = ["init", "--config", str(inipath), "chat.example.org"]
assert main(args) == 0
capsys.readouterr()
assert main(["init", "chat.example.org"]) == 1
assert main(args) == 1
out, err = capsys.readouterr()
assert "path exists" in out.lower()
assert main(["init", "chat.example.org", "--force"]) == 0
args.insert(1, "--force")
assert main(args) == 0
out, err = capsys.readouterr()
assert "deleting config file" in out.lower()

View File

@@ -3,7 +3,7 @@ from copy import deepcopy
import pytest
from cmdeploy import remote
from cmdeploy.dns import check_full_zone, check_initial_remote_data
from cmdeploy.dns import check_full_zone, check_initial_remote_data, parse_zone_records
@pytest.fixture
@@ -125,18 +125,49 @@ class TestPerformInitialChecks:
assert not l
def test_parse_zone_records():
text = """
; This is a comment
some.domain. 3600 IN A 1.1.1.1
; Another comment
www.some.domain. 3600 IN CNAME some.domain.
; Multi-word rdata
some.domain. 3600 IN MX 10 mail.some.domain.
; DKIM record (single line, multi-word TXT rdata)
dkim._domainkey.some.domain. 3600 IN TXT "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG" "9w0BAQEFAAOCAQ8AMIIBCgKCAQEA"
; Another TXT record
_dmarc.some.domain. 3600 IN TXT "v=DMARC1;p=reject"
"""
records = list(parse_zone_records(text))
assert records == [
("some.domain", "3600", "A", "1.1.1.1"),
("www.some.domain", "3600", "CNAME", "some.domain."),
("some.domain", "3600", "MX", "10 mail.some.domain."),
(
"dkim._domainkey.some.domain",
"3600",
"TXT",
'"v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG" "9w0BAQEFAAOCAQ8AMIIBCgKCAQEA"',
),
("_dmarc.some.domain", "3600", "TXT", '"v=DMARC1;p=reject"'),
]
def test_parse_zone_records_invalid_line():
text = "invalid line"
with pytest.raises(ValueError, match="Bad zone record line"):
list(parse_zone_records(text))
def parse_zonefile_into_dict(zonefile, mockdns_base, only_required=False):
for zf_line in zonefile.split("\n"):
if zf_line.startswith("#"):
if "Recommended" in zf_line and only_required:
return
continue
if not zf_line.strip():
continue
zf_domain, zf_typ, zf_value = zf_line.split(maxsplit=2)
zf_domain = zf_domain.rstrip(".")
zf_value = zf_value.strip()
mockdns_base.setdefault(zf_typ, {})[zf_domain] = zf_value
if only_required:
zonefile = zonefile.split("; Recommended")[0]
for name, ttl, rtype, rdata in parse_zone_records(zonefile):
mockdns_base.setdefault(rtype, {})[name] = rdata
class MockSSHExec:

View File

@@ -1,264 +0,0 @@
Docker installation
===================
This section provides instructions for installing a chatmail relay
using Docker Compose.
.. note::
- Docker support is experimental, CI builds and tests the image automatically, but please report bugs.
- The image wraps the cmdeploy process detailed in the :doc:`getting_started` instructions in a Debian-systemd image with r/w access to `/sys/fs`
- Currently amd64-only (arm64 should work but is untested).
Setup Preparation
-----------------
We use ``chat.example.org`` as the chatmail domain in the following
steps. Please substitute it with your own domain.
1. Install docker and docker compose v2 (check with `docker compose version`), install, e.g., on
- Debian 12 through the `official install instructions <https://docs.docker.com/engine/install/debian/#install-using-the-repository>`_
- Debian 13+ with `apt install docker docker-compose`
2. Setup the initial DNS records.
The following is an example in the familiar BIND zone file format with
a TTL of 1 hour (3600 seconds).
Please substitute your domain and IP addresses.
::
chat.example.org. 3600 IN A 198.51.100.5
chat.example.org. 3600 IN AAAA 2001:db8::5
www.chat.example.org. 3600 IN CNAME chat.example.org.
mta-sts.chat.example.org. 3600 IN CNAME chat.example.org.
3. Configure kernel parameters on the host, as these can not be set from the container::
echo "fs.inotify.max_user_instances=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
echo "fs.inotify.max_user_watches=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
sudo sysctl --system
Docker Compose Setup
--------------------
Pre-built images are available from GitHub Container Registry. The
``main`` branch and tagged releases are pushed automatically by CI::
docker pull ghcr.io/chatmail/relay:main # latest main branch
docker pull ghcr.io/chatmail/relay:1.2.3 # tagged release
Create service directory
^^^^^^^^^^^^^^^^^^^^^^^^
Either:
- Create a service directory and download the compose files::
mkdir -p /srv/chatmail-relay && cd /srv/chatmail-relay
wget https://raw.githubusercontent.com/chatmail/relay/refs/heads/main/docker/docker-compose.yaml
wget https://raw.githubusercontent.com/chatmail/relay/refs/heads/main/docker/docker-compose.override.yaml.example -O docker-compose.override.yaml
- or clone the chatmail repo and enter the docker directory::
git clone https://github.com/chatmail/relay
cd relay/docker
Customize and start
^^^^^^^^^^^^^^^^^^^
1. Set the fully qualified domain name of the relay::
echo 'MAIL_DOMAIN=chat.example.org' > .env
The container generates a ``chatmail.ini`` with defaults from
``MAIL_DOMAIN`` on first start. To customize chatmail settings, mount
your own ``chatmail.ini`` instead (see `Custom chatmail.ini`_ below).
2. All local customizations (data paths, extra volumes, config mounts) go in
``docker-compose.override.yaml``, which Compose merges automatically with
the base file. By default, all data is stored in docker volumes, you will
likely want to at least create and configure the mail storage location, but
you might also want to configure external TLS certificates there.
3. Start the container::
docker compose up -d
docker compose logs -f chatmail # view logs, Ctrl+C to exit
4. After installation is complete, open ``https://chat.example.org`` in
your browser.
Finish install and test
-----------------------
You can test the installation with::
pip install cmping chat.example.org # or
uvx cmping chat.example.org # if you use https://docs.astral.sh/uv/
You should check and extend your DNS records for better interoperability::
# Show required DNS records
docker exec chatmail cmdeploy dns --ssh-host @local
You can check server status with::
docker exec chatmail cmdeploy status --ssh-host @local
You can run some benchmarks (can also run from any machine with cmdeploy installed)::
docker exec chatmail cmdeploy bench
You can run the test suite with::
docker exec chatmail cmdeploy test --ssh-host localhost
You can look at logs::
docker exec chatmail journalctl -fu postfix@-
Customization
-------------
Website
^^^^^^^^^^^^^^
You can customize the chatmail landing page by mounting a directory with
your own website source files.
1. Create a directory with your custom website source::
mkdir -p ./custom/www/src
nano ./custom/www/src/index.md
2. Add the volume mount in ``docker-compose.override.yaml``::
services:
chatmail:
volumes:
- ./custom/www:/opt/chatmail-www
3. Restart the service::
docker compose down
docker compose up -d
Custom chatmail.ini
^^^^^^^^^^^^^^^^^^^
If you want to go beyond simply setting the ``MAIL_DOMAIN`` in ``.env``, you
can use a regular `chatmail.ini` to give you full control.
1. Extract the generated config from a running container::
docker cp chatmail:/etc/chatmail/chatmail.ini ./chatmail.ini
2. Edit ``chatmail.ini`` as needed.
3. Add the volume mount in ``docker-compose.override.yaml`` ::
services:
chatmail:
volumes:
- ./chatmail.ini:/etc/chatmail/chatmail.ini
4. Restart the container, the container skips generating a new one: ::
docker compose down && docker compose up -d
External TLS certificates
^^^^^^^^^^^^^^^^^^^^^^^^^
If TLS certificates are managed outside the container (e.g. by certbot,
acmetool, or Traefik on the host), mount them into the container and set
``TLS_EXTERNAL_CERT_AND_KEY`` in ``docker-compose.override.yaml``.
Changed certificates are picked up automatically via inotify.
See the examples in the example override and :ref:`external-tls` in the getting started guide for details.
Migrating from a bare-metal install
------------------------------------
If you have an existing bare-metal chatmail installation and want to
switch to Docker:
1. Stop all existing services::
systemctl stop postfix dovecot doveauth nginx opendkim unbound \
acmetool-redirector filtermail filtermail-incoming chatmail-turn \
iroh-relay chatmail-metadata lastlogin mtail
systemctl disable postfix dovecot doveauth nginx opendkim unbound \
acmetool-redirector filtermail filtermail-incoming chatmail-turn \
iroh-relay chatmail-metadata lastlogin mtail
2. Copy your existing ``chatmail.ini`` and mount it into the container
(see `Custom chatmail.ini`_ above)::
cp /usr/local/lib/chatmaild/chatmail.ini ./chatmail.ini
3. Copy persistent data into the ``./data/`` subdirectories (for example, as configured in `Customize and start`_) ::
mkdir -p data/dkim data/certs data/mail
# DKIM keys
cp -a /etc/dkimkeys/* data/dkim/
# TLS certificates
rsync -a /var/lib/acme/ data/certs/
Note that ownership of dkim and acme is adjusted on container start.
For the mail directory::
rsync -a /home/vmail/ data/mail/
Alternatively, mount ``/home/vmail`` directly by changing the volume
in ``docker-compose-override.yaml``::
- /home/vmail:/home/vmail
The three ``./data/`` subdirectories cover all persistent state.
Everything else is regenerated by the ``configure`` and ``activate``
stages on container start.
Building the image
------------------
Clone the repository and build the Docker image::
git clone https://github.com/chatmail/relay
cd relay
docker/build.sh
The build bakes all binaries, Python packages, and the install stage
into the image. After building, only the ``docker/`` directory and a ``.env``
with ``MAIL_DOMAIN`` are needed to run the container. The `build.sh` passes the
git hash onto the docker build so it can be determined if there has been a
change that warrants a redeploy.
You can transfer a locally built image to your server directly (pigz is parallel `gzip` which can be used instead as well) ::
docker save chatmail-relay:latest | pigz | ssh chat.example.org 'pigz -d | docker load'
Forcing a full reinstall
------------------------
On container start, only the ``configure`` and ``activate`` stages run by default.
To force a full reinstall (e.g. after updating the source), either
rebuild the image::
docker compose build chatmail
docker compose up -d
Or override the stages at runtime without rebuilding::
CMDEPLOY_STAGES="install,configure,activate" docker compose up -d

View File

@@ -98,12 +98,6 @@ steps. Please substitute it with your own domain.
configure at your DNS provider (it can take some time until they are
public).
Docker installation
-------------------
There is experimental support for running chatmail via Docker Compose.
See :doc:`docker` for full setup instructions.
Other helpful commands
----------------------

View File

@@ -13,7 +13,6 @@ Contributions and feedback welcome through the https://github.com/chatmail/relay
:maxdepth: 5
getting_started
docker
proxy
migrate
overview

View File

@@ -1,9 +0,0 @@
#!/bin/sh
# Build the chatmail Docker image with the current git hash baked in.
# Usage: ./docker/build.sh [extra docker-compose build args...]
#
# .git/ is excluded from the build context (.dockerignore) so the hash
# must be passed as a build arg from the host.
export GIT_HASH=$(git rev-parse HEAD)
exec docker compose -f docker/docker-compose.yaml build "$@"

View File

@@ -1,14 +0,0 @@
[Unit]
Description=Run container setup commands
After=multi-user.target
ConditionPathExists=/chatmail-init.sh
[Service]
Type=oneshot
ExecStart=/bin/bash /chatmail-init.sh
RemainAfterExit=true
WorkingDirectory=/opt/chatmail
PassEnvironment=<envs_list>
[Install]
WantedBy=multi-user.target

View File

@@ -1,85 +0,0 @@
#!/bin/bash
set -euo pipefail
export CHATMAIL_INI="${CHATMAIL_INI:-/etc/chatmail/chatmail.ini}"
CMDEPLOY=/opt/cmdeploy/bin/cmdeploy
if [ -z "$MAIL_DOMAIN" ]; then
echo "ERROR: Environment variable 'MAIL_DOMAIN' must be set!" >&2
exit 1
fi
# Generate DKIM keys if not mounted
if [ ! -f /etc/dkimkeys/opendkim.private ]; then
/usr/sbin/opendkim-genkey -D /etc/dkimkeys -d "$MAIL_DOMAIN" -s opendkim
fi
# Fix ownership for bind-mounted keys (host opendkim UID may differ from container)
chown -R opendkim:opendkim /etc/dkimkeys
# Create chatmail.ini, skip if mounted
mkdir -p "$(dirname "$CHATMAIL_INI")"
if [ ! -f "$CHATMAIL_INI" ]; then
$CMDEPLOY init --config "$CHATMAIL_INI" "$MAIL_DOMAIN"
fi
# Auto-detect IPv6: if the host has no IPv6 connectivity, set disable_ipv6
# in the ini so dovecot/postfix/nginx bind to IPv4 only.
# Uses network_mode:host so /proc/net/if_inet6 reflects the host's stack.
if [ ! -e /proc/net/if_inet6 ]; then
if grep -q '^disable_ipv6 = False' "$CHATMAIL_INI"; then
sed -i 's/^disable_ipv6 = False/disable_ipv6 = True/' "$CHATMAIL_INI"
echo "[INFO] IPv6 not available, set disable_ipv6 = True"
fi
fi
# Inject external TLS paths from env var unless defined in chatmail.ini
if [ -n "${TLS_EXTERNAL_CERT_AND_KEY:-}" ]; then
if ! grep -q '^tls_external_cert_and_key' "$CHATMAIL_INI"; then
echo "tls_external_cert_and_key = $TLS_EXTERNAL_CERT_AND_KEY" >> "$CHATMAIL_INI"
fi
fi
# Ensure mailboxes directory exists (chatmail-metadata needs it at startup,
# but Dovecot only creates it on first mail delivery)
mkdir -p "/home/vmail/mail/${MAIL_DOMAIN}"
chown vmail:vmail "/home/vmail/mail/${MAIL_DOMAIN}"
# --- Deploy fingerprint: skip cmdeploy run if nothing changed ---
# On restart with identical image+config, systemd already brings up all
# enabled services only configure+activate are needed here.
IMAGE_VERSION_FILE="/etc/chatmail-image-version"
FINGERPRINT_FILE="/etc/chatmail/.deploy-fingerprint"
image_ver="none"
[ -f "$IMAGE_VERSION_FILE" ] && image_ver=$(cat "$IMAGE_VERSION_FILE")
config_hash=$(sha256sum "$CHATMAIL_INI" | cut -c1-16)
current_fp="${image_ver}:${config_hash}"
# CMDEPLOY_STAGES non-empty in env = operator override -> always run.
# Otherwise, if fingerprint matches the last successful deploy, skip.
if [ -z "${CMDEPLOY_STAGES:-}" ] \
&& [ -f "$FINGERPRINT_FILE" ] \
&& [ "$(cat "$FINGERPRINT_FILE")" = "$current_fp" ]; then
echo "[INFO] No changes detected ($current_fp), skipping deploy."
else
export CMDEPLOY_STAGES="${CMDEPLOY_STAGES:-configure,activate}"
# Skip DNS check when MAIL_DOMAIN is a bare IP address
SKIP_DNS=""
if [[ "$MAIL_DOMAIN" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]] || [[ "$MAIL_DOMAIN" =~ : ]]; then
SKIP_DNS="--skip-dns-check"
fi
$CMDEPLOY run --config "$CHATMAIL_INI" --ssh-host @local $SKIP_DNS
# Restore the build-time hash
cp /etc/chatmail-image-version /etc/chatmail-version
echo "$current_fp" > "$FINGERPRINT_FILE"
fi
# Signal success to Docker healthcheck
touch /run/chatmail-init.done
# Forward journald to console so `docker compose logs` works
grep -q '^ForwardToConsole=yes' /etc/systemd/journald.conf \
|| echo "ForwardToConsole=yes" >> /etc/systemd/journald.conf
systemctl restart systemd-journald

View File

@@ -1,108 +0,0 @@
# syntax=docker/dockerfile:1
FROM jrei/systemd-debian:12 AS base
ENV LANG=en_US.UTF-8
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt/lists,sharing=locked \
echo 'APT::Install-Recommends "0";' > /etc/apt/apt.conf.d/01norecommend && \
echo 'APT::Install-Suggests "0";' >> /etc/apt/apt.conf.d/01norecommend && \
apt-get update && \
DEBIAN_FRONTEND=noninteractive TZ=UTC \
apt-get install -y \
ca-certificates \
gcc \
git \
python3 \
python3-dev \
python3-venv \
tzdata \
locales && \
sed -i -e "s/# $LANG.*/$LANG UTF-8/" /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=$LANG
# --- Build-time: install cmdeploy venv and run install stage ---
# Editable install so importlib.resources reads directly from the source tree.
# On container start only "configure,activate" stages run.
# Copy dependency metadata first so pip install layer is cached
COPY cmdeploy/pyproject.toml /opt/chatmail/cmdeploy/pyproject.toml
COPY chatmaild/pyproject.toml /opt/chatmail/chatmaild/pyproject.toml
# Dummy scaffolding so editable install can discover packages
RUN mkdir -p /opt/chatmail/cmdeploy/src/cmdeploy \
/opt/chatmail/chatmaild/src/chatmaild && \
touch /opt/chatmail/cmdeploy/src/cmdeploy/__init__.py \
/opt/chatmail/chatmaild/src/chatmaild/__init__.py
# Dummy git repo: .git/ is excluded from the build context (.dockerignore)
# but setuptools calls `git ls-files` when building the sdist.
WORKDIR /opt/chatmail
RUN --mount=type=cache,target=/root/.cache/pip \
git init -q && \
python3 -m venv /opt/cmdeploy && \
/opt/cmdeploy/bin/pip install -e chatmaild/ -e cmdeploy/
# Full source copy (editable install's .egg-link still points here)
COPY . /opt/chatmail/
# Minimal chatmail.ini
RUN printf '[params]\nmail_domain = build.local\n' > /tmp/chatmail.ini
RUN CMDEPLOY_STAGES=install \
CHATMAIL_INI=/tmp/chatmail.ini \
/opt/cmdeploy/bin/pyinfra @local \
/opt/chatmail/cmdeploy/src/cmdeploy/run.py -y
RUN cp -a www/ /opt/chatmail-www/
# Remove build-only packages — not needed at runtime.
# Keep git: test_deployed_state needs `git rev-parse HEAD` to verify the
# deployed version hash matches /etc/chatmail-version.
RUN apt-get purge -y gcc python3-dev && \
apt-get autoremove -y && \
rm -f /tmp/chatmail.ini
# Record image version (used in deploy fingerprint at runtime).
# GIT_HASH is passed as a build arg (from docker-compose or CI) so that
# .git/ can be excluded from the build context via .dockerignore.
# Two files: chatmail-image-version is the immutable build hash (survives
# deploys); chatmail-version is overwritten by cmdeploy run and restored
# from the image version after each deploy in chatmail-init.sh.
ARG GIT_HASH=unknown
RUN echo "$GIT_HASH" > /etc/chatmail-image-version && \
echo "$GIT_HASH" > /etc/chatmail-version
# Mock git HEAD so `git rev-parse HEAD` returns the source repo's commit hash.
# The .git/ dir was created by `git init` earlier (for setuptools); we just
# write the build hash into whatever branch HEAD points to.
RUN head_ref=$(sed 's/^ref: //' /opt/chatmail/.git/HEAD) && \
mkdir -p "/opt/chatmail/.git/$(dirname "$head_ref")" && \
echo "$GIT_HASH" > "/opt/chatmail/.git/$head_ref"
# --- End build-time install ---
ENV TZ=:/etc/localtime
ENV PATH="/opt/cmdeploy/bin:${PATH}"
RUN ln -s /etc/chatmail/chatmail.ini /opt/chatmail/chatmail.ini
ARG CHATMAIL_INIT_SERVICE_PATH=/lib/systemd/system/chatmail-init.service
COPY ./docker/chatmail-init.service "$CHATMAIL_INIT_SERVICE_PATH"
RUN ln -sf "$CHATMAIL_INIT_SERVICE_PATH" "/etc/systemd/system/multi-user.target.wants/chatmail-init.service"
# Remove default nginx site config at build time (not in entrypoint)
RUN rm -f /etc/nginx/sites-enabled/default
COPY --chmod=555 ./docker/chatmail-init.sh /chatmail-init.sh
COPY --chmod=555 ./docker/entrypoint.sh /entrypoint.sh
COPY --chmod=555 ./docker/healthcheck.sh /healthcheck.sh
HEALTHCHECK --interval=10s --start-period=180s --timeout=10s --retries=3 \
CMD /healthcheck.sh
STOPSIGNAL SIGRTMIN+3
ENTRYPOINT ["/entrypoint.sh"]
CMD [ "--default-standard-output=journal+console", \
"--default-standard-error=journal+console" ]

View File

@@ -1,70 +0,0 @@
# Traefik reverse-proxy example — use as a compose override:
#
# docker compose -f docker-compose.yaml -f docker-compose-traefik.yaml up -d
#
# Traefik handles HTTP→HTTPS redirect and ACME certificate issuance.
# traefik-certs-dumper extracts the certificates to the filesystem so
# chatmail's Postfix/Dovecot/nginx can use them via TLS_EXTERNAL_CERT_AND_KEY.
#
# Prerequisites:
# mkdir -p traefik/data traefik/dynamic-configs
# touch traefik/data/acme.json && chmod 600 traefik/data/acme.json
# cp traefik/config.yaml.example traefik/config.yaml # see below
#
# Required .env variables (in addition to MAIL_DOMAIN):
# ACME_EMAIL=admin@example.org
services:
chatmail:
environment:
# Point chatmail at the certs dumped by traefik-certs-dumper.
# The container's tls-cert-reload.path watches for changes.
TLS_EXTERNAL_CERT_AND_KEY: >-
/traefik-certs/${MAIL_DOMAIN}/certificate.crt
/traefik-certs/${MAIL_DOMAIN}/privatekey.key
volumes:
- traefik-certs:/traefik-certs:ro
depends_on:
- traefik-certs-dumper
labels:
- traefik.enable=true
- traefik.http.services.chatmail.loadbalancer.server.scheme=https
- traefik.http.services.chatmail.loadbalancer.server.port=443
- traefik.http.routers.chatmail.rule=Host(`${MAIL_DOMAIN}`) || Host(`mta-sts.${MAIL_DOMAIN}`) || Host(`www.${MAIL_DOMAIN}`)
- traefik.http.routers.chatmail.tls=true
- traefik.http.routers.chatmail.tls.certresolver=letsEncrypt
traefik:
image: traefik:v3.3
container_name: traefik
restart: unless-stopped
network_mode: host
command:
- "--configFile=/config.yaml"
- "--certificatesresolvers.letsEncrypt.acme.email=${ACME_EMAIL}"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik/config.yaml:/config.yaml:ro
- ./traefik/data/acme.json:/acme.json
- ./traefik/dynamic-configs:/dynamic/conf:ro
traefik-certs-dumper:
image: ldez/traefik-certs-dumper:v2.10.0
restart: unless-stopped
depends_on:
- traefik
entrypoint: sh -c '
apk add openssl
&& while ! [ -e /data/acme.json ]
|| ! [ $$(jq ".[] | .Certificates | length" /data/acme.json | jq -s "add") != 0 ]; do
sleep 1;
done
&& traefik-certs-dumper file
--version v3 --watch --domain-subdir=true
--source /data/acme.json --dest /certs'
volumes:
- ./traefik/data/acme.json:/data/acme.json:ro
- traefik-certs:/certs
volumes:
traefik-certs:

View File

@@ -1,11 +0,0 @@
# Used by .github/workflows/docker-ci.yaml
# The GHCR image is set via CHATMAIL_IMAGE env var at deploy time.
services:
chatmail:
image: ${CHATMAIL_IMAGE:-chatmail-relay:latest}
volumes:
- /srv/chatmail/chatmail.ini:/etc/chatmail/chatmail.ini
- /srv/chatmail/dkim:/etc/dkimkeys
- /srv/chatmail/certs:/var/lib/acme
environment:
TLS_EXTERNAL_CERT_AND_KEY: /var/lib/acme/live/${MAIL_DOMAIN}/fullchain /var/lib/acme/live/${MAIL_DOMAIN}/privkey

View File

@@ -1,44 +0,0 @@
# Local overrides: copy to docker-compose.override.yaml in this directory.
# Compose automatically merges this with docker-compose.yaml.
#
# cp docker-compose.override.yaml.example docker-compose.override.yaml
#
# Volumes are APPENDED to the base file's volumes list, environment and other scalar keys are MERGED by key.
services:
chatmail:
volumes:
## Data paths — bind-mount to host directories for easy access/backup.
# - ./data/dkim:/etc/dkimkeys
# - ./data/certs:/var/lib/acme
# - ./data/mail:/home/vmail
## Or mount from an existing bare-metal install.
# - /home/vmail:/home/vmail
## Mount your own chatmail.ini (skips auto-generation):
# - ./chatmail.ini:/etc/chatmail/chatmail.ini
## Custom website:
# - ./custom/www:/opt/chatmail-www
## Debug — mount scripts for live editing:
# - ./chatmail-init.sh:/chatmail-init.sh
# - ./entrypoint.sh:/entrypoint.sh
# environment:
## Mount certs (above) and set TLS_EXTERNAL_CERT_AND_KEY to in-container paths.
## A tls-cert-reload.path watcher inside the container reloads services
## when the cert file changes. However, inotify does not cross bind-mount
## boundaries, so host-side renewals (certbot, acmetool, etc.) must
## notify the container explicitly. Add this to your renewal hook:
##
## docker exec chatmail systemctl start tls-cert-reload.service
##
## Host acmetool (bare-metal migration): create mount above, and
## rsync -a /var/lib/acme/live data/certs
# TLS_EXTERNAL_CERT_AND_KEY: "/var/lib/acme/live/${MAIL_DOMAIN}/fullchain /var/lib/acme/live/${MAIL_DOMAIN}/privkey"
##
## (Untested) Traefik certs-dumper (see docker/docker-compose-traefik.yaml) - also add volume:
## - traefik-certs:/certs:ro
# TLS_EXTERNAL_CERT_AND_KEY: "/certs/${MAIL_DOMAIN}/certificate.crt /certs/${MAIL_DOMAIN}/privatekey.key"

View File

@@ -1,48 +0,0 @@
# Base compose file — do not edit. Put customizations (data paths, extra
# volumes, env overrides) in docker-compose.override.yaml instead.
# See docker-compose.override.yaml.example in this directory for a starting point.
#
# Security notes: this container uses
# - network_mode:host chatmail needs many ports (25, 53, 80, 143, 443, 465,
# 587, 993, 3340, 8443) and needs to operate from the real IP, which bridging
# would make tricky
# - cgroup:host (required for systemd).
# Together these give the container near-host-level access. This is acceptable
# for a dedicated mail server, but be aware that the container can bind any
# port and see all host network traffic.
services:
chatmail:
build:
context: ../
dockerfile: docker/chatmail_relay.dockerfile
args:
GIT_HASH: ${GIT_HASH:-unknown}
image: chatmail-relay:latest
restart: unless-stopped
container_name: chatmail
# Required for systemd — use only one of the following:
cgroup: host # compose v2
# privileged: true # compose v1 (less restricted)
tty: true # required for logs
tmpfs: # required for systemd
- /tmp
- /run
- /run/lock
logging:
driver: none
environment:
MAIL_DOMAIN: $MAIL_DOMAIN
network_mode: "host"
volumes:
## system (required)
- /sys/fs/cgroup:/sys/fs/cgroup:rw
## data (defaults — override in docker-compose.override.yaml)
- mail:/home/vmail
- dkim:/etc/dkimkeys
- certs:/var/lib/acme
volumes:
mail:
dkim:
certs:

View File

@@ -1,9 +0,0 @@
#!/bin/bash
set -eo pipefail
CHATMAIL_INIT_SERVICE_PATH="${CHATMAIL_INIT_SERVICE_PATH:-/lib/systemd/system/chatmail-init.service}"
env_vars="MAIL_DOMAIN CMDEPLOY_STAGES CHATMAIL_INI TLS_EXTERNAL_CERT_AND_KEY PATH"
sed -i "s|<envs_list>|$env_vars|g" "$CHATMAIL_INIT_SERVICE_PATH"
exec /lib/systemd/systemd "$@"

View File

@@ -1 +0,0 @@
MAIL_DOMAIN=chat.example.com

View File

@@ -1,16 +0,0 @@
#!/bin/bash
# returns 0 when chatmail-init succeeded and all expected services are running.
set -e
test -f /run/chatmail-init.done
# Core services
services="chatmail-metadata doveauth dovecot filtermail filtermail-incoming nginx postfix unbound"
# Optional services
for svc in iroh-relay turnserver; do
systemctl is-enabled "$svc" 2>/dev/null && services="$services $svc"
done
exec systemctl is-active $services