mirror of
https://github.com/chatmail/relay.git
synced 2026-09-27 17:50:10 +00:00
Compare commits
280
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9bb94ad110 | ||
|
|
9b92d4f4bd | ||
|
|
27e1efc253 | ||
|
|
34276807cb | ||
|
|
76c86d9050 | ||
|
|
52e262f712 | ||
|
|
ccd2baf13d | ||
|
|
10a2035a3e | ||
|
|
682bdfad2b | ||
|
|
2acd76d61d | ||
|
|
577a9a5893 | ||
|
|
1bcefbc81c | ||
|
|
cf76dfd358 | ||
|
|
8d722cb067 | ||
|
|
ae13d1e446 | ||
|
|
f064b9ec2c | ||
|
|
4f92a110b5 | ||
|
|
135ea2e637 | ||
|
|
e1a6c5cfda | ||
|
|
230415914b | ||
|
|
8f4b1461d2 | ||
|
|
af5e641905 | ||
|
|
acc00badd4 | ||
|
|
5d37a73795 | ||
|
|
cfc6eb9b21 | ||
|
|
ecbaf1b255 | ||
|
|
dc383cf5f3 | ||
|
|
7df8274a96 | ||
|
|
2e98d192d9 | ||
|
|
28b93801fc | ||
|
|
0177502929 | ||
|
|
7ea891657a | ||
|
|
8cc578a5bb | ||
|
|
ea8af25840 | ||
|
|
a398029e2a | ||
|
|
5fddfdbc0c | ||
|
|
089d9e2372 | ||
|
|
0c0e84f649 | ||
|
|
bb835594cc | ||
|
|
b75717c205 | ||
|
|
b1b58a6612 | ||
|
|
d12e1d53f7 | ||
|
|
bd995918ba | ||
|
|
a1d54dca93 | ||
|
|
4d6f6af20b | ||
|
|
7d7bccd44b | ||
|
|
7c9d4422a5 | ||
|
|
9896541fbb | ||
|
|
54ecc59345 | ||
|
|
24922fbada | ||
|
|
02e68ac146 | ||
|
|
b0c452afe5 | ||
|
|
0b34522b22 | ||
|
|
49b65b676a | ||
|
|
0473f5a588 | ||
|
|
d07a51522d | ||
|
|
0c917d4d87 | ||
|
|
2e80eb651f | ||
|
|
8742ead65e | ||
|
|
24ecb41dee | ||
|
|
ababa88a2d | ||
|
|
5312341404 | ||
|
|
3ee0bfd35b | ||
|
|
a30c8b3995 | ||
|
|
2ddd5d0cc7 | ||
|
|
3c32af1c47 | ||
|
|
4131fd0823 | ||
|
|
6121d46234 | ||
|
|
99efe7a371 | ||
|
|
e29ffd1179 | ||
|
|
82a2a8e39f | ||
|
|
080cd0118d | ||
|
|
82cdf49e62 | ||
|
|
25c8987904 | ||
|
|
0755bb931f | ||
|
|
21b4c0d1a8 | ||
|
|
1518537ae4 | ||
|
|
07dd9c88c6 | ||
|
|
dc6b4f4b75 | ||
|
|
b689fd2f75 | ||
|
|
4fc1ddbeae | ||
|
|
8f6f5a9f81 | ||
|
|
7f1521d7cd | ||
|
|
8927fc5c58 | ||
|
|
7f83b82f14 | ||
|
|
26128aca27 | ||
|
|
af581d558f | ||
|
|
79e1d78e5b | ||
|
|
d4acf064a9 | ||
|
|
5b6140a30a | ||
|
|
96d53b7303 | ||
|
|
cf8e6bb6a1 | ||
|
|
85395860bc | ||
|
|
f1515a4d64 | ||
|
|
e9fdbf39d4 | ||
|
|
a4dacc2768 | ||
|
|
c8e727c03c | ||
|
|
3b4e91f492 | ||
|
|
44ddab8f38 | ||
|
|
8187796e1d | ||
|
|
199434e602 | ||
|
|
721280867a | ||
|
|
b2ab18476a | ||
|
|
de47a627fd | ||
|
|
07e807bdfe | ||
|
|
62ce8d474d | ||
|
|
842e0baa61 | ||
|
|
eea39c7bab | ||
|
|
b17900bca1 | ||
|
|
93c0c3a891 | ||
|
|
b05164a86e | ||
|
|
d64d546582 | ||
|
|
9cf0bba7ed | ||
|
|
4e274ae280 | ||
|
|
8fefc1cc94 | ||
|
|
83aa54bb82 | ||
|
|
7f6215b875 | ||
|
|
9d22b2b79a | ||
|
|
60ce20d3aa | ||
|
|
d823b28390 | ||
|
|
4fde0d5c5f | ||
|
|
5b24879573 | ||
|
|
d8681d237f | ||
|
|
223fa3c67b | ||
|
|
1192531afc | ||
|
|
3dfac56db9 | ||
|
|
23074f0898 | ||
|
|
dc97e85ebe | ||
|
|
d59d1c8f8f | ||
|
|
d0891ef3f1 | ||
|
|
9206fae6d3 | ||
|
|
5c603db830 | ||
|
|
51f5f36478 | ||
|
|
f34642e96c | ||
|
|
0dfc3fae4e | ||
|
|
e7becac980 | ||
|
|
ebaa3587a6 | ||
|
|
a5b1cada7b | ||
|
|
f2941c9807 | ||
|
|
f4d9a7e747 | ||
|
|
e99f71aadd | ||
|
|
71739d3334 | ||
|
|
fa762d0cb5 | ||
|
|
5dc49d20dd | ||
|
|
ac5042a3d5 | ||
|
|
ac3ea7fd8e | ||
|
|
62de6d824c | ||
|
|
02976942b9 | ||
|
|
1d981ccbf4 | ||
|
|
8eee767e85 | ||
|
|
44c8c1906f | ||
|
|
8ba2bc22c5 | ||
|
|
3c9fd2718d | ||
|
|
0f8a02d4d2 | ||
|
|
3ac927ed2d | ||
|
|
6f5dd21c8e | ||
|
|
2350cbabeb | ||
|
|
ba11f37eed | ||
|
|
91f5fbf193 | ||
|
|
cfd1092d21 | ||
|
|
94b582681e | ||
|
|
324a6f3da9 | ||
|
|
a8119e125f | ||
|
|
1e16b4de52 | ||
|
|
e0b24715e0 | ||
|
|
11034c9374 | ||
|
|
b5f7169337 | ||
|
|
81b4aecce9 | ||
|
|
61261170ad | ||
|
|
33f9cddb1b | ||
|
|
dc8e0a34a2 | ||
|
|
efc24fcdf3 | ||
|
|
9a9bda80b1 | ||
|
|
74f4721f2b | ||
|
|
14f829003d | ||
|
|
5da9aaeb37 | ||
|
|
d068052f0c | ||
|
|
fa5afadaaa | ||
|
|
eb862f1645 | ||
|
|
6f3039509c | ||
|
|
504fb249a9 | ||
|
|
deca1ef2a3 | ||
|
|
0323e757a9 | ||
|
|
752957dae8 | ||
|
|
3a553f2286 | ||
|
|
cb1e4ff5bb | ||
|
|
8d7870db55 | ||
|
|
5a31ff475f | ||
|
|
8c18aea18e | ||
|
|
ebf5a51964 | ||
|
|
f596d4b56d | ||
|
|
8e3c18019b | ||
|
|
9da3f5c235 | ||
|
|
6def189d16 | ||
|
|
24612e9121 | ||
|
|
a9dd9fe3e0 | ||
|
|
aa846c3478 | ||
|
|
921080125f | ||
|
|
d898f41064 | ||
|
|
e9e012234b | ||
|
|
bb40c5bb21 | ||
|
|
a229f1bc45 | ||
|
|
4ba19b0031 | ||
|
|
5eab3a5a25 | ||
|
|
30729d9be0 | ||
|
|
4b04aae83b | ||
|
|
0eed92171c | ||
|
|
a5b9a98baa | ||
|
|
ab2d807084 | ||
|
|
ce05b26c77 | ||
|
|
77ed93fb7a | ||
|
|
39d1ecaa03 | ||
|
|
a266ffd060 | ||
|
|
a47bb94143 | ||
|
|
43ae9fee5c | ||
|
|
42dc781d7d | ||
|
|
ed664cd9cd | ||
|
|
26a13fbc26 | ||
|
|
d054fbb5aa | ||
|
|
def08c52f4 | ||
|
|
32cfa9c76c | ||
|
|
c0b207c320 | ||
|
|
4ebde2825d | ||
|
|
6a7e6ce9e7 | ||
|
|
8db668c037 | ||
|
|
45fafa10a9 | ||
|
|
ee435a7ef7 | ||
|
|
8fafd4e79f | ||
|
|
129b8a20bc | ||
|
|
a1f64ebd96 | ||
|
|
fb64be97b5 | ||
|
|
b05e26819f | ||
|
|
1db586b3eb | ||
|
|
44fe2dc08f | ||
|
|
8721600d13 | ||
|
|
dfed2b4681 | ||
|
|
f5fd286663 | ||
|
|
16b00da373 | ||
|
|
75606f5eb8 | ||
|
|
d256538f81 | ||
|
|
fdf8e5e345 | ||
|
|
81a161d433 | ||
|
|
454ac6248a | ||
|
|
85915652b3 | ||
|
|
1e8c56e08a | ||
|
|
a65f082817 | ||
|
|
6c18d37772 | ||
|
|
df4ff92133 | ||
|
|
825831ee81 | ||
|
|
0aa08b7413 | ||
|
|
14dfabf2ff | ||
|
|
0a77b3339b | ||
|
|
001d8c80fc | ||
|
|
1e376f7945 | ||
|
|
1ae92e0639 | ||
|
|
56386c231b | ||
|
|
2bdfecff72 | ||
|
|
cef739e3b3 | ||
|
|
3d128d3c64 | ||
|
|
79f68342f4 | ||
|
|
54863453c2 | ||
|
|
74326a8c54 | ||
|
|
59e5dea597 | ||
|
|
d7d89d66c1 | ||
|
|
00d723bd6e | ||
|
|
c257bfca4b | ||
|
|
82c9831369 | ||
|
|
b835318ce9 | ||
|
|
b4a46d23e6 | ||
|
|
c6d9d27a84 | ||
|
|
4521f03c99 | ||
|
|
c78859aec6 | ||
|
|
98bd5944cc | ||
|
|
e8933c455f | ||
|
|
d3a483c403 | ||
|
|
e687120d96 | ||
|
|
7409bd3452 | ||
|
|
1a34172487 | ||
|
|
38246ca8ea | ||
|
|
2635ac7e6d |
@@ -0,0 +1,15 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: "cargo"
|
||||
directory: "/filtermail"
|
||||
schedule:
|
||||
interval: "monthly"
|
||||
cooldown:
|
||||
default-days: 7
|
||||
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
cooldown:
|
||||
default-days: 7
|
||||
@@ -0,0 +1,43 @@
|
||||
name: chatmaild CI
|
||||
|
||||
on:
|
||||
# Triggers when a PR is merged into main or a direct push occurs
|
||||
push:
|
||||
branches: [ "main" ]
|
||||
|
||||
# Triggers for any PR (and its subsequent commits) targeting the main branch
|
||||
pull_request:
|
||||
branches: [ "main" ]
|
||||
|
||||
permissions: {}
|
||||
|
||||
# Newest push wins: Prevents multiple runs from clashing and wasting runner efforts
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
tox:
|
||||
name: isolated chatmaild tests
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
# Checkout pull request HEAD commit instead of merge commit
|
||||
# Otherwise `test_deployed_state` will be unhappy.
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
persist-credentials: false
|
||||
- uses: dtolnay/rust-toolchain@2c7215f132e9ebf062739d9130488b56d53c060c # master
|
||||
with:
|
||||
toolchain: stable
|
||||
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
|
||||
- name: compile filtermail
|
||||
run: |
|
||||
sudo apt-get update && sudo apt-get install -y musl-tools
|
||||
rustup target add x86_64-unknown-linux-musl
|
||||
cd filtermail && CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=musl-gcc RUSTFLAGS="-Ctarget-feature=+crt-static -Clink-self-contained=yes" cargo build --release --target x86_64-unknown-linux-musl
|
||||
mv target/x86_64-unknown-linux-musl/release/filtermail /usr/local/bin/filtermail && chmod +x /usr/local/bin/filtermail
|
||||
- name: run chatmaild tests
|
||||
working-directory: chatmaild
|
||||
run: pipx run tox
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
name: No-DNS
|
||||
|
||||
on:
|
||||
# Triggers when a PR is merged into main or a direct push occurs
|
||||
push:
|
||||
branches: [ "main" ]
|
||||
|
||||
# Triggers for any PR (and its subsequent commits) targeting the main branch
|
||||
pull_request:
|
||||
branches: [ "main" ]
|
||||
|
||||
permissions: {}
|
||||
|
||||
# Newest push wins: Prevents multiple runs from clashing and wasting runner efforts
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
|
||||
jobs:
|
||||
no-dns:
|
||||
name: LXC deploy and test
|
||||
uses: chatmail/cmlxc/.github/workflows/lxc-test.yml@main
|
||||
with:
|
||||
cmlxc_version: main
|
||||
cmlxc_commands: |
|
||||
cmlxc init
|
||||
# single cmdeploy relay test
|
||||
cmlxc -v deploy-cmdeploy --source ./repo --type ipv4 cm0
|
||||
cmlxc -v test-cmdeploy cm0
|
||||
|
||||
# cross cmdeploy relay test (two ipv4 relays)
|
||||
cmlxc -v deploy-cmdeploy --source ./repo --ipv4-only --type ipv4 cm1
|
||||
cmlxc -v test-cmdeploy cm0 cm1
|
||||
|
||||
# cross cmdeploy/madmail relay tests
|
||||
cmlxc -v deploy-madmail mad0
|
||||
cmlxc -v test-cmdeploy cm0 mad0
|
||||
cmlxc -v test-mini mad0 cm0
|
||||
cmlxc -v test-mini cm0 mad0
|
||||
+44
-20
@@ -1,30 +1,31 @@
|
||||
name: CI
|
||||
name: cmdeploy CI
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
# Triggers when a PR is merged into main or a direct push occurs
|
||||
push:
|
||||
branches: [ "main" ]
|
||||
|
||||
# Triggers for any PR (and its subsequent commits) targeting the main branch
|
||||
pull_request:
|
||||
branches: [ "main" ]
|
||||
|
||||
permissions: {}
|
||||
|
||||
# Newest push wins: Prevents multiple runs from clashing and wasting runner efforts
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
|
||||
jobs:
|
||||
tox:
|
||||
name: isolated chatmaild tests
|
||||
cmdeploy:
|
||||
name: cmdeploy tests
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
# Checkout pull request HEAD commit instead of merge commit
|
||||
# Otherwise `test_deployed_state` will be unhappy.
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
- name: download filtermail
|
||||
run: curl -L https://github.com/chatmail/filtermail/releases/download/v0.6.0/filtermail-x86_64 -o /usr/local/bin/filtermail && chmod +x /usr/local/bin/filtermail
|
||||
- name: run chatmaild tests
|
||||
working-directory: chatmaild
|
||||
run: pipx run tox
|
||||
|
||||
scripts:
|
||||
name: deploy-chatmail tests
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
persist-credentials: false
|
||||
|
||||
- name: initenv
|
||||
run: scripts/initenv.sh
|
||||
@@ -38,5 +39,28 @@ jobs:
|
||||
- name: run deploy-chatmail offline tests
|
||||
run: pytest --pyargs cmdeploy
|
||||
|
||||
# all other cmdeploy commands require a staging server
|
||||
# see https://github.com/deltachat/chatmail/issues/100
|
||||
lxc-test:
|
||||
name: LXC deploy and test
|
||||
uses: chatmail/cmlxc/.github/workflows/lxc-test.yml@main
|
||||
with:
|
||||
cmlxc_version: main
|
||||
cmlxc_commands: |
|
||||
sudo apt-get update && sudo apt-get install -y musl-tools
|
||||
rustup target add x86_64-unknown-linux-musl
|
||||
cd repo/filtermail && CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=musl-gcc RUSTFLAGS="-Ctarget-feature=+crt-static -Clink-self-contained=yes" cargo build --release --target x86_64-unknown-linux-musl
|
||||
cmlxc init
|
||||
# single cmdeploy relay test
|
||||
cmlxc -v deploy-cmdeploy --source ./repo --filtermail repo/filtermail/target/x86_64-unknown-linux-musl/release/filtermail cm0
|
||||
cmlxc -v test-mini cm0
|
||||
cmlxc -v test-cmdeploy cm0
|
||||
|
||||
# cross cmdeploy relay test
|
||||
cmlxc -v deploy-cmdeploy --source ./repo --filtermail repo/filtermail/target/x86_64-unknown-linux-musl/release/filtermail --ipv4-only cm1
|
||||
cmlxc -v test-cmdeploy cm0 cm1
|
||||
|
||||
# cross cmdeploy/madmail relay tests
|
||||
cmlxc -v deploy-madmail mad0
|
||||
cmlxc -v test-cmdeploy cm0 mad0
|
||||
cmlxc -v test-mini cm0 mad0
|
||||
cmlxc -v test-mini mad0 cm0
|
||||
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# Notify the docker repo to build and test a new image after relay CI passes.
|
||||
#
|
||||
# Sends a repository_dispatch event to chatmail/docker with the relay ref
|
||||
# and short SHA, which triggers docker-ci.yaml to build, push to GHCR,
|
||||
# and run integration tests via cmlxc.
|
||||
|
||||
name: Trigger Docker build
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
tags: ['[0-9]+.[0-9]+.[0-9]+']
|
||||
workflow_dispatch:
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
dispatch:
|
||||
name: Dispatch build to chatmail/docker
|
||||
runs-on: ubuntu-latest
|
||||
if: github.repository == 'chatmail/relay'
|
||||
steps:
|
||||
- name: Compute short SHA
|
||||
id: sha
|
||||
run: echo "short=$(echo '${{ github.sha }}' | cut -c1-7)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Send repository_dispatch
|
||||
uses: peter-evans/repository-dispatch@ff45666b9427631e3450c54a1bcbee4d9ff4d7c0 # v3
|
||||
with:
|
||||
token: ${{ secrets.CHATMAIL_DOCKER_DISPATCH_TOKEN }}
|
||||
repository: chatmail/docker
|
||||
event-type: relay-updated
|
||||
client-payload: >-
|
||||
{
|
||||
"relay_ref": "${{ github.ref_name }}",
|
||||
"relay_sha": "${{ github.sha }}",
|
||||
"relay_sha_short": "${{ steps.sha.outputs.short }}"
|
||||
}
|
||||
@@ -7,6 +7,8 @@ on:
|
||||
- 'scripts/build-docs.sh'
|
||||
- '.github/workflows/docs-preview.yaml'
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
scripts:
|
||||
name: build
|
||||
@@ -16,6 +18,8 @@ jobs:
|
||||
url: https://staging.chatmail.at/doc/relay/${{ steps.prepare.outputs.prid }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: initenv
|
||||
run: scripts/initenv.sh
|
||||
@@ -34,18 +38,22 @@ jobs:
|
||||
- name: Get Pullrequest ID
|
||||
id: prepare
|
||||
run: |
|
||||
export PULLREQUEST_ID=$(echo "${{ github.ref }}" | cut -d "/" -f3)
|
||||
export PULLREQUEST_ID=$(echo "${GITHUB_REF}" | cut -d "/" -f3)
|
||||
echo "prid=$PULLREQUEST_ID" >> $GITHUB_OUTPUT
|
||||
if [ $(expr length "${{ secrets.USERNAME }}") -gt "1" ]; then echo "uploadtoserver=true" >> $GITHUB_OUTPUT; fi
|
||||
- run: |
|
||||
echo "baseurl: /${{ steps.prepare.outputs.prid }}" >> _config.yml
|
||||
echo "baseurl: /${STEPS_PREPARE_OUTPUTS_PRID}" >> _config.yml
|
||||
env:
|
||||
STEPS_PREPARE_OUTPUTS_PRID: ${{ steps.prepare.outputs.prid }}
|
||||
|
||||
- name: Upload preview
|
||||
run: |
|
||||
mkdir -p "$HOME/.ssh"
|
||||
echo "${{ secrets.CHATMAIL_STAGING_SSHKEY }}" > "$HOME/.ssh/key"
|
||||
chmod 600 "$HOME/.ssh/key"
|
||||
rsync -rILvh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/doc/build/ "${{ secrets.USERNAME }}@chatmail.at:/var/www/html/staging.chatmail.at/doc/relay/${{ steps.prepare.outputs.prid }}/"
|
||||
rsync -rILvh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/doc/build/ "${{ secrets.USERNAME }}@chatmail.at:${STEPS_PREPARE_OUTPUTS_PRID}/"
|
||||
env:
|
||||
STEPS_PREPARE_OUTPUTS_PRID: ${{ steps.prepare.outputs.prid }}
|
||||
|
||||
- name: check links
|
||||
working-directory: doc
|
||||
|
||||
@@ -10,6 +10,8 @@ on:
|
||||
- 'scripts/build-docs.sh'
|
||||
- '.github/workflows/docs.yaml'
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
scripts:
|
||||
name: build
|
||||
@@ -19,6 +21,8 @@ jobs:
|
||||
url: https://chatmail.at/doc/relay/
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: initenv
|
||||
run: scripts/initenv.sh
|
||||
@@ -43,5 +47,5 @@ jobs:
|
||||
mkdir -p "$HOME/.ssh"
|
||||
echo "${{ secrets.CHATMAIL_STAGING_SSHKEY }}" > "$HOME/.ssh/key"
|
||||
chmod 600 "$HOME/.ssh/key"
|
||||
rsync -rILvh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/doc/build/ "${{ secrets.USERNAME }}@chatmail.at:/var/www/html/chatmail.at/doc/relay/"
|
||||
rsync -rILvh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/doc/build/ "${{ secrets.USERNAME }}@chatmail.at:"
|
||||
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
name: filtermail CI
|
||||
|
||||
permissions: {}
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "filtermail/**"
|
||||
- ".github/workflows/filtermail-ci.yml"
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "filtermail/**"
|
||||
- ".github/workflows/filtermail-ci.yml"
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
fmt:
|
||||
name: Check Formatting
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: dtolnay/rust-toolchain@2c7215f132e9ebf062739d9130488b56d53c060c # master
|
||||
with:
|
||||
toolchain: nightly
|
||||
components: rustfmt
|
||||
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
|
||||
- uses: taiki-e/install-action@2ca9b94c269419b7b0c711c09d0b21c4e1d51145 # v2.83.1
|
||||
with:
|
||||
tool: taplo-cli
|
||||
- run: cd filtermail && taplo format --check && cargo fmt --check
|
||||
|
||||
cargo-machete:
|
||||
name: Check Unused Dependencies
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: bnjbvr/cargo-machete@ac30a525c0a8d163a92d727b3ff079ee3f6ecb08 # v0.9.2
|
||||
|
||||
lint-clippy:
|
||||
name: Check Clippy
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
toolchain: ["stable", "beta"]
|
||||
continue-on-error: ${{ matrix.toolchain == 'beta' }}
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: dtolnay/rust-toolchain@2c7215f132e9ebf062739d9130488b56d53c060c # master
|
||||
with:
|
||||
toolchain: ${{ matrix.toolchain }}
|
||||
components: clippy
|
||||
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
|
||||
- run: cd filtermail && cargo clippy
|
||||
env:
|
||||
RUSTUP_TOOLCHAIN: ${{ matrix.toolchain }}
|
||||
|
||||
rust-test:
|
||||
name: Run rust tests
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: dtolnay/rust-toolchain@2c7215f132e9ebf062739d9130488b56d53c060c # master
|
||||
with:
|
||||
toolchain: stable
|
||||
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
|
||||
- run: cd filtermail && cargo test
|
||||
@@ -0,0 +1,75 @@
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Build binaries.
|
||||
if: startsWith(github.event.release.tag_name, 'filtermail-')
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: dtolnay/rust-toolchain@2c7215f132e9ebf062739d9130488b56d53c060c # master
|
||||
with:
|
||||
toolchain: stable
|
||||
targets: aarch64-unknown-linux-musl,x86_64-unknown-linux-musl
|
||||
- uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2.2.1
|
||||
with:
|
||||
use-cache: false
|
||||
- uses: taiki-e/install-action@2ca9b94c269419b7b0c711c09d0b21c4e1d51145 # v2.83.1
|
||||
with:
|
||||
tool: cargo-zigbuild
|
||||
- name: Build
|
||||
run: |
|
||||
cd filtermail
|
||||
RUSTFLAGS="-Ctarget-feature=+crt-static -Clink-self-contained=yes" \
|
||||
cargo zigbuild \
|
||||
--release \
|
||||
--target x86_64-unknown-linux-musl \
|
||||
--target aarch64-unknown-linux-musl
|
||||
- name: Upload x86_64 binary
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: filtermail-x86_64
|
||||
path: filtermail/target/x86_64-unknown-linux-musl/release/filtermail
|
||||
if-no-files-found: error
|
||||
- name: Upload aarch64 binary
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: filtermail-aarch64
|
||||
path: filtermail/target/aarch64-unknown-linux-musl/release/filtermail
|
||||
if-no-files-found: error
|
||||
|
||||
publish:
|
||||
name: Upload binaries to the release
|
||||
needs: ["build"]
|
||||
permissions:
|
||||
contents: write
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Download x86_64 binary
|
||||
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
||||
with:
|
||||
name: filtermail-x86_64
|
||||
path: filtermail-x86_64
|
||||
- name: Download aarch64 binary
|
||||
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
||||
with:
|
||||
name: filtermail-aarch64
|
||||
path: filtermail-aarch64
|
||||
- name: Upload binaries to the GitHub release
|
||||
if: github.event_name == 'release'
|
||||
env:
|
||||
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
|
||||
REF_NAME: ${{ github.ref_name }}
|
||||
run: |
|
||||
mkdir dist
|
||||
mv filtermail-x86_64/filtermail dist/filtermail-x86_64
|
||||
mv filtermail-aarch64/filtermail dist/filtermail-aarch64
|
||||
gh release upload "$REF_NAME" \
|
||||
--repo ${{ github.repository }} \
|
||||
dist/*
|
||||
@@ -1,104 +0,0 @@
|
||||
name: deploy on staging-ipv4.testrun.org, and run tests
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
paths-ignore:
|
||||
- 'scripts/**'
|
||||
- '**/README.md'
|
||||
- 'CHANGELOG.md'
|
||||
- 'LICENSE'
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
name: deploy on staging-ipv4.testrun.org, and run tests
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
environment:
|
||||
name: staging-ipv4.testrun.org
|
||||
url: https://staging-ipv4.testrun.org/
|
||||
concurrency: staging-ipv4.testrun.org
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: prepare SSH
|
||||
run: |
|
||||
mkdir ~/.ssh
|
||||
echo "${{ secrets.STAGING_SSH_KEY }}" >> ~/.ssh/id_ed25519
|
||||
chmod 600 ~/.ssh/id_ed25519
|
||||
ssh-keyscan staging-ipv4.testrun.org > ~/.ssh/known_hosts
|
||||
# save previous acme & dkim state
|
||||
rsync -avz root@staging-ipv4.testrun.org:/var/lib/acme acme-ipv4 || true
|
||||
rsync -avz root@staging-ipv4.testrun.org:/etc/dkimkeys dkimkeys-ipv4 || true
|
||||
# store previous acme & dkim state on ns.testrun.org, if it contains useful certs
|
||||
if [ -f dkimkeys-ipv4/dkimkeys/opendkim.private ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" dkimkeys-ipv4 root@ns.testrun.org:/tmp/ || true; fi
|
||||
if [ "$(ls -A acme-ipv4/acme/certs)" ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" acme-ipv4 root@ns.testrun.org:/tmp/ || true; fi
|
||||
# make sure CAA record isn't set
|
||||
scp -o StrictHostKeyChecking=accept-new .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org nsd-checkzone staging-ipv4.testrun.org /etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org systemctl reload nsd
|
||||
|
||||
- name: rebuild staging-ipv4.testrun.org to have a clean VPS
|
||||
run: |
|
||||
curl -X POST \
|
||||
-H "Authorization: Bearer ${{ secrets.HETZNER_API_TOKEN }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"image":"debian-12"}' \
|
||||
"https://api.hetzner.cloud/v1/servers/${{ secrets.STAGING_IPV4_SERVER_ID }}/actions/rebuild"
|
||||
|
||||
- run: scripts/initenv.sh
|
||||
|
||||
- name: append venv/bin to PATH
|
||||
run: echo venv/bin >>$GITHUB_PATH
|
||||
|
||||
- name: upload TLS cert after rebuilding
|
||||
run: |
|
||||
echo " --- wait until staging-ipv4.testrun.org VPS is rebuilt --- "
|
||||
rm ~/.ssh/known_hosts
|
||||
while ! ssh -o ConnectTimeout=180 -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org id -u ; do sleep 1 ; done
|
||||
ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org id -u
|
||||
# download acme & dkim state from ns.testrun.org
|
||||
rsync -e "ssh -o StrictHostKeyChecking=accept-new" -avz root@ns.testrun.org:/tmp/acme-ipv4/acme acme-restore || true
|
||||
rsync -avz root@ns.testrun.org:/tmp/dkimkeys-ipv4/dkimkeys dkimkeys-restore || true
|
||||
# restore acme & dkim state to staging2.testrun.org
|
||||
rsync -avz acme-restore/acme root@staging-ipv4.testrun.org:/var/lib/ || true
|
||||
rsync -avz dkimkeys-restore/dkimkeys root@staging-ipv4.testrun.org:/etc/ || true
|
||||
ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org chown root:root -R /var/lib/acme || true
|
||||
|
||||
- name: run deploy-chatmail offline tests
|
||||
run: pytest --pyargs cmdeploy
|
||||
|
||||
- name: setup dependencies
|
||||
run: |
|
||||
ssh root@staging-ipv4.testrun.org apt update
|
||||
ssh root@staging-ipv4.testrun.org apt install -y git python3.11-venv python3-dev gcc
|
||||
ssh root@staging-ipv4.testrun.org git clone https://github.com/chatmail/relay
|
||||
ssh root@staging-ipv4.testrun.org "cd relay && git checkout " ${{ github.head_ref }}
|
||||
ssh root@staging-ipv4.testrun.org "cd relay && scripts/initenv.sh"
|
||||
|
||||
- name: initialize config
|
||||
run: |
|
||||
ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy init staging-ipv4.testrun.org"
|
||||
ssh root@staging-ipv4.testrun.org "sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' relay/chatmail.ini"
|
||||
ssh root@staging-ipv4.testrun.org "sed -i 's/#\s*mtail_address/mtail_address/' relay/chatmail.ini"
|
||||
|
||||
- run: ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy run --verbose --skip-dns-check --ssh-host localhost"
|
||||
|
||||
- name: set DNS entries
|
||||
run: |
|
||||
ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy dns --zonefile staging-generated.zone --ssh-host localhost"
|
||||
ssh root@staging-ipv4.testrun.org cat relay/staging-generated.zone >> .github/workflows/staging-ipv4.testrun.org-default.zone
|
||||
cat .github/workflows/staging-ipv4.testrun.org-default.zone
|
||||
scp .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org nsd-checkzone staging-ipv4.testrun.org /etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org systemctl reload nsd
|
||||
|
||||
- name: cmdeploy test
|
||||
run: ssh root@staging-ipv4.testrun.org "cd relay && CHATMAIL_DOMAIN2=ci-chatmail.testrun.org scripts/cmdeploy test --slow --ssh-host localhost"
|
||||
|
||||
- name: cmdeploy dns
|
||||
run: ssh root@staging-ipv4.testrun.org "cd relay && scripts/cmdeploy dns -v --ssh-host localhost"
|
||||
|
||||
@@ -1,97 +0,0 @@
|
||||
name: deploy on staging2.testrun.org, and run tests
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
paths-ignore:
|
||||
- 'scripts/**'
|
||||
- '**/README.md'
|
||||
- 'CHANGELOG.md'
|
||||
- 'LICENSE'
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
name: deploy on staging2.testrun.org, and run tests
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
environment:
|
||||
name: staging2.testrun.org
|
||||
url: https://staging2.testrun.org/
|
||||
concurrency: staging2.testrun.org
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: prepare SSH
|
||||
run: |
|
||||
mkdir ~/.ssh
|
||||
echo "${{ secrets.STAGING_SSH_KEY }}" >> ~/.ssh/id_ed25519
|
||||
chmod 600 ~/.ssh/id_ed25519
|
||||
ssh-keyscan staging2.testrun.org > ~/.ssh/known_hosts
|
||||
# save previous acme & dkim state
|
||||
rsync -avz root@staging2.testrun.org:/var/lib/acme . || true
|
||||
rsync -avz root@staging2.testrun.org:/etc/dkimkeys . || true
|
||||
# store previous acme & dkim state on ns.testrun.org, if it contains useful certs
|
||||
if [ -f dkimkeys/opendkim.private ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" dkimkeys root@ns.testrun.org:/tmp/ || true; fi
|
||||
if [ "$(ls -A acme/certs)" ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" acme root@ns.testrun.org:/tmp/ || true; fi
|
||||
# make sure CAA record isn't set
|
||||
scp -o StrictHostKeyChecking=accept-new .github/workflows/staging.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging2.testrun.org.zone
|
||||
ssh root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging2.testrun.org.zone
|
||||
ssh root@ns.testrun.org nsd-checkzone staging2.testrun.org /etc/nsd/staging2.testrun.org.zone
|
||||
ssh root@ns.testrun.org systemctl reload nsd
|
||||
|
||||
- name: rebuild staging2.testrun.org to have a clean VPS
|
||||
run: |
|
||||
curl -X POST \
|
||||
-H "Authorization: Bearer ${{ secrets.HETZNER_API_TOKEN }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"image":"debian-12"}' \
|
||||
"https://api.hetzner.cloud/v1/servers/${{ secrets.STAGING_SERVER_ID }}/actions/rebuild"
|
||||
|
||||
- run: scripts/initenv.sh
|
||||
|
||||
- name: append venv/bin to PATH
|
||||
run: echo venv/bin >>$GITHUB_PATH
|
||||
|
||||
- name: upload TLS cert after rebuilding
|
||||
run: |
|
||||
echo " --- wait until staging2.testrun.org VPS is rebuilt --- "
|
||||
rm ~/.ssh/known_hosts
|
||||
while ! ssh -o ConnectTimeout=180 -o StrictHostKeyChecking=accept-new -v root@staging2.testrun.org id -u ; do sleep 1 ; done
|
||||
ssh -o StrictHostKeyChecking=accept-new -v root@staging2.testrun.org id -u
|
||||
# download acme & dkim state from ns.testrun.org
|
||||
rsync -e "ssh -o StrictHostKeyChecking=accept-new" -avz root@ns.testrun.org:/tmp/acme acme-restore || true
|
||||
rsync -avz root@ns.testrun.org:/tmp/dkimkeys dkimkeys-restore || true
|
||||
# restore acme & dkim state to staging2.testrun.org
|
||||
rsync -avz acme-restore/acme root@staging2.testrun.org:/var/lib/ || true
|
||||
rsync -avz dkimkeys-restore/dkimkeys root@staging2.testrun.org:/etc/ || true
|
||||
ssh -o StrictHostKeyChecking=accept-new -v root@staging2.testrun.org chown root:root -R /var/lib/acme || true
|
||||
|
||||
- name: add hpk42 key to staging server
|
||||
run: ssh root@staging2.testrun.org 'curl -s https://github.com/hpk42.keys >> .ssh/authorized_keys'
|
||||
|
||||
- name: run deploy-chatmail offline tests
|
||||
run: pytest --pyargs cmdeploy
|
||||
|
||||
- run: |
|
||||
cmdeploy init staging2.testrun.org
|
||||
sed -i 's/#\s*mtail_address/mtail_address/' chatmail.ini
|
||||
|
||||
- run: cmdeploy run --verbose --skip-dns-check
|
||||
|
||||
- name: set DNS entries
|
||||
run: |
|
||||
cmdeploy dns --zonefile staging-generated.zone --verbose
|
||||
cat staging-generated.zone >> .github/workflows/staging.testrun.org-default.zone
|
||||
cat .github/workflows/staging.testrun.org-default.zone
|
||||
scp .github/workflows/staging.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging2.testrun.org.zone
|
||||
ssh root@ns.testrun.org nsd-checkzone staging2.testrun.org /etc/nsd/staging2.testrun.org.zone
|
||||
ssh root@ns.testrun.org systemctl reload nsd
|
||||
|
||||
- name: cmdeploy test
|
||||
run: CHATMAIL_DOMAIN2=ci-chatmail.testrun.org cmdeploy test --slow
|
||||
|
||||
- name: cmdeploy dns
|
||||
run: cmdeploy dns -v
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
name: GitHub Actions Security Analysis with zizmor
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["main"]
|
||||
pull_request:
|
||||
branches: ["**"]
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
zizmor:
|
||||
name: Run zizmor
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
security-events: write # Required for upload-sarif (used by zizmor-action) to upload SARIF files.
|
||||
contents: read
|
||||
actions: read
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Run zizmor
|
||||
uses: zizmorcore/zizmor-action@b1d7e1fb5de872772f31590499237e7cce841e8e # v0.5.3
|
||||
@@ -0,0 +1,7 @@
|
||||
rules:
|
||||
unpinned-uses:
|
||||
config:
|
||||
policies:
|
||||
actions/*: ref-pin
|
||||
dependabot/*: ref-pin
|
||||
chatmail/*: ref-pin
|
||||
+1
-1
@@ -5,7 +5,6 @@ __pycache__/
|
||||
*.swp
|
||||
*qr-*.png
|
||||
chatmail*.ini
|
||||
lxconfigs/
|
||||
|
||||
|
||||
# C extensions
|
||||
@@ -79,6 +78,7 @@ docs/_build/
|
||||
# PyBuilder
|
||||
.pybuilder/
|
||||
target/
|
||||
filtermail/target/
|
||||
|
||||
# Jupyter Notebook
|
||||
.ipynb_checkpoints
|
||||
|
||||
+182
-1
@@ -1,4 +1,185 @@
|
||||
# Changelog for chatmail deployment
|
||||
# Changelog for chatmail deployment
|
||||
|
||||
## [1.12.0] - 2026-07-31
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- [**breaking**] Introduce configurable system limits to reject new address creation and limit imap/smtp connections.
|
||||
Dovecot default connection limit lowered from 50k to 10k,
|
||||
Postfix default connection limit lowered from 5k to 1k,
|
||||
larger relays need to adjust their settings.
|
||||
|
||||
### Features
|
||||
|
||||
- Reduce maximal_queue_lifetime from 5d to 2d
|
||||
- Disable negative cache in unbound (#992)
|
||||
- *(mtail)* Add incoming_mailer_daemon_mail_count
|
||||
- *(postfix)* Disable processing of MIME headers
|
||||
- *(dovecot)* Advertise privacy_mail as admin contact, drop server comment
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Set relay restrictions per smtpd service with default reject
|
||||
- Reduce maxproc for filtermail-transport LMTP client to 500
|
||||
- Core 2.50.0 does not have delete_server_after config anymore.
|
||||
- Check if all required ports are available for filtermail (#983)
|
||||
- Always deploy unbound.conf.d/chatmail.conf (#993)
|
||||
- Expire empty directories (#994)
|
||||
- Crypt-r dependency was declared for wrong Python version
|
||||
- Always overwrite /etc/resolv.conf, even if it is a symbolic link
|
||||
- Pass kwargs to files.put()
|
||||
- List Iroh proxy endpoints used by 0.35 and 1.0, drop stale /relay/probe from earlier versions
|
||||
- Fix port discovery when ss -tulpn shows dovecot before stats
|
||||
|
||||
### Documentation
|
||||
|
||||
- Add scripts/initenv.sh to upgrade instructions
|
||||
- Update overview diagrams (#995)
|
||||
- *(overview)* Remove mermaid styles from 'Accepting and delivering mail' (#1009)
|
||||
- *(README.md)* Clarify security enforcement (#1011)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- *(ci)* Auto-trigger docker build on release tag push
|
||||
- *(acmetool)* Update let's encrypt ToS link to 1.8
|
||||
- *(ci)* Update doc staging upload path
|
||||
- *(ci)* Fix docs upload path
|
||||
|
||||
### Refactor
|
||||
|
||||
- *(postfix)* Remove unused "filter" lmtp service
|
||||
- Install dns-root-data instead of using unbound-anchor
|
||||
- *(deps)* Remove domain-validator dependency
|
||||
|
||||
### Testing
|
||||
|
||||
- Set socket security for IMAP and SMTP to "TLS" in "dclogin"
|
||||
|
||||
## [1.11.0] - 2026-05-15
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- [**breaking**] Drop passthrough_sender and passthrough_recipients chatmail.ini options to eliminate one more source of unencrypted messages
|
||||
|
||||
### Features
|
||||
|
||||
- Use filtermail for delivery to remote MTAs
|
||||
- Expose metadata "maxsmtprecipients" value
|
||||
- Support setup without domain, with only an IPv4 address (#963)
|
||||
- *(doc/docker)* Introduce docker images in documentation
|
||||
- DKIM-sign bounce messages (mainly "user does not exist")
|
||||
- *(config)* Load default values from Config(), not chatmail.ini.f (#853)
|
||||
- Make turn_socket_path configurable, and cleanup tests and turnserver code.
|
||||
- Warn about any unused chatmail.ini parameter at the end of "cmdeploy run"
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Make www tests work with editable instead of just plain installs
|
||||
- Use path with no leading slash for mxdeliv
|
||||
- Increase filtermail-transport concurrency limit
|
||||
- Fix #972 by increasing file descriptors for filtermail
|
||||
- *(mtail)* Correct boot ordering and deploy restart logic
|
||||
- *(cmdeploy)* Stop and disable unbound-resolvconf
|
||||
- *(nginx)* Properly redirect www to mail_domain
|
||||
- *(dns)* Query correct NS if MNAME server is hidden (#954)
|
||||
- Legacy token metadata storage used list type, but if no new setmetadata happened, the user would not be notified at all.
|
||||
- *(logging)* Log all http requests to syslog
|
||||
|
||||
### Documentation
|
||||
|
||||
- Document how to upgrade to new version (#965)
|
||||
|
||||
### Other
|
||||
|
||||
- *(deps)* Upgrade to filtermail v0.6.4
|
||||
|
||||
### Refactor
|
||||
|
||||
- Introduce automated change-tracking across deployers
|
||||
|
||||
## 1.10.0 2026-04-30
|
||||
|
||||
* start mtail after networking is fully up <https://github.com/chatmail/relay/pull/942>
|
||||
* support specifying custom filtermail binary through environment variable <https://github.com/chatmail/relay/pull/941>
|
||||
* add automated zizmor scanning of github workflows <https://github.com/chatmail/relay/pull/938>
|
||||
* added dispatch for *automated builds of chatmail relay docker images* <https://github.com/chatmail/relay/pull/934>
|
||||
* do not bind SMTP client sockets to public addresses <https://github.com/chatmail/relay/pull/932>
|
||||
* underline in docs that scripts/initenv.sh should be used for building the docs <https://github.com/chatmail/relay/pull/933>
|
||||
* automatic oldest-first message removal from mailboxes to always stay under max_mailbox_size <https://github.com/chatmail/relay/pull/929>
|
||||
* remove --slow from cmdeploy test <https://github.com/chatmail/relay/pull/931>
|
||||
* handle missing inotify sysctl keys in containers <https://github.com/chatmail/relay/pull/930>
|
||||
* replace resolvconf with static resolv.conf <https://github.com/chatmail/relay/pull/928>
|
||||
* disable fsync for LMTP and IMAP services <https://github.com/chatmail/relay/pull/925>
|
||||
* re-use cmlxc workflow, replacing CI with hetzner staging servers with local lxc containers <https://github.com/chatmail/relay/pull/917>
|
||||
* explicitly install resolvconf <https://github.com/chatmail/relay/pull/924>
|
||||
* detect stale dovecot binary and force restart in activate() <https://github.com/chatmail/relay/pull/922>
|
||||
* Rename filtermail_http_port to filtermail_http_port_incoming <https://github.com/chatmail/relay/pull/921>
|
||||
* consolidated is_in_container() check https://github.com/chatmail/relay/pull/920>
|
||||
* restart dovecot after package replacement (rebase, test condense) <https://github.com/chatmail/relay/pull/913>
|
||||
* Set permissions on dovecot pin prefs <https://github.com/chatmail/relay/pull/915>
|
||||
* Route `/mxdeliv/` to configurable port <https://github.com/chatmail/relay/pull/901>
|
||||
* fix VM detection, automated testing fixes, use newer chatmail-turn and move to standard BIND DNS zone format <https://github.com/chatmail/relay/pull/912>
|
||||
* Upgrade to filtermail 0.6.1 <https://github.com/chatmail/relay/pull/910>
|
||||
* pin dovecot packages to prevent apt upgrades <https://github.com/chatmail/relay/pull/908>
|
||||
* add rpc server to cmdeploy along with client <https://github.com/chatmail/relay/pull/906>
|
||||
* remove unused deps from chatmaild <https://github.com/chatmail/relay/pull/905>
|
||||
* set default smtp_tls_security_level to "verify" unconditionally <https://github.com/chatmail/relay/pull/902>
|
||||
* featprefer IPv4 in SMTP client <https://github.com/chatmail/relay/pull/900>
|
||||
* Install dovecot .deb packages atomically <https://github.com/chatmail/relay/pull/899>
|
||||
* stop installing cron package <https://github.com/chatmail/relay/pull/898>
|
||||
* Rewrite dovecot install logic, update <https://github.com/chatmail/relay/pull/862>
|
||||
* fix a test and some linting fixes <https://github.com/chatmail/relay/pull/897>
|
||||
* Disable IP verification on domain-literal addresses <https://github.com/chatmail/relay/pull/895>
|
||||
* disable installing recommended packages globally on the relay <https://github.com/chatmail/relay/pull/887>
|
||||
* multiple bug fixes across chatmaild and cmdeploy <https://github.com/chatmail/relay/pull/883>
|
||||
* remove /metrics from the website <https://github.com/chatmail/relay/pull/703>
|
||||
* add Prometheus textfile output to fsreport <https://github.com/chatmail/relay/pull/881>
|
||||
* chown opendkim: private key <https://github.com/chatmail/relay/pull/879>
|
||||
* make sure chatmail-metadata was started <https://github.com/chatmail/relay/pull/882>
|
||||
* dovecot update url <https://github.com/chatmail/relay/pull/880>
|
||||
* upgrade to filtermail v0.5.2 <https://github.com/chatmail/relay/pull/876>
|
||||
* download dovecot packages from github release <https://github.com/chatmail/relay/pull/875>
|
||||
* replace DKIM verification with filtermail v0.5 <https://github.com/chatmail/relay/pull/831>
|
||||
* remove CFFI deltachat bindings usage, and consolidate test support with rpc-bindings <https://github.com/chatmail/relay/pull/872>
|
||||
* prepare chatmaild/cmdeploy changes for Docker support <https://github.com/chatmail/relay/pull/857>
|
||||
* stabilize online benchmark timing adding rate-limit-aware cooldown between iterations <https://github.com/chatmail/relay/pull/867>
|
||||
* move rate-limit cooldown to benchmark fixture <https://github.com/chatmail/relay/pull/868>
|
||||
* reconfigure acmetool from redirector to proxy mode <https://github.com/chatmail/relay/pull/861>
|
||||
* make tests work with `--ssh-host localhost` <https://github.com/chatmail/relay/pull/856>
|
||||
* mark f-string with f prefix in test_expunged <https://github.com/chatmail/relay/pull/863>
|
||||
* install also if dovecot.service=False in SystemdEnabled Fact <https://github.com/chatmail/relay/pull/841>
|
||||
* Introduce support for self-signed chatmail relays <https://github.com/chatmail/relay/pull/855>
|
||||
* Strip Received headers before delivery <https://github.com/chatmail/relay/pull/849>
|
||||
* upgrade to filtermail v0.3 <https://github.com/chatmail/relay/pull/850>
|
||||
* fix link to Maddy and update madmail URL <https://github.com/chatmail/relay/pull/847>
|
||||
* accept self-signed certificates for IP-only relays <https://github.com/chatmail/relay/pull/846>
|
||||
* enforce sending from public IP addresses <https://github.com/chatmail/relay/pull/845>
|
||||
* port check: check addresses, fix single services <https://github.com/chatmail/relay/pull/844>
|
||||
* remediates issue with improper concat on resolver injection <https://github.com/chatmail/relay/pull/834>
|
||||
* ipv6 boolean not being respected during operations <https://github.com/chatmail/relay/pull/832>
|
||||
* upgrade to filtermail v0.2 by <https://github.com/chatmail/relay/pull/825>
|
||||
* fix link to filtermail <https://github.com/chatmail/relay/pull/824>
|
||||
* print timestamps when sending messages <https://github.com/chatmail/relay/pull/823>
|
||||
* fix flaky test_exceed_rate_limit <https://github.com/chatmail/relay/pull/822>
|
||||
* Replace filtermail with rust reimplementation <https://github.com/chatmail/relay/pull/808>
|
||||
* Set default internal SMTP ports in Config <https://github.com/chatmail/relay/pull/819>
|
||||
* separate metrics for incoming and outgoing messages <https://github.com/chatmail/relay/pull/820>
|
||||
* disable appending the Received header <https://github.com/chatmail/relay/pull/815>
|
||||
* fail on errors in postfix/dovecot config <https://github.com/chatmail/relay/pull/813>
|
||||
* tweak idle/hibernate metrics some more <https://github.com/chatmail/relay/pull/811>
|
||||
* add config flag to export statistics <https://github.com/chatmail/relay/pull/806>
|
||||
* add --website-only option to run subcommand <https://github.com/chatmail/relay/pull/768>
|
||||
* Strip DKIM-Signature header before LMTP <https://github.com/chatmail/relay/pull/803>
|
||||
* properly make sure that postfix gets restarted on failure <https://github.com/chatmail/relay/pull/802>
|
||||
* expire.py: use absolute path to maildirsize <https://github.com/chatmail/relay/pull/807>
|
||||
* pin Dovecot documentation URLs to version 2.3 <https://github.com/chatmail/relay/pull/800>
|
||||
* try to use "build machine" and "deployment server" consistently <https://github.com/chatmail/relay/pull/797>
|
||||
* adds instructions for migrating control machines <https://github.com/chatmail/relay/pull/795>
|
||||
* use consistent naming schema in getting started <https://github.com/chatmail/relay/pull/793>
|
||||
* remove jsok/serialize-workflow-action dependency <https://github.com/chatmail/relay/pull/790>
|
||||
* streamline migration guide wording, provide titled steps <https://github.com/chatmail/relay/pull/789>
|
||||
* increases default max mailbox size <https://github.com/chatmail/relay/pull/792>
|
||||
* use daemon_name for OpenDKIM sign-verify decision instead of IP <https://github.com/chatmail/relay/pull/784>
|
||||
|
||||
## 1.9.0 2025-12-18
|
||||
|
||||
|
||||
@@ -5,3 +5,6 @@ We use [git-cliff] to generate the changelog from commit messages before the rel
|
||||
|
||||
[Conventional Commits]: https://www.conventionalcommits.org/
|
||||
[git-cliff]: https://git-cliff.org/
|
||||
|
||||
To update client app version information,
|
||||
edit [chatmaild/src/chatmaild/defaults/appversions.json](chatmaild/src/chatmaild/defaults/appversions.json).
|
||||
|
||||
@@ -1,19 +1,24 @@
|
||||
|
||||
# Chatmail relays for end-to-end encrypted email
|
||||
|
||||
Chatmail relay servers are interoperable Mail Transport Agents (MTAs) designed for:
|
||||
Chatmail relay servers are interoperable Mail Transport Agents (MTAs) designed for:
|
||||
|
||||
- **Zero State:** no private data or metadata collected, messages are auto-deleted, low disk usage
|
||||
|
||||
- **Instant/Realtime:** sub-second message delivery, realtime P2P
|
||||
streaming, privacy-preserving Push Notifications for Apple, Google, and Huawei;
|
||||
|
||||
- **Security Enforcement**: only strict TLS, DKIM and OpenPGP with minimized metadata accepted
|
||||
- **Security Enforcement**: Only connections with strict TLS are accepted;
|
||||
all messages must be correctly signed with DKIM and OpenPGP-encrypted with minimized metadata.
|
||||
There are experimental exceptions for no-DNS relays,
|
||||
which are allowed use self-signed TLS certificates
|
||||
and which do not need to DKIM-sign their messages.
|
||||
Unencrypted messages are allowed in neither case.
|
||||
|
||||
- **Reliable Federation and Decentralization:** No spam or IP reputation checks, federating
|
||||
depends on established IETF standards and protocols.
|
||||
|
||||
This repository contains everything needed to setup a ready-to-use chatmail relay on an ssh-reachable host.
|
||||
This repository contains everything needed to setup a ready-to-use chatmail relay on an ssh-reachable host.
|
||||
For getting started and more information please refer to the web version of this repositories' documentation at
|
||||
|
||||
[https://chatmail.at/doc/relay](https://chatmail.at/doc/relay)
|
||||
|
||||
+23
-5
@@ -1,15 +1,33 @@
|
||||
# Releasing a new version of chatmail relay
|
||||
|
||||
For example, to release version 1.9.0 of chatmail relay, do the following steps.
|
||||
For example, to release version 1.13.0 of chatmail relay, do the following steps.
|
||||
|
||||
1. Update the changelog: `git cliff --unreleased --tag 1.9.0 --prepend CHANGELOG.md` or `git cliff -u -t 1.9.0 -p CHANGELOG.md`.
|
||||
1. Update the changelog: `git cliff --unreleased --tag 1.13.0 --prepend CHANGELOG.md` or `git cliff -u -t 1.13.0 -p CHANGELOG.md`.
|
||||
|
||||
2. Open the changelog in the editor, edit it if required.
|
||||
|
||||
3. Commit the changes to the changelog with a commit message `chore(release): prepare for 1.9.0`.
|
||||
|
||||
3. Tag the release: `git tag --annotate 1.9.0`.
|
||||
4. Open a PR with the new commit, merge it to main after review.
|
||||
|
||||
4. Push the release tag: `git push origin 1.9.0`.
|
||||
5. In the web interface, create a GitHub release, tell it to create a new tag.
|
||||
|
||||
5. Create a GitHub release: `gh release create 1.9.0`.
|
||||
## Releasing filtermail
|
||||
|
||||
filtermail is versioned independently and released from the same repo,
|
||||
using `filtermail-` prefixed tags. To release filtermail 0.7.5:
|
||||
|
||||
1. Switch to `filtermail/` directory: `cd filtermail`
|
||||
|
||||
2. Update the changelog:
|
||||
`git cliff u -t filtermail-0.7.5 -p CHANGELOG.md`
|
||||
|
||||
3. Bump `version` in `Cargo.toml` and commit `Cargo.lock`.
|
||||
|
||||
4. Commit with the message: `feat(release): prepare filtermail for 0.7.5`
|
||||
|
||||
5. Open a PR with the new commit, merge it to main after review.
|
||||
|
||||
6. In the web interface, create a GitHub release, tell it to create a new tag `filtermail-0.7.5`. Do not push the tag by hand: the release must exist before the upload job runs.
|
||||
|
||||
7. Bump the pinned version and sha256sums in `cmdeploy/src/cmdeploy/filtermail/deployer.py`.
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
include src/chatmaild/defaults/*.json
|
||||
include src/chatmaild/ini/*.ini.f
|
||||
include src/chatmaild/ini/*.ini
|
||||
include src/chatmaild/tests/mail-data/*
|
||||
|
||||
@@ -6,13 +6,11 @@ build-backend = "setuptools.build_meta"
|
||||
name = "chatmaild"
|
||||
version = "0.3"
|
||||
dependencies = [
|
||||
"aiosmtpd",
|
||||
"iniconfig",
|
||||
"deltachat-rpc-server",
|
||||
"deltachat-rpc-client",
|
||||
"filelock",
|
||||
"psutil",
|
||||
"requests",
|
||||
"crypt-r >= 3.13.1 ; python_version >= '3.11'",
|
||||
"crypt-r >= 3.13.1 ; python_version >= '3.13'",
|
||||
]
|
||||
|
||||
[tool.setuptools]
|
||||
@@ -24,10 +22,10 @@ where = ['src']
|
||||
[project.scripts]
|
||||
doveauth = "chatmaild.doveauth:main"
|
||||
chatmail-metadata = "chatmaild.metadata:main"
|
||||
chatmail-expire = "chatmaild.expire:main"
|
||||
chatmail-expire = "chatmaild.expire:daily_expire_main"
|
||||
chatmail-quota-expire = "chatmaild.expire:quota_expire_main"
|
||||
chatmail-fsreport = "chatmaild.fsreport:main"
|
||||
lastlogin = "chatmaild.lastlogin:main"
|
||||
turnserver = "chatmaild.turnserver:main"
|
||||
|
||||
[project.entry-points.pytest11]
|
||||
"chatmaild.testplugin" = "chatmaild.tests.plugin"
|
||||
@@ -70,6 +68,7 @@ commands =
|
||||
deps = pytest
|
||||
pdbpp
|
||||
pytest-localserver
|
||||
aiosmtpd
|
||||
execnet
|
||||
commands = pytest -v -rsXx {posargs}
|
||||
"""
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import os
|
||||
import ipaddress
|
||||
from pathlib import Path
|
||||
|
||||
import iniconfig
|
||||
@@ -9,63 +9,88 @@ from chatmaild.user import User
|
||||
def read_config(inipath):
|
||||
assert Path(inipath).exists(), inipath
|
||||
cfg = iniconfig.IniConfig(inipath)
|
||||
params = cfg.sections["params"]
|
||||
default_config_content = get_default_config_content(params["mail_domain"])
|
||||
df_params = iniconfig.IniConfig("ini", data=default_config_content)["params"]
|
||||
new_params = dict(df_params.items())
|
||||
new_params.update(params)
|
||||
return Config(inipath, params=new_params)
|
||||
return Config(inipath, params=cfg.sections["params"])
|
||||
|
||||
|
||||
class Config:
|
||||
def __init__(self, inipath, params):
|
||||
self._inipath = inipath
|
||||
self.mail_domain = params["mail_domain"]
|
||||
self.max_user_send_per_minute = int(params.get("max_user_send_per_minute", 60))
|
||||
self.max_user_send_burst_size = int(params.get("max_user_send_burst_size", 10))
|
||||
self.max_mailbox_size = params["max_mailbox_size"]
|
||||
self.max_message_size = int(params.get("max_message_size", "31457280"))
|
||||
self.delete_mails_after = params["delete_mails_after"]
|
||||
self.delete_large_after = params["delete_large_after"]
|
||||
self.delete_inactive_users_after = int(params["delete_inactive_users_after"])
|
||||
self.username_min_length = int(params["username_min_length"])
|
||||
self.username_max_length = int(params["username_max_length"])
|
||||
self.password_min_length = int(params["password_min_length"])
|
||||
self.passthrough_senders = params["passthrough_senders"].split()
|
||||
self.passthrough_recipients = params["passthrough_recipients"].split()
|
||||
self.www_folder = params.get("www_folder", "")
|
||||
self.filtermail_smtp_port = int(params.get("filtermail_smtp_port", "10080"))
|
||||
params = dict(params)
|
||||
raw_domain = params.pop("mail_domain")
|
||||
self.mail_domain_bare = raw_domain
|
||||
|
||||
if is_valid_ipv4(raw_domain):
|
||||
self.ipv4_relay = raw_domain
|
||||
self.mail_domain = f"[{raw_domain}]"
|
||||
self.postfix_myhostname = ipaddress.IPv4Address(raw_domain).reverse_pointer
|
||||
else:
|
||||
self.ipv4_relay = None
|
||||
self.mail_domain = raw_domain
|
||||
self.postfix_myhostname = raw_domain
|
||||
|
||||
self.max_user_send_per_minute = int(params.pop("max_user_send_per_minute", 60))
|
||||
self.max_user_send_burst_size = int(params.pop("max_user_send_burst_size", 10))
|
||||
self.max_mailbox_size = params.pop("max_mailbox_size", "500M")
|
||||
self.max_message_size = int(params.pop("max_message_size", 31457280))
|
||||
self.delete_mails_after = params.pop("delete_mails_after", "20")
|
||||
self.delete_large_after = params.pop("delete_large_after", "7")
|
||||
self.delete_inactive_users_after = int(
|
||||
params.pop("delete_inactive_users_after", 90)
|
||||
)
|
||||
self.username_min_length = int(params.pop("username_min_length", 9))
|
||||
self.username_max_length = int(params.pop("username_max_length", 9))
|
||||
self.password_min_length = int(params.pop("password_min_length", 9))
|
||||
self.www_folder = params.pop("www_folder", "")
|
||||
self.filtermail_smtp_port = int(params.pop("filtermail_smtp_port", "10080"))
|
||||
self.filtermail_smtp_port_incoming = int(
|
||||
params.get("filtermail_smtp_port_incoming", "10081")
|
||||
params.pop("filtermail_smtp_port_incoming", "10081")
|
||||
)
|
||||
self.postfix_reinject_port = int(params.get("postfix_reinject_port", "10025"))
|
||||
self.filtermail_http_port_incoming = int(
|
||||
params.pop("filtermail_http_port_incoming", "10082")
|
||||
)
|
||||
self.filtermail_lmtp_port_transport = int(
|
||||
params.pop("filtermail_lmtp_port_transport", "10083")
|
||||
)
|
||||
self.postfix_reinject_port = int(params.pop("postfix_reinject_port", "10025"))
|
||||
self.postfix_reinject_port_incoming = int(
|
||||
params.get("postfix_reinject_port_incoming", "10026")
|
||||
params.pop("postfix_reinject_port_incoming", "10026")
|
||||
)
|
||||
self.mtail_address = params.get("mtail_address")
|
||||
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
|
||||
self.addr_v4 = os.environ.get("CHATMAIL_ADDR_V4", "")
|
||||
self.addr_v6 = os.environ.get("CHATMAIL_ADDR_V6", "")
|
||||
self.acme_email = params.get("acme_email", "")
|
||||
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
|
||||
self.imap_compress = params.get("imap_compress", "false").lower() == "true"
|
||||
if "iroh_relay" not in params:
|
||||
self.iroh_relay = "https://" + params["mail_domain"]
|
||||
self.mtail_address = params.pop("mtail_address", None)
|
||||
self.disable_ipv6 = params.pop("disable_ipv6", "false").lower() == "true"
|
||||
self.acme_email = params.pop("acme_email", "")
|
||||
self.imap_rawlog = params.pop("imap_rawlog", "false").lower() == "true"
|
||||
self.imap_compress = params.pop("imap_compress", "false").lower() == "true"
|
||||
self.turn_socket_path = params.pop(
|
||||
"turn_socket_path", "/run/chatmail-turn/turn.socket"
|
||||
)
|
||||
iroh_relay = params.pop("iroh_relay", None)
|
||||
if iroh_relay is None:
|
||||
self.iroh_relay = "https://" + raw_domain
|
||||
self.enable_iroh_relay = True
|
||||
else:
|
||||
self.iroh_relay = params["iroh_relay"].strip()
|
||||
self.iroh_relay = iroh_relay.strip()
|
||||
self.enable_iroh_relay = False
|
||||
self.privacy_postal = params.get("privacy_postal")
|
||||
self.privacy_mail = params.get("privacy_mail")
|
||||
self.privacy_pdo = params.get("privacy_pdo")
|
||||
self.privacy_supervisor = params.get("privacy_supervisor")
|
||||
self.privacy_postal = params.pop("privacy_postal", None)
|
||||
self.privacy_mail = params.pop("privacy_mail", None)
|
||||
self.privacy_pdo = params.pop("privacy_pdo", None)
|
||||
self.privacy_supervisor = params.pop("privacy_supervisor", None)
|
||||
|
||||
self.max_load_1m = float(params.pop("max_load_1m", 5))
|
||||
self.min_available_memory_mb = parse_size_mb(
|
||||
params.pop("min_available_memory", "200M")
|
||||
)
|
||||
self.min_free_disk_space_mb = parse_size_mb(
|
||||
params.pop("min_free_disk_space", "1G")
|
||||
)
|
||||
self.max_imap_connections = int(params.pop("max_imap_connections", 10000))
|
||||
self.max_smtp_connections = int(params.pop("max_smtp_connections", 1000))
|
||||
|
||||
# TLS certificate management.
|
||||
# If tls_external_cert_and_key is set, use externally managed certs.
|
||||
# Otherwise derived from the domain name:
|
||||
# - Domains starting with "_" use self-signed certificates
|
||||
# - All other domains use ACME.
|
||||
external = params.get("tls_external_cert_and_key", "").strip()
|
||||
external = params.pop("tls_external_cert_and_key", "").strip()
|
||||
|
||||
if external:
|
||||
parts = external.split()
|
||||
@@ -76,21 +101,27 @@ class Config:
|
||||
)
|
||||
self.tls_cert_mode = "external"
|
||||
self.tls_cert_path, self.tls_key_path = parts
|
||||
elif self.mail_domain.startswith("_"):
|
||||
elif raw_domain.startswith("_") or self.ipv4_relay:
|
||||
self.tls_cert_mode = "self"
|
||||
self.tls_cert_path = "/etc/ssl/certs/mailserver.pem"
|
||||
self.tls_key_path = "/etc/ssl/private/mailserver.key"
|
||||
else:
|
||||
self.tls_cert_mode = "acme"
|
||||
self.tls_cert_path = f"/var/lib/acme/live/{self.mail_domain}/fullchain"
|
||||
self.tls_key_path = f"/var/lib/acme/live/{self.mail_domain}/privkey"
|
||||
self.tls_cert_path = f"/var/lib/acme/live/{raw_domain}/fullchain"
|
||||
self.tls_key_path = f"/var/lib/acme/live/{raw_domain}/privkey"
|
||||
|
||||
# deprecated option
|
||||
mbdir = params.get("mailboxes_dir", f"/home/vmail/mail/{self.mail_domain}")
|
||||
mbdir = params.pop("mailboxes_dir", f"/home/vmail/mail/{raw_domain}")
|
||||
self.mailboxes_dir = Path(mbdir.strip())
|
||||
|
||||
# old unused option (except for first migration from sqlite to maildir store)
|
||||
self.passdb_path = Path(params.get("passdb_path", "/home/vmail/passdb.sqlite"))
|
||||
self.passdb_path = Path(params.pop("passdb_path", "/home/vmail/passdb.sqlite"))
|
||||
self._unused_keys = list(params)
|
||||
|
||||
@property
|
||||
def max_mailbox_size_mb(self):
|
||||
"""Return max_mailbox_size as an integer in megabytes."""
|
||||
return parse_size_mb(self.max_mailbox_size)
|
||||
|
||||
def _getbytefile(self):
|
||||
return open(self._inipath, "rb")
|
||||
@@ -105,6 +136,16 @@ class Config:
|
||||
return User(maildir, addr, password_path, uid="vmail", gid="vmail")
|
||||
|
||||
|
||||
def parse_size_mb(limit):
|
||||
"""Parse a size string like ``500M`` or ``2G`` and return megabytes."""
|
||||
value = limit.strip().upper().removesuffix("B")
|
||||
if value.endswith("G"):
|
||||
return int(value[:-1]) * 1024
|
||||
if value.endswith("M"):
|
||||
return int(value[:-1])
|
||||
return int(value)
|
||||
|
||||
|
||||
def write_initial_config(inipath, mail_domain, overrides):
|
||||
"""Write out default config file, using the specified config value overrides."""
|
||||
content = get_default_config_content(mail_domain, **overrides)
|
||||
@@ -132,28 +173,13 @@ def get_default_config_content(mail_domain, **overrides):
|
||||
for name, value in extra.items():
|
||||
new_line = f"{name} = {value}"
|
||||
new_lines.append(new_line)
|
||||
return "\n".join(new_lines)
|
||||
|
||||
content = "\n".join(new_lines)
|
||||
|
||||
# apply testrun privacy overrides
|
||||
|
||||
if mail_domain.endswith(".testrun.org"):
|
||||
override_inipath = inidir.joinpath("override-testrun.ini")
|
||||
privacy = iniconfig.IniConfig(override_inipath)["privacy"]
|
||||
lines = []
|
||||
for line in content.split("\n"):
|
||||
for key, value in privacy.items():
|
||||
value_lines = value.format(mail_domain=mail_domain).strip().split("\n")
|
||||
if not line.startswith(f"{key} =") or not value_lines:
|
||||
continue
|
||||
if len(value_lines) == 1:
|
||||
lines.append(f"{key} = {value}")
|
||||
else:
|
||||
lines.append(f"{key} =")
|
||||
for vl in value_lines:
|
||||
lines.append(f" {vl}")
|
||||
break
|
||||
else:
|
||||
lines.append(line)
|
||||
content = "\n".join(lines)
|
||||
return content
|
||||
def is_valid_ipv4(address: str) -> bool:
|
||||
"""Check if a mail_domain is an IPv4 address."""
|
||||
try:
|
||||
ipaddress.IPv4Address(address)
|
||||
return True
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"clients": [
|
||||
{
|
||||
"clientId": "deltachat",
|
||||
"sources": [
|
||||
{
|
||||
"sourceId": "gplay",
|
||||
"versionInteger": 754,
|
||||
"versionString": "2.57.0",
|
||||
"downloadUrl": "https://github.com/deltachat/deltachat-android/releases/download/v2.57.0/deltachat-gplay-release-2.57.0.apk"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -14,6 +14,7 @@ except ImportError:
|
||||
from .config import Config, read_config
|
||||
from .dictproxy import DictProxy
|
||||
from .migrate_db import migrate_from_db_to_maildir
|
||||
from .syslimits import has_sufficient_resources
|
||||
|
||||
NOCREATE_FILE = "/etc/chatmail-nocreate"
|
||||
VALID_LOCALPART_RE = re.compile(r"^[a-z0-9._-]+$")
|
||||
@@ -147,6 +148,8 @@ class AuthDictProxy(DictProxy):
|
||||
return userdata
|
||||
if not is_allowed_to_create(self.config, addr, cleartext_password):
|
||||
return
|
||||
if not has_sufficient_resources(self.config):
|
||||
return
|
||||
|
||||
lock = filelock.FileLock(str(user.password_path) + ".lock", timeout=5)
|
||||
with lock:
|
||||
|
||||
@@ -4,17 +4,26 @@ Expire old messages and addresses.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
from argparse import ArgumentParser
|
||||
from collections import namedtuple
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from stat import S_ISREG
|
||||
|
||||
from chatmaild.config import read_config
|
||||
|
||||
FileEntry = namedtuple("FileEntry", ("path", "mtime", "size"))
|
||||
QuotaFileEntry = namedtuple("QuotaFileEntry", ("mtime", "quota_size", "path"))
|
||||
|
||||
# Quota cleanup factor of max_mailbox_size. The mailbox is reset to this size.
|
||||
QUOTA_CLEANUP_FACTOR = 0.7
|
||||
|
||||
# e.g. "cur/1775324677.M448978P3029757.exam,S=3235,W=3305:2,S"
|
||||
_dovecot_fn_rex = re.compile(r".+/(\d+)\..+,S=(\d+)")
|
||||
|
||||
|
||||
def iter_mailboxes(basedir, maxnum):
|
||||
@@ -74,6 +83,42 @@ class MailboxStat:
|
||||
self.extrafiles.sort(key=lambda x: -x.size)
|
||||
|
||||
|
||||
def parse_dovecot_filename(relpath):
|
||||
m = _dovecot_fn_rex.match(relpath)
|
||||
if not m:
|
||||
return None
|
||||
return QuotaFileEntry(int(m.group(1)), int(m.group(2)), relpath)
|
||||
|
||||
|
||||
def scan_mailbox_messages(mbox):
|
||||
messages = []
|
||||
for sub in ("cur", "new"):
|
||||
for name in os_listdir_if_exists(mbox / sub):
|
||||
if entry := parse_dovecot_filename(f"{sub}/{name}"):
|
||||
messages.append(entry)
|
||||
return messages
|
||||
|
||||
|
||||
def expire_to_target(mbox, target_bytes):
|
||||
messages = scan_mailbox_messages(mbox)
|
||||
total_size = sum(m.quota_size for m in messages)
|
||||
# Keep recent 24 hours of messages protected from expiry because
|
||||
# likely something is wrong with interactions on that address
|
||||
# and quota-full signal can help the address owner's device to notice it
|
||||
undeletable_messages_cutoff = time.time() - (3600 * 24)
|
||||
removed = 0
|
||||
for entry in sorted(messages):
|
||||
if total_size <= target_bytes:
|
||||
break
|
||||
if entry.mtime > undeletable_messages_cutoff:
|
||||
break
|
||||
(mbox / entry.path).unlink(missing_ok=True)
|
||||
total_size -= entry.quota_size
|
||||
removed += 1
|
||||
|
||||
return removed
|
||||
|
||||
|
||||
def print_info(msg):
|
||||
print(msg, file=sys.stderr)
|
||||
|
||||
@@ -123,6 +168,16 @@ class Expiry:
|
||||
if mbox.last_login and mbox.last_login < cutoff_without_login:
|
||||
self.remove_mailbox(mbox.basedir)
|
||||
return
|
||||
elif mbox.last_login is None:
|
||||
try:
|
||||
if not self.dry:
|
||||
os.rmdir(mbox.basedir)
|
||||
self.del_mboxes += 1
|
||||
except OSError:
|
||||
print_info(
|
||||
f"Skipped deleting {mbox.basedir}, doesn't have last_login but isn't empty"
|
||||
)
|
||||
return
|
||||
|
||||
mboxname = os.path.basename(mbox.basedir)
|
||||
if self.verbose:
|
||||
@@ -143,6 +198,19 @@ class Expiry:
|
||||
else:
|
||||
continue
|
||||
changed = True
|
||||
|
||||
target_bytes = (
|
||||
self.config.max_mailbox_size_mb * 1024 * 1024 * QUOTA_CLEANUP_FACTOR
|
||||
)
|
||||
removed = expire_to_target(Path(mbox.basedir), target_bytes)
|
||||
if removed:
|
||||
changed = True
|
||||
self.del_files += removed
|
||||
if self.verbose:
|
||||
print_info(
|
||||
f"quota-expire: removed {removed} message(s) from {mboxname}"
|
||||
)
|
||||
|
||||
if changed:
|
||||
self.remove_file(f"{mbox.basedir}/maildirsize")
|
||||
|
||||
@@ -154,9 +222,9 @@ class Expiry:
|
||||
)
|
||||
|
||||
|
||||
def main(args=None):
|
||||
def daily_expire_main(args=None):
|
||||
"""Expire mailboxes and messages according to chatmail config"""
|
||||
parser = ArgumentParser(description=main.__doc__)
|
||||
parser = ArgumentParser(description=daily_expire_main.__doc__)
|
||||
ini = "/usr/local/lib/chatmaild/chatmail.ini"
|
||||
parser.add_argument(
|
||||
"chatmail_ini",
|
||||
@@ -202,5 +270,33 @@ def main(args=None):
|
||||
print(exp.get_summary())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv[1:])
|
||||
def quota_expire_main(args=None):
|
||||
"""Remove mailbox messages to stay within a megabyte target.
|
||||
|
||||
This entry point is called by dovecot when a quota threshold is passed.
|
||||
"""
|
||||
|
||||
parser = ArgumentParser(description=quota_expire_main.__doc__)
|
||||
parser.add_argument(
|
||||
"target_mb",
|
||||
type=int,
|
||||
help="target mailbox size in megabytes",
|
||||
)
|
||||
parser.add_argument(
|
||||
"mailbox_path",
|
||||
type=Path,
|
||||
help="path to a user mailbox",
|
||||
)
|
||||
args = parser.parse_args(args)
|
||||
|
||||
target_bytes = args.target_mb * 1024 * 1024
|
||||
|
||||
removed_count = expire_to_target(args.mailbox_path, target_bytes)
|
||||
if removed_count:
|
||||
(args.mailbox_path / "maildirsize").unlink(missing_ok=True)
|
||||
print(
|
||||
f"quota-expire: removed {removed_count} message(s)"
|
||||
f" from {args.mailbox_path.name}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 0
|
||||
|
||||
@@ -12,41 +12,62 @@ mail_domain = {mail_domain}
|
||||
#
|
||||
|
||||
# email sending rate per user and minute
|
||||
max_user_send_per_minute = 60
|
||||
#max_user_send_per_minute = 60
|
||||
|
||||
# per-user max burst size for sending rate limiting (GCRA bucket capacity)
|
||||
max_user_send_burst_size = 10
|
||||
#max_user_send_burst_size = 10
|
||||
|
||||
# maximum mailbox size of a chatmail address
|
||||
max_mailbox_size = 500M
|
||||
# (Oldest messages will be removed automatically, so mailboxes never run full)
|
||||
#max_mailbox_size = 500M
|
||||
|
||||
# maximum message size for an e-mail in bytes
|
||||
max_message_size = 31457280
|
||||
#max_message_size = 31457280
|
||||
|
||||
# days after which mails are unconditionally deleted
|
||||
delete_mails_after = 20
|
||||
#delete_mails_after = 20
|
||||
|
||||
# days after which large messages (>200k) are unconditionally deleted
|
||||
delete_large_after = 7
|
||||
#delete_large_after = 7
|
||||
|
||||
# days after which users without a successful login are deleted (database and mails)
|
||||
delete_inactive_users_after = 90
|
||||
#delete_inactive_users_after = 90
|
||||
|
||||
# minimum length a username must have
|
||||
username_min_length = 9
|
||||
#username_min_length = 9
|
||||
|
||||
# maximum length a username can have
|
||||
username_max_length = 9
|
||||
#username_max_length = 9
|
||||
|
||||
# minimum length a password must have
|
||||
password_min_length = 9
|
||||
#password_min_length = 9
|
||||
|
||||
# list of chatmail addresses which can send outbound un-encrypted mail
|
||||
passthrough_senders =
|
||||
#
|
||||
# System resource limits
|
||||
#
|
||||
|
||||
# list of e-mail recipients for which to accept outbound un-encrypted mails
|
||||
# (space-separated, item may start with "@" to whitelist whole recipient domains)
|
||||
passthrough_recipients =
|
||||
# The following three limits refuse creation of new addresses
|
||||
# while existing addresses keep working.
|
||||
# Rejections are logged by the doveauth service.
|
||||
|
||||
# Maximum 1-minute load average, as reported by "uptime";
|
||||
# it counts processes waiting for disk I/O as well as for CPU.
|
||||
#max_load_1m = 5
|
||||
|
||||
# Minimum memory available without swapping.
|
||||
#min_available_memory = 200M
|
||||
|
||||
# Minimum free disk space on the file system holding the mailboxes.
|
||||
#min_free_disk_space = 1G
|
||||
|
||||
# Maximum number of concurrent IMAP connections
|
||||
# (the Dovecot imap process limit).
|
||||
#max_imap_connections = 10000
|
||||
|
||||
# Maximum number of concurrent SMTP connections
|
||||
# on each of the submission and smtps ports (the Postfix process limit).
|
||||
# A single client IP may use up to a fifth of this.
|
||||
#max_smtp_connections = 1000
|
||||
|
||||
# Use externally managed TLS certificates instead of built-in acmetool.
|
||||
# Paths refer to files on the deployment server (not the build machine).
|
||||
@@ -62,19 +83,11 @@ passthrough_recipients =
|
||||
# Deployment Details
|
||||
#
|
||||
|
||||
# SMTP outgoing filtermail and reinjection
|
||||
filtermail_smtp_port = 10080
|
||||
postfix_reinject_port = 10025
|
||||
|
||||
# SMTP incoming filtermail and reinjection
|
||||
filtermail_smtp_port_incoming = 10081
|
||||
postfix_reinject_port_incoming = 10026
|
||||
|
||||
# if set to "True" IPv6 is disabled
|
||||
disable_ipv6 = False
|
||||
#disable_ipv6 = False
|
||||
|
||||
# Your email adress, which will be used in acmetool to manage Let's Encrypt SSL certificates
|
||||
acme_email =
|
||||
#acme_email =
|
||||
|
||||
# Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail
|
||||
# service.
|
||||
@@ -107,13 +120,13 @@ acme_email =
|
||||
# in per-maildir ".in/.out" files.
|
||||
# Note that you need to manually cleanup these files
|
||||
# so use this option with caution on production servers.
|
||||
imap_rawlog = false
|
||||
#imap_rawlog = false
|
||||
|
||||
# set to true if you want to enable the IMAP COMPRESS Extension,
|
||||
# which allows IMAP connections to be efficiently compressed.
|
||||
# WARNING: Enabling this makes it impossible to hibernate IMAP
|
||||
# processes which will result in much higher memory/RAM usage.
|
||||
imap_compress = false
|
||||
#imap_compress = false
|
||||
|
||||
|
||||
#
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
|
||||
[privacy]
|
||||
|
||||
passthrough_recipients = privacy@testrun.org echo@{mail_domain}
|
||||
|
||||
privacy_postal =
|
||||
Merlinux GmbH, Represented by the managing director H. Krekel,
|
||||
Reichgrafen Str. 20, 79102 Freiburg, Germany
|
||||
|
||||
privacy_mail = privacy@testrun.org
|
||||
privacy_pdo =
|
||||
Prof. Dr. Fabian Schmieder, lexICT UG (limited), Ostfeldstr. 49, 30559 Hannover.
|
||||
You can contact him at *delta-privacy@merlinux.eu* (Keyword: DPO)
|
||||
privacy_supervisor =
|
||||
State Commissioner for Data Protection and Freedom of Information of
|
||||
Baden-Württemberg in 70173 Stuttgart, Germany.
|
||||
@@ -1,13 +1,35 @@
|
||||
import json
|
||||
import logging
|
||||
import socket
|
||||
import sys
|
||||
import time
|
||||
from contextlib import contextmanager
|
||||
from importlib.resources import files
|
||||
|
||||
from .config import read_config
|
||||
from .dictproxy import DictProxy
|
||||
from .filedict import FileDict
|
||||
from .notifier import Notifier
|
||||
from .turnserver import turn_credentials
|
||||
|
||||
|
||||
def turn_credentials(turn_socket_path):
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client_socket:
|
||||
client_socket.settimeout(5)
|
||||
client_socket.connect(turn_socket_path)
|
||||
with client_socket.makefile("rb") as file:
|
||||
return file.readline().decode("utf-8").strip()
|
||||
|
||||
|
||||
def read_appversions(path):
|
||||
try:
|
||||
data = json.loads(path.read_bytes())
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except (OSError, ValueError):
|
||||
logging.exception(f"failed to read {path}")
|
||||
return None
|
||||
# the dict protocol is line-based, keep the value single-line
|
||||
return json.dumps(data, separators=(",", ":"))
|
||||
|
||||
|
||||
def _is_valid_token_timestamp(timestamp, now):
|
||||
@@ -70,44 +92,57 @@ class Metadata:
|
||||
# Some tokens have expired, remove them.
|
||||
with self._modify_tokens(addr) as _tokens:
|
||||
pass
|
||||
elif isinstance(tokens, list):
|
||||
with self._modify_tokens(addr) as tokens:
|
||||
token_list = list(tokens.keys())
|
||||
else:
|
||||
token_list = []
|
||||
return token_list
|
||||
|
||||
|
||||
class MetadataDictProxy(DictProxy):
|
||||
def __init__(self, notifier, metadata, iroh_relay=None, turn_hostname=None):
|
||||
def __init__(
|
||||
self,
|
||||
notifier,
|
||||
metadata,
|
||||
iroh_relay=None,
|
||||
turn_hostname=None,
|
||||
turn_socket_path=None,
|
||||
):
|
||||
super().__init__()
|
||||
self.notifier = notifier
|
||||
self.metadata = metadata
|
||||
self.iroh_relay = iroh_relay
|
||||
self.turn_hostname = turn_hostname
|
||||
self.turn_socket_path = turn_socket_path
|
||||
self.appversions_path = files(__package__).joinpath("defaults/appversions.json")
|
||||
|
||||
def handle_lookup(self, parts):
|
||||
# Lpriv/43f5f508a7ea0366dff30200c15250e3/devicetoken\tlkj123poi@c2.testrun.org
|
||||
keyparts = parts[0].split("/", 2)
|
||||
if keyparts[0] == "priv":
|
||||
keyname = keyparts[2]
|
||||
addr = parts[1]
|
||||
if keyname == self.metadata.DEVICETOKEN_KEY:
|
||||
match parts[0].split("/", 2):
|
||||
case ["priv", _, keyname] if keyname == self.metadata.DEVICETOKEN_KEY:
|
||||
addr = parts[1]
|
||||
res = " ".join(self.metadata.get_tokens_for_addr(addr))
|
||||
return f"O{res}\n"
|
||||
elif keyparts[0] == "shared":
|
||||
keyname = keyparts[2]
|
||||
if (
|
||||
keyname == "vendor/vendor.dovecot/pvt/server/vendor/deltachat/irohrelay"
|
||||
and self.iroh_relay
|
||||
):
|
||||
# Handle `GETMETADATA "" /shared/vendor/deltachat/irohrelay`
|
||||
return f"O{self.iroh_relay}\n"
|
||||
elif keyname == "vendor/vendor.dovecot/pvt/server/vendor/deltachat/turn":
|
||||
try:
|
||||
res = turn_credentials()
|
||||
except Exception:
|
||||
logging.exception("failed to get TURN credentials")
|
||||
return "N\n"
|
||||
port = 3478
|
||||
return f"O{self.turn_hostname}:{port}:{res}\n"
|
||||
case ["shared", _, keyname]:
|
||||
prefix = "vendor/vendor.dovecot/pvt/server/vendor/deltachat/"
|
||||
if keyname.startswith(prefix):
|
||||
match keyname[len(prefix) :]:
|
||||
case "irohrelay" if self.iroh_relay:
|
||||
return f"O{self.iroh_relay}\n"
|
||||
case "turn":
|
||||
try:
|
||||
res = turn_credentials(self.turn_socket_path)
|
||||
except Exception:
|
||||
logging.exception("failed to get TURN credentials")
|
||||
return "N\n"
|
||||
return f"O{self.turn_hostname}:3478:{res}\n"
|
||||
case "maxsmtprecipients":
|
||||
# postfix default (see "postconf smtpd_recipient_limit")
|
||||
return "O1000\n"
|
||||
case "appversions":
|
||||
value = read_appversions(self.appversions_path)
|
||||
return f"O{value}\n" if value else "N\n"
|
||||
|
||||
logging.warning(f"lookup ignored: {parts!r}")
|
||||
return "N\n"
|
||||
@@ -117,12 +152,13 @@ class MetadataDictProxy(DictProxy):
|
||||
# https://github.com/dovecot/core/blob/main/src/lib-storage/mailbox-attribute.h
|
||||
keyname = parts[1].split("/")
|
||||
value = parts[2] if len(parts) > 2 else ""
|
||||
if keyname[0] == "priv" and keyname[2] == self.metadata.DEVICETOKEN_KEY:
|
||||
self.metadata.add_token_to_addr(addr, value)
|
||||
return True
|
||||
elif keyname[0] == "priv" and keyname[2] == "messagenew":
|
||||
self.notifier.new_message_for_addr(addr, self.metadata)
|
||||
return True
|
||||
match keyname:
|
||||
case ["priv", _, key] if key == self.metadata.DEVICETOKEN_KEY:
|
||||
self.metadata.add_token_to_addr(addr, value)
|
||||
return True
|
||||
case ["priv", _, "messagenew"]:
|
||||
self.notifier.new_message_for_addr(addr, self.metadata)
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
@@ -133,6 +169,7 @@ def main():
|
||||
config = read_config(config_path)
|
||||
iroh_relay = config.iroh_relay
|
||||
mail_domain = config.mail_domain
|
||||
socket_path = config.turn_socket_path
|
||||
|
||||
vmail_dir = config.mailboxes_dir
|
||||
if not vmail_dir.exists():
|
||||
@@ -150,6 +187,7 @@ def main():
|
||||
metadata=metadata,
|
||||
iroh_relay=iroh_relay,
|
||||
turn_hostname=mail_domain,
|
||||
turn_socket_path=socket_path,
|
||||
)
|
||||
|
||||
dictproxy.serve_forever_from_socket(socket)
|
||||
|
||||
@@ -25,13 +25,19 @@ def create_newemail_dict(config: Config):
|
||||
return dict(email=f"{user}@{config.mail_domain}", password=f"{password}")
|
||||
|
||||
|
||||
def create_dclogin_url(email, password):
|
||||
def create_dclogin_url(config, email, password):
|
||||
"""Build a dclogin: URL with credentials and self-signed cert acceptance.
|
||||
|
||||
Uses ic=3 (AcceptInvalidCertificates) so chatmail clients
|
||||
can connect to servers with self-signed TLS certificates.
|
||||
"""
|
||||
return f"dclogin:{quote(email, safe='@')}?p={quote(password, safe='')}&v=1&ic=3"
|
||||
if config.ipv4_relay:
|
||||
imap_host = "&ih=" + config.ipv4_relay
|
||||
smtp_host = "&sh=" + config.ipv4_relay
|
||||
else:
|
||||
imap_host = ""
|
||||
smtp_host = ""
|
||||
return f"dclogin:{quote(email, safe='@[]')}?p={quote(password, safe='')}&v=1{imap_host}{smtp_host}&ic=3"
|
||||
|
||||
|
||||
def print_new_account():
|
||||
@@ -40,7 +46,9 @@ def print_new_account():
|
||||
|
||||
result = dict(email=creds["email"], password=creds["password"])
|
||||
if config.tls_cert_mode == "self":
|
||||
result["dclogin_url"] = create_dclogin_url(creds["email"], creds["password"])
|
||||
result["dclogin_url"] = create_dclogin_url(
|
||||
config, creds["email"], creds["password"]
|
||||
)
|
||||
|
||||
print("Content-Type: application/json")
|
||||
print("")
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Detect whether the system is at its limits."""
|
||||
|
||||
import logging
|
||||
|
||||
import psutil
|
||||
|
||||
MB = 1024 * 1024
|
||||
|
||||
|
||||
def read_value(getter):
|
||||
try:
|
||||
return getter()
|
||||
except Exception as e:
|
||||
logging.warning("ignoring unreadable system limit: %s", e)
|
||||
return None
|
||||
|
||||
|
||||
def has_sufficient_resources(config):
|
||||
"""Return False if load, memory or disk exceeds a configured limit."""
|
||||
load = read_value(lambda: psutil.getloadavg()[0])
|
||||
mem = read_value(lambda: psutil.virtual_memory().available // MB)
|
||||
disk = read_value(lambda: psutil.disk_usage(str(config.mailboxes_dir)).free // MB)
|
||||
if load is not None and load > config.max_load_1m:
|
||||
msg = f"load avg {load:.2f} > {config.max_load_1m:.2f}"
|
||||
elif mem is not None and mem < config.min_available_memory_mb:
|
||||
msg = f"available memory {mem}MB < {config.min_available_memory_mb}MB"
|
||||
elif disk is not None and disk < config.min_free_disk_space_mb:
|
||||
msg = f"free disk {disk}MB < {config.min_free_disk_space_mb}MB"
|
||||
else:
|
||||
return True
|
||||
logging.warning("registration rejected: %s", msg)
|
||||
return False
|
||||
@@ -20,6 +20,10 @@ def make_config(tmp_path):
|
||||
basedir.mkdir(parents=True, exist_ok=True)
|
||||
overrides = settings.copy() if settings else {}
|
||||
overrides["mailboxes_dir"] = str(basedir)
|
||||
# permissive resource limits so tests never depend on host load/memory/disk
|
||||
overrides.setdefault("max_load_1m", "99999")
|
||||
overrides.setdefault("min_available_memory", "0")
|
||||
overrides.setdefault("min_free_disk_space", "0")
|
||||
write_initial_config(inipath, mail_domain, overrides=overrides)
|
||||
return read_config(inipath)
|
||||
|
||||
@@ -31,6 +35,11 @@ def example_config(make_config):
|
||||
return make_config("chat.example.org")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ipv4_config(make_config):
|
||||
return make_config("1.3.3.7")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def maildomain(example_config):
|
||||
return example_config.mail_domain
|
||||
@@ -85,13 +94,13 @@ def mockout():
|
||||
captured_green = []
|
||||
captured_plain = []
|
||||
|
||||
def red(self, msg, **kw):
|
||||
def red(self, msg):
|
||||
self.captured_red.append(msg)
|
||||
|
||||
def green(self, msg, **kw):
|
||||
def green(self, msg):
|
||||
self.captured_green.append(msg)
|
||||
|
||||
def print(self, msg="", **kw):
|
||||
def __call__(self, msg):
|
||||
self.captured_plain.append(msg)
|
||||
|
||||
return MockOut()
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from chatmaild.metadata import MetadataDictProxy
|
||||
|
||||
ALLOWED_URL_PREFIXES = (
|
||||
"https://github.com/deltachat/",
|
||||
"https://download.delta.chat/",
|
||||
)
|
||||
|
||||
|
||||
def check_string(value):
|
||||
assert isinstance(value, str), value
|
||||
assert value
|
||||
|
||||
|
||||
def check_version_integer(value):
|
||||
# core parses this as u32, see https://github.com/chatmail/core/pull/8557
|
||||
assert isinstance(value, int) and not isinstance(value, bool), value
|
||||
assert 0 <= value < 2**32, value
|
||||
|
||||
|
||||
def check_appversions(data):
|
||||
"""Verifies the file the way core parses it.
|
||||
|
||||
core deserializes into typed structs and drops the whole payload
|
||||
of a relay if a single value has an unexpected type,
|
||||
while missing or misspelled keys silently turn into defaults.
|
||||
"""
|
||||
assert set(data) == {"clients"}, data
|
||||
assert isinstance(data["clients"], list)
|
||||
assert data["clients"]
|
||||
client_ids = []
|
||||
for client in data["clients"]:
|
||||
assert set(client) == {"clientId", "sources"}, client
|
||||
check_string(client["clientId"])
|
||||
client_ids.append(client["clientId"])
|
||||
assert isinstance(client["sources"], list)
|
||||
assert client["sources"]
|
||||
source_ids = []
|
||||
for source in client["sources"]:
|
||||
assert set(source) == {
|
||||
"sourceId",
|
||||
"versionInteger",
|
||||
"versionString",
|
||||
"downloadUrl",
|
||||
}, source
|
||||
check_string(source["sourceId"])
|
||||
source_ids.append(source["sourceId"])
|
||||
check_version_integer(source["versionInteger"])
|
||||
check_string(source["versionString"])
|
||||
check_string(source["downloadUrl"])
|
||||
assert source["downloadUrl"].startswith(ALLOWED_URL_PREFIXES)
|
||||
# core takes the first matching source, later duplicates never surface
|
||||
assert len(set(source_ids)) == len(source_ids), source_ids
|
||||
assert len(set(client_ids)) == len(client_ids), client_ids
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def appversions():
|
||||
# check the file which chatmail-metadata actually serves
|
||||
path = MetadataDictProxy(notifier=None, metadata=None).appversions_path
|
||||
return json.loads(path.read_text())
|
||||
|
||||
|
||||
def test_appversions_schema(appversions):
|
||||
check_appversions(appversions)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [True, -1, 2**32, "754", 754.0, None])
|
||||
def test_version_integer_rejected(appversions, value):
|
||||
appversions["clients"][0]["sources"][0]["versionInteger"] = value
|
||||
with pytest.raises(AssertionError):
|
||||
check_appversions(appversions)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("key", ["clientId", "sources"])
|
||||
def test_misspelled_client_key_rejected(appversions, key):
|
||||
client = appversions["clients"][0]
|
||||
client[key + "s"] = client.pop(key)
|
||||
with pytest.raises(AssertionError):
|
||||
check_appversions(appversions)
|
||||
|
||||
|
||||
def test_duplicate_source_id_rejected(appversions):
|
||||
sources = appversions["clients"][0]["sources"]
|
||||
sources.append(dict(sources[0]))
|
||||
with pytest.raises(AssertionError):
|
||||
check_appversions(appversions)
|
||||
|
||||
|
||||
def test_foreign_download_url_rejected(appversions):
|
||||
appversions["clients"][0]["sources"][0]["downloadUrl"] = "https://example.org/x.apk"
|
||||
with pytest.raises(AssertionError):
|
||||
check_appversions(appversions)
|
||||
@@ -1,6 +1,10 @@
|
||||
import pytest
|
||||
|
||||
from chatmaild.config import read_config
|
||||
from chatmaild.config import (
|
||||
is_valid_ipv4,
|
||||
parse_size_mb,
|
||||
read_config,
|
||||
)
|
||||
|
||||
|
||||
def test_read_config_basic(example_config):
|
||||
@@ -9,10 +13,21 @@ def test_read_config_basic(example_config):
|
||||
assert not example_config.privacy_pdo and not example_config.privacy_postal
|
||||
|
||||
inipath = example_config._inipath
|
||||
inipath.write_text(inipath.read_text().replace("60", "37"))
|
||||
inipath.write_text(
|
||||
inipath.read_text().replace(
|
||||
"#max_user_send_per_minute = 60",
|
||||
"max_user_send_per_minute = 37",
|
||||
)
|
||||
)
|
||||
example_config = read_config(inipath)
|
||||
assert example_config.max_user_send_per_minute == 37
|
||||
assert example_config.mail_domain == "chat.example.org"
|
||||
assert example_config.ipv4_relay is None
|
||||
|
||||
|
||||
def test_read_config_ipv4(ipv4_config):
|
||||
assert ipv4_config.ipv4_relay == "1.3.3.7"
|
||||
assert ipv4_config.mail_domain == "[1.3.3.7]"
|
||||
|
||||
|
||||
def test_read_config_basic_using_defaults(tmp_path, maildomain):
|
||||
@@ -21,26 +36,23 @@ def test_read_config_basic_using_defaults(tmp_path, maildomain):
|
||||
example_config = read_config(inipath)
|
||||
assert example_config.max_user_send_per_minute == 60
|
||||
assert example_config.filtermail_smtp_port_incoming == 10081
|
||||
assert example_config.filtermail_smtp_port == 10080
|
||||
assert example_config.postfix_reinject_port == 10025
|
||||
assert example_config.max_user_send_per_minute == 60
|
||||
assert example_config.max_mailbox_size == "500M"
|
||||
assert example_config.delete_mails_after == "20"
|
||||
assert example_config.delete_large_after == "7"
|
||||
assert example_config.username_min_length == 9
|
||||
assert example_config.username_max_length == 9
|
||||
assert example_config.password_min_length == 9
|
||||
assert example_config.max_imap_connections == 10000
|
||||
assert example_config.max_smtp_connections == 1000
|
||||
assert example_config._unused_keys == []
|
||||
|
||||
|
||||
def test_read_config_testrun(make_config):
|
||||
config = make_config("something.testrun.org")
|
||||
assert config.mail_domain == "something.testrun.org"
|
||||
assert len(config.privacy_postal.split("\n")) > 1
|
||||
assert len(config.privacy_supervisor.split("\n")) > 1
|
||||
assert len(config.privacy_pdo.split("\n")) > 1
|
||||
assert config.privacy_mail == "privacy@testrun.org"
|
||||
assert config.filtermail_smtp_port == 10080
|
||||
assert config.postfix_reinject_port == 10025
|
||||
assert config.max_user_send_per_minute == 60
|
||||
assert config.max_mailbox_size == "500M"
|
||||
assert config.delete_mails_after == "20"
|
||||
assert config.delete_large_after == "7"
|
||||
assert config.username_min_length == 9
|
||||
assert config.username_max_length == 9
|
||||
assert config.password_min_length == 9
|
||||
assert "privacy@testrun.org" in config.passthrough_recipients
|
||||
assert config.passthrough_senders == []
|
||||
def test_config_unused_keys(make_config):
|
||||
config = make_config("chat.example.org", {"passthrough_senders": "x@y.org"})
|
||||
assert config._unused_keys == ["passthrough_senders"]
|
||||
|
||||
|
||||
def test_config_userstate_paths(make_config, tmp_path):
|
||||
@@ -121,3 +133,31 @@ def test_config_tls_external_bad_format(make_config):
|
||||
"tls_external_cert_and_key": "/only/one/path.pem",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_parse_size_mb():
|
||||
assert parse_size_mb("500M") == 500
|
||||
assert parse_size_mb("2G") == 2048
|
||||
assert parse_size_mb(" 1g ") == 1024
|
||||
assert parse_size_mb("100MB") == 100
|
||||
assert parse_size_mb("256") == 256
|
||||
|
||||
|
||||
def test_max_mailbox_size_mb(make_config):
|
||||
config = make_config("chat.example.org")
|
||||
assert config.max_mailbox_size == "500M"
|
||||
assert config.max_mailbox_size_mb == 500
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
["input", "result"],
|
||||
[
|
||||
("example.org", False),
|
||||
("1.3.3.7", True),
|
||||
("fe::1", False),
|
||||
("ad.1e.dag.adf", False),
|
||||
("12394142", False),
|
||||
],
|
||||
)
|
||||
def test_is_valid_ipv4(input, result):
|
||||
assert result == is_valid_ipv4(input)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import time
|
||||
|
||||
from chatmaild.doveauth import AuthDictProxy
|
||||
from chatmaild.expire import main as main_expire
|
||||
from chatmaild.expire import daily_expire_main as main_expire
|
||||
|
||||
|
||||
def test_login_timestamps(example_config):
|
||||
|
||||
@@ -202,3 +202,17 @@ def test_50_concurrent_lookups_different_accounts(gencreds, dictproxy):
|
||||
res = results.get()
|
||||
if res is not None:
|
||||
pytest.fail(f"concurrent lookup failed\n{res}")
|
||||
|
||||
|
||||
def test_insufficient_resources_block_creation_not_existing_logins(
|
||||
dictproxy, gencreds, monkeypatch
|
||||
):
|
||||
addr, password = gencreds()
|
||||
assert dictproxy.lookup_passdb(addr, password)
|
||||
|
||||
monkeypatch.setattr(
|
||||
chatmaild.doveauth, "has_sufficient_resources", lambda config: False
|
||||
)
|
||||
newaddr, newpassword = gencreds()
|
||||
assert not dictproxy.lookup_passdb(newaddr, newpassword)
|
||||
assert dictproxy.lookup_passdb(addr, password)
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
import itertools
|
||||
import os
|
||||
import random
|
||||
import shutil
|
||||
import time
|
||||
from datetime import datetime
|
||||
from fnmatch import fnmatch
|
||||
from pathlib import Path
|
||||
@@ -7,15 +10,22 @@ from pathlib import Path
|
||||
import pytest
|
||||
|
||||
from chatmaild.expire import (
|
||||
Expiry,
|
||||
FileEntry,
|
||||
MailboxStat,
|
||||
expire_to_target,
|
||||
get_file_entry,
|
||||
iter_mailboxes,
|
||||
os_listdir_if_exists,
|
||||
parse_dovecot_filename,
|
||||
quota_expire_main,
|
||||
scan_mailbox_messages,
|
||||
)
|
||||
from chatmaild.expire import main as expiry_main
|
||||
from chatmaild.expire import daily_expire_main as expiry_main
|
||||
from chatmaild.fsreport import main as report_main
|
||||
|
||||
MB = 1024 * 1024
|
||||
|
||||
|
||||
def fill_mbox(folderdir):
|
||||
password = folderdir.joinpath("password")
|
||||
@@ -96,6 +106,32 @@ def test_stats_mailbox(mbox1):
|
||||
assert mbox3.last_login is None
|
||||
|
||||
|
||||
def test_mbox_without_password(mbox1, example_config, capsys):
|
||||
password = Path(mbox1.basedir).joinpath("password")
|
||||
os.remove(password)
|
||||
mbox_rescan = MailboxStat(mbox1.basedir)
|
||||
assert mbox_rescan.last_login is None
|
||||
|
||||
exp = Expiry(
|
||||
example_config, dry=False, now=datetime.now().timestamp(), verbose=False
|
||||
)
|
||||
exp.process_mailbox_stat(mbox_rescan)
|
||||
out, err = capsys.readouterr()
|
||||
assert "doesn't have last_login but isn't empty" in err
|
||||
assert os.path.isdir(mbox_rescan.basedir)
|
||||
|
||||
for entry in os.scandir(mbox_rescan.basedir):
|
||||
if os.path.isdir(entry):
|
||||
shutil.rmtree(entry)
|
||||
else:
|
||||
os.remove(entry)
|
||||
|
||||
exp.process_mailbox_stat(mbox_rescan)
|
||||
out, err = capsys.readouterr()
|
||||
assert "doesn't have last_login but isn't empty" not in err
|
||||
assert not os.path.isdir(mbox_rescan.basedir)
|
||||
|
||||
|
||||
def test_report_no_mailboxes(example_config):
|
||||
args = (str(example_config._inipath),)
|
||||
report_main(args)
|
||||
@@ -196,3 +232,51 @@ def test_os_listdir_if_exists(tmp_path):
|
||||
tmp_path.joinpath("x").write_text("hello")
|
||||
assert len(os_listdir_if_exists(str(tmp_path))) == 1
|
||||
assert len(os_listdir_if_exists(str(tmp_path.joinpath("123123")))) == 0
|
||||
|
||||
|
||||
# --- quota expire tests ---
|
||||
|
||||
_msg_counter = itertools.count(1)
|
||||
|
||||
|
||||
def _create_message(basedir, sub, size, days_old=0, disk_size=None):
|
||||
seq = next(_msg_counter)
|
||||
mtime = int(time.time() - days_old * 86400)
|
||||
name = f"{mtime}.M1P1Q{seq}.hostname,S={size},W={size}:2,S"
|
||||
path = basedir / sub / name
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(b"x" * (disk_size if disk_size is not None else size))
|
||||
os.utime(path, (mtime, mtime))
|
||||
return path
|
||||
|
||||
|
||||
def test_parse_dovecot_filename():
|
||||
e = parse_dovecot_filename("cur/1775324677.M448978P3029757.exam,S=3235,W=3305:2,S")
|
||||
assert e.path == "cur/1775324677.M448978P3029757.exam,S=3235,W=3305:2,S"
|
||||
assert e.mtime == 1775324677
|
||||
assert e.quota_size == 3235
|
||||
assert parse_dovecot_filename("cur/msg_without_structure") is None
|
||||
|
||||
|
||||
def test_expire_to_target(tmp_path):
|
||||
_create_message(tmp_path, "cur", MB, days_old=10, disk_size=100)
|
||||
_create_message(tmp_path, "new", MB, days_old=5)
|
||||
_create_message(tmp_path, "cur", MB, days_old=0) # undeletable (<1 hour)
|
||||
assert len(scan_mailbox_messages(tmp_path)) == 3
|
||||
# removes oldest first, uses S= size not disk size
|
||||
removed = expire_to_target(tmp_path, MB)
|
||||
assert removed == 2
|
||||
msgs = scan_mailbox_messages(tmp_path)
|
||||
assert len(msgs) == 1
|
||||
# the surviving message is the fresh undeletable one
|
||||
assert msgs[0].mtime > time.time() - 3600
|
||||
|
||||
|
||||
def test_quota_expire_main(tmp_path, capsys):
|
||||
mbox = tmp_path / "user@example.org"
|
||||
_create_message(mbox, "cur", 2 * MB, days_old=5)
|
||||
(mbox / "maildirsize").write_text("x")
|
||||
quota_expire_main([str(1), str(mbox)])
|
||||
_, err = capsys.readouterr()
|
||||
assert "quota-expire: removed 1 message(s) from user@example.org" in err
|
||||
assert not (mbox / "maildirsize").exists()
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import io
|
||||
import json
|
||||
import time
|
||||
|
||||
import pytest
|
||||
@@ -7,6 +8,7 @@ import requests
|
||||
from chatmaild.metadata import (
|
||||
Metadata,
|
||||
MetadataDictProxy,
|
||||
read_appversions,
|
||||
)
|
||||
from chatmaild.notifier import (
|
||||
Notifier,
|
||||
@@ -324,7 +326,7 @@ def test_turn_credentials_exception_returns_N(notifier, metadata, monkeypatch):
|
||||
turn_hostname="turn.example.org",
|
||||
)
|
||||
|
||||
def mock_turn_credentials():
|
||||
def mock_turn_credentials(turn_socket_path):
|
||||
raise ConnectionRefusedError("socket not available")
|
||||
|
||||
monkeypatch.setattr(chatmaild.metadata, "turn_credentials", mock_turn_credentials)
|
||||
@@ -348,7 +350,9 @@ def test_turn_credentials_success(notifier, metadata, monkeypatch):
|
||||
turn_hostname="turn.example.org",
|
||||
)
|
||||
|
||||
monkeypatch.setattr(chatmaild.metadata, "turn_credentials", lambda: "user:pass")
|
||||
monkeypatch.setattr(
|
||||
chatmaild.metadata, "turn_credentials", lambda path: "user:pass"
|
||||
)
|
||||
|
||||
transactions = {}
|
||||
res = dictproxy.handle_dovecot_request(
|
||||
@@ -360,15 +364,65 @@ def test_turn_credentials_success(notifier, metadata, monkeypatch):
|
||||
|
||||
|
||||
def test_iroh_relay(dictproxy):
|
||||
rfile = io.BytesIO(
|
||||
b"\n".join(
|
||||
[
|
||||
b"H",
|
||||
b"Lshared/0123/vendor/vendor.dovecot/pvt/server/vendor/deltachat/irohrelay\tuser@example.org",
|
||||
]
|
||||
)
|
||||
)
|
||||
wfile = io.BytesIO()
|
||||
key = b"Lshared/0123/vendor/vendor.dovecot/pvt/server/vendor/deltachat/irohrelay\tuser@example.org"
|
||||
rfile, wfile = io.BytesIO(b"H\n" + key), io.BytesIO()
|
||||
dictproxy.iroh_relay = "https://example.org/"
|
||||
dictproxy.loop_forever(rfile, wfile)
|
||||
assert wfile.getvalue() == b"Ohttps://example.org/\n"
|
||||
|
||||
|
||||
def test_read_appversions(tmp_path):
|
||||
path = tmp_path.joinpath("appversions.json")
|
||||
assert read_appversions(path) is None
|
||||
|
||||
path.write_text('{\n "clients": []\n}')
|
||||
assert read_appversions(path) == '{"clients":[]}'
|
||||
|
||||
# the value travels as a single dict protocol line
|
||||
path.write_text('{"clients": [{"clientId": "one\\ntwo"}]}')
|
||||
assert read_appversions(path) == '{"clients":[{"clientId":"one\\ntwo"}]}'
|
||||
|
||||
path.write_text("bad json")
|
||||
assert read_appversions(path) is None
|
||||
|
||||
|
||||
def test_appversions_lookup(dictproxy):
|
||||
# the version information shipped with chatmaild is served as a single line
|
||||
key = b"Lshared/0123/vendor/vendor.dovecot/pvt/server/vendor/deltachat/appversions"
|
||||
key += b"\tuser@example.org"
|
||||
rfile, wfile = io.BytesIO(b"H\n" + key), io.BytesIO()
|
||||
dictproxy.loop_forever(rfile, wfile)
|
||||
value = wfile.getvalue()
|
||||
assert value.startswith(b"O") and value.endswith(b"\n")
|
||||
assert json.loads(value[1:])["clients"]
|
||||
|
||||
|
||||
def test_legacy_token_migration(metadata, testaddr):
|
||||
with metadata.get_metadata_dict(testaddr).modify() as data:
|
||||
data[metadata.DEVICETOKEN_KEY] = ["oldtoken1", "oldtoken2"]
|
||||
|
||||
assert metadata.get_tokens_for_addr(testaddr) == ["oldtoken1", "oldtoken2"]
|
||||
mdict = metadata.get_metadata_dict(testaddr).read()
|
||||
tokens = mdict[metadata.DEVICETOKEN_KEY]
|
||||
assert isinstance(tokens, dict)
|
||||
assert "oldtoken1" in tokens and "oldtoken2" in tokens
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"suffix, expected",
|
||||
[
|
||||
(b"vendor/deltachat/maxsmtprecipients", b"O1000\n"),
|
||||
(b"wrong/prefix/key", b"N\n"),
|
||||
(b"vendor/deltachat/unknown", b"N\n"),
|
||||
],
|
||||
ids=["maxsmtprecipients", "prefix_mismatch", "unknown_name"],
|
||||
)
|
||||
def test_shared_lookup(dictproxy, suffix, expected):
|
||||
key = (
|
||||
b"Lshared/0123/vendor/vendor.dovecot/pvt/server/"
|
||||
+ suffix
|
||||
+ b"\tuser@example.org"
|
||||
)
|
||||
rfile, wfile = io.BytesIO(b"H\n" + key), io.BytesIO()
|
||||
dictproxy.loop_forever(rfile, wfile)
|
||||
assert wfile.getvalue() == expected
|
||||
|
||||
@@ -48,6 +48,8 @@ def test_migration(tmp_path, example_config, caplog):
|
||||
assert passdb_path.stat().st_size > 10000
|
||||
|
||||
example_config.passdb_path = passdb_path
|
||||
# ensure logging.info records are captured regardless of global configuration
|
||||
caplog.set_level("INFO")
|
||||
|
||||
assert not caplog.records
|
||||
|
||||
|
||||
@@ -19,18 +19,35 @@ def test_create_newemail_dict(example_config):
|
||||
assert ac1["password"] != ac2["password"]
|
||||
|
||||
|
||||
def test_create_dclogin_url():
|
||||
url = create_dclogin_url("user@example.org", "p@ss w+rd")
|
||||
def test_create_newemail_dict_ip(ipv4_config):
|
||||
ac = create_newemail_dict(ipv4_config)
|
||||
assert ac["email"].endswith("@[1.3.3.7]")
|
||||
|
||||
|
||||
def test_create_dclogin_url(example_config):
|
||||
addr = "user@example.org"
|
||||
password = "p@ss w+rd"
|
||||
url = create_dclogin_url(example_config, addr, password)
|
||||
assert url.startswith("dclogin:")
|
||||
assert "v=1" in url
|
||||
assert "ic=3" in url
|
||||
|
||||
assert "user@example.org" in url
|
||||
assert addr in url
|
||||
# password special chars must be encoded
|
||||
assert "p%40ss" in url
|
||||
assert "w%2Brd" in url
|
||||
|
||||
|
||||
def test_create_dclogin_url_ipv4(ipv4_config):
|
||||
addr = "user@[1.3.3.7]"
|
||||
password = "p@ss w+rd"
|
||||
url = create_dclogin_url(ipv4_config, addr, password)
|
||||
assert url.startswith("dclogin:")
|
||||
assert "v=1" in url
|
||||
assert "ic=3" in url
|
||||
assert addr in url
|
||||
|
||||
|
||||
def test_print_new_account(capsys, monkeypatch, maildomain, tmpdir, example_config):
|
||||
monkeypatch.setattr(chatmaild.newemail, "CONFIG_PATH", str(example_config._inipath))
|
||||
print_new_account()
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
import shutil
|
||||
|
||||
import psutil
|
||||
|
||||
from chatmaild.syslimits import has_sufficient_resources
|
||||
|
||||
PERMISSIVE = {
|
||||
"max_load_1m": "99999",
|
||||
"min_available_memory": "0",
|
||||
"min_free_disk_space": "0",
|
||||
}
|
||||
|
||||
|
||||
def test_rejects_constrained_system(make_config, caplog):
|
||||
assert has_sufficient_resources(make_config("chat.example.org", PERMISSIVE))
|
||||
for settings in (
|
||||
{"max_load_1m": "-1.0"},
|
||||
{"min_available_memory": "99999999G"},
|
||||
{"min_free_disk_space": "99999999G"},
|
||||
):
|
||||
config = make_config("chat.example.org", PERMISSIVE | settings)
|
||||
caplog.clear()
|
||||
assert not has_sufficient_resources(config), settings
|
||||
assert "registration rejected" in caplog.text
|
||||
|
||||
|
||||
def test_unreadable_disk_does_not_reject(make_config, caplog):
|
||||
config = make_config(
|
||||
"chat.example.org", PERMISSIVE | {"min_free_disk_space": "99999999G"}
|
||||
)
|
||||
shutil.rmtree(config.mailboxes_dir)
|
||||
assert has_sufficient_resources(config)
|
||||
assert "ignoring" in caplog.text
|
||||
|
||||
|
||||
def test_one_unreadable_value_keeps_other_checks(make_config, monkeypatch, caplog):
|
||||
def raise_error(*args):
|
||||
raise psutil.Error("dud")
|
||||
|
||||
monkeypatch.setattr(psutil, "getloadavg", raise_error)
|
||||
config = make_config(
|
||||
"chat.example.org", PERMISSIVE | {"min_free_disk_space": "99999999G"}
|
||||
)
|
||||
assert not has_sufficient_resources(config)
|
||||
assert "ignoring" in caplog.text
|
||||
@@ -0,0 +1,46 @@
|
||||
import socket
|
||||
import threading
|
||||
|
||||
import pytest
|
||||
|
||||
from chatmaild.metadata import turn_credentials
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def turn_socket(tmp_path):
|
||||
sock_path = str(tmp_path / "turn.socket")
|
||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
server.bind(sock_path)
|
||||
server.listen(1)
|
||||
yield sock_path, server
|
||||
server.close()
|
||||
|
||||
|
||||
def test_turn_credentials_timeout(turn_socket):
|
||||
sock_path, server = turn_socket
|
||||
with pytest.raises(socket.timeout):
|
||||
# Inside turn_credentials the kernel listen backlog (1)
|
||||
# completes connect() without accept()
|
||||
# so the client blocks on readline() until the 5s timeout fires.
|
||||
turn_credentials(sock_path)
|
||||
|
||||
|
||||
def test_turn_credentials_connection_refused_on_not_existing_socket(tmp_path):
|
||||
missing = str(tmp_path / "nonexistent.socket")
|
||||
with pytest.raises((ConnectionRefusedError, FileNotFoundError)):
|
||||
turn_credentials(missing)
|
||||
|
||||
|
||||
def test_turn_credentials_socket_success(turn_socket):
|
||||
sock_path, server = turn_socket
|
||||
|
||||
def respond():
|
||||
conn, _ = server.accept()
|
||||
conn.sendall(b"testuser:testpass\n")
|
||||
conn.close()
|
||||
|
||||
t = threading.Thread(target=respond, daemon=True)
|
||||
t.start()
|
||||
|
||||
result = turn_credentials(sock_path)
|
||||
assert result == "testuser:testpass"
|
||||
@@ -1,73 +0,0 @@
|
||||
import socket
|
||||
import threading
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from chatmaild.turnserver import turn_credentials
|
||||
|
||||
SOCKET_PATH = "/run/chatmail-turn/turn.socket"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def turn_socket(tmp_path):
|
||||
"""Create a real Unix socket server at a temp path."""
|
||||
sock_path = str(tmp_path / "turn.socket")
|
||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
server.bind(sock_path)
|
||||
server.listen(1)
|
||||
yield sock_path, server
|
||||
server.close()
|
||||
|
||||
|
||||
def _call_turn_credentials(sock_path):
|
||||
"""Call turn_credentials but connect to sock_path instead of hardcoded path."""
|
||||
original_connect = socket.socket.connect
|
||||
|
||||
def patched_connect(self, address):
|
||||
if address == SOCKET_PATH:
|
||||
address = sock_path
|
||||
return original_connect(self, address)
|
||||
|
||||
with patch.object(socket.socket, "connect", patched_connect):
|
||||
return turn_credentials()
|
||||
|
||||
|
||||
def test_turn_credentials_timeout(turn_socket):
|
||||
"""Server accepts but never responds — must raise socket.timeout."""
|
||||
sock_path, server = turn_socket
|
||||
|
||||
def accept_and_hang():
|
||||
conn, _ = server.accept()
|
||||
time.sleep(30)
|
||||
conn.close()
|
||||
|
||||
t = threading.Thread(target=accept_and_hang, daemon=True)
|
||||
t.start()
|
||||
|
||||
with pytest.raises(socket.timeout):
|
||||
_call_turn_credentials(sock_path)
|
||||
|
||||
|
||||
def test_turn_credentials_connection_refused(tmp_path):
|
||||
"""Socket file doesn't exist — must raise ConnectionRefusedError or FileNotFoundError."""
|
||||
missing = str(tmp_path / "nonexistent.socket")
|
||||
with pytest.raises((ConnectionRefusedError, FileNotFoundError)):
|
||||
_call_turn_credentials(missing)
|
||||
|
||||
|
||||
def test_turn_credentials_success(turn_socket):
|
||||
"""Server responds with credentials — must return stripped string."""
|
||||
sock_path, server = turn_socket
|
||||
|
||||
def respond():
|
||||
conn, _ = server.accept()
|
||||
conn.sendall(b"testuser:testpass\n")
|
||||
conn.close()
|
||||
|
||||
t = threading.Thread(target=respond, daemon=True)
|
||||
t.start()
|
||||
|
||||
result = _call_turn_credentials(sock_path)
|
||||
assert result == "testuser:testpass"
|
||||
@@ -1,10 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
import socket
|
||||
|
||||
|
||||
def turn_credentials() -> str:
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client_socket:
|
||||
client_socket.settimeout(5)
|
||||
client_socket.connect("/run/chatmail-turn/turn.socket")
|
||||
with client_socket.makefile("rb") as file:
|
||||
return file.readline().decode("utf-8").strip()
|
||||
@@ -92,3 +92,5 @@ topo_order_commits = true
|
||||
sort_commits = "oldest"
|
||||
# Process submodules commits
|
||||
recurse_submodules = false
|
||||
exclude_paths = ["filtermail/"]
|
||||
tag_pattern = '^[0-9]+\.[0-9]+\.[0-9]+$'
|
||||
|
||||
@@ -10,7 +10,6 @@ dependencies = [
|
||||
"pillow",
|
||||
"qrcode",
|
||||
"markdown",
|
||||
"pytest",
|
||||
"setuptools>=68",
|
||||
"termcolor",
|
||||
"build",
|
||||
@@ -21,6 +20,7 @@ dependencies = [
|
||||
"execnet",
|
||||
"imap_tools",
|
||||
"deltachat-rpc-client",
|
||||
"deltachat-rpc-server",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import importlib.resources
|
||||
|
||||
from pyinfra.operations import apt, files, server, systemd
|
||||
from pyinfra.operations import apt, server
|
||||
|
||||
from ..basedeploy import Deployer
|
||||
|
||||
@@ -9,9 +7,6 @@ class AcmetoolDeployer(Deployer):
|
||||
def __init__(self, email, domains):
|
||||
self.domains = domains
|
||||
self.email = email
|
||||
self.need_restart_redirector = False
|
||||
self.need_restart_reconcile_service = False
|
||||
self.need_restart_reconcile_timer = False
|
||||
|
||||
def install(self):
|
||||
apt.packages(
|
||||
@@ -19,121 +14,41 @@ class AcmetoolDeployer(Deployer):
|
||||
packages=["acmetool"],
|
||||
)
|
||||
|
||||
files.file(
|
||||
name="Remove old acmetool cronjob, it is replaced with systemd timer.",
|
||||
path="/etc/cron.d/acmetool",
|
||||
present=False,
|
||||
)
|
||||
self.remove_file("/etc/cron.d/acmetool")
|
||||
|
||||
files.put(
|
||||
name="Install acmetool hook.",
|
||||
src=importlib.resources.files(__package__)
|
||||
.joinpath("acmetool.hook")
|
||||
.open("rb"),
|
||||
dest="/etc/acme/hooks/nginx",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="755",
|
||||
)
|
||||
files.file(
|
||||
name="Remove acmetool hook from the wrong location where it was previously installed.",
|
||||
path="/usr/lib/acme/hooks/nginx",
|
||||
present=False,
|
||||
)
|
||||
self.put_executable("acmetool/acmetool.hook", "/etc/acme/hooks/nginx")
|
||||
self.remove_file("/usr/lib/acme/hooks/nginx")
|
||||
|
||||
def configure(self):
|
||||
files.template(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"response-file.yaml.j2"
|
||||
),
|
||||
dest="/var/lib/acme/conf/responses",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.put_template(
|
||||
"acmetool/response-file.yaml.j2",
|
||||
"/var/lib/acme/conf/responses",
|
||||
email=self.email,
|
||||
)
|
||||
|
||||
files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("target.yaml.j2"),
|
||||
dest="/var/lib/acme/conf/target",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.put_template(
|
||||
"acmetool/target.yaml.j2",
|
||||
"/var/lib/acme/conf/target",
|
||||
)
|
||||
|
||||
server.shell(
|
||||
name=f"Remove old acmetool desired files for {self.domains[0]}",
|
||||
commands=[f"rm -f /var/lib/acme/desired/{self.domains[0]}-*"],
|
||||
)
|
||||
files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("desired.yaml.j2"),
|
||||
dest=f"/var/lib/acme/desired/{self.domains[0]}", # 0 is mailhost TLD
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.put_template(
|
||||
"acmetool/desired.yaml.j2",
|
||||
f"/var/lib/acme/desired/{self.domains[0]}",
|
||||
domains=self.domains,
|
||||
)
|
||||
|
||||
service_file = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"acmetool-redirector.service"
|
||||
),
|
||||
dest="/etc/systemd/system/acmetool-redirector.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart_redirector = service_file.changed
|
||||
|
||||
reconcile_service_file = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"acmetool-reconcile.service"
|
||||
),
|
||||
dest="/etc/systemd/system/acmetool-reconcile.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart_reconcile_service = reconcile_service_file.changed
|
||||
|
||||
reconcile_timer_file = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"acmetool-reconcile.timer"
|
||||
),
|
||||
dest="/etc/systemd/system/acmetool-reconcile.timer",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart_reconcile_timer = reconcile_timer_file.changed
|
||||
self.ensure_systemd_unit("acmetool/acmetool-redirector.service")
|
||||
self.ensure_systemd_unit("acmetool/acmetool-reconcile.service")
|
||||
self.ensure_systemd_unit("acmetool/acmetool-reconcile.timer")
|
||||
|
||||
def activate(self):
|
||||
systemd.service(
|
||||
name="Setup acmetool-redirector service",
|
||||
service="acmetool-redirector.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=self.need_restart_redirector,
|
||||
)
|
||||
self.need_restart_redirector = False
|
||||
|
||||
systemd.service(
|
||||
name="Setup acmetool-reconcile service",
|
||||
service="acmetool-reconcile.service",
|
||||
running=False,
|
||||
enabled=False,
|
||||
daemon_reload=self.need_restart_reconcile_service,
|
||||
)
|
||||
self.need_restart_reconcile_service = False
|
||||
|
||||
systemd.service(
|
||||
name="Setup acmetool-reconcile timer",
|
||||
service="acmetool-reconcile.timer",
|
||||
running=True,
|
||||
enabled=True,
|
||||
daemon_reload=self.need_restart_reconcile_timer,
|
||||
)
|
||||
self.need_restart_reconcile_timer = False
|
||||
self.ensure_service("acmetool-redirector.service")
|
||||
self.ensure_service("acmetool-reconcile.service", running=False, enabled=False)
|
||||
self.ensure_service("acmetool-reconcile.timer")
|
||||
|
||||
server.shell(
|
||||
name=f"Reconcile certificates for: {', '.join(self.domains)}",
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
"acme-enter-email": "{{ email }}"
|
||||
"acme-agreement:https://letsencrypt.org/documents/LE-SA-v1.6-August-18-2025.pdf": true
|
||||
"acme-agreement:https://letsencrypt.org/documents/LE-SA-v1.8-July-06-2026.pdf": true
|
||||
|
||||
@@ -3,6 +3,9 @@ import io
|
||||
import os
|
||||
from contextlib import contextmanager
|
||||
|
||||
from pyinfra import host
|
||||
from pyinfra.facts.files import Sha256File
|
||||
from pyinfra.facts.server import Command
|
||||
from pyinfra.operations import files, server, systemd
|
||||
|
||||
|
||||
@@ -11,6 +14,17 @@ def has_systemd():
|
||||
return os.path.isdir("/run/systemd/system")
|
||||
|
||||
|
||||
def is_in_container() -> bool:
|
||||
"""Return True if running inside a container (Docker, LXC, etc.)."""
|
||||
return (
|
||||
host.get_fact(
|
||||
Command,
|
||||
"systemd-detect-virt --container --quiet 2>/dev/null && echo yes || true",
|
||||
)
|
||||
== "yes"
|
||||
)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def blocked_service_startup():
|
||||
"""Prevent services from auto-starting during package installation.
|
||||
@@ -37,11 +51,10 @@ def get_resource(arg, pkg=__package__):
|
||||
return importlib.resources.files(pkg).joinpath(arg)
|
||||
|
||||
|
||||
def configure_remote_units(mail_domain, units) -> None:
|
||||
def configure_remote_units(deployer, mail_domain, units) -> None:
|
||||
remote_base_dir = "/usr/local/lib/chatmaild"
|
||||
remote_venv_dir = f"{remote_base_dir}/venv"
|
||||
remote_chatmail_inipath = f"{remote_base_dir}/chatmail.ini"
|
||||
root_owned = dict(user="root", group="root", mode="644")
|
||||
|
||||
# install systemd units
|
||||
for fn in units:
|
||||
@@ -57,15 +70,13 @@ def configure_remote_units(mail_domain, units) -> None:
|
||||
source_path = get_resource(f"service/{basename}.f")
|
||||
content = source_path.read_text().format(**params).encode()
|
||||
|
||||
files.put(
|
||||
name=f"Upload {basename}",
|
||||
deployer.put_file(
|
||||
src=io.BytesIO(content),
|
||||
dest=f"/etc/systemd/system/{basename}",
|
||||
**root_owned,
|
||||
)
|
||||
|
||||
|
||||
def activate_remote_units(units) -> None:
|
||||
def activate_remote_units(deployer, units) -> None:
|
||||
# activate systemd units
|
||||
for fn in units:
|
||||
basename = fn if "." in fn else f"{fn}.service"
|
||||
@@ -75,14 +86,8 @@ def activate_remote_units(units) -> None:
|
||||
enabled = False
|
||||
else:
|
||||
enabled = True
|
||||
systemd.service(
|
||||
name=f"Setup {basename}",
|
||||
service=basename,
|
||||
running=enabled,
|
||||
enabled=enabled,
|
||||
restarted=enabled,
|
||||
daemon_reload=True,
|
||||
)
|
||||
|
||||
deployer.ensure_service(basename, running=enabled, enabled=enabled)
|
||||
|
||||
|
||||
class Deployment:
|
||||
@@ -128,6 +133,7 @@ class Deployment:
|
||||
|
||||
class Deployer:
|
||||
need_restart = False
|
||||
daemon_reload = False
|
||||
|
||||
def install(self):
|
||||
pass
|
||||
@@ -137,3 +143,114 @@ class Deployer:
|
||||
|
||||
def activate(self):
|
||||
pass
|
||||
|
||||
def ensure_service(self, service, running=True, enabled=True):
|
||||
if running:
|
||||
verb = "Start and enable"
|
||||
else:
|
||||
verb = "Stop"
|
||||
systemd.service(
|
||||
name=f"{verb} {service}",
|
||||
service=service,
|
||||
running=running,
|
||||
enabled=enabled,
|
||||
restarted=self.need_restart if running else False,
|
||||
daemon_reload=self.daemon_reload,
|
||||
)
|
||||
self.daemon_reload = False
|
||||
|
||||
def ensure_systemd_unit(self, src, **kwargs):
|
||||
dest_name = src.split("/")[-1].replace(".j2", "")
|
||||
dest = f"/etc/systemd/system/{dest_name}"
|
||||
if src.endswith(".j2"):
|
||||
return self.put_template(src, dest, **kwargs)
|
||||
return self.put_file(src, dest)
|
||||
|
||||
def put_file(self, src, dest, mode="644", **kwargs):
|
||||
if isinstance(src, str):
|
||||
src = get_resource(src)
|
||||
res = files.put(
|
||||
name=f"Upload {dest}",
|
||||
src=src,
|
||||
dest=dest,
|
||||
user="root",
|
||||
group="root",
|
||||
mode=mode,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
return self._update_restart_signals(dest, res)
|
||||
|
||||
def put_executable(self, src, dest):
|
||||
return self.put_file(src, dest, mode="755")
|
||||
|
||||
def put_template(self, src, dest, owner="root", **kwargs):
|
||||
if isinstance(src, str):
|
||||
src = get_resource(src)
|
||||
res = files.template(
|
||||
name=f"Upload {dest}",
|
||||
src=src,
|
||||
dest=dest,
|
||||
user=owner,
|
||||
group=owner,
|
||||
mode="644",
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
return self._update_restart_signals(dest, res)
|
||||
|
||||
def remove_file(self, dest):
|
||||
res = files.file(name=f"Remove {dest}", path=dest, present=False)
|
||||
return self._update_restart_signals(dest, res)
|
||||
|
||||
def ensure_line(self, path, line, **kwargs):
|
||||
name = kwargs.pop("name", f"Ensure line in {path}")
|
||||
res = files.line(name=name, path=path, line=line, **kwargs)
|
||||
return self._update_restart_signals(path, res)
|
||||
|
||||
def ensure_directory(self, path, owner="root", mode="755", **kwargs):
|
||||
name = kwargs.pop("name", f"Ensure directory {path}")
|
||||
res = files.directory(
|
||||
name=name,
|
||||
path=path,
|
||||
user=owner,
|
||||
group=owner,
|
||||
mode=mode,
|
||||
present=True,
|
||||
**kwargs,
|
||||
)
|
||||
return self._update_restart_signals(path, res)
|
||||
|
||||
def remove_directory(self, path, **kwargs):
|
||||
name = kwargs.pop("name", f"Remove directory {path}")
|
||||
res = files.directory(name=name, path=path, present=False, **kwargs)
|
||||
return self._update_restart_signals(path, res)
|
||||
|
||||
def download_executable(self, url, dest, sha256sum, extract=None):
|
||||
existing = host.get_fact(Sha256File, dest)
|
||||
if existing == sha256sum:
|
||||
return
|
||||
|
||||
tmp = f"{dest}.new"
|
||||
if extract:
|
||||
dl_cmd = f"curl -fSL {url} | {extract} >{tmp}"
|
||||
else:
|
||||
dl_cmd = f"curl -fSL {url} -o {tmp}"
|
||||
|
||||
server.shell(
|
||||
name=f"Download {dest}",
|
||||
commands=[
|
||||
f"({dl_cmd}"
|
||||
f" && echo '{sha256sum} {tmp}' | sha256sum -c"
|
||||
f" && mv {tmp} {dest})",
|
||||
f"chmod 755 {dest}",
|
||||
],
|
||||
)
|
||||
self.need_restart = True
|
||||
|
||||
def _update_restart_signals(self, path, res):
|
||||
if res.changed:
|
||||
self.need_restart = True
|
||||
if str(path).startswith("/etc/systemd/system/"):
|
||||
self.daemon_reload = True
|
||||
return res
|
||||
|
||||
@@ -15,27 +15,10 @@ from pathlib import Path
|
||||
import pyinfra
|
||||
from chatmaild.config import read_config, write_initial_config
|
||||
from packaging import version
|
||||
from termcolor import colored
|
||||
|
||||
from . import dns, remote
|
||||
from .lxc.cli import (
|
||||
lxc_start_cmd,
|
||||
lxc_start_cmd_options,
|
||||
lxc_status_cmd,
|
||||
lxc_status_cmd_options,
|
||||
lxc_stop_cmd,
|
||||
lxc_stop_cmd_options,
|
||||
lxc_test_cmd,
|
||||
lxc_test_cmd_options,
|
||||
)
|
||||
from .lxc.incus import DNSConfigurationError
|
||||
from .sshexec import (
|
||||
LocalExec,
|
||||
SSHExec,
|
||||
resolve_host_from_ssh_config,
|
||||
resolve_key_from_ssh_config,
|
||||
)
|
||||
from .util import Out
|
||||
from .www import main as webdev_main
|
||||
from .sshexec import LocalExec, SSHExec
|
||||
|
||||
#
|
||||
# cmdeploy sub commands and options
|
||||
@@ -99,21 +82,29 @@ def run_cmd_options(parser):
|
||||
help="disable checks nslookup for dns",
|
||||
)
|
||||
add_ssh_host_option(parser)
|
||||
add_ssh_config_option(parser)
|
||||
|
||||
|
||||
def _warn_unused_settings(unused_keys, out):
|
||||
if unused_keys:
|
||||
names = ", ".join(unused_keys)
|
||||
out.red(
|
||||
f"WARNING: chatmail.ini contains settings that have no effect: {names}\n"
|
||||
"Please remove them from chatmail.ini."
|
||||
)
|
||||
|
||||
|
||||
def run_cmd(args, out):
|
||||
"""Deploy chatmail services on the remote server."""
|
||||
|
||||
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
|
||||
sshexec = get_sshexec(ssh_host, ssh_config=args.ssh_config)
|
||||
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain_bare
|
||||
sshexec = get_sshexec(ssh_host)
|
||||
require_iroh = args.config.enable_iroh_relay
|
||||
strict_tls = args.config.tls_cert_mode == "acme"
|
||||
if args.config.ipv4_relay:
|
||||
args.dns_check_disabled = True
|
||||
if not args.dns_check_disabled:
|
||||
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
|
||||
if not dns.check_initial_remote_data(
|
||||
remote_data, strict_tls=strict_tls, print=out.red
|
||||
):
|
||||
if not dns.check_initial_remote_data(remote_data, strict_tls=strict_tls, print=out.red):
|
||||
return 1
|
||||
|
||||
env = os.environ.copy()
|
||||
@@ -121,31 +112,11 @@ def run_cmd(args, out):
|
||||
env["CHATMAIL_WEBSITE_ONLY"] = "True" if args.website_only else ""
|
||||
env["CHATMAIL_DISABLE_MAIL"] = "True" if args.disable_mail else ""
|
||||
env["CHATMAIL_REQUIRE_IROH"] = "True" if require_iroh else ""
|
||||
if not args.dns_check_disabled:
|
||||
env["CHATMAIL_ADDR_V4"] = remote_data.get("A") or ""
|
||||
env["CHATMAIL_ADDR_V6"] = remote_data.get("AAAA") or ""
|
||||
env["DEBIAN_FRONTEND"] = "noninteractive"
|
||||
env["TERM"] = "linux"
|
||||
deploy_path = importlib.resources.files(__package__).joinpath("run.py").resolve()
|
||||
pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
|
||||
|
||||
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
|
||||
ssh_config = args.ssh_config
|
||||
if ssh_config:
|
||||
ssh_config = str(Path(ssh_config).resolve())
|
||||
|
||||
# Use pyinfra's native SSH data keys to configure the connection directly
|
||||
# rather than relying on paramiko config parsing (see also sshexec.py)
|
||||
ip = resolve_host_from_ssh_config(ssh_host, ssh_config)
|
||||
key = resolve_key_from_ssh_config(ssh_host, ssh_config)
|
||||
data_args = f"--data ssh_hostname={ip} --data ssh_known_hosts_file=/dev/null"
|
||||
if key:
|
||||
data_args += f" --data ssh_key={key}"
|
||||
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y {data_args}"
|
||||
if ssh_host in ["localhost", "@docker"]:
|
||||
if ssh_host == "@docker":
|
||||
env["CHATMAIL_NOPORTCHECK"] = "True"
|
||||
env["CHATMAIL_NOSYSCTL"] = "True"
|
||||
if ssh_host == "localhost":
|
||||
cmd = f"{pyinf} @local {deploy_path} -y"
|
||||
|
||||
if version.parse(pyinfra.__version__) < version.parse("3"):
|
||||
@@ -153,21 +124,17 @@ def run_cmd(args, out):
|
||||
return 1
|
||||
|
||||
try:
|
||||
ret = out.shell(cmd, env=env)
|
||||
if ret:
|
||||
out.red("Deploy failed")
|
||||
return 1
|
||||
out.check_call(cmd, env=env)
|
||||
if args.website_only:
|
||||
out.green("Website deployment completed.")
|
||||
elif (
|
||||
not args.dns_check_disabled
|
||||
and strict_tls
|
||||
and not remote_data["acme_account_url"]
|
||||
):
|
||||
elif not args.dns_check_disabled and strict_tls and not remote_data["acme_account_url"]:
|
||||
out.red("Deploy completed but letsencrypt not configured")
|
||||
out.red("Run 'cmdeploy run' again")
|
||||
elif args.config.ipv4_relay:
|
||||
out.green("Deploy completed.")
|
||||
else:
|
||||
out.green("Deploy completed, call `cmdeploy dns` next.")
|
||||
_warn_unused_settings(args.config._unused_keys, out)
|
||||
return 0
|
||||
except subprocess.CalledProcessError:
|
||||
out.red("Deploy failed")
|
||||
@@ -180,16 +147,19 @@ def dns_cmd_options(parser):
|
||||
dest="zonefile",
|
||||
type=pathlib.Path,
|
||||
default=None,
|
||||
help="write DNS records in standard BIND format to the given file",
|
||||
help="write out a zonefile",
|
||||
)
|
||||
add_ssh_host_option(parser)
|
||||
add_ssh_config_option(parser)
|
||||
|
||||
|
||||
def dns_cmd(args, out):
|
||||
"""Check DNS entries and optionally generate dns zone file."""
|
||||
if args.config.ipv4_relay:
|
||||
ipv4 = args.config.ipv4_relay
|
||||
print(f"[WARNING] {ipv4} is not a domain, skipping DNS checks.")
|
||||
return 0
|
||||
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
|
||||
sshexec = get_sshexec(ssh_host, verbose=args.verbose, ssh_config=args.ssh_config)
|
||||
sshexec = get_sshexec(ssh_host, verbose=args.verbose)
|
||||
tls_cert_mode = args.config.tls_cert_mode
|
||||
strict_tls = tls_cert_mode == "acme"
|
||||
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
|
||||
@@ -220,14 +190,13 @@ def dns_cmd(args, out):
|
||||
|
||||
def status_cmd_options(parser):
|
||||
add_ssh_host_option(parser)
|
||||
add_ssh_config_option(parser)
|
||||
|
||||
|
||||
def status_cmd(args, out):
|
||||
"""Display status for online chatmail instance."""
|
||||
|
||||
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
|
||||
sshexec = get_sshexec(ssh_host, verbose=args.verbose, ssh_config=args.ssh_config)
|
||||
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain_bare
|
||||
sshexec = get_sshexec(ssh_host, verbose=args.verbose)
|
||||
|
||||
out.green(f"chatmail domain: {args.config.mail_domain}")
|
||||
if args.config.privacy_mail:
|
||||
@@ -240,21 +209,16 @@ def status_cmd(args, out):
|
||||
|
||||
|
||||
def test_cmd_options(parser):
|
||||
parser.add_argument(
|
||||
"--slow",
|
||||
dest="slow",
|
||||
action="store_true",
|
||||
help="also run slow tests",
|
||||
)
|
||||
add_ssh_host_option(parser)
|
||||
add_ssh_config_option(parser)
|
||||
|
||||
|
||||
def test_cmd(args, out):
|
||||
"""Run local and online tests for chatmail deployment."""
|
||||
|
||||
env = os.environ.copy()
|
||||
env["CHATMAIL_INI"] = str(args.inipath.resolve())
|
||||
env["CHATMAIL_INI"] = str(args.inipath.absolute())
|
||||
if args.ssh_host:
|
||||
env["CHATMAIL_SSH"] = args.ssh_host
|
||||
|
||||
pytest_path = shutil.which("pytest")
|
||||
pytest_args = [
|
||||
@@ -266,13 +230,7 @@ def test_cmd(args, out):
|
||||
"-v",
|
||||
"--durations=5",
|
||||
]
|
||||
if args.slow:
|
||||
pytest_args.append("--slow")
|
||||
if args.ssh_host:
|
||||
pytest_args.extend(["--ssh-host", args.ssh_host])
|
||||
if args.ssh_config:
|
||||
pytest_args.extend(["--ssh-config", str(Path(args.ssh_config).resolve())])
|
||||
ret = out.shell(" ".join(pytest_args), env=env)
|
||||
ret = out.run_ret(pytest_args, env=env)
|
||||
return ret
|
||||
|
||||
|
||||
@@ -309,8 +267,8 @@ def fmt_cmd(args, out):
|
||||
format_args.extend(sources)
|
||||
check_args.extend(sources)
|
||||
|
||||
out.shell(" ".join(format_args), quiet=not args.verbose)
|
||||
out.shell(" ".join(check_args), quiet=not args.verbose)
|
||||
out.check_call(" ".join(format_args), quiet=not args.verbose)
|
||||
out.check_call(" ".join(check_args), quiet=not args.verbose)
|
||||
|
||||
|
||||
def bench_cmd(args, out):
|
||||
@@ -323,7 +281,9 @@ def bench_cmd(args, out):
|
||||
|
||||
def webdev_cmd(args, out):
|
||||
"""Run local web development loop for static web pages."""
|
||||
webdev_main()
|
||||
from .www import main
|
||||
|
||||
main()
|
||||
|
||||
|
||||
#
|
||||
@@ -331,25 +291,41 @@ def webdev_cmd(args, out):
|
||||
#
|
||||
|
||||
|
||||
class Out:
|
||||
"""Convenience output printer providing coloring."""
|
||||
|
||||
def red(self, msg, file=sys.stderr):
|
||||
print(colored(msg, "red"), file=file)
|
||||
|
||||
def green(self, msg, file=sys.stderr):
|
||||
print(colored(msg, "green"), file=file)
|
||||
|
||||
def __call__(self, msg, red=False, green=False, file=sys.stdout):
|
||||
color = "red" if red else ("green" if green else None)
|
||||
print(colored(msg, color), file=file)
|
||||
|
||||
def check_call(self, arg, env=None, quiet=False):
|
||||
if not quiet:
|
||||
self(f"[$ {arg}]", file=sys.stderr)
|
||||
return subprocess.check_call(arg, shell=True, env=env)
|
||||
|
||||
def run_ret(self, args, env=None, quiet=False):
|
||||
if not quiet:
|
||||
cmdstring = " ".join(args)
|
||||
self(f"[$ {cmdstring}]", file=sys.stderr)
|
||||
proc = subprocess.run(args, env=env, check=False)
|
||||
return proc.returncode
|
||||
|
||||
|
||||
def add_ssh_host_option(parser):
|
||||
parser.add_argument(
|
||||
"--ssh-host",
|
||||
dest="ssh_host",
|
||||
help="Run commands on 'localhost', via '@docker', or on a specific SSH host "
|
||||
help="Run commands on 'localhost' or on a specific SSH host "
|
||||
"instead of chatmail.ini's mail_domain.",
|
||||
)
|
||||
|
||||
|
||||
def add_ssh_config_option(parser):
|
||||
parser.add_argument(
|
||||
"--ssh-config",
|
||||
dest="ssh_config",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Path to an SSH config file (e.g. lxconfigs/ssh-config).",
|
||||
)
|
||||
|
||||
|
||||
def add_config_option(parser):
|
||||
parser.add_argument(
|
||||
"--config",
|
||||
@@ -359,26 +335,25 @@ def add_config_option(parser):
|
||||
type=Path,
|
||||
help="path to the chatmail.ini file",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose",
|
||||
"-v",
|
||||
dest="verbose",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="provide verbose logging",
|
||||
)
|
||||
|
||||
|
||||
def add_subcommand(subparsers, func, add_config=True):
|
||||
def add_subcommand(subparsers, func):
|
||||
name = func.__name__
|
||||
assert name.endswith("_cmd")
|
||||
name = name[:-4].replace("_", "-")
|
||||
name = name[:-4]
|
||||
doc = func.__doc__.strip()
|
||||
help = doc.split("\n")[0].strip(".")
|
||||
p = subparsers.add_parser(name, description=doc, help=help)
|
||||
p.set_defaults(func=func)
|
||||
if add_config:
|
||||
add_config_option(p)
|
||||
p.add_argument(
|
||||
"-v",
|
||||
"--verbose",
|
||||
dest="verbose",
|
||||
action="count",
|
||||
default=0,
|
||||
help="increase verbosity (can be repeated: -v, -vv)",
|
||||
)
|
||||
add_config_option(p)
|
||||
return p
|
||||
|
||||
|
||||
@@ -387,60 +362,43 @@ Setup your chatmail server configuration and
|
||||
deploy it via SSH to your remote location.
|
||||
"""
|
||||
|
||||
# Explicit subcommand registry: (cmd_func, options_func_or_None, needs_config).
|
||||
# LXC commands don't need a chatmail.ini (no config); all others do.
|
||||
SUBCOMMANDS = [
|
||||
(init_cmd, init_cmd_options, True),
|
||||
(run_cmd, run_cmd_options, True),
|
||||
(dns_cmd, dns_cmd_options, True),
|
||||
(status_cmd, status_cmd_options, True),
|
||||
(test_cmd, test_cmd_options, True),
|
||||
(fmt_cmd, fmt_cmd_options, True),
|
||||
(bench_cmd, None, True),
|
||||
(webdev_cmd, None, True),
|
||||
(lxc_start_cmd, lxc_start_cmd_options, False),
|
||||
(lxc_stop_cmd, lxc_stop_cmd_options, False),
|
||||
(lxc_status_cmd, lxc_status_cmd_options, False),
|
||||
(lxc_test_cmd, lxc_test_cmd_options, False),
|
||||
]
|
||||
|
||||
|
||||
def get_parser():
|
||||
"""Return an ArgumentParser for the 'cmdeploy' CLI"""
|
||||
|
||||
parser = argparse.ArgumentParser(description=description.strip())
|
||||
parser.set_defaults(func=None, inipath=None)
|
||||
subparsers = parser.add_subparsers(title="subcommands")
|
||||
|
||||
for func, addopts, needs_config in SUBCOMMANDS:
|
||||
subparser = add_subcommand(subparsers, func, add_config=needs_config)
|
||||
if addopts is not None:
|
||||
addopts(subparser)
|
||||
# find all subcommands in the module namespace
|
||||
glob = globals()
|
||||
for name, func in glob.items():
|
||||
if name.endswith("_cmd"):
|
||||
subparser = add_subcommand(subparsers, func)
|
||||
addopts = glob.get(name + "_options")
|
||||
if addopts is not None:
|
||||
addopts(subparser)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def get_sshexec(ssh_host: str, verbose=True, ssh_config=None):
|
||||
def get_sshexec(ssh_host: str, verbose=True):
|
||||
if ssh_host in ["localhost", "@local"]:
|
||||
return LocalExec(verbose, docker=False)
|
||||
elif ssh_host == "@docker":
|
||||
return LocalExec(verbose, docker=True)
|
||||
return LocalExec(verbose)
|
||||
if verbose:
|
||||
print(f"[ssh] login to {ssh_host}")
|
||||
return SSHExec(ssh_host, verbose=verbose, ssh_config=ssh_config)
|
||||
return SSHExec(ssh_host, verbose=verbose)
|
||||
|
||||
|
||||
def main(args=None):
|
||||
"""Provide main entry point for 'cmdeploy' CLI invocation."""
|
||||
parser = get_parser()
|
||||
args = parser.parse_args(args=args)
|
||||
if args.func is None:
|
||||
if not hasattr(args, "func"):
|
||||
return parser.parse_args(["-h"])
|
||||
|
||||
out = Out(verbosity=args.verbose)
|
||||
out = Out()
|
||||
kwargs = {}
|
||||
|
||||
if args.inipath is not None and args.func.__name__ not in ("init_cmd", "fmt_cmd"):
|
||||
if args.func.__name__ not in ("init_cmd", "fmt_cmd"):
|
||||
if not args.inipath.exists():
|
||||
out.red(f"expecting {args.inipath} to exist, run init first?")
|
||||
raise SystemExit(1)
|
||||
@@ -455,9 +413,6 @@ def main(args=None):
|
||||
if res is None:
|
||||
res = 0
|
||||
return res
|
||||
except DNSConfigurationError as exc:
|
||||
out.red(str(exc))
|
||||
return 1
|
||||
except KeyboardInterrupt:
|
||||
out.red("KeyboardInterrupt")
|
||||
sys.exit(130)
|
||||
|
||||
+120
-215
@@ -2,7 +2,6 @@
|
||||
Chat Mail pyinfra deploy.
|
||||
"""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
@@ -13,10 +12,11 @@ from chatmaild.config import read_config
|
||||
from pyinfra import facts, host, logger
|
||||
from pyinfra.api import FactBase
|
||||
from pyinfra.facts import hardware
|
||||
from pyinfra.facts.files import Sha256File
|
||||
from pyinfra.facts.systemd import SystemdEnabled
|
||||
from pyinfra.operations import apt, files, pip, server, systemd
|
||||
|
||||
from cmdeploy.cmdeploy import Out
|
||||
|
||||
from .acmetool import AcmetoolDeployer
|
||||
from .basedeploy import (
|
||||
Deployer,
|
||||
@@ -24,8 +24,8 @@ from .basedeploy import (
|
||||
activate_remote_units,
|
||||
blocked_service_startup,
|
||||
configure_remote_units,
|
||||
get_resource,
|
||||
has_systemd,
|
||||
is_in_container,
|
||||
)
|
||||
from .dovecot.deployer import DovecotDeployer
|
||||
from .external.deployer import ExternalTlsDeployer
|
||||
@@ -35,7 +35,6 @@ from .nginx.deployer import NginxDeployer
|
||||
from .opendkim.deployer import OpendkimDeployer
|
||||
from .postfix.deployer import PostfixDeployer
|
||||
from .selfsigned.deployer import SelfSignedTlsDeployer
|
||||
from .util import Out, get_version_string
|
||||
from .www import build_webpages, find_merge_conflict, get_paths
|
||||
|
||||
|
||||
@@ -81,25 +80,22 @@ def remove_legacy_artifacts():
|
||||
)
|
||||
|
||||
|
||||
def _install_remote_venv_with_chatmaild() -> None:
|
||||
def _install_remote_venv_with_chatmaild(deployer) -> None:
|
||||
remove_legacy_artifacts()
|
||||
dist_file = _build_chatmaild(dist_dir=Path("chatmaild/dist"))
|
||||
remote_base_dir = "/usr/local/lib/chatmaild"
|
||||
remote_dist_file = f"{remote_base_dir}/dist/{dist_file.name}"
|
||||
remote_venv_dir = f"{remote_base_dir}/venv"
|
||||
root_owned = dict(user="root", group="root", mode="644")
|
||||
|
||||
apt.packages(
|
||||
name="apt install python3-virtualenv",
|
||||
packages=["python3-virtualenv"],
|
||||
)
|
||||
|
||||
files.put(
|
||||
name="Upload chatmaild source package",
|
||||
deployer.ensure_directory(f"{remote_base_dir}/dist")
|
||||
deployer.put_file(
|
||||
src=dist_file.open("rb"),
|
||||
dest=remote_dist_file,
|
||||
create_remote_dir=True,
|
||||
**root_owned,
|
||||
)
|
||||
|
||||
pip.virtualenv(
|
||||
@@ -121,73 +117,76 @@ def _install_remote_venv_with_chatmaild() -> None:
|
||||
)
|
||||
|
||||
|
||||
def _configure_remote_venv_with_chatmaild(config) -> None:
|
||||
def _configure_remote_venv_with_chatmaild(deployer, config) -> None:
|
||||
remote_base_dir = "/usr/local/lib/chatmaild"
|
||||
remote_chatmail_inipath = f"{remote_base_dir}/chatmail.ini"
|
||||
root_owned = dict(user="root", group="root", mode="644")
|
||||
|
||||
files.put(
|
||||
name=f"Upload {remote_chatmail_inipath}",
|
||||
deployer.put_file(
|
||||
src=config._getbytefile(),
|
||||
dest=remote_chatmail_inipath,
|
||||
**root_owned,
|
||||
)
|
||||
|
||||
files.file(
|
||||
path="/etc/cron.d/chatmail-metrics",
|
||||
present=False,
|
||||
)
|
||||
files.file(
|
||||
path="/var/www/html/metrics",
|
||||
present=False,
|
||||
)
|
||||
deployer.remove_file("/etc/cron.d/chatmail-metrics")
|
||||
deployer.remove_file("/var/www/html/metrics")
|
||||
|
||||
|
||||
class UnboundDeployer(Deployer):
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
self.need_restart = False
|
||||
|
||||
def install(self):
|
||||
# Run local DNS resolver `unbound`. `resolvconf` takes care of
|
||||
# setting up /etc/resolv.conf to use 127.0.0.1 as the resolver.
|
||||
|
||||
# On an IPv4-only system, if unbound is started but not configured,
|
||||
# it causes subsequent steps to fail to resolve hosts.
|
||||
with blocked_service_startup():
|
||||
# dns-root-data is an optional package
|
||||
# that contains /usr/share/dns/root.key
|
||||
#
|
||||
# This file is copied into /var/lib/unbound/root.key
|
||||
# at the start of "unbound" systemd unit
|
||||
# by /usr/libexec/unbound-helper shell script
|
||||
# from the "unbound" package as of version 1.17.1-2+deb12u4
|
||||
#
|
||||
# The same /var/lib/unbound/root.key can be retrieved directly
|
||||
# following the procedure from
|
||||
# <https://www.rfc-editor.org/info/rfc7958/#section-3.1>
|
||||
# with "unbound-anchor -a /var/lib/unbound/root.key"
|
||||
# We don't install and use "unbound-anchor".
|
||||
apt.packages(
|
||||
name="Install unbound",
|
||||
packages=["unbound", "unbound-anchor", "dnsutils"],
|
||||
packages=["unbound", "dns-root-data", "dnsutils"],
|
||||
)
|
||||
|
||||
def configure(self):
|
||||
# Remove dynamic resolver managers that compete for /etc/resolv.conf.
|
||||
apt.packages(
|
||||
name="Purge resolvconf",
|
||||
packages=["resolvconf"],
|
||||
present=False,
|
||||
extra_uninstall_args="--purge",
|
||||
)
|
||||
# systemd-resolved can't be purged due to dependencies; stop and mask.
|
||||
server.shell(
|
||||
name="Generate root keys for validating DNSSEC",
|
||||
name="Stop and mask systemd-resolved",
|
||||
commands=[
|
||||
"unbound-anchor -a /var/lib/unbound/root.key || true",
|
||||
"systemctl stop systemd-resolved.service || true",
|
||||
"systemctl mask systemd-resolved.service",
|
||||
],
|
||||
)
|
||||
if self.config.disable_ipv6:
|
||||
files.directory(
|
||||
path="/etc/unbound/unbound.conf.d",
|
||||
present=True,
|
||||
user="root",
|
||||
group="root",
|
||||
mode="755",
|
||||
)
|
||||
conf = files.put(
|
||||
src=get_resource("unbound/unbound.conf.j2"),
|
||||
dest="/etc/unbound/unbound.conf.d/chatmail.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
else:
|
||||
conf = files.file(
|
||||
path="/etc/unbound/unbound.conf.d/chatmail.conf",
|
||||
present=False,
|
||||
)
|
||||
self.need_restart |= conf.changed
|
||||
# Configure unbound resolver with Quad9 fallback and a trailing newline
|
||||
# (SolusVM bug).
|
||||
self.put_file(
|
||||
src=BytesIO(b"nameserver 127.0.0.1\nnameserver 9.9.9.9\n"),
|
||||
dest="/etc/resolv.conf",
|
||||
force=True,
|
||||
)
|
||||
self.ensure_directory(
|
||||
path="/etc/unbound/unbound.conf.d",
|
||||
)
|
||||
self.put_template(
|
||||
"unbound/unbound.conf.j2",
|
||||
"/etc/unbound/unbound.conf.d/chatmail.conf",
|
||||
disable_ipv6=self.config.disable_ipv6,
|
||||
)
|
||||
|
||||
def activate(self):
|
||||
server.shell(
|
||||
@@ -197,27 +196,25 @@ class UnboundDeployer(Deployer):
|
||||
],
|
||||
)
|
||||
|
||||
systemd.service(
|
||||
name="Start and enable unbound",
|
||||
service="unbound.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=self.need_restart,
|
||||
self.ensure_service("unbound.service")
|
||||
|
||||
self.ensure_service(
|
||||
"unbound-resolvconf.service",
|
||||
running=False,
|
||||
enabled=False,
|
||||
)
|
||||
|
||||
|
||||
class MtastsDeployer(Deployer):
|
||||
def configure(self):
|
||||
# Remove configuration.
|
||||
files.file("/etc/mta-sts-daemon.yml", present=False)
|
||||
files.directory("/usr/local/lib/postfix-mta-sts-resolver", present=False)
|
||||
files.file("/etc/systemd/system/mta-sts-daemon.service", present=False)
|
||||
self.remove_file("/etc/mta-sts-daemon.yml")
|
||||
self.remove_directory("/usr/local/lib/postfix-mta-sts-resolver")
|
||||
self.remove_file("/etc/systemd/system/mta-sts-daemon.service")
|
||||
|
||||
def activate(self):
|
||||
systemd.service(
|
||||
name="Stop MTA-STS daemon",
|
||||
service="mta-sts-daemon.service",
|
||||
daemon_reload=True,
|
||||
self.ensure_service(
|
||||
"mta-sts-daemon.service",
|
||||
running=False,
|
||||
enabled=False,
|
||||
)
|
||||
@@ -228,14 +225,7 @@ class WebsiteDeployer(Deployer):
|
||||
self.config = config
|
||||
|
||||
def install(self):
|
||||
files.directory(
|
||||
name="Ensure /var/www exists",
|
||||
path="/var/www",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="755",
|
||||
present=True,
|
||||
)
|
||||
self.ensure_directory("/var/www")
|
||||
|
||||
def configure(self):
|
||||
www_path, src_dir, build_dir = get_paths(self.config)
|
||||
@@ -254,14 +244,8 @@ class WebsiteDeployer(Deployer):
|
||||
logger.warning("Web page build failed, skipping website deployment")
|
||||
return
|
||||
# if it is not a hugo page, upload it as is
|
||||
# pyinfra files.rsync (experimental) causes problems with ssh-config configuration
|
||||
# the stable files.sync should do
|
||||
files.sync(
|
||||
src=str(www_path),
|
||||
dest="/var/www/html",
|
||||
user="www-data",
|
||||
group="www-data",
|
||||
delete=True,
|
||||
files.rsync(
|
||||
f"{www_path}/", "/var/www/html", flags=["-avz", "--chown=www-data"]
|
||||
)
|
||||
|
||||
|
||||
@@ -269,17 +253,20 @@ class LegacyRemoveDeployer(Deployer):
|
||||
def install(self):
|
||||
apt.packages(name="Remove rspamd", packages="rspamd", present=False)
|
||||
|
||||
# unbound-anchor was used to download /var/lib/unbound/root.key
|
||||
# It is replaced by dns-root-data which contains /usr/share/dns/root.key.
|
||||
# unbound systemd unit copies /usr/share/dns/root.key
|
||||
# into /var/lib/unbound/root.key automatically on start
|
||||
# as long as /usr/share/dns/root.key is present.
|
||||
apt.packages(name="Remove unbound-anchor", packages="unbound-anchor", present=False)
|
||||
|
||||
# remove historic expunge script
|
||||
# which is now implemented through a systemd timer (chatmail-expire)
|
||||
files.file(
|
||||
path="/etc/cron.d/expunge",
|
||||
present=False,
|
||||
)
|
||||
self.remove_file("/etc/cron.d/expunge")
|
||||
|
||||
# Remove OBS repository key that is no longer used.
|
||||
files.file("/etc/apt/keyrings/obs-home-deltachat.gpg", present=False)
|
||||
files.line(
|
||||
name="Remove DeltaChat OBS home repository from sources.list",
|
||||
self.remove_file("/etc/apt/keyrings/obs-home-deltachat.gpg")
|
||||
self.ensure_line(
|
||||
path="/etc/apt/sources.list",
|
||||
line="deb [signed-by=/etc/apt/keyrings/obs-home-deltachat.gpg] https://download.opensuse.org/repositories/home:/deltachat/Debian_12/ ./",
|
||||
escape_regex_characters=True,
|
||||
@@ -287,11 +274,7 @@ class LegacyRemoveDeployer(Deployer):
|
||||
)
|
||||
|
||||
# prior relay versions used filelogging
|
||||
files.directory(
|
||||
name="Ensure old logs on disk are deleted",
|
||||
path="/var/log/journal/",
|
||||
present=False,
|
||||
)
|
||||
self.remove_directory("/var/log/journal/")
|
||||
# remove echobot if it is still running
|
||||
if has_systemd() and host.get_fact(SystemdEnabled).get("echobot.service"):
|
||||
systemd.service(
|
||||
@@ -333,22 +316,13 @@ class TurnDeployer(Deployer):
|
||||
"0fb3e792419494e21ecad536464929dba706bb2c88884ed8f1788141d26fc756",
|
||||
),
|
||||
}[host.get_fact(facts.server.Arch)]
|
||||
|
||||
existing_sha256sum = host.get_fact(Sha256File, "/usr/local/bin/chatmail-turn")
|
||||
if existing_sha256sum != sha256sum:
|
||||
server.shell(
|
||||
name="Download chatmail-turn",
|
||||
commands=[
|
||||
f"(curl -L {url} >/usr/local/bin/chatmail-turn.new && (echo '{sha256sum} /usr/local/bin/chatmail-turn.new' | sha256sum -c) && mv /usr/local/bin/chatmail-turn.new /usr/local/bin/chatmail-turn)",
|
||||
"chmod 755 /usr/local/bin/chatmail-turn",
|
||||
],
|
||||
)
|
||||
self.download_executable(url, "/usr/local/bin/chatmail-turn", sha256sum)
|
||||
|
||||
def configure(self):
|
||||
configure_remote_units(self.mail_domain, self.units)
|
||||
configure_remote_units(self, self.mail_domain, self.units)
|
||||
|
||||
def activate(self):
|
||||
activate_remote_units(self.units)
|
||||
activate_remote_units(self, self.units)
|
||||
|
||||
|
||||
class IrohDeployer(Deployer):
|
||||
@@ -366,78 +340,38 @@ class IrohDeployer(Deployer):
|
||||
"f8ef27631fac213b3ef668d02acd5b3e215292746a3fc71d90c63115446008b1",
|
||||
),
|
||||
}[host.get_fact(facts.server.Arch)]
|
||||
|
||||
existing_sha256sum = host.get_fact(Sha256File, "/usr/local/bin/iroh-relay")
|
||||
if existing_sha256sum != sha256sum:
|
||||
server.shell(
|
||||
name="Download iroh-relay",
|
||||
commands=[
|
||||
f"(curl -L {url} | gunzip | tar -x -f - ./iroh-relay -O >/usr/local/bin/iroh-relay.new && (echo '{sha256sum} /usr/local/bin/iroh-relay.new' | sha256sum -c) && mv /usr/local/bin/iroh-relay.new /usr/local/bin/iroh-relay)",
|
||||
"chmod 755 /usr/local/bin/iroh-relay",
|
||||
],
|
||||
)
|
||||
|
||||
self.need_restart = True
|
||||
self.download_executable(
|
||||
url,
|
||||
"/usr/local/bin/iroh-relay",
|
||||
sha256sum,
|
||||
extract="gunzip | tar -xf - ./iroh-relay -O",
|
||||
)
|
||||
|
||||
def configure(self):
|
||||
systemd_unit = files.put(
|
||||
name="Upload iroh-relay systemd unit",
|
||||
src=get_resource("iroh-relay.service"),
|
||||
dest="/etc/systemd/system/iroh-relay.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart |= systemd_unit.changed
|
||||
|
||||
iroh_config = files.put(
|
||||
name="Upload iroh-relay config",
|
||||
src=get_resource("iroh-relay.toml"),
|
||||
dest="/etc/iroh-relay.toml",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart |= iroh_config.changed
|
||||
self.ensure_systemd_unit("iroh-relay.service")
|
||||
self.put_file("iroh-relay.toml", "/etc/iroh-relay.toml")
|
||||
|
||||
def activate(self):
|
||||
systemd.service(
|
||||
name="Start and enable iroh-relay",
|
||||
service="iroh-relay.service",
|
||||
running=True,
|
||||
self.ensure_service(
|
||||
"iroh-relay.service",
|
||||
enabled=self.enable_iroh_relay,
|
||||
restarted=self.need_restart,
|
||||
)
|
||||
self.need_restart = False
|
||||
|
||||
|
||||
class JournaldDeployer(Deployer):
|
||||
def configure(self):
|
||||
journald_conf = files.put(
|
||||
name="Configure journald",
|
||||
src=get_resource("journald.conf"),
|
||||
dest="/etc/systemd/journald.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart = journald_conf.changed
|
||||
self.put_file("journald.conf", "/etc/systemd/journald.conf")
|
||||
|
||||
def activate(self):
|
||||
systemd.service(
|
||||
name="Start and enable journald",
|
||||
service="systemd-journald.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=self.need_restart,
|
||||
)
|
||||
self.need_restart = False
|
||||
self.ensure_service("systemd-journald.service")
|
||||
|
||||
|
||||
class ChatmailVenvDeployer(Deployer):
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
self.units = (
|
||||
# doveauth must restart when chatmaild/ini file changes
|
||||
"doveauth",
|
||||
"chatmail-metadata",
|
||||
"lastlogin",
|
||||
"chatmail-expire",
|
||||
@@ -447,14 +381,14 @@ class ChatmailVenvDeployer(Deployer):
|
||||
)
|
||||
|
||||
def install(self):
|
||||
_install_remote_venv_with_chatmaild()
|
||||
_install_remote_venv_with_chatmaild(self)
|
||||
|
||||
def configure(self):
|
||||
_configure_remote_venv_with_chatmaild(self.config)
|
||||
configure_remote_units(self.config.mail_domain, self.units)
|
||||
_configure_remote_venv_with_chatmaild(self, self.config)
|
||||
configure_remote_units(self, self.config.mail_domain_bare, self.units)
|
||||
|
||||
def activate(self):
|
||||
activate_remote_units(self.units)
|
||||
activate_remote_units(self, self.units)
|
||||
|
||||
|
||||
class ChatmailDeployer(Deployer):
|
||||
@@ -468,13 +402,9 @@ class ChatmailDeployer(Deployer):
|
||||
self.mail_domain = config.mail_domain
|
||||
|
||||
def install(self):
|
||||
files.put(
|
||||
name="Disable installing recommended packages globally",
|
||||
self.put_file(
|
||||
src=BytesIO(b'APT::Install-Recommends "false";\n'),
|
||||
dest="/etc/apt/apt.conf.d/00InstallRecommends",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
apt.update(name="apt update", cache_time=24 * 3600)
|
||||
apt.upgrade(name="upgrade apt packages", auto_remove=True)
|
||||
@@ -488,21 +418,13 @@ class ChatmailDeployer(Deployer):
|
||||
name="Install rsync",
|
||||
packages=["rsync"],
|
||||
)
|
||||
apt.packages(
|
||||
name="Ensure cron is installed",
|
||||
packages=["cron"],
|
||||
)
|
||||
|
||||
def configure(self):
|
||||
# Ensure the per-domain mailbox directory exists before
|
||||
# chatmail-metadata starts (it crashes without it).
|
||||
files.directory(
|
||||
name="Ensure vmail mailbox directory exists",
|
||||
path=f"/home/vmail/mail/{self.mail_domain}",
|
||||
user="vmail",
|
||||
group="vmail",
|
||||
# metadata crashes if the mailboxes dir does not exist
|
||||
self.ensure_directory(
|
||||
str(self.config.mailboxes_dir),
|
||||
owner="vmail",
|
||||
mode="700",
|
||||
present=True,
|
||||
)
|
||||
|
||||
# This file is used by auth proxy.
|
||||
@@ -514,15 +436,6 @@ class ChatmailDeployer(Deployer):
|
||||
],
|
||||
)
|
||||
|
||||
files.directory(
|
||||
name=f"Ensure mailboxes directory {self.config.mailboxes_dir} exists",
|
||||
path=str(self.config.mailboxes_dir),
|
||||
user="vmail",
|
||||
group="vmail",
|
||||
mode="700",
|
||||
present=True,
|
||||
)
|
||||
|
||||
|
||||
class FcgiwrapDeployer(Deployer):
|
||||
def install(self):
|
||||
@@ -532,22 +445,20 @@ class FcgiwrapDeployer(Deployer):
|
||||
)
|
||||
|
||||
def activate(self):
|
||||
systemd.service(
|
||||
name="Start and enable fcgiwrap",
|
||||
service="fcgiwrap.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
)
|
||||
self.ensure_service("fcgiwrap.service")
|
||||
|
||||
|
||||
class GithashDeployer(Deployer):
|
||||
def activate(self):
|
||||
files.put(
|
||||
name="Upload chatmail relay git commit hash",
|
||||
src=StringIO(get_version_string()),
|
||||
dest="/etc/chatmail-version",
|
||||
mode="700",
|
||||
)
|
||||
try:
|
||||
git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode()
|
||||
except Exception:
|
||||
git_hash = "unknown\n"
|
||||
try:
|
||||
git_diff = subprocess.check_output(["git", "diff"]).decode()
|
||||
except Exception:
|
||||
git_diff = ""
|
||||
self.put_file(src=StringIO(git_hash + git_diff), dest="/etc/chatmail-version")
|
||||
|
||||
|
||||
def get_tls_deployer(config, mail_domain):
|
||||
@@ -573,20 +484,12 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
|
||||
"""
|
||||
config = read_config(config_path)
|
||||
check_config(config)
|
||||
mail_domain = config.mail_domain
|
||||
bare_host = config.mail_domain_bare
|
||||
|
||||
if website_only:
|
||||
Deployment().perform_stages([WebsiteDeployer(config)])
|
||||
return
|
||||
|
||||
if host.get_fact(Port, port=53) != "unbound":
|
||||
files.line(
|
||||
name="Add 9.9.9.9 to resolv.conf",
|
||||
path="/etc/resolv.conf",
|
||||
# Guard against resolv.conf missing a trailing newline (SolusVM bug).
|
||||
line="\nnameserver 9.9.9.9",
|
||||
)
|
||||
|
||||
# Check if mtail_address interface is available (if configured)
|
||||
if config.mtail_address and config.mtail_address not in (
|
||||
"127.0.0.1",
|
||||
@@ -601,7 +504,7 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
|
||||
)
|
||||
exit(1)
|
||||
|
||||
if not os.environ.get("CHATMAIL_NOPORTCHECK"):
|
||||
if not is_in_container():
|
||||
port_services = [
|
||||
(["master", "smtpd"], 25),
|
||||
("unbound", 53),
|
||||
@@ -617,15 +520,17 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
|
||||
("nginx", 443),
|
||||
(["master", "smtpd"], 465),
|
||||
(["master", "smtpd"], 587),
|
||||
(["imap-login", "dovecot"], 993),
|
||||
(["dovecot", "imap-login"], 993),
|
||||
("iroh-relay", 3340),
|
||||
("mtail", 3903),
|
||||
("stats", 3904),
|
||||
(["dovecot", "stats"], 3904),
|
||||
("nginx", 8443),
|
||||
(["master", "smtpd"], config.postfix_reinject_port),
|
||||
(["master", "smtpd"], config.postfix_reinject_port_incoming),
|
||||
("filtermail", config.filtermail_smtp_port),
|
||||
("filtermail", config.filtermail_smtp_port_incoming),
|
||||
("filtermail", config.filtermail_http_port_incoming),
|
||||
("filtermail", config.filtermail_lmtp_port_transport),
|
||||
]
|
||||
for service, port in port_services:
|
||||
print(f"Checking if port {port} is available for {service}...")
|
||||
@@ -638,7 +543,7 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
|
||||
)
|
||||
exit(1)
|
||||
|
||||
tls_deployer = get_tls_deployer(config, mail_domain)
|
||||
tls_deployer = get_tls_deployer(config, bare_host)
|
||||
|
||||
all_deployers = [
|
||||
ChatmailDeployer(config),
|
||||
@@ -646,13 +551,13 @@ def deploy_chatmail(config_path: Path, disable_mail: bool, website_only: bool) -
|
||||
FiltermailDeployer(),
|
||||
JournaldDeployer(),
|
||||
UnboundDeployer(config),
|
||||
TurnDeployer(mail_domain),
|
||||
TurnDeployer(bare_host),
|
||||
IrohDeployer(config.enable_iroh_relay),
|
||||
tls_deployer,
|
||||
WebsiteDeployer(config),
|
||||
ChatmailVenvDeployer(config),
|
||||
MtastsDeployer(),
|
||||
OpendkimDeployer(mail_domain),
|
||||
*([] if config.ipv4_relay else [OpendkimDeployer(bare_host)]),
|
||||
# Dovecot should be started before Postfix
|
||||
# because it creates authentication socket
|
||||
# required by Postfix.
|
||||
|
||||
@@ -4,11 +4,7 @@ from . import remote
|
||||
|
||||
|
||||
def parse_zone_records(text):
|
||||
"""Yield ``(name, ttl, rtype, rdata)`` from standard BIND-format text.
|
||||
|
||||
Skips comment lines (starting with ``;``) and blank lines.
|
||||
Each record line must have the format ``name TTL IN type rdata``.
|
||||
"""
|
||||
"""Yield ``(name, ttl, rtype, rdata)`` from standard BIND-format text."""
|
||||
for raw_line in text.splitlines():
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith(";"):
|
||||
@@ -47,33 +43,36 @@ def get_filled_zone_file(remote_data):
|
||||
remote_data["sts_id"] = datetime.datetime.now().strftime("%Y%m%d%H%M")
|
||||
|
||||
d = remote_data["mail_domain"]
|
||||
|
||||
def append_record(name, rtype, rdata, ttl=3600):
|
||||
lines.append(f"{name:<40} {ttl:<6} IN {rtype:<5} {rdata}")
|
||||
|
||||
lines = ["; Required DNS entries"]
|
||||
if remote_data.get("A"):
|
||||
lines.append(f"{d}. 3600 IN A {remote_data['A']}")
|
||||
append_record(f"{d}.", "A", remote_data["A"])
|
||||
if remote_data.get("AAAA"):
|
||||
lines.append(f"{d}. 3600 IN AAAA {remote_data['AAAA']}")
|
||||
lines.append(f"{d}. 3600 IN MX 10 {d}.")
|
||||
append_record(f"{d}.", "AAAA", remote_data["AAAA"])
|
||||
append_record(f"{d}.", "MX", f"10 {d}.")
|
||||
if remote_data.get("strict_tls"):
|
||||
lines.append(
|
||||
f'_mta-sts.{d}. 3600 IN TXT "v=STSv1; id={remote_data["sts_id"]}"'
|
||||
)
|
||||
lines.append(f"mta-sts.{d}. 3600 IN CNAME {d}.")
|
||||
lines.append(f"www.{d}. 3600 IN CNAME {d}.")
|
||||
append_record(f"_mta-sts.{d}.", "TXT", f'"v=STSv1; id={remote_data["sts_id"]}"')
|
||||
append_record(f"mta-sts.{d}.", "CNAME", f"{d}.")
|
||||
append_record(f"www.{d}.", "CNAME", f"{d}.")
|
||||
lines.append(remote_data["dkim_entry"])
|
||||
lines.append("")
|
||||
lines.append("; Recommended DNS entries")
|
||||
lines.append(f'{d}. 3600 IN TXT "v=spf1 a ~all"')
|
||||
lines.append(f'_dmarc.{d}. 3600 IN TXT "v=DMARC1;p=reject;adkim=s;aspf=s"')
|
||||
append_record(f"{d}.", "TXT", '"v=spf1 a ~all"')
|
||||
append_record(f"_dmarc.{d}.", "TXT", '"v=DMARC1;p=reject;adkim=s;aspf=s"')
|
||||
if remote_data.get("acme_account_url"):
|
||||
lines.append(
|
||||
f"{d}. 3600 IN CAA 0 issue"
|
||||
f' "letsencrypt.org;accounturi={remote_data["acme_account_url"]}"'
|
||||
append_record(
|
||||
f"{d}.",
|
||||
"CAA",
|
||||
f'0 issue "letsencrypt.org;accounturi={remote_data["acme_account_url"]}"',
|
||||
)
|
||||
lines.append(f'_adsp._domainkey.{d}. 3600 IN TXT "dkim=discardable"')
|
||||
lines.append(f"_submission._tcp.{d}. 3600 IN SRV 0 1 587 {d}.")
|
||||
lines.append(f"_submissions._tcp.{d}. 3600 IN SRV 0 1 465 {d}.")
|
||||
lines.append(f"_imap._tcp.{d}. 3600 IN SRV 0 1 143 {d}.")
|
||||
lines.append(f"_imaps._tcp.{d}. 3600 IN SRV 0 1 993 {d}.")
|
||||
append_record(f"_adsp._domainkey.{d}.", "TXT", '"dkim=discardable"')
|
||||
append_record(f"_submission._tcp.{d}.", "SRV", f"0 1 587 {d}.")
|
||||
append_record(f"_submissions._tcp.{d}.", "SRV", f"0 1 465 {d}.")
|
||||
append_record(f"_imap._tcp.{d}.", "SRV", f"0 1 143 {d}.")
|
||||
append_record(f"_imaps._tcp.{d}.", "SRV", f"0 1 993 {d}.")
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
@@ -91,19 +90,19 @@ def check_full_zone(sshexec, remote_data, out, zonefile) -> int:
|
||||
if required_diff:
|
||||
out.red("Please set required DNS entries at your DNS provider:\n")
|
||||
for line in required_diff:
|
||||
out.print(line)
|
||||
out.print()
|
||||
out(line)
|
||||
out("")
|
||||
returncode = 1
|
||||
if remote_data.get("dkim_entry") in required_diff:
|
||||
out.print(
|
||||
out(
|
||||
"If the DKIM entry above does not work with your DNS provider,"
|
||||
" you can try this one:\n"
|
||||
)
|
||||
out.print(remote_data.get("web_dkim_entry") + "\n")
|
||||
out(remote_data.get("web_dkim_entry") + "\n")
|
||||
if recommended_diff:
|
||||
out.print("WARNING: these recommended DNS entries are not set:\n")
|
||||
out("WARNING: these recommended DNS entries are not set:\n")
|
||||
for line in recommended_diff:
|
||||
out.print(line)
|
||||
out(line)
|
||||
|
||||
if not (recommended_diff or required_diff):
|
||||
out.green("Great! All your DNS entries are verified and correct.")
|
||||
|
||||
@@ -1,20 +1,32 @@
|
||||
import io
|
||||
import urllib.request
|
||||
|
||||
from chatmaild.config import Config
|
||||
from pyinfra import host
|
||||
from pyinfra.facts.deb import DebPackages
|
||||
from pyinfra.facts.server import Arch, Command, Sysctl
|
||||
from pyinfra.facts.systemd import SystemdEnabled
|
||||
from pyinfra.operations import apt, files, server, systemd
|
||||
from pyinfra.operations import apt, files, server
|
||||
|
||||
from cmdeploy.basedeploy import (
|
||||
Deployer,
|
||||
activate_remote_units,
|
||||
blocked_service_startup,
|
||||
configure_remote_units,
|
||||
get_resource,
|
||||
has_systemd,
|
||||
is_in_container,
|
||||
)
|
||||
|
||||
DOVECOT_ARCHIVE_VERSION = "2.3.21+dfsg1-3"
|
||||
DOVECOT_PACKAGE_VERSION = f"1:{DOVECOT_ARCHIVE_VERSION}"
|
||||
|
||||
DOVECOT_SHA256 = {
|
||||
("core", "amd64"): "dd060706f52a306fa863d874717210b9fe10536c824afe1790eec247ded5b27d",
|
||||
("core", "arm64"): "e7548e8a82929722e973629ecc40fcfa886894cef3db88f23535149e7f730dc9",
|
||||
("imapd", "amd64"): "8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86",
|
||||
("imapd", "arm64"): "178fa877ddd5df9930e8308b518f4b07df10e759050725f8217a0c1fb3fd707f",
|
||||
("lmtpd", "amd64"): "2f69ba5e35363de50962d42cccbfe4ed8495265044e244007d7ccddad77513ab",
|
||||
("lmtpd", "arm64"): "89f52fb36524f5877a177dff4a713ba771fd3f91f22ed0af7238d495e143b38f",
|
||||
}
|
||||
|
||||
|
||||
class DovecotDeployer(Deployer):
|
||||
daemon_reload = False
|
||||
@@ -22,38 +34,63 @@ class DovecotDeployer(Deployer):
|
||||
def __init__(self, config, disable_mail):
|
||||
self.config = config
|
||||
self.disable_mail = disable_mail
|
||||
self.units = ["doveauth"]
|
||||
self.units = []
|
||||
|
||||
def install(self):
|
||||
arch = host.get_fact(Arch)
|
||||
if has_systemd() and "dovecot.service" in host.get_fact(SystemdEnabled):
|
||||
return # already installed and running
|
||||
|
||||
with blocked_service_startup():
|
||||
_install_dovecot_package("core", arch)
|
||||
_install_dovecot_package("imapd", arch)
|
||||
_install_dovecot_package("lmtpd", arch)
|
||||
debs = []
|
||||
for pkg in ("core", "imapd", "lmtpd"):
|
||||
deb, changed = _download_dovecot_package(pkg, arch)
|
||||
self.need_restart |= changed
|
||||
if deb:
|
||||
debs.append(deb)
|
||||
if debs:
|
||||
deb_list = " ".join(debs)
|
||||
# First dpkg may fail on missing dependencies (stderr suppressed);
|
||||
# apt-get --fix-broken pulls them in, then dpkg retries cleanly.
|
||||
server.shell(
|
||||
name="Install dovecot packages",
|
||||
commands=[
|
||||
f"dpkg --force-confdef --force-confold -i {deb_list} 2> /dev/null || true",
|
||||
"DEBIAN_FRONTEND=noninteractive apt-get -y --fix-broken install",
|
||||
f"dpkg --force-confdef --force-confold -i {deb_list}",
|
||||
],
|
||||
)
|
||||
self.need_restart = True
|
||||
self.put_file(
|
||||
src=io.StringIO(
|
||||
"Package: dovecot-*\n"
|
||||
"Pin: version *\n"
|
||||
"Pin-Priority: -1\n"
|
||||
),
|
||||
dest="/etc/apt/preferences.d/pin-dovecot",
|
||||
)
|
||||
|
||||
def configure(self):
|
||||
configure_remote_units(self.config.mail_domain, self.units)
|
||||
self.need_restart, self.daemon_reload = _configure_dovecot(self.config)
|
||||
configure_remote_units(self, self.config.mail_domain_bare, self.units)
|
||||
_configure_dovecot(self, self.config)
|
||||
|
||||
def activate(self):
|
||||
activate_remote_units(self.units)
|
||||
activate_remote_units(self, self.units)
|
||||
|
||||
restart = False if self.disable_mail else self.need_restart
|
||||
# Detect stale binary: package installed but service still runs old (deleted) binary.
|
||||
if not self.disable_mail and not self.need_restart:
|
||||
stale = host.get_fact(
|
||||
Command,
|
||||
'pid=$(systemctl show -p MainPID --value dovecot.service 2>/dev/null);'
|
||||
' [ "${pid:-0}" != "0" ] && readlink "/proc/$pid/exe" 2>/dev/null | grep -q "(deleted)"'
|
||||
" && echo STALE || true",
|
||||
)
|
||||
if stale == "STALE":
|
||||
self.need_restart = True
|
||||
|
||||
systemd.service(
|
||||
name="Disable dovecot for now"
|
||||
if self.disable_mail
|
||||
else "Start and enable Dovecot",
|
||||
service="dovecot.service",
|
||||
running=False if self.disable_mail else True,
|
||||
enabled=False if self.disable_mail else True,
|
||||
restarted=restart,
|
||||
daemon_reload=self.daemon_reload,
|
||||
active = not self.disable_mail
|
||||
self.ensure_service(
|
||||
"dovecot.service",
|
||||
running=active,
|
||||
enabled=active,
|
||||
)
|
||||
self.need_restart = False
|
||||
|
||||
|
||||
def _pick_url(primary, fallback):
|
||||
@@ -65,86 +102,63 @@ def _pick_url(primary, fallback):
|
||||
return fallback
|
||||
|
||||
|
||||
def _install_dovecot_package(package: str, arch: str):
|
||||
def _download_dovecot_package(package: str, arch: str) -> tuple[str | None, bool]:
|
||||
"""Download a dovecot .deb if needed, return (path, changed)."""
|
||||
arch = "amd64" if arch == "x86_64" else arch
|
||||
arch = "arm64" if arch == "aarch64" else arch
|
||||
primary_url = f"https://download.delta.chat/dovecot/dovecot-{package}_2.3.21%2Bdfsg1-3_{arch}.deb"
|
||||
fallback_url = f"https://github.com/chatmail/dovecot/releases/download/upstream%2F2.3.21%2Bdfsg1/dovecot-{package}_2.3.21%2Bdfsg1-3_{arch}.deb"
|
||||
url = _pick_url(primary_url, fallback_url)
|
||||
deb_filename = "/root/" + url.split("/")[-1]
|
||||
|
||||
match (package, arch):
|
||||
case ("core", "amd64"):
|
||||
sha256 = "dd060706f52a306fa863d874717210b9fe10536c824afe1790eec247ded5b27d"
|
||||
case ("core", "arm64"):
|
||||
sha256 = "e7548e8a82929722e973629ecc40fcfa886894cef3db88f23535149e7f730dc9"
|
||||
case ("imapd", "amd64"):
|
||||
sha256 = "8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86"
|
||||
case ("imapd", "arm64"):
|
||||
sha256 = "178fa877ddd5df9930e8308b518f4b07df10e759050725f8217a0c1fb3fd707f"
|
||||
case ("lmtpd", "amd64"):
|
||||
sha256 = "2f69ba5e35363de50962d42cccbfe4ed8495265044e244007d7ccddad77513ab"
|
||||
case ("lmtpd", "arm64"):
|
||||
sha256 = "89f52fb36524f5877a177dff4a713ba771fd3f91f22ed0af7238d495e143b38f"
|
||||
case _:
|
||||
apt.packages(packages=[f"dovecot-{package}"])
|
||||
return
|
||||
pkg_name = f"dovecot-{package}"
|
||||
sha256 = DOVECOT_SHA256.get((package, arch))
|
||||
if sha256 is None:
|
||||
op = apt.packages(packages=[pkg_name])
|
||||
return None, bool(getattr(op, "changed", False))
|
||||
|
||||
installed_versions = host.get_fact(DebPackages).get(pkg_name, [])
|
||||
if DOVECOT_PACKAGE_VERSION in installed_versions:
|
||||
return None, False
|
||||
|
||||
url_version = DOVECOT_ARCHIVE_VERSION.replace("+", "%2B")
|
||||
deb_base = f"{pkg_name}_{url_version}_{arch}.deb"
|
||||
primary_url = f"https://download.delta.chat/dovecot/{deb_base}"
|
||||
fallback_url = f"https://github.com/chatmail/dovecot/releases/download/upstream%2F{url_version}/{deb_base}"
|
||||
url = _pick_url(primary_url, fallback_url)
|
||||
deb_filename = f"/root/{deb_base}"
|
||||
|
||||
files.download(
|
||||
name=f"Download dovecot-{package}",
|
||||
name=f"Download {pkg_name}",
|
||||
src=url,
|
||||
dest=deb_filename,
|
||||
sha256sum=sha256,
|
||||
cache_time=60 * 60 * 24 * 365 * 10, # never redownload the package
|
||||
)
|
||||
|
||||
apt.deb(name=f"Install dovecot-{package}", src=deb_filename)
|
||||
return deb_filename, True
|
||||
|
||||
|
||||
def _configure_dovecot(config: Config, debug: bool = False) -> (bool, bool):
|
||||
def _configure_dovecot(deployer, config: Config, debug: bool = False):
|
||||
"""Configures Dovecot IMAP server."""
|
||||
need_restart = False
|
||||
daemon_reload = False
|
||||
|
||||
main_config = files.template(
|
||||
src=get_resource("dovecot/dovecot.conf.j2"),
|
||||
dest="/etc/dovecot/dovecot.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
deployer.put_template(
|
||||
"dovecot/dovecot.conf.j2",
|
||||
"/etc/dovecot/dovecot.conf",
|
||||
config=config,
|
||||
debug=debug,
|
||||
disable_ipv6=config.disable_ipv6,
|
||||
)
|
||||
need_restart |= main_config.changed
|
||||
auth_config = files.put(
|
||||
src=get_resource("dovecot/auth.conf"),
|
||||
dest="/etc/dovecot/auth.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
deployer.put_file("dovecot/auth.conf", "/etc/dovecot/auth.conf")
|
||||
deployer.put_file(
|
||||
"dovecot/push_notification.lua", "/etc/dovecot/push_notification.lua"
|
||||
)
|
||||
need_restart |= auth_config.changed
|
||||
lua_push_notification_script = files.put(
|
||||
src=get_resource("dovecot/push_notification.lua"),
|
||||
dest="/etc/dovecot/push_notification.lua",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= lua_push_notification_script.changed
|
||||
|
||||
# as per https://doc.dovecot.org/2.3/configuration_manual/os/
|
||||
# it is recommended to set the following inotify limits
|
||||
can_modify = host.get_fact(Command, "systemd-detect-virt -c || true") == "none"
|
||||
can_modify = not is_in_container()
|
||||
for name in ("max_user_instances", "max_user_watches"):
|
||||
key = f"fs.inotify.{name}"
|
||||
value = host.get_fact(Sysctl)[key]
|
||||
value = host.get_fact(Sysctl).get(key, 0)
|
||||
if value > 65534:
|
||||
continue
|
||||
if not can_modify:
|
||||
print(
|
||||
"\n!!!! refusing to attempt sysctl setting in shared-kernel containers\n"
|
||||
"\n!!!! refusing to attempt sysctl setting in containers\n"
|
||||
f"!!!! dovecot: sysctl {key!r}={value}, should be >65534 for production setups\n"
|
||||
"!!!!"
|
||||
)
|
||||
@@ -156,25 +170,20 @@ def _configure_dovecot(config: Config, debug: bool = False) -> (bool, bool):
|
||||
persist=True,
|
||||
)
|
||||
|
||||
timezone_env = files.line(
|
||||
deployer.ensure_line(
|
||||
name="Set TZ environment variable",
|
||||
path="/etc/environment",
|
||||
line="TZ=:/etc/localtime",
|
||||
)
|
||||
need_restart |= timezone_env.changed
|
||||
|
||||
restart_conf = files.put(
|
||||
name="dovecot: restart automatically on failure",
|
||||
src=get_resource("service/10_restart.conf"),
|
||||
dest="/etc/systemd/system/dovecot.service.d/10_restart.conf",
|
||||
deployer.put_file(
|
||||
"service/10_restart_on_failure.conf",
|
||||
"/etc/systemd/system/dovecot.service.d/10_restart.conf",
|
||||
)
|
||||
daemon_reload |= restart_conf.changed
|
||||
|
||||
# Validate dovecot configuration before restart
|
||||
if need_restart:
|
||||
if deployer.need_restart:
|
||||
server.shell(
|
||||
name="Validate dovecot configuration",
|
||||
commands=["doveconf -n >/dev/null"],
|
||||
)
|
||||
|
||||
return need_restart, daemon_reload
|
||||
|
||||
@@ -7,6 +7,7 @@ listen = 0.0.0.0
|
||||
protocols = imap lmtp
|
||||
|
||||
auth_mechanisms = plain
|
||||
auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@[]
|
||||
|
||||
{% if debug == true %}
|
||||
auth_verbose = yes
|
||||
@@ -36,11 +37,15 @@ default_client_limit = 20000
|
||||
# the following warning will be logged:
|
||||
# Warning: service(imap): process_limit (1024) reached, client connections are being dropped
|
||||
service imap {
|
||||
process_limit = 50000
|
||||
process_limit = {{ config.max_imap_connections }}
|
||||
}
|
||||
|
||||
mail_server_admin = mailto:root@{{ config.mail_domain }}
|
||||
mail_server_comment = Chatmail server
|
||||
{% if config.privacy_mail %}
|
||||
# Advertised to clients as IMAP METADATA /shared/admin (RFC 5464).
|
||||
# The privacy_mail contact from chatmail.ini is used because it is
|
||||
# the only address set by an operator.
|
||||
mail_server_admin = mailto:{{ config.privacy_mail }}
|
||||
{% endif %}
|
||||
|
||||
# `zlib` enables compressing messages stored in the maildir.
|
||||
# See
|
||||
@@ -133,6 +138,11 @@ protocol lmtp {
|
||||
# mail_lua and push_notification_lua are needed for Lua push notification handler.
|
||||
# <https://doc.dovecot.org/2.3/configuration_manual/push_notification/#configuration>
|
||||
mail_plugins = $mail_plugins mail_lua notify push_notification push_notification_lua
|
||||
|
||||
# Disable fsync for LMTP. May lose delivered message,
|
||||
# but unlikely to cause problems with multiple relays.
|
||||
# https://doc.dovecot.org/2.3/admin_manual/mailbox_formats/#fsyncing
|
||||
mail_fsync = never
|
||||
}
|
||||
|
||||
plugin {
|
||||
@@ -144,12 +154,26 @@ plugin {
|
||||
}
|
||||
|
||||
plugin {
|
||||
# for now we define static quota-rules for all users
|
||||
quota = maildir:User quota
|
||||
quota_rule = *:storage={{ config.max_mailbox_size }}
|
||||
quota_max_mail_size={{ config.max_message_size }}
|
||||
quota_grace = 0
|
||||
# quota_over_flag_value = TRUE
|
||||
|
||||
quota_rule = *:storage={{ config.max_mailbox_size_mb }}M
|
||||
|
||||
# Trigger at 75%% of quota, expire oldest messages down to 70%%.
|
||||
# The percentages are chosen to prevent current Delta Chat users
|
||||
# from seeing "quota warnings" which trigger at 80% and 95%.
|
||||
|
||||
quota_warning = storage=75%% quota-warning {{ config.max_mailbox_size_mb * 70 // 100 }} {{ config.mailboxes_dir }}/%u
|
||||
}
|
||||
|
||||
service quota-warning {
|
||||
executable = script /usr/local/lib/chatmaild/venv/bin/chatmail-quota-expire
|
||||
user = vmail
|
||||
unix_listener quota-warning {
|
||||
user = vmail
|
||||
mode = 0600
|
||||
}
|
||||
}
|
||||
|
||||
# push_notification configuration
|
||||
@@ -252,6 +276,9 @@ protocol imap {
|
||||
# sort -sn <(sed 's/ / C: /' *.in) <(sed 's/ / S: /' cat *.out)
|
||||
|
||||
rawlog_dir = %h
|
||||
|
||||
# Disable fsync for IMAP. May lose IMAP changes like setting flags.
|
||||
mail_fsync = never
|
||||
}
|
||||
{% endif %}
|
||||
|
||||
|
||||
+12
-38
@@ -1,10 +1,7 @@
|
||||
import io
|
||||
|
||||
from pyinfra import host
|
||||
from pyinfra.facts.files import File
|
||||
from pyinfra.operations import files, systemd
|
||||
|
||||
from cmdeploy.basedeploy import Deployer, get_resource
|
||||
from ..basedeploy import Deployer
|
||||
|
||||
|
||||
class ExternalTlsDeployer(Deployer):
|
||||
@@ -23,45 +20,22 @@ class ExternalTlsDeployer(Deployer):
|
||||
def configure(self):
|
||||
# Verify cert and key exist on the remote host using pyinfra facts.
|
||||
for path in (self.cert_path, self.key_path):
|
||||
info = host.get_fact(File, path=path)
|
||||
if info is None:
|
||||
if host.get_fact(File, path=path) is None:
|
||||
raise Exception(f"External TLS file not found on server: {path}")
|
||||
|
||||
# Deploy the .path unit (templated with the cert path).
|
||||
# pkg=__package__ is required here because the resource files
|
||||
# live in cmdeploy.external, not the default cmdeploy package.
|
||||
source = get_resource("tls-cert-reload.path.f", pkg=__package__)
|
||||
content = source.read_text().format(cert_path=self.cert_path).encode()
|
||||
|
||||
path_unit = files.put(
|
||||
name="Upload tls-cert-reload.path",
|
||||
src=io.BytesIO(content),
|
||||
dest="/etc/systemd/system/tls-cert-reload.path",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.ensure_systemd_unit(
|
||||
"external/tls-cert-reload.path.j2",
|
||||
cert_path=self.cert_path,
|
||||
)
|
||||
|
||||
service_unit = files.put(
|
||||
name="Upload tls-cert-reload.service",
|
||||
src=get_resource("tls-cert-reload.service", pkg=__package__),
|
||||
dest="/etc/systemd/system/tls-cert-reload.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.ensure_systemd_unit(
|
||||
"external/tls-cert-reload.service",
|
||||
)
|
||||
|
||||
if path_unit.changed or service_unit.changed:
|
||||
self.need_restart = True
|
||||
|
||||
def activate(self):
|
||||
systemd.service(
|
||||
name="Enable tls-cert-reload path watcher",
|
||||
service="tls-cert-reload.path",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=self.need_restart,
|
||||
daemon_reload=self.need_restart,
|
||||
)
|
||||
# No explicit reload needed here: dovecot/nginx read the cert
|
||||
# on startup, and the .path watcher handles live changes.
|
||||
self.ensure_service(
|
||||
"tls-cert-reload.path",
|
||||
running=True,
|
||||
enabled=True,
|
||||
)
|
||||
|
||||
+1
-1
@@ -9,7 +9,7 @@
|
||||
Description=Watch TLS certificate for changes
|
||||
|
||||
[Path]
|
||||
PathChanged={cert_path}
|
||||
PathChanged={{ cert_path }}
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,52 +1,40 @@
|
||||
from pyinfra import facts, host
|
||||
from pyinfra.operations import files, systemd
|
||||
import os
|
||||
|
||||
from cmdeploy.basedeploy import Deployer, get_resource
|
||||
from pyinfra import facts, host
|
||||
|
||||
from cmdeploy.basedeploy import Deployer
|
||||
|
||||
|
||||
class FiltermailDeployer(Deployer):
|
||||
services = ["filtermail", "filtermail-incoming"]
|
||||
services = ["filtermail", "filtermail-incoming", "filtermail-transport"]
|
||||
bin_path = "/usr/local/bin/filtermail"
|
||||
config_path = "/usr/local/lib/chatmaild/chatmail.ini"
|
||||
|
||||
def __init__(self):
|
||||
self.need_restart = False
|
||||
|
||||
def install(self):
|
||||
local_bin = os.environ.get("CHATMAIL_FILTERMAIL_BINARY")
|
||||
if local_bin:
|
||||
self.put_executable(
|
||||
src=local_bin,
|
||||
dest=self.bin_path,
|
||||
)
|
||||
return
|
||||
|
||||
arch = host.get_fact(facts.server.Arch)
|
||||
url = f"https://github.com/chatmail/filtermail/releases/download/v0.6.0/filtermail-{arch}"
|
||||
url = f"https://github.com/chatmail/filtermail/releases/download/v0.7.4/filtermail-{arch}"
|
||||
sha256sum = {
|
||||
"x86_64": "3fd8b18282252c75a5bbfa603d8c1b65f6563e5e920bddf3e64e451b7cdb43ce",
|
||||
"aarch64": "2bd191de205f7fd60158dd8e3516ab7e3efb14627696f3d7dc186bdcd9e10a43",
|
||||
"x86_64": "484cb8dff083134aefba9fce4a6b7ef4784a0f0e28e5108ecf8bb9e58a44fd2c",
|
||||
"aarch64": "66aa0ca2ca9add7a12d92883d76f8786384092adfde24a3d3a1d0b1f30d23a9e",
|
||||
}[arch]
|
||||
self.need_restart |= files.download(
|
||||
name="Download filtermail",
|
||||
src=url,
|
||||
sha256sum=sha256sum,
|
||||
dest=self.bin_path,
|
||||
mode="755",
|
||||
).changed
|
||||
self.download_executable(url, self.bin_path, sha256sum)
|
||||
|
||||
def configure(self):
|
||||
for service in self.services:
|
||||
self.need_restart |= files.template(
|
||||
src=get_resource(f"filtermail/{service}.service.j2"),
|
||||
dest=f"/etc/systemd/system/{service}.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.ensure_systemd_unit(
|
||||
f"filtermail/{service}.service.j2",
|
||||
bin_path=self.bin_path,
|
||||
config_path=self.config_path,
|
||||
).changed
|
||||
)
|
||||
|
||||
def activate(self):
|
||||
for service in self.services:
|
||||
systemd.service(
|
||||
name=f"Start and enable {service}",
|
||||
service=f"{service}.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=self.need_restart,
|
||||
daemon_reload=True,
|
||||
)
|
||||
self.need_restart = False
|
||||
self.ensure_service(f"{service}.service")
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
[Unit]
|
||||
Description=Chatmail transport service
|
||||
|
||||
[Service]
|
||||
ExecStart={{ bin_path }} {{ config_path }} transport
|
||||
Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
LimitNOFILE=524288
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,475 +0,0 @@
|
||||
"""lxc-start/stop/status/test subcommands for testing with local containers."""
|
||||
|
||||
import os
|
||||
import time
|
||||
|
||||
from ..util import get_git_hash, get_version_string, shell
|
||||
from .incus import RELAY_IMAGE_ALIAS, Incus, RelayContainer
|
||||
|
||||
RELAY_NAMES = ("test0", "test1")
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# lxc-start
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
|
||||
def lxc_start_cmd_options(parser):
|
||||
_add_name_args(
|
||||
parser,
|
||||
help_text="User relay name(s) to create (default: test0).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--ipv4-only",
|
||||
dest="ipv4_only",
|
||||
action="store_true",
|
||||
help="Create an IPv4-only container.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--run",
|
||||
action="store_true",
|
||||
help="Run 'cmdeploy run' on each container after starting it.",
|
||||
)
|
||||
|
||||
|
||||
def lxc_start_cmd(args, out):
|
||||
"""Create/Ensure and start LXC relay and DNS containers."""
|
||||
|
||||
with out.section("Preparing container setup"):
|
||||
_lxc_start_cmd(args, out)
|
||||
|
||||
|
||||
def _lxc_start_cmd(args, out):
|
||||
ix = Incus(out)
|
||||
sub = out.new_prefixed_out()
|
||||
out.green("Ensuring base image ...")
|
||||
ix.ensure_base_image()
|
||||
out.green("Ensuring DNS container (ns-localchat) ...")
|
||||
dns_ct = ix.get_dns_container()
|
||||
dns_ct.ensure()
|
||||
sub.print(f"DNS container IP: {dns_ct.ipv4}")
|
||||
|
||||
names = args.names if args.names else RELAY_NAMES
|
||||
relays = list(ix.get_container(n) for n in names)
|
||||
for ct in relays:
|
||||
out.green(f"Ensuring container {ct.name!r} ({ct.domain}) ...")
|
||||
ct.ensure()
|
||||
ip = ct.ipv4
|
||||
|
||||
sub.print("Configuring container hostname ...")
|
||||
ct.configure_hosts(ip)
|
||||
|
||||
sub.print(f"Writing {ct.ini.name} ...")
|
||||
ct.write_ini(disable_ipv6=args.ipv4_only)
|
||||
sub.print(f"Config: {ct.ini}")
|
||||
if args.ipv4_only:
|
||||
ct.disable_ipv6()
|
||||
ipv6 = None
|
||||
else:
|
||||
output = ct.bash(
|
||||
"ip -6 addr show scope global -deprecated"
|
||||
" | grep -oP '(?<=inet6 )[^/]+'",
|
||||
check=False,
|
||||
)
|
||||
ipv6 = output.strip() if output else None
|
||||
sub.print(f"{_format_addrs(ip, ipv6)}")
|
||||
|
||||
sub.green(f"Container {ct.name!r} ready: {ct.domain} -> {ip}")
|
||||
out.print()
|
||||
|
||||
# Reset DNS zones only for the containers we just started
|
||||
started_cnames = {ct.name for ct in relays}
|
||||
managed = ix.list_managed()
|
||||
started = [c for c in managed if c["name"] in started_cnames]
|
||||
|
||||
if started:
|
||||
out.print(
|
||||
f"Resetting DNS zones for {len(started)} domain(s) (A + AAAA records) ..."
|
||||
)
|
||||
dns_ct.reset_dns_records(dns_ct.ipv4, started)
|
||||
|
||||
for ct in relays:
|
||||
if ct.name in started_cnames:
|
||||
sub.print(f"Configuring DNS in {ct.name} ...")
|
||||
ct.configure_dns(dns_ct.ipv4)
|
||||
|
||||
# Generate the unified SSH config
|
||||
out.green("Writing ssh-config ...")
|
||||
ssh_cfg = ix.write_ssh_config()
|
||||
sub.print(f"{ssh_cfg}")
|
||||
|
||||
# Verify SSH via the generated config
|
||||
for ct in relays:
|
||||
sub.print(f"Verifying SSH to {ct.name} via ssh-config ...")
|
||||
if ct.verify_ssh(ssh_cfg):
|
||||
sub.print(f"SSH OK: ssh -F lxconfigs/ssh-config {ct.domain}")
|
||||
else:
|
||||
sub.red(f"WARNING: SSH verification failed for {ct.name}")
|
||||
|
||||
# Print integration suggestions
|
||||
ssh_cfg = ix.ssh_config_path
|
||||
if not ix.check_ssh_include():
|
||||
sub.green(
|
||||
"\n(Optional) To use containers from any SSH client, add to ~/.ssh/config:"
|
||||
)
|
||||
sub.green(f" Include {ssh_cfg}")
|
||||
|
||||
# Optionally run cmdeploy run + dns on each relay
|
||||
if args.run:
|
||||
for ct in relays:
|
||||
with out.section(f"cmdeploy run: {ct.sname} ({ct.domain})"):
|
||||
ret = _run_cmdeploy("run", ct, ix, out, extra=["--skip-dns-check"])
|
||||
if ret:
|
||||
out.red(f"Deploy to {ct.sname} failed (exit {ret})")
|
||||
return ret
|
||||
|
||||
with out.section("loading DNS zones"):
|
||||
for ct in relays:
|
||||
ret = _run_cmdeploy(
|
||||
"dns", ct, ix, out,
|
||||
extra=["--zonefile", str(ct.zone)],
|
||||
)
|
||||
if ret:
|
||||
out.red(f"DNS for {ct.sname} failed (exit {ret})")
|
||||
return ret
|
||||
if ct.zone.exists():
|
||||
dns_ct.set_dns_records(ct.zone.read_text())
|
||||
out.print(f"Restarting filtermail-incoming on {ct.name}")
|
||||
ct.bash("systemctl restart filtermail-incoming")
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# lxc-stop
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
|
||||
def lxc_stop_cmd_options(parser):
|
||||
parser.add_argument(
|
||||
"--destroy",
|
||||
action="store_true",
|
||||
help="Delete containers and their config files after stopping.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--destroy-all",
|
||||
dest="destroy_all",
|
||||
action="store_true",
|
||||
help="Like --destroy, but also remove the ns-localchat DNS container.",
|
||||
)
|
||||
_add_name_args(
|
||||
parser,
|
||||
help_text="Container name(s) to stop (default: test0 + test1).",
|
||||
)
|
||||
|
||||
|
||||
def lxc_stop_cmd(args, out):
|
||||
"""Stop (and optionally destroy) local LXC relay containers."""
|
||||
ix = Incus(out)
|
||||
names = args.names or RELAY_NAMES
|
||||
destroy = args.destroy or args.destroy_all
|
||||
|
||||
for ct in map(ix.get_container, names):
|
||||
if destroy:
|
||||
out.green(f"Destroying container {ct.name!r} ...")
|
||||
ct.destroy()
|
||||
else:
|
||||
out.green(f"Stopping container {ct.name!r} ...")
|
||||
ct.stop(force=True)
|
||||
|
||||
if args.destroy_all:
|
||||
dns_ct = ix.get_dns_container()
|
||||
out.green(f"Destroying DNS container {dns_ct.name!r} ...")
|
||||
dns_ct.destroy()
|
||||
ix.delete_images()
|
||||
|
||||
if destroy:
|
||||
ix.write_ssh_config()
|
||||
out.green("LXC containers destroyed.")
|
||||
else:
|
||||
out.green("LXC containers stopped.")
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# lxc-test
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
|
||||
def lxc_test_cmd_options(parser):
|
||||
parser.add_argument(
|
||||
"--one",
|
||||
action="store_true",
|
||||
help="Only deploy and test against test0 (skip test1).",
|
||||
)
|
||||
|
||||
|
||||
def lxc_test_cmd(args, out):
|
||||
"""Run full LXC pipeline: start, deploy, DNS, zone files, and tests.
|
||||
|
||||
All commands run directly on the host using
|
||||
``--ssh-config lxconfigs/ssh-config`` for SSH access.
|
||||
"""
|
||||
ix = Incus(out)
|
||||
t_total = time.time()
|
||||
relay_names = list(RELAY_NAMES)
|
||||
if args.one:
|
||||
relay_names = relay_names[:1]
|
||||
|
||||
local_hash = get_git_hash()
|
||||
|
||||
# Per-relay: start, deploy, then snapshot the first relay as a
|
||||
# reusable image so the second relay launches pre-deployed.
|
||||
ipv4_only_flags = {RELAY_NAMES[0]: False, RELAY_NAMES[1]: True}
|
||||
|
||||
for ct in map(ix.get_container, relay_names):
|
||||
name = ct.sname
|
||||
ipv4_only = ipv4_only_flags.get(name, False)
|
||||
v_flag = " -" + "v" * out.verbosity if out.verbosity > 0 else ""
|
||||
start_cmd = f"cmdeploy lxc-start{v_flag} {name}"
|
||||
if ipv4_only:
|
||||
start_cmd += " --ipv4-only"
|
||||
with out.section(f"cmdeploy lxc-start: {name}"):
|
||||
ret = out.shell(start_cmd, cwd=str(ix.project_root))
|
||||
if ret:
|
||||
return ret
|
||||
|
||||
status = _deploy_status(ct, local_hash, ix)
|
||||
with out.section(f"cmdeploy run: {name}"):
|
||||
if "IN-SYNC" in status:
|
||||
out.print(f"{name} is {status}, skipping")
|
||||
else:
|
||||
ret = _run_cmdeploy("run", ct, ix, out, extra=["--skip-dns-check"])
|
||||
if ret:
|
||||
out.red(f"Deploy to {name} failed (exit {ret})")
|
||||
return ret
|
||||
|
||||
# Snapshot the first relay so subsequent ones launch pre-deployed
|
||||
if not ix.find_image([RELAY_IMAGE_ALIAS]):
|
||||
with out.section("lxc-test: caching relay image"):
|
||||
ct.publish_as_relay_image()
|
||||
|
||||
for ct in map(ix.get_container, relay_names):
|
||||
with out.section(f"cmdeploy dns: {ct.sname} ({ct.domain})"):
|
||||
ret = _run_cmdeploy("dns", ct, ix, out, extra=["--zonefile", str(ct.zone)])
|
||||
if ret:
|
||||
out.red(f"DNS for {ct.sname} failed (exit {ret})")
|
||||
return ret
|
||||
|
||||
with out.section(f"lxc-test: loading DNS zones {' & '.join(relay_names)}"):
|
||||
dns_ct = ix.get_dns_container()
|
||||
for ct in map(ix.get_container, relay_names):
|
||||
if ct.zone.exists():
|
||||
zone_data = ct.zone.read_text()
|
||||
out.print(f"Loading {ct.zone} into PowerDNS ...")
|
||||
dns_ct.set_dns_records(zone_data)
|
||||
|
||||
# Restart filtermail so its in-process DNS cache
|
||||
# does not hold stale negative DKIM responses
|
||||
# from before the zones were loaded.
|
||||
for ct in map(ix.get_container, relay_names):
|
||||
out.print(f"Restarting filtermail-incoming on {ct.name} ...")
|
||||
ct.bash("systemctl restart filtermail-incoming")
|
||||
|
||||
with out.section("cmdeploy test"):
|
||||
first = ix.get_container(relay_names[0])
|
||||
env = None
|
||||
if len(relay_names) > 1:
|
||||
env = os.environ.copy()
|
||||
env["CHATMAIL_DOMAIN2"] = ix.get_container(relay_names[1]).domain
|
||||
ret = _run_cmdeploy("test", first, ix, out, **({"env": env} if env else {}))
|
||||
if ret:
|
||||
out.red(f"Tests failed (exit {ret})")
|
||||
return ret
|
||||
|
||||
elapsed = time.time() - t_total
|
||||
out.section_line(f"lxc-test complete ({elapsed:.1f}s)")
|
||||
if out.section_timings:
|
||||
out.print("Section timings:")
|
||||
for name, secs in out.section_timings:
|
||||
out.print(f" {name:.<50s} {secs:5.1f}s")
|
||||
out.print(f" {'total':.<50s} {elapsed:5.1f}s")
|
||||
out.section_timings.clear()
|
||||
return 0
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# lxc-status
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
|
||||
def lxc_status_cmd_options(parser):
|
||||
pass
|
||||
|
||||
|
||||
def lxc_status_cmd(args, out):
|
||||
"""Show status of local LXC chatmail containers."""
|
||||
ix = Incus(out)
|
||||
containers = ix.list_managed()
|
||||
if not containers:
|
||||
out.red("No LXC containers found. Run 'cmdeploy lxc-start' first.")
|
||||
return 1
|
||||
|
||||
local_hash = get_git_hash()
|
||||
|
||||
# Get storage pool path for display
|
||||
storage_path = None
|
||||
data = ix.run_json(["storage", "show", "default"], check=False)
|
||||
if data:
|
||||
storage_path = data.get("config", {}).get("source")
|
||||
msg = "Container status"
|
||||
if storage_path:
|
||||
msg += f": {storage_path}"
|
||||
out.section_line(msg)
|
||||
|
||||
dns_ip = None
|
||||
for c in containers:
|
||||
_print_container_status(out, c, ix, local_hash)
|
||||
if c["name"] == ix.get_dns_container().name:
|
||||
dns_ip = c["ip"]
|
||||
|
||||
out.section_line("Host ssh and DNS configuration")
|
||||
_print_ssh_status(out, ix)
|
||||
_print_dns_forwarding_status(out, dns_ip)
|
||||
return 0
|
||||
|
||||
|
||||
def _print_container_status(out, c, ix, local_hash):
|
||||
"""Print name/status, domain/IPs, and RAM for one container."""
|
||||
cname = c["name"]
|
||||
is_running = c.get("status") == "Running"
|
||||
ct = ix.get_container(cname)
|
||||
|
||||
# First line: name + running/STOPPED + deploy status
|
||||
if not is_running:
|
||||
tag = "STOPPED"
|
||||
elif not isinstance(ct, RelayContainer):
|
||||
tag = "running"
|
||||
else:
|
||||
tag = f"running {_deploy_status(ct, local_hash, ix)}"
|
||||
out.print(f"{cname:20s} {tag}")
|
||||
|
||||
# Second line: domain, IPv4, IPv6
|
||||
domain = c.get("domain", "")
|
||||
ip = c.get("ip") or "?"
|
||||
ipv6 = c.get("ipv6")
|
||||
out.print(f"{domain:20s} {_format_addrs(ip, ipv6)}")
|
||||
|
||||
# Third line: RAM (RSS), config
|
||||
detail_out = out.new_prefixed_out(" " * 21)
|
||||
try:
|
||||
used, total = ct.rss_mib()
|
||||
except Exception:
|
||||
ram_str = "RSS ?"
|
||||
else:
|
||||
ram_str = f"RSS {used}/{total} MiB ({used * 100 // total}%)"
|
||||
|
||||
if isinstance(ct, RelayContainer):
|
||||
detail = f"{ram_str}, config: {os.path.relpath(ct.ini)}"
|
||||
else:
|
||||
detail = ram_str
|
||||
|
||||
detail_out.print(detail)
|
||||
out.print()
|
||||
|
||||
|
||||
def _print_ssh_status(out, ix):
|
||||
"""Print SSH integration status."""
|
||||
ssh_cfg = ix.ssh_config_path
|
||||
if ix.check_ssh_include():
|
||||
out.green("SSH: ~/.ssh/config includes lxconfigs/ssh-config ✓")
|
||||
else:
|
||||
out.red("SSH: ~/.ssh/config does NOT include lxconfigs/ssh-config")
|
||||
sub = out.new_prefixed_out()
|
||||
sub.print("Add to ~/.ssh/config:")
|
||||
sub.print(f" Include {ssh_cfg}")
|
||||
|
||||
|
||||
def _print_dns_forwarding_status(out, dns_ip):
|
||||
"""Print host DNS forwarding status for .localchat."""
|
||||
sub = out.new_prefixed_out()
|
||||
if not dns_ip:
|
||||
out.red("DNS: ns-localchat container not found")
|
||||
return
|
||||
try:
|
||||
rv = shell("resolvectl status incusbr0")
|
||||
dns_ok = dns_ip in rv.stdout and "localchat" in rv.stdout
|
||||
except Exception:
|
||||
dns_ok = None
|
||||
if dns_ok is True:
|
||||
out.green(f"DNS: .localchat forwarding to {dns_ip} ✓")
|
||||
elif dns_ok is False:
|
||||
out.red("DNS: .localchat forwarding NOT configured")
|
||||
sub.print("Run:")
|
||||
sub.print(f" sudo resolvectl dns incusbr0 {dns_ip}")
|
||||
sub.print(" sudo resolvectl domain incusbr0 ~localchat")
|
||||
else:
|
||||
sub.print("DNS: .localchat forwarding status UNKNOWN")
|
||||
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# -------------------------------------------------------------------
|
||||
|
||||
|
||||
def _format_addrs(ip, ipv6=None):
|
||||
parts = [f"IPv4 {ip}"]
|
||||
if ipv6:
|
||||
parts.append(f"IPv6 {ipv6}")
|
||||
return ", ".join(parts)
|
||||
|
||||
|
||||
def _deploy_status(ct, local_hash, ix):
|
||||
"""Return a human-readable deploy status string.
|
||||
|
||||
Compares the full deployed version (hash + diff) against
|
||||
the local state built by :func:`~cmdeploy.util.get_version_string`.
|
||||
"""
|
||||
deployed = ct.deployed_version()
|
||||
if deployed is None:
|
||||
return "NOT DEPLOYED"
|
||||
|
||||
# A container launched from the relay image has the same
|
||||
# git hash but a different domain — always redeploy.
|
||||
deployed_domain = ct.deployed_domain()
|
||||
if deployed_domain and deployed_domain != ct.domain:
|
||||
return f"DOMAIN-MISMATCH (deployed: {deployed_domain})"
|
||||
|
||||
deployed_lines = deployed.splitlines()
|
||||
deployed_hash = deployed_lines[0] if deployed_lines else ""
|
||||
short = deployed_hash[:12]
|
||||
|
||||
if not local_hash:
|
||||
return f"UNKNOWN (deployed: {short})"
|
||||
|
||||
local_short = local_hash[:12]
|
||||
if deployed_hash != local_hash:
|
||||
return f"STALE (deployed: {short}, local: {local_short})"
|
||||
|
||||
# Hash matches — check for uncommitted diffs
|
||||
local_version = get_version_string()
|
||||
if deployed != local_version:
|
||||
return f"DIRTY ({local_short}, undeployed changes)"
|
||||
|
||||
return f"IN-SYNC ({short})"
|
||||
|
||||
|
||||
def _add_name_args(parser, help_text):
|
||||
parser.add_argument("names", nargs="*", metavar="NAME", help=help_text)
|
||||
|
||||
|
||||
def _run_cmdeploy(subcmd, ct, ix, out, extra=None, **kwargs):
|
||||
"""Run ``cmdeploy <subcmd>`` with standard --config/--ssh flags.
|
||||
|
||||
*ct* is a Container (uses ``ct.ini`` and ``ct.domain``).
|
||||
Returns the subprocess exit code.
|
||||
"""
|
||||
extra_str = " ".join(extra) if extra else ""
|
||||
v_flag = " -" + "v" * out.verbosity if out.verbosity > 0 else ""
|
||||
cmd = f"""
|
||||
cmdeploy {subcmd}{v_flag}
|
||||
--config {ct.ini}
|
||||
--ssh-config {ix.ssh_config_path}
|
||||
--ssh-host {ct.domain}
|
||||
{extra_str}
|
||||
"""
|
||||
if "cwd" not in kwargs:
|
||||
kwargs["cwd"] = str(ix.project_root)
|
||||
return out.shell(cmd, **kwargs)
|
||||
@@ -1,768 +0,0 @@
|
||||
"""Core Incus operations for local chatmail LXC containers."""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import textwrap
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from ..util import shell
|
||||
|
||||
LABEL_KEY = "user.localchat-managed"
|
||||
SSH_KEY_NAME = "id_localchat"
|
||||
DOMAIN_SUFFIX = ".localchat"
|
||||
UPSTREAM_IMAGE = "images:debian/12"
|
||||
BASE_IMAGE_ALIAS = "localchat-base"
|
||||
BASE_SETUP_NAME = "localchat-base-setup"
|
||||
RELAY_IMAGE_ALIAS = "localchat-relay"
|
||||
|
||||
DNS_CONTAINER_NAME = "ns-localchat"
|
||||
DNS_DOMAIN = "ns.localchat"
|
||||
|
||||
|
||||
class DNSConfigurationError(Exception):
|
||||
"""Raised when the DNS container is not reachable or not answering."""
|
||||
|
||||
|
||||
def _extract_ip(net_data, family="inet"):
|
||||
"""Extract the first global-scope IP of *family* from network state data.
|
||||
|
||||
*net_data* is the ``state.network`` dict from ``incus list --format=json``.
|
||||
*family* is ``"inet"`` for IPv4 or ``"inet6"`` for IPv6.
|
||||
Returns the address string, or None.
|
||||
"""
|
||||
for iface_name, iface in net_data.items():
|
||||
if iface_name == "lo":
|
||||
continue
|
||||
for addr in iface.get("addresses", []):
|
||||
if addr["family"] == family and addr["scope"] == "global":
|
||||
return addr["address"]
|
||||
return None
|
||||
|
||||
|
||||
class Incus:
|
||||
"""Gateway for all Incus container operations.
|
||||
|
||||
Instantiated once per CLI command and passed around so that
|
||||
all modules share a single entry point for Incus interactions.
|
||||
"""
|
||||
|
||||
def __init__(self, out):
|
||||
self.out = out
|
||||
self.project_root = Path(__file__).resolve().parent.parent.parent.parent.parent
|
||||
self.lxconfigs_dir = self.project_root / "lxconfigs"
|
||||
self.lxconfigs_dir.mkdir(exist_ok=True)
|
||||
self.ssh_key_path = self.lxconfigs_dir / SSH_KEY_NAME
|
||||
if not self.ssh_key_path.exists():
|
||||
shell(
|
||||
f"ssh-keygen -t ed25519 -f {self.ssh_key_path} -N '' -C localchat",
|
||||
check=True,
|
||||
)
|
||||
self.ssh_config_path = self.lxconfigs_dir / "ssh-config"
|
||||
|
||||
def write_ssh_config(self):
|
||||
"""Write ``lxconfigs/ssh-config`` mapping all containers to their IPs.
|
||||
|
||||
Each Host block maps the container name, the domain name, and the
|
||||
short relay name (e.g. ``_test0``) to the container's IP, using the
|
||||
shared localchat SSH key. Returns the path to the file.
|
||||
"""
|
||||
containers = self.list_managed()
|
||||
key_path = self.ssh_key_path
|
||||
lines = ["# Auto-generated by cmdeploy lxc-start — do not edit\n"]
|
||||
for c in containers:
|
||||
hosts = [c["name"]]
|
||||
domain = c.get("domain", "")
|
||||
if domain and domain != c["name"]:
|
||||
hosts.append(domain)
|
||||
short = domain.split(".")[0]
|
||||
if short and short not in hosts:
|
||||
hosts.append(short)
|
||||
lines.append(f"\nHost {' '.join(hosts)}\n")
|
||||
lines.append(f" Hostname {c['ip']}\n")
|
||||
lines.append(" User root\n")
|
||||
lines.append(f" IdentityFile {key_path}\n")
|
||||
lines.append(" IdentitiesOnly yes\n")
|
||||
lines.append(" StrictHostKeyChecking accept-new\n")
|
||||
lines.append(" UserKnownHostsFile /dev/null\n")
|
||||
lines.append(" LogLevel ERROR\n")
|
||||
path = self.ssh_config_path
|
||||
path.write_text("".join(lines))
|
||||
return path
|
||||
|
||||
def check_ssh_include(self):
|
||||
"""Check if the user's ~/.ssh/config already includes our ssh-config."""
|
||||
user_ssh_config = Path.home() / ".ssh" / "config"
|
||||
if not user_ssh_config.exists():
|
||||
return False
|
||||
lines = user_ssh_config.read_text().splitlines()
|
||||
target = f"include {self.ssh_config_path}".lower()
|
||||
return any(line.strip().lower() == target for line in lines)
|
||||
|
||||
def get_host_nameservers(self):
|
||||
"""Return upstream nameservers found on the host."""
|
||||
ns = []
|
||||
for path in ["/run/systemd/resolve/resolv.conf", "/etc/resolv.conf"]:
|
||||
p = Path(path)
|
||||
if p.exists():
|
||||
for line in p.read_text().splitlines():
|
||||
if line.strip().startswith("nameserver "):
|
||||
addr = line.split()[1]
|
||||
if addr not in ("127.0.0.1", "127.0.0.53", "::1"):
|
||||
if addr not in ns:
|
||||
ns.append(addr)
|
||||
if ns:
|
||||
break
|
||||
return ns
|
||||
|
||||
def run(self, args, check=True, capture=True, input=None):
|
||||
"""Run an incus command.
|
||||
|
||||
When *capture* is True and *verbosity* >= 1, output is streamed
|
||||
to the terminal line-by-line while also being captured for
|
||||
later return via result.stdout.
|
||||
"""
|
||||
cmd = ["incus", "--quiet"] + list(args)
|
||||
sub = self.out.new_prefixed_out(" ")
|
||||
|
||||
if not capture:
|
||||
# Simple case: let subprocess handle streams (no capture)
|
||||
if self.out.verbosity >= 1:
|
||||
sub.print(f"$ {' '.join(cmd)}")
|
||||
return subprocess.run(
|
||||
cmd, text=True, input=input, check=check, stdout=None, stderr=None
|
||||
)
|
||||
|
||||
# Capture case: we may need to stream while capturing
|
||||
if sub.verbosity >= 1:
|
||||
cmd_lines = " ".join(cmd).splitlines()
|
||||
sub.print(f"$ {cmd_lines.pop(0)}")
|
||||
if sub.verbosity >= 2:
|
||||
for line in cmd_lines:
|
||||
sub.print(f" {line}")
|
||||
|
||||
proc = subprocess.Popen(
|
||||
cmd,
|
||||
text=True,
|
||||
stdin=subprocess.PIPE if input else subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
|
||||
stdout_lines = []
|
||||
if input:
|
||||
proc.stdin.write(input)
|
||||
proc.stdin.close()
|
||||
|
||||
for line in proc.stdout:
|
||||
stdout_lines.append(line)
|
||||
if sub.verbosity >= 2:
|
||||
sub.print(f" > {line.rstrip()}")
|
||||
|
||||
stderr = proc.stderr.read()
|
||||
ret = proc.wait()
|
||||
stdout = "".join(stdout_lines)
|
||||
if check and ret != 0:
|
||||
full_output = stdout + stderr
|
||||
for line in full_output.splitlines():
|
||||
if sub.verbosity < 1: # and we haven't printed it yet
|
||||
sub.red(line)
|
||||
raise subprocess.CalledProcessError(ret, cmd, output=stdout, stderr=stderr)
|
||||
|
||||
return subprocess.CompletedProcess(cmd, ret, stdout=stdout, stderr=stderr)
|
||||
|
||||
def run_json(self, args, check=True):
|
||||
"""Run an incus command with ``--format=json``.
|
||||
|
||||
Returns the parsed JSON on success.
|
||||
When *check* is True raises ``subprocess.CalledProcessError``
|
||||
on non-zero exit; when False returns *None* instead.
|
||||
"""
|
||||
result = self.run(
|
||||
list(args) + ["--format=json"],
|
||||
check=check,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return None
|
||||
return json.loads(result.stdout)
|
||||
|
||||
def run_output(self, args, check=True):
|
||||
"""Run an incus command and return its stripped stdout.
|
||||
|
||||
When *check* is False, returns *None* on non-zero exit
|
||||
instead of raising.
|
||||
"""
|
||||
result = self.run(args, check=check)
|
||||
if result.returncode != 0:
|
||||
return None
|
||||
return result.stdout.strip()
|
||||
|
||||
def find_image(self, aliases):
|
||||
"""Return the first alias from *aliases* that exists, else None."""
|
||||
images = self.run_json(["image", "list"], check=False) or []
|
||||
existing = {a.get("name") for img in images for a in img.get("aliases", [])}
|
||||
for alias in aliases:
|
||||
if alias in existing:
|
||||
return alias
|
||||
return None
|
||||
|
||||
def delete_images(self):
|
||||
"""Delete the cached base and relay images."""
|
||||
for alias in (RELAY_IMAGE_ALIAS, BASE_IMAGE_ALIAS):
|
||||
self.run(["image", "delete", alias], check=False) # ok if absent
|
||||
|
||||
def list_managed(self):
|
||||
"""Return list of dicts with name, ip, ipv6, domain, status, memory_usage."""
|
||||
containers = []
|
||||
for ct in self.run_json(["list"]):
|
||||
config = ct.get("config", {})
|
||||
if config.get(LABEL_KEY) != "true":
|
||||
continue
|
||||
name = ct["name"]
|
||||
state = ct.get("state", {})
|
||||
net = state.get("network") or {}
|
||||
containers.append(
|
||||
{
|
||||
"name": name,
|
||||
"ip": _extract_ip(net, "inet"),
|
||||
"ipv6": _extract_ip(net, "inet6"),
|
||||
"domain": config.get(
|
||||
"user.localchat-domain", f"{name}{DOMAIN_SUFFIX}"
|
||||
),
|
||||
"status": ct.get("status", "Unknown"),
|
||||
"memory_usage": state.get("memory", {}).get("usage", 0),
|
||||
}
|
||||
)
|
||||
return containers
|
||||
|
||||
def ensure_base_image(self):
|
||||
"""Build and cache a base image with openssh and the SSH key.
|
||||
|
||||
The image is published as a local incus image with alias
|
||||
'localchat-base'. Subsequent container launches use this
|
||||
image instead of the upstream Debian 12, skipping the
|
||||
slow apt-get install step.
|
||||
Returns the image alias.
|
||||
"""
|
||||
if self.find_image([BASE_IMAGE_ALIAS]):
|
||||
self.out.print(f" Base image '{BASE_IMAGE_ALIAS}' already cached.")
|
||||
return BASE_IMAGE_ALIAS
|
||||
|
||||
self.out.print(" Building base image (one-time setup) ...")
|
||||
|
||||
self.run(["delete", BASE_SETUP_NAME, "--force"], check=False)
|
||||
self.run(["image", "delete", BASE_IMAGE_ALIAS], check=False)
|
||||
self.run(["launch", UPSTREAM_IMAGE, BASE_SETUP_NAME])
|
||||
|
||||
ct = Container(self, BASE_SETUP_NAME)
|
||||
ct.wait_ready()
|
||||
|
||||
key_path = self.ssh_key_path
|
||||
pub_key = key_path.with_suffix(".pub").read_text().strip()
|
||||
host_ns = self.get_host_nameservers()
|
||||
ns_lines = "\n".join(f"nameserver {n}" for n in host_ns)
|
||||
ct.bash(f"""
|
||||
printf '{ns_lines}\n' > /etc/resolv.conf
|
||||
apt-get -o DPkg::Lock::Timeout=60 update
|
||||
DEBIAN_FRONTEND=noninteractive apt-get install -y openssh-server python3
|
||||
systemctl enable ssh
|
||||
apt-get clean
|
||||
mkdir -p /root/.ssh
|
||||
chmod 700 /root/.ssh
|
||||
echo '{pub_key}' > /root/.ssh/authorized_keys
|
||||
chmod 600 /root/.ssh/authorized_keys
|
||||
""")
|
||||
|
||||
self.run(["stop", BASE_SETUP_NAME])
|
||||
self.run(["publish", BASE_SETUP_NAME, f"--alias={BASE_IMAGE_ALIAS}"])
|
||||
self.run(["delete", BASE_SETUP_NAME, "--force"])
|
||||
self.out.print(f" Base image '{BASE_IMAGE_ALIAS}' ready.")
|
||||
return BASE_IMAGE_ALIAS
|
||||
|
||||
def get_container(self, name):
|
||||
"""Return a container handle for the given name.
|
||||
|
||||
Accepts both short relay names (``test0``) and full Incus
|
||||
container names (``test0-localchat``). Returns
|
||||
``DNSContainer`` for the DNS container and
|
||||
``RelayContainer`` for everything else.
|
||||
"""
|
||||
if name == DNS_CONTAINER_NAME:
|
||||
return DNSContainer(self)
|
||||
return RelayContainer(self, name.removesuffix("-localchat"))
|
||||
|
||||
def get_dns_container(self):
|
||||
"""Return a DNSContainer handle."""
|
||||
return DNSContainer(self)
|
||||
|
||||
|
||||
class Container:
|
||||
"""The base container handle wraps all interactions with incus."""
|
||||
|
||||
def __init__(self, incus, name, domain=None):
|
||||
self.incus = incus
|
||||
self.out = incus.out
|
||||
self.name = name
|
||||
self.domain = domain or f"{name}{DOMAIN_SUFFIX}"
|
||||
self.ipv4 = None
|
||||
self.ipv6 = None
|
||||
|
||||
def bash(self, script, check=True):
|
||||
"""Returns stdout from executing ``bash -ec <script>`` inside this container.
|
||||
|
||||
*script* is dedented and stripped so callers can use triple-quoted strings.
|
||||
When *check* is False, returns *None* on non-zero exit instead of raising.
|
||||
"""
|
||||
script = textwrap.dedent(script).strip()
|
||||
cmd = ["exec", self.name, "--", "bash", "-ec", script]
|
||||
return self.incus.run_output(cmd, check=check)
|
||||
|
||||
def run_cmd(self, *args, check=True):
|
||||
"""Return stdout from running a command directly in the container (no shell).
|
||||
|
||||
When *check* is False, returns *None* on non-zero exit instead of raising.
|
||||
"""
|
||||
return self.incus.run_output(
|
||||
["exec", self.name, "--", *args],
|
||||
check=check,
|
||||
)
|
||||
|
||||
def start(self):
|
||||
self.incus.run(["start", self.name])
|
||||
|
||||
def stop(self, force=False):
|
||||
cmd = ["stop", self.name]
|
||||
if force:
|
||||
cmd.append("--force")
|
||||
self.incus.run(cmd, check=False)
|
||||
|
||||
def launch(self):
|
||||
"""Launch from the best available image, return the alias used."""
|
||||
image = self.incus.find_image([RELAY_IMAGE_ALIAS, BASE_IMAGE_ALIAS])
|
||||
if not image:
|
||||
raise RuntimeError(
|
||||
f"No base image '{BASE_IMAGE_ALIAS}' found. "
|
||||
"Call ensure_base_image() before launching containers."
|
||||
)
|
||||
self.out.print(f" Launching from '{image}' image ...")
|
||||
cfg = []
|
||||
cfg += ("-c", f"{LABEL_KEY}=true")
|
||||
cfg += ("-c", f"user.localchat-domain={self.domain}")
|
||||
self.incus.run(["launch", image, self.name, *cfg])
|
||||
return image
|
||||
|
||||
def ensure(self):
|
||||
"""Create/start this container from the cached base image.
|
||||
|
||||
On first call, builds the base image (~30s).
|
||||
Subsequent containers launch in ~2s from the cached image.
|
||||
Returns ``self`` for chaining.
|
||||
"""
|
||||
data = self.incus.run_json(["list", self.name], check=False) or []
|
||||
|
||||
existing = [c for c in data if c["name"] == self.name]
|
||||
if existing:
|
||||
if existing[0]["status"] != "Running":
|
||||
self.start()
|
||||
else:
|
||||
self.launch()
|
||||
self.wait_ready()
|
||||
return self
|
||||
|
||||
def destroy(self):
|
||||
"""Stop, delete, and clean up config files."""
|
||||
self.stop(force=True)
|
||||
self.incus.run(["delete", self.name, "--force"], check=False)
|
||||
|
||||
def push_file_content(self, dest_path, content):
|
||||
"""Write *content* to *dest_path* inside the container.
|
||||
|
||||
*content* is dedented and stripped so callers can use
|
||||
indented triple-quoted strings.
|
||||
"""
|
||||
content = textwrap.dedent(content).strip() + "\n"
|
||||
self.incus.run(
|
||||
["file", "push", "-", f"{self.name}{dest_path}"],
|
||||
input=content,
|
||||
)
|
||||
self.bash(f"chmod 644 {dest_path}")
|
||||
|
||||
def wait_ready(self, timeout=60):
|
||||
"""Wait until the container is running with an IPv4 address.
|
||||
|
||||
Sets ``self.ipv4`` and ``self.ipv6`` (may be *None*),
|
||||
or raises ``TimeoutError``.
|
||||
"""
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
data = self.incus.run_json(
|
||||
["list", self.name],
|
||||
check=False,
|
||||
)
|
||||
if data and data[0].get("status") == "Running":
|
||||
net = data[0].get("state", {}).get("network", {})
|
||||
self.ipv4 = _extract_ip(net, "inet")
|
||||
self.ipv6 = _extract_ip(net, "inet6")
|
||||
if self.ipv4:
|
||||
return
|
||||
time.sleep(1)
|
||||
raise TimeoutError(
|
||||
f"Container {self.name!r} did not become ready within {timeout}s"
|
||||
)
|
||||
|
||||
def rss_mib(self):
|
||||
"""Return ``(used, total)`` memory from container (or None if unobtainable)."""
|
||||
output = self.bash("free -m", check=False)
|
||||
if output:
|
||||
for line in output.splitlines():
|
||||
if line.startswith("Mem:"):
|
||||
parts = line.split()
|
||||
return int(parts[2]), int(parts[1])
|
||||
|
||||
|
||||
class RelayContainer(Container):
|
||||
"""Container handle for a chatmail relay.
|
||||
|
||||
Accepts the short relay name (e.g. ``test0``) and derives
|
||||
the Incus container name and mail domain automatically.
|
||||
"""
|
||||
|
||||
def __init__(self, incus, name):
|
||||
super().__init__(
|
||||
incus,
|
||||
f"{name}-localchat",
|
||||
domain=f"_{name}{DOMAIN_SUFFIX}",
|
||||
)
|
||||
self.sname = name
|
||||
self.ini = incus.lxconfigs_dir / f"chatmail-{name}.ini"
|
||||
self.zone = incus.lxconfigs_dir / f"{name}.zone"
|
||||
|
||||
def launch(self):
|
||||
"""Launch (from a potentially cached image) and clear inherited chatmail-version."""
|
||||
image = super().launch()
|
||||
self.bash("rm -f /etc/chatmail-version")
|
||||
return image
|
||||
|
||||
def destroy(self):
|
||||
"""Stop, delete, and clean up config files."""
|
||||
super().destroy()
|
||||
if self.ini.exists():
|
||||
self.ini.unlink()
|
||||
|
||||
def disable_ipv6(self):
|
||||
"""Disable IPv6 inside the container via sysctl."""
|
||||
# incus provides net.* virtualization for LXC containers so that
|
||||
# these sysctls only affect the container's network namespace.
|
||||
self.bash("""
|
||||
sysctl -w net.ipv6.conf.all.disable_ipv6=1
|
||||
sysctl -w net.ipv6.conf.default.disable_ipv6=1
|
||||
""")
|
||||
self.push_file_content(
|
||||
"/etc/sysctl.d/99-disable-ipv6.conf",
|
||||
"""
|
||||
net.ipv6.conf.all.disable_ipv6=1
|
||||
net.ipv6.conf.default.disable_ipv6=1
|
||||
""",
|
||||
)
|
||||
|
||||
def configure_hosts(self, ip):
|
||||
"""Set hostname and /etc/hosts inside the container."""
|
||||
self.bash(f"""
|
||||
echo '{self.name}' > /etc/hostname
|
||||
hostname {self.name}
|
||||
sed -i '/ {self.domain}$/d' /etc/hosts
|
||||
echo '{ip} {self.name} {self.domain}' >> /etc/hosts
|
||||
""")
|
||||
|
||||
def publish_as_relay_image(self):
|
||||
"""Publish this container as a reusable relay image.
|
||||
|
||||
Stops the container, 'publishes' it as 'localchat-relay', then restarts it.
|
||||
"""
|
||||
if self.incus.find_image([RELAY_IMAGE_ALIAS]):
|
||||
return
|
||||
self.out.print(
|
||||
f" Locally caching {self.name!r} as '{RELAY_IMAGE_ALIAS}' image ..."
|
||||
)
|
||||
self.incus.run(
|
||||
["publish", self.name, f"--alias={RELAY_IMAGE_ALIAS}", "--force"]
|
||||
)
|
||||
self.wait_ready()
|
||||
self.out.print(f" Relay image '{RELAY_IMAGE_ALIAS}' ready.")
|
||||
|
||||
def deployed_version(self):
|
||||
"""Read /etc/chatmail-version, or None if absent."""
|
||||
return self.bash("cat /etc/chatmail-version", check=False)
|
||||
|
||||
def deployed_domain(self):
|
||||
"""Read the domain deployed on the container (postfix myhostname)."""
|
||||
return self.bash(
|
||||
"postconf -h myhostname 2>/dev/null",
|
||||
check=False,
|
||||
)
|
||||
|
||||
def verify_ssh(self, ssh_config):
|
||||
"""Verify SSH connectivity to this container."""
|
||||
cmd = f"ssh -F {ssh_config} -o ConnectTimeout=60 root@{self.domain} hostname"
|
||||
return shell(cmd, timeout=60).returncode == 0
|
||||
|
||||
def configure_dns(self, dns_ip):
|
||||
"""Point this container's resolver at *dns_ip* and verify DNS is reachable."""
|
||||
self.bash(f"""
|
||||
systemctl disable --now systemd-resolved 2>/dev/null || true
|
||||
rm -f /etc/resolv.conf
|
||||
printf 'nameserver {dns_ip}\\n' >/etc/resolv.conf
|
||||
mkdir -p /etc/unbound/unbound.conf.d
|
||||
""")
|
||||
self.push_file_content(
|
||||
"/etc/unbound/unbound.conf.d/localchat-forward.conf",
|
||||
f"""
|
||||
server:
|
||||
domain-insecure: "localchat"
|
||||
|
||||
forward-zone:
|
||||
name: "localchat"
|
||||
forward-addr: {dns_ip}
|
||||
""",
|
||||
)
|
||||
self.bash("systemctl restart unbound 2>/dev/null || true")
|
||||
self._wait_dns_reachable(dns_ip)
|
||||
|
||||
def _wait_dns_reachable(self, dns_ip, timeout=10):
|
||||
"""Poll until *dns_ip* answers a DNS query from this container."""
|
||||
if self.bash("which dig", check=False) is None:
|
||||
self.bash(
|
||||
"DEBIAN_FRONTEND=noninteractive "
|
||||
"apt-get install -y dnsutils 2>/dev/null || true"
|
||||
)
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
result = self.bash(
|
||||
f"dig @{dns_ip} . SOA +short +time=1 +tries=1",
|
||||
check=False,
|
||||
)
|
||||
if result and result.strip():
|
||||
return
|
||||
time.sleep(0.5)
|
||||
raise DNSConfigurationError(
|
||||
f"DNS at {dns_ip} not reachable from {self.name} after {timeout}s"
|
||||
)
|
||||
|
||||
def write_ini(self, disable_ipv6=False):
|
||||
"""Generate a chatmail.ini config file in lxconfigs/."""
|
||||
from chatmaild.config import write_initial_config
|
||||
|
||||
overrides = {
|
||||
"max_user_send_per_minute": 600,
|
||||
"max_user_send_burst_size": 100,
|
||||
"mtail_address": "127.0.0.1",
|
||||
}
|
||||
if disable_ipv6:
|
||||
overrides["disable_ipv6"] = "True"
|
||||
write_initial_config(self.ini, self.domain, overrides)
|
||||
return self.ini
|
||||
|
||||
|
||||
class DNSContainer(Container):
|
||||
"""Container handle for the PowerDNS name server.
|
||||
|
||||
Manages the authoritative and recursive DNS services required for
|
||||
name resolution in the local testing environment.
|
||||
"""
|
||||
|
||||
def __init__(self, incus):
|
||||
super().__init__(incus, DNS_CONTAINER_NAME, domain=DNS_DOMAIN)
|
||||
|
||||
def pdnsutil(self, *args, check=True):
|
||||
"""Run ``pdnsutil <args>`` inside the DNS container."""
|
||||
return self.run_cmd("pdnsutil", *args, check=check)
|
||||
|
||||
def replace_rrset(self, zone, name, rtype, ttl, rdata):
|
||||
"""Shortcut for ``pdnsutil replace-rrset``."""
|
||||
self.pdnsutil("replace-rrset", zone, name, rtype, ttl, rdata)
|
||||
|
||||
def restart_services(self):
|
||||
"""Restart pdns and pdns-recursor, then wait until DNS is answering."""
|
||||
self.bash("""
|
||||
systemctl restart pdns
|
||||
systemctl restart pdns-recursor || true
|
||||
""")
|
||||
self._wait_dns_ready()
|
||||
|
||||
def _wait_dns_ready(self, timeout=60):
|
||||
"""Poll until the recursor answers a query on port 53."""
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
result = self.bash(
|
||||
"dig @127.0.0.1 . SOA +short +time=1 +tries=1",
|
||||
check=False,
|
||||
)
|
||||
if result and result.strip():
|
||||
return
|
||||
time.sleep(0.5)
|
||||
raise DNSConfigurationError(f"DNS recursor not answering after {timeout}s")
|
||||
|
||||
def ensure(self):
|
||||
"""Create the DNS container with PowerDNS if needed.
|
||||
|
||||
Calls ``super().ensure()`` to create/start the container
|
||||
and set up SSH, then installs PowerDNS and configures
|
||||
the Incus bridge to use this container as DNS.
|
||||
"""
|
||||
super().ensure()
|
||||
self._install_powerdns()
|
||||
self.incus.run(
|
||||
["network", "set", "incusbr0", "dns.mode=none"],
|
||||
check=False,
|
||||
)
|
||||
self.incus.run(
|
||||
["network", "set", "incusbr0", f"raw.dnsmasq=dhcp-option=6,{self.ipv4}"],
|
||||
check=False,
|
||||
)
|
||||
|
||||
def destroy(self):
|
||||
"""Stop, delete, and reset bridge DNS config."""
|
||||
super().destroy()
|
||||
self.incus.run(["network", "unset", "incusbr0", "dns.mode"], check=False)
|
||||
self.incus.run(["network", "unset", "incusbr0", "raw.dnsmasq"], check=False)
|
||||
|
||||
def _install_powerdns(self):
|
||||
"""Install and configure PowerDNS if not already present."""
|
||||
if self.run_cmd("which", "pdns_server", check=False) is not None:
|
||||
return
|
||||
|
||||
host_ns = self.incus.get_host_nameservers()
|
||||
ns_lines = "\n".join(f"nameserver {n}" for n in host_ns)
|
||||
|
||||
self.bash(f"""
|
||||
systemctl disable --now systemd-resolved 2>/dev/null || true
|
||||
rm -f /etc/resolv.conf
|
||||
printf '{ns_lines}\n' > /etc/resolv.conf
|
||||
|
||||
# Block automatic service startup during package installation
|
||||
printf '#!/bin/sh\\nexit 101\\n' > /usr/sbin/policy-rc.d
|
||||
chmod +x /usr/sbin/policy-rc.d
|
||||
|
||||
apt-get -o DPkg::Lock::Timeout=60 update
|
||||
DEBIAN_FRONTEND=noninteractive apt-get install -y \
|
||||
pdns-server pdns-backend-sqlite3 sqlite3 pdns-recursor dnsutils
|
||||
|
||||
# Remove the startup block
|
||||
rm /usr/sbin/policy-rc.d
|
||||
|
||||
systemctl stop pdns pdns-recursor || true
|
||||
mkdir -p /var/lib/powerdns
|
||||
sqlite3 /var/lib/powerdns/pdns.sqlite3 \
|
||||
</usr/share/doc/pdns-backend-sqlite3/schema.sqlite3.sql
|
||||
chown -R pdns:pdns /var/lib/powerdns
|
||||
""")
|
||||
|
||||
self.push_file_content(
|
||||
"/etc/powerdns/pdns.conf",
|
||||
"""
|
||||
launch=gsqlite3
|
||||
gsqlite3-database=/var/lib/powerdns/pdns.sqlite3
|
||||
local-address=127.0.0.1
|
||||
local-port=5353
|
||||
""",
|
||||
)
|
||||
|
||||
self.push_file_content(
|
||||
"/etc/powerdns/recursor.conf",
|
||||
"""
|
||||
local-address=0.0.0.0
|
||||
local-port=53
|
||||
forward-zones=localchat=127.0.0.1:5353
|
||||
allow-from=0.0.0.0/0
|
||||
dont-query=
|
||||
dnssec=off
|
||||
""",
|
||||
)
|
||||
|
||||
self.bash("""
|
||||
systemctl start pdns
|
||||
systemctl start pdns-recursor
|
||||
echo 'nameserver 127.0.0.1' > /etc/resolv.conf
|
||||
""")
|
||||
self._wait_dns_ready()
|
||||
|
||||
def reset_dns_records(self, dns_ip, domains):
|
||||
"""Create DNS zones with initial A records via pdnsutil.
|
||||
|
||||
Only sets SOA, NS, and A records as the minimal set
|
||||
needed for SSH connectivity. Full records (MX, TXT, SRV,
|
||||
CNAME, DKIM) are added later by ``cmdeploy dns``.
|
||||
|
||||
Args:
|
||||
dns_ip: IP of the DNS container
|
||||
domains: list of dicts with 'name', 'domain', 'ip'
|
||||
"""
|
||||
for d in domains:
|
||||
domain = d["domain"]
|
||||
ip = d["ip"]
|
||||
self.out.print(f" {domain} -> {ip}")
|
||||
|
||||
# Delete and recreate zone fresh (removes stale records)
|
||||
self.pdnsutil("delete-zone", domain, check=False)
|
||||
self.pdnsutil("create-zone", domain, f"ns.{domain}")
|
||||
|
||||
serial = str(int(time.time()))
|
||||
soa = f"ns.{domain} hostmaster.{domain} {serial} 3600 900 604800 300"
|
||||
self.replace_rrset(domain, ".", "SOA", "3600", soa)
|
||||
self.replace_rrset(domain, ".", "NS", "3600", f"ns.{domain}.")
|
||||
self.replace_rrset(domain, ".", "A", "3600", ip)
|
||||
self.replace_rrset(domain, "ns", "A", "3600", dns_ip)
|
||||
|
||||
# AAAA (domain -> container IPv6, if available)
|
||||
ipv6 = d.get("ipv6")
|
||||
if ipv6:
|
||||
self.replace_rrset(domain, ".", "AAAA", "3600", ipv6)
|
||||
self.out.print(f" zone reset: SOA, NS, A, AAAA ({ip}, {ipv6})")
|
||||
else:
|
||||
# Remove any stale AAAA record
|
||||
self.pdnsutil("delete-rrset", domain, ".", "AAAA", check=False)
|
||||
self.out.print(f" zone reset: SOA, NS, A ({ip}, IPv4-only)")
|
||||
|
||||
self.restart_services()
|
||||
|
||||
def set_dns_records(self, text):
|
||||
"""Add or overwrite DNS records from standard BIND format.
|
||||
|
||||
Uses ``cmdeploy.dns.parse_zone_records`` to parse.
|
||||
Zones are created automatically from the record names.
|
||||
"""
|
||||
from ..dns import parse_zone_records
|
||||
|
||||
zones_seen = set()
|
||||
|
||||
for name, ttl, rtype, rdata in parse_zone_records(text):
|
||||
# Derive zone from name: find top-level .localchat domain
|
||||
name_parts = name.split(".")
|
||||
zone = name # fallback
|
||||
for i in range(len(name_parts) - 1):
|
||||
if name_parts[i + 1 :] == ["localchat"]:
|
||||
zone = ".".join(name_parts[i:])
|
||||
break
|
||||
|
||||
# Create zone if first time seeing it
|
||||
if zone not in zones_seen:
|
||||
self.pdnsutil(
|
||||
"create-zone",
|
||||
zone,
|
||||
f"ns.{zone}",
|
||||
check=False,
|
||||
)
|
||||
zones_seen.add(zone)
|
||||
|
||||
# Figure out the record name relative to zone
|
||||
if name == zone:
|
||||
relative = "."
|
||||
elif name.endswith(f".{zone}"):
|
||||
relative = name[: -(len(zone) + 1)]
|
||||
else:
|
||||
relative = name
|
||||
|
||||
self.replace_rrset(zone, relative, rtype, ttl, rdata)
|
||||
|
||||
if zones_seen:
|
||||
self.restart_services()
|
||||
@@ -28,6 +28,13 @@ counter created_nonci_accounts
|
||||
}
|
||||
}
|
||||
|
||||
# doveauth refusing new addresses because a chatmail.ini
|
||||
# system resource limit is exceeded.
|
||||
counter rejected_registrations
|
||||
/registration rejected: / {
|
||||
rejected_registrations++
|
||||
}
|
||||
|
||||
counter postfix_timeouts
|
||||
/timeout after DATA/ {
|
||||
postfix_timeouts++
|
||||
@@ -73,8 +80,21 @@ counter incoming_unencrypted_mail_count
|
||||
filtered_incoming_mail_count++
|
||||
}
|
||||
|
||||
counter incoming_mailer_daemon_mail_count
|
||||
/Incoming: Filtering mailer-daemon message from/ {
|
||||
incoming_mailer_daemon_mail_count++
|
||||
filtered_incoming_mail_count++
|
||||
}
|
||||
|
||||
counter rejected_unencrypted_mail_count
|
||||
/Rejected unencrypted mail/ {
|
||||
rejected_unencrypted_mail_count++
|
||||
}
|
||||
|
||||
counter quota_expire_runs
|
||||
counter quota_expire_removed_files
|
||||
|
||||
/quota-expire: removed (?P<count>\d+) message\(s\)/ {
|
||||
quota_expire_runs++
|
||||
quota_expire_removed_files += $count
|
||||
}
|
||||
|
||||
@@ -1,10 +1,7 @@
|
||||
from pyinfra import facts, host
|
||||
from pyinfra.operations import apt, files, server, systemd
|
||||
from pyinfra.operations import apt
|
||||
|
||||
from cmdeploy.basedeploy import (
|
||||
Deployer,
|
||||
get_resource,
|
||||
)
|
||||
from cmdeploy.basedeploy import Deployer
|
||||
|
||||
|
||||
class MtailDeployer(Deployer):
|
||||
@@ -18,51 +15,30 @@ class MtailDeployer(Deployer):
|
||||
(url, sha256sum) = {
|
||||
"x86_64": (
|
||||
"https://github.com/google/mtail/releases/download/v3.0.8/mtail_3.0.8_linux_amd64.tar.gz",
|
||||
"123c2ee5f48c3eff12ebccee38befd2233d715da736000ccde49e3d5607724e4",
|
||||
"d55cb601049c5e61eabab29998dbbcea95d480e5448544f9470337ba2eea882e",
|
||||
),
|
||||
"aarch64": (
|
||||
"https://github.com/google/mtail/releases/download/v3.0.8/mtail_3.0.8_linux_arm64.tar.gz",
|
||||
"aa04811c0929b6754408676de520e050c45dddeb3401881888a092c9aea89cae",
|
||||
"f748db8ad2a1e0b63684d4c8868cf6a373a20f7e6922e5ece601fff0ee00eb1a",
|
||||
),
|
||||
}[host.get_fact(facts.server.Arch)]
|
||||
|
||||
server.shell(
|
||||
name="Download mtail",
|
||||
commands=[
|
||||
f"(echo '{sha256sum} /usr/local/bin/mtail' | sha256sum -c) || (curl -L {url} | gunzip | tar -x -f - mtail -O >/usr/local/bin/mtail.new && mv /usr/local/bin/mtail.new /usr/local/bin/mtail)",
|
||||
"chmod 755 /usr/local/bin/mtail",
|
||||
],
|
||||
self.download_executable(
|
||||
url,
|
||||
"/usr/local/bin/mtail",
|
||||
sha256sum,
|
||||
extract="gunzip | tar -xf - mtail -O",
|
||||
)
|
||||
|
||||
def configure(self):
|
||||
# Using our own systemd unit instead of `/usr/lib/systemd/system/mtail.service`.
|
||||
# This allows to read from journalctl instead of log files.
|
||||
files.template(
|
||||
src=get_resource("mtail/mtail.service.j2"),
|
||||
dest="/etc/systemd/system/mtail.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.ensure_systemd_unit(
|
||||
"mtail/mtail.service.j2",
|
||||
address=self.mtail_address or "127.0.0.1",
|
||||
port=3903,
|
||||
)
|
||||
|
||||
mtail_conf = files.put(
|
||||
name="Mtail configuration",
|
||||
src=get_resource("mtail/delivered_mail.mtail"),
|
||||
dest="/etc/mtail/delivered_mail.mtail",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart = mtail_conf.changed
|
||||
self.put_file("mtail/delivered_mail.mtail", "/etc/mtail/delivered_mail.mtail")
|
||||
|
||||
def activate(self):
|
||||
systemd.service(
|
||||
name="Start and enable mtail",
|
||||
service="mtail.service",
|
||||
running=bool(self.mtail_address),
|
||||
enabled=bool(self.mtail_address),
|
||||
restarted=self.need_restart,
|
||||
)
|
||||
self.need_restart = False
|
||||
active = bool(self.mtail_address)
|
||||
self.ensure_service("mtail.service", running=active, enabled=active)
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# filtermail.mtail: filtermail process event counters
|
||||
|
||||
#
|
||||
# Counters
|
||||
#
|
||||
|
||||
# Connection errors in the transport path
|
||||
# Error::Io / Error::ConnectionFailed / Error::Tls: the reason string filtermail strips
|
||||
counter filtermail_transport_error_total by reason
|
||||
|
||||
# Unexpected transport errors: DNS failure, HTTP error, config error.
|
||||
counter filtermail_transport_unexpected_total
|
||||
|
||||
# Silent drops: inbound
|
||||
# "unencrypted" overlaps with rejected_unencrypted_mail_count in delivered_mail.mtail.
|
||||
counter filtermail_inbound_drop_total by reason
|
||||
|
||||
# Silent drops: outbound
|
||||
# "unencrypted" overlaps with rejected_unencrypted_mail_count in delivered_mail.mtail;
|
||||
# "sender_disabled" and "all_recipients_disabled" are new.
|
||||
counter filtermail_outbound_drop_total by reason
|
||||
|
||||
# Reinject failures
|
||||
counter filtermail_reinject_error_total by direction
|
||||
|
||||
# SMTP-level connection errors (client dropped mid-session)
|
||||
counter filtermail_smtp_error_total by reason
|
||||
|
||||
#
|
||||
# filtermail::transport
|
||||
#
|
||||
|
||||
/filtermail\[\d+\]: \[WARN\s+filtermail::transport::worker\] Connection error relaying to mail server \S+: / {
|
||||
/timed out/ {
|
||||
filtermail_transport_error_total["connection_timeout"]++
|
||||
} otherwise {
|
||||
/Failed to connect to any of the following addresses/ {
|
||||
filtermail_transport_error_total["connection_refused"]++
|
||||
} otherwise {
|
||||
/Connection refused/ {
|
||||
filtermail_transport_error_total["connection_refused"]++
|
||||
} otherwise {
|
||||
/invalid certificate/ {
|
||||
filtermail_transport_error_total["tls_cert_not_verified"]++
|
||||
} otherwise {
|
||||
/peer sent fatal alert|handshake/ {
|
||||
filtermail_transport_error_total["tls_handshake"]++
|
||||
} otherwise {
|
||||
/close_notify|connection closed/ {
|
||||
filtermail_transport_error_total["lost_connection"]++
|
||||
} otherwise {
|
||||
filtermail_transport_error_total["other"]++
|
||||
}}}}}}}
|
||||
|
||||
/filtermail\[\d+\]: \[WARN\s+filtermail::transport::worker\] Unexpected error while delivering/ {
|
||||
filtermail_transport_unexpected_total++
|
||||
}
|
||||
|
||||
#
|
||||
# filtermail::smtp_server
|
||||
#
|
||||
|
||||
/filtermail(?:-incoming)?\[\d+\]: \[WARN\s+filtermail::smtp_server\] Unexpected EoF while receiving DATA/ {
|
||||
filtermail_smtp_error_total["unexpected_eof"]++
|
||||
}
|
||||
|
||||
/filtermail(?:-incoming)?\[\d+\]: \[WARN\s+filtermail::smtp_server\] Malformed DATA line without CRLF/ {
|
||||
filtermail_smtp_error_total["malformed_data_line"]++
|
||||
}
|
||||
|
||||
#
|
||||
# filtermail::outbound
|
||||
#
|
||||
|
||||
/filtermail\[\d+\]: \[WARN\s+filtermail::outbound\] Rejected unencrypted mail/ {
|
||||
filtermail_outbound_drop_total["unencrypted"]++
|
||||
}
|
||||
|
||||
/filtermail\[\d+\]: \[WARN\s+filtermail::outbound\] Dropping mail; Sender .* is disabled/ {
|
||||
filtermail_outbound_drop_total["sender_disabled"]++
|
||||
}
|
||||
|
||||
/filtermail\[\d+\]: \[WARN\s+filtermail::outbound\] Dropping mail; All recipients disabled/ {
|
||||
filtermail_outbound_drop_total["all_recipients_disabled"]++
|
||||
}
|
||||
|
||||
/filtermail\[\d+\]: \[WARN\s+filtermail::outbound\] Failed to re.inject mail/ {
|
||||
filtermail_reinject_error_total["outbound"]++
|
||||
}
|
||||
|
||||
#
|
||||
# filtermail::inbound
|
||||
#
|
||||
|
||||
/filtermail(?:-incoming)?\[\d+\]: \[WARN\s+filtermail::inbound\] Rejected unencrypted mail/ {
|
||||
filtermail_inbound_drop_total["unencrypted"]++
|
||||
}
|
||||
|
||||
/filtermail(?:-incoming)?\[\d+\]: \[WARN\s+filtermail::inbound\] Failed to re.inject mail/ {
|
||||
filtermail_reinject_error_total["inbound"]++
|
||||
}
|
||||
@@ -1,10 +1,13 @@
|
||||
[Unit]
|
||||
Description=mtail
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/bin/sh -c "journalctl -f -o short-iso -n 0 | /usr/local/bin/mtail --address={{ address }} --port={{ port }} --progs /etc/mtail --logtostderr --logs -"
|
||||
Restart=on-failure
|
||||
RestartSec=2s
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from chatmaild.config import Config
|
||||
from pyinfra.operations import apt, files, systemd
|
||||
from pyinfra.operations import apt
|
||||
|
||||
from cmdeploy.basedeploy import (
|
||||
Deployer,
|
||||
@@ -31,87 +31,50 @@ class NginxDeployer(Deployer):
|
||||
# For documentation about policy-rc.d, see:
|
||||
# https://people.debian.org/~hmh/invokerc.d-policyrc.d-specification.txt
|
||||
#
|
||||
files.put(
|
||||
src=get_resource("policy-rc.d"),
|
||||
dest="/usr/sbin/policy-rc.d",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="755",
|
||||
)
|
||||
self.put_executable(src="policy-rc.d", dest="/usr/sbin/policy-rc.d")
|
||||
|
||||
apt.packages(
|
||||
name="Install nginx",
|
||||
packages=["nginx", "libnginx-mod-stream"],
|
||||
)
|
||||
|
||||
files.file("/usr/sbin/policy-rc.d", present=False)
|
||||
self.remove_file("/usr/sbin/policy-rc.d")
|
||||
|
||||
def configure(self):
|
||||
self.need_restart = _configure_nginx(self.config)
|
||||
_configure_nginx(self, self.config)
|
||||
|
||||
def activate(self):
|
||||
systemd.service(
|
||||
name="Start and enable nginx",
|
||||
service="nginx.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=self.need_restart,
|
||||
)
|
||||
self.need_restart = False
|
||||
self.ensure_service("nginx.service")
|
||||
|
||||
|
||||
def _configure_nginx(config: Config, debug: bool = False) -> bool:
|
||||
def _configure_nginx(deployer, config: Config, debug: bool = False):
|
||||
"""Configures nginx HTTP server."""
|
||||
need_restart = False
|
||||
|
||||
main_config = files.template(
|
||||
src=get_resource("nginx/nginx.conf.j2"),
|
||||
dest="/etc/nginx/nginx.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
deployer.put_template(
|
||||
"nginx/nginx.conf.j2",
|
||||
"/etc/nginx/nginx.conf",
|
||||
config=config,
|
||||
disable_ipv6=config.disable_ipv6,
|
||||
)
|
||||
need_restart |= main_config.changed
|
||||
|
||||
autoconfig = files.template(
|
||||
src=get_resource("nginx/autoconfig.xml.j2"),
|
||||
dest="/var/www/html/.well-known/autoconfig/mail/config-v1.1.xml",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
deployer.put_template(
|
||||
"nginx/autoconfig.xml.j2",
|
||||
"/var/www/html/.well-known/autoconfig/mail/config-v1.1.xml",
|
||||
config=config,
|
||||
)
|
||||
need_restart |= autoconfig.changed
|
||||
|
||||
mta_sts_config = files.template(
|
||||
src=get_resource("nginx/mta-sts.txt.j2"),
|
||||
dest="/var/www/html/.well-known/mta-sts.txt",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
deployer.put_template(
|
||||
"nginx/mta-sts.txt.j2",
|
||||
"/var/www/html/.well-known/mta-sts.txt",
|
||||
config=config,
|
||||
)
|
||||
need_restart |= mta_sts_config.changed
|
||||
|
||||
# install CGI newemail script
|
||||
#
|
||||
cgi_dir = "/usr/lib/cgi-bin"
|
||||
files.directory(
|
||||
name=f"Ensure {cgi_dir} exists",
|
||||
path=cgi_dir,
|
||||
user="root",
|
||||
group="root",
|
||||
)
|
||||
deployer.ensure_directory(cgi_dir)
|
||||
|
||||
files.put(
|
||||
name="Upload cgi newemail.py script",
|
||||
deployer.put_executable(
|
||||
src=get_resource("newemail.py", pkg="chatmaild").open("rb"),
|
||||
dest=f"{cgi_dir}/newemail.py",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="755",
|
||||
)
|
||||
|
||||
return need_restart
|
||||
|
||||
@@ -42,6 +42,9 @@ stream {
|
||||
}
|
||||
|
||||
http {
|
||||
# access_log setting is inherited by all server sections
|
||||
access_log syslog:server=unix:/dev/log,facility=local7;
|
||||
|
||||
{% if config.tls_cert_mode == "self" %}
|
||||
limit_req_zone $binary_remote_addr zone=newaccount:10m rate=2r/s;
|
||||
{% endif %}
|
||||
@@ -69,9 +72,11 @@ http {
|
||||
|
||||
index index.html index.htm;
|
||||
|
||||
server_name {{ config.mail_domain }} www.{{ config.mail_domain }} mta-sts.{{ config.mail_domain }};
|
||||
server_name {{ config.mail_domain }} mta-sts.{{ config.mail_domain }};
|
||||
|
||||
access_log syslog:server=unix:/dev/log,facility=local7;
|
||||
location /mxdeliv {
|
||||
proxy_pass http://127.0.0.1:{{ config.filtermail_http_port_incoming }};
|
||||
}
|
||||
|
||||
location / {
|
||||
# First attempt to serve request as file, then
|
||||
@@ -123,7 +128,10 @@ http {
|
||||
proxy_set_header Connection "upgrade";
|
||||
}
|
||||
|
||||
location /relay/probe {
|
||||
# Endpoints Iroh uses for net_report probes and that clients
|
||||
# probe to tell whether this relay works. Both paths are served
|
||||
# by iroh-relay 0.35 and by the 1.0 line.
|
||||
location /ping {
|
||||
proxy_pass http://127.0.0.1:3340;
|
||||
proxy_http_version 1.1;
|
||||
}
|
||||
@@ -139,7 +147,6 @@ http {
|
||||
listen 127.0.0.1:8443 ssl;
|
||||
server_name www.{{ config.mail_domain }};
|
||||
return 301 $scheme://{{ config.mail_domain }}$request_uri;
|
||||
access_log syslog:server=unix:/dev/log,facility=local7;
|
||||
}
|
||||
|
||||
server {
|
||||
|
||||
@@ -4,9 +4,9 @@ Installs OpenDKIM
|
||||
|
||||
from pyinfra import host
|
||||
from pyinfra.facts.files import File
|
||||
from pyinfra.operations import apt, files, server, systemd
|
||||
from pyinfra.operations import apt, files, server
|
||||
|
||||
from cmdeploy.basedeploy import Deployer, get_resource
|
||||
from cmdeploy.basedeploy import Deployer
|
||||
|
||||
|
||||
class OpendkimDeployer(Deployer):
|
||||
@@ -25,65 +25,39 @@ class OpendkimDeployer(Deployer):
|
||||
domain = self.mail_domain
|
||||
dkim_selector = "opendkim"
|
||||
"""Configures OpenDKIM"""
|
||||
need_restart = False
|
||||
|
||||
main_config = files.template(
|
||||
src=get_resource("opendkim/opendkim.conf"),
|
||||
dest="/etc/opendkim.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.put_template(
|
||||
"opendkim/opendkim.conf",
|
||||
"/etc/opendkim.conf",
|
||||
config={"domain_name": domain, "opendkim_selector": dkim_selector},
|
||||
)
|
||||
need_restart |= main_config.changed
|
||||
|
||||
screen_script = files.file(
|
||||
path="/etc/opendkim/screen.lua",
|
||||
present=False,
|
||||
)
|
||||
need_restart |= screen_script.changed
|
||||
self.remove_file("/etc/opendkim/screen.lua")
|
||||
self.remove_file("/etc/opendkim/final.lua")
|
||||
|
||||
final_script = files.file(
|
||||
path="/etc/opendkim/final.lua",
|
||||
present=False,
|
||||
)
|
||||
need_restart |= final_script.changed
|
||||
|
||||
files.directory(
|
||||
name="Add opendkim directory to /etc",
|
||||
path="/etc/opendkim",
|
||||
user="opendkim",
|
||||
group="opendkim",
|
||||
self.ensure_directory(
|
||||
"/etc/opendkim",
|
||||
owner="opendkim",
|
||||
mode="750",
|
||||
present=True,
|
||||
)
|
||||
|
||||
keytable = files.template(
|
||||
src=get_resource("opendkim/KeyTable"),
|
||||
dest="/etc/dkimkeys/KeyTable",
|
||||
user="opendkim",
|
||||
group="opendkim",
|
||||
mode="644",
|
||||
self.put_template(
|
||||
"opendkim/KeyTable",
|
||||
"/etc/dkimkeys/KeyTable",
|
||||
owner="opendkim",
|
||||
config={"domain_name": domain, "opendkim_selector": dkim_selector},
|
||||
)
|
||||
need_restart |= keytable.changed
|
||||
|
||||
signing_table = files.template(
|
||||
src=get_resource("opendkim/SigningTable"),
|
||||
dest="/etc/dkimkeys/SigningTable",
|
||||
user="opendkim",
|
||||
group="opendkim",
|
||||
mode="644",
|
||||
self.put_template(
|
||||
"opendkim/SigningTable",
|
||||
"/etc/dkimkeys/SigningTable",
|
||||
owner="opendkim",
|
||||
config={"domain_name": domain, "opendkim_selector": dkim_selector},
|
||||
)
|
||||
need_restart |= signing_table.changed
|
||||
files.directory(
|
||||
name="Add opendkim socket directory to /var/spool/postfix",
|
||||
path="/var/spool/postfix/opendkim",
|
||||
user="opendkim",
|
||||
group="opendkim",
|
||||
self.ensure_directory(
|
||||
"/var/spool/postfix/opendkim",
|
||||
owner="opendkim",
|
||||
mode="750",
|
||||
present=True,
|
||||
)
|
||||
|
||||
if not host.get_fact(File, f"/etc/dkimkeys/{dkim_selector}.private"):
|
||||
@@ -96,12 +70,10 @@ class OpendkimDeployer(Deployer):
|
||||
_su_user="opendkim",
|
||||
)
|
||||
|
||||
service_file = files.put(
|
||||
name="Configure opendkim to restart once a day",
|
||||
src=get_resource("opendkim/systemd.conf"),
|
||||
dest="/etc/systemd/system/opendkim.service.d/10-prevent-memory-leak.conf",
|
||||
self.put_file(
|
||||
"opendkim/systemd.conf",
|
||||
"/etc/systemd/system/opendkim.service.d/10-prevent-memory-leak.conf",
|
||||
)
|
||||
need_restart |= service_file.changed
|
||||
|
||||
files.file(
|
||||
name="chown opendkim: /etc/dkimkeys/opendkim.private",
|
||||
@@ -110,15 +82,5 @@ class OpendkimDeployer(Deployer):
|
||||
group="opendkim",
|
||||
)
|
||||
|
||||
self.need_restart = need_restart
|
||||
|
||||
def activate(self):
|
||||
systemd.service(
|
||||
name="Start and enable OpenDKIM",
|
||||
service="opendkim.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
daemon_reload=self.need_restart,
|
||||
restarted=self.need_restart,
|
||||
)
|
||||
self.need_restart = False
|
||||
self.ensure_service("opendkim.service")
|
||||
|
||||
@@ -1,11 +1,10 @@
|
||||
from pyinfra.operations import apt, files, server, systemd
|
||||
from pyinfra.operations import apt, server
|
||||
|
||||
from cmdeploy.basedeploy import Deployer, get_resource
|
||||
from cmdeploy.basedeploy import Deployer
|
||||
|
||||
|
||||
class PostfixDeployer(Deployer):
|
||||
required_users = [("postfix", None, ["opendkim"])]
|
||||
daemon_reload = False
|
||||
|
||||
def __init__(self, config, disable_mail):
|
||||
self.config = config
|
||||
@@ -19,81 +18,46 @@ class PostfixDeployer(Deployer):
|
||||
|
||||
def configure(self):
|
||||
config = self.config
|
||||
need_restart = False
|
||||
|
||||
main_config = files.template(
|
||||
src=get_resource("postfix/main.cf.j2"),
|
||||
dest="/etc/postfix/main.cf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.put_template(
|
||||
"postfix/main.cf.j2",
|
||||
"/etc/postfix/main.cf",
|
||||
config=config,
|
||||
disable_ipv6=config.disable_ipv6,
|
||||
)
|
||||
need_restart |= main_config.changed
|
||||
|
||||
master_config = files.template(
|
||||
src=get_resource("postfix/master.cf.j2"),
|
||||
dest="/etc/postfix/master.cf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.put_template(
|
||||
"postfix/master.cf.j2",
|
||||
"/etc/postfix/master.cf",
|
||||
debug=False,
|
||||
config=config,
|
||||
)
|
||||
need_restart |= master_config.changed
|
||||
|
||||
header_cleanup = files.put(
|
||||
src=get_resource("postfix/submission_header_cleanup"),
|
||||
dest="/etc/postfix/submission_header_cleanup",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
self.put_file(
|
||||
"postfix/submission_header_cleanup",
|
||||
"/etc/postfix/submission_header_cleanup",
|
||||
)
|
||||
need_restart |= header_cleanup.changed
|
||||
self.put_file("postfix/lmtp_header_cleanup", "/etc/postfix/lmtp_header_cleanup")
|
||||
|
||||
lmtp_header_cleanup = files.put(
|
||||
src=get_resource("postfix/lmtp_header_cleanup"),
|
||||
dest="/etc/postfix/lmtp_header_cleanup",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
res = self.put_file(
|
||||
"postfix/smtp_tls_policy_map", "/etc/postfix/smtp_tls_policy_map"
|
||||
)
|
||||
need_restart |= lmtp_header_cleanup.changed
|
||||
|
||||
tls_policy_map = files.put(
|
||||
name="Upload SMTP TLS Policy that accepts self-signed certificates for IP-only hosts",
|
||||
src=get_resource("postfix/smtp_tls_policy_map"),
|
||||
dest="/etc/postfix/smtp_tls_policy_map",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= tls_policy_map.changed
|
||||
if tls_policy_map.changed:
|
||||
tls_policy_changed = res.changed
|
||||
if tls_policy_changed:
|
||||
server.shell(
|
||||
commands=["postmap /etc/postfix/smtp_tls_policy_map"],
|
||||
)
|
||||
|
||||
# Login map that 1:1 maps email address to login.
|
||||
login_map = files.put(
|
||||
src=get_resource("postfix/login_map"),
|
||||
dest="/etc/postfix/login_map",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= login_map.changed
|
||||
self.put_file("postfix/login_map", "/etc/postfix/login_map")
|
||||
|
||||
restart_conf = files.put(
|
||||
name="postfix: restart automatically on failure",
|
||||
src=get_resource("service/10_restart.conf"),
|
||||
dest="/etc/systemd/system/postfix@.service.d/10_restart.conf",
|
||||
self.put_file(
|
||||
"service/10_restart_on_failure.conf",
|
||||
"/etc/systemd/system/postfix@.service.d/10_restart.conf",
|
||||
)
|
||||
self.daemon_reload = restart_conf.changed
|
||||
|
||||
# Validate postfix configuration before restart
|
||||
if need_restart:
|
||||
if self.need_restart:
|
||||
server.shell(
|
||||
name="Validate postfix configuration",
|
||||
# Extract stderr and quit with error if non-zero
|
||||
@@ -101,19 +65,11 @@ class PostfixDeployer(Deployer):
|
||||
"""bash -c 'w=$(postconf 2>&1 >/dev/null); [[ -z "$w" ]] || { echo "$w"; false; }'"""
|
||||
],
|
||||
)
|
||||
self.need_restart = need_restart
|
||||
|
||||
def activate(self):
|
||||
restart = False if self.disable_mail else self.need_restart
|
||||
|
||||
systemd.service(
|
||||
name="disable postfix for now"
|
||||
if self.disable_mail
|
||||
else "Start and enable Postfix",
|
||||
service="postfix.service",
|
||||
running=False if self.disable_mail else True,
|
||||
enabled=False if self.disable_mail else True,
|
||||
restarted=restart,
|
||||
daemon_reload=self.daemon_reload,
|
||||
active = not self.disable_mail
|
||||
self.ensure_service(
|
||||
"postfix.service",
|
||||
running=active,
|
||||
enabled=active,
|
||||
)
|
||||
self.need_restart = False
|
||||
|
||||
@@ -20,7 +20,7 @@ smtpd_tls_key_file={{ config.tls_key_path }}
|
||||
smtpd_tls_security_level=may
|
||||
|
||||
smtp_tls_CApath=/etc/ssl/certs
|
||||
smtp_tls_security_level={{ "verify" if config.tls_cert_mode == "acme" else "encrypt" }}
|
||||
smtp_tls_security_level=verify
|
||||
# Send SNI extension when connecting to other servers.
|
||||
# <https://www.postfix.org/postconf.5.html#smtp_tls_servername>
|
||||
smtp_tls_servername = hostname
|
||||
@@ -53,15 +53,19 @@ smtpd_tls_exclude_ciphers = aNULL, RC4, MD5, DES
|
||||
# See <https://www.postfix.org/FORWARD_SECRECY_README.html#server_fs>.
|
||||
tls_preempt_cipherlist = yes
|
||||
|
||||
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
|
||||
myhostname = {{ config.mail_domain }}
|
||||
# Reject by default, override per smtpd in master.cf
|
||||
smtpd_relay_restrictions = reject
|
||||
myhostname = {{ config.postfix_myhostname }}
|
||||
alias_maps = hash:/etc/aliases
|
||||
alias_database = hash:/etc/aliases
|
||||
|
||||
# Postfix does not deliver mail for any domain by itself.
|
||||
# Primary domain is listed in `virtual_mailbox_domains` instead
|
||||
# and handed over to Dovecot.
|
||||
mydestination =
|
||||
# When postfix receives mail for $mydestination,
|
||||
# it hands it over to dovecot via $local_transport.
|
||||
# Note: IP literals must be handled via local delivery / mydestination.
|
||||
mydestination = {{ config.mail_domain }}
|
||||
local_transport = lmtp:unix:private/dovecot-lmtp
|
||||
# postfix doesn't check whether local users exist or not:
|
||||
local_recipient_maps =
|
||||
|
||||
relayhost =
|
||||
{% if disable_ipv6 %}
|
||||
@@ -69,15 +73,6 @@ mynetworks = 127.0.0.0/8
|
||||
{% else %}
|
||||
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
|
||||
{% endif %}
|
||||
{% if config.addr_v4 %}
|
||||
smtp_bind_address = {{ config.addr_v4 }}
|
||||
{% endif %}
|
||||
{% if config.addr_v6 %}
|
||||
smtp_bind_address6 = {{ config.addr_v6 }}
|
||||
{% endif %}
|
||||
{% if config.addr_v4 or config.addr_v6 %}
|
||||
smtp_bind_address_enforce = yes
|
||||
{% endif %}
|
||||
mailbox_size_limit = 0
|
||||
message_size_limit = {{config.max_message_size}}
|
||||
recipient_delimiter = +
|
||||
@@ -88,10 +83,16 @@ inet_protocols = ipv4
|
||||
inet_protocols = all
|
||||
{% endif %}
|
||||
|
||||
virtual_transport = lmtp:unix:private/dovecot-lmtp
|
||||
virtual_mailbox_domains = {{ config.mail_domain }}
|
||||
lmtp_header_checks = regexp:/etc/postfix/lmtp_header_cleanup
|
||||
|
||||
# Do not apply header checks to MIME headers
|
||||
# and other headers that are actually part of the message body.
|
||||
# Reference:
|
||||
# <https://www.postfix.org/postconf.5.html#disable_mime_input_processing>
|
||||
# <https://www.postfix.org/header_checks.5.html>
|
||||
# <https://stump.io/blog/2020/11/29/a-little-gotcha-with-postfixs-header_checks/>
|
||||
disable_mime_input_processing = yes
|
||||
|
||||
mua_client_restrictions = permit_sasl_authenticated, reject
|
||||
mua_sender_restrictions = reject_sender_login_mismatch, permit_sasl_authenticated, reject
|
||||
mua_helo_restrictions = permit_mynetworks, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, permit
|
||||
@@ -102,3 +103,35 @@ smtpd_sender_login_maps = regexp:/etc/postfix/login_map
|
||||
# Do not lookup SMTP client hostnames to reduce delays
|
||||
# and avoid unnecessary DNS requests.
|
||||
smtpd_peername_lookup = no
|
||||
|
||||
# Use filtermail-transport to relay messages.
|
||||
# We can't force postfix to split messages per destination,
|
||||
# when specifying a custom next-hop,
|
||||
# so instead this is handled in filtermail.
|
||||
# We use LMTP instead SMTP so we can communicate per-recipient errors back to postfix.
|
||||
default_transport = lmtp-filtermail:inet:[127.0.0.1]:{{ config.filtermail_lmtp_port_transport }}
|
||||
|
||||
# All deliveries over lmtp-filtermail are treated
|
||||
# as having the same destination [127.0.0.1],
|
||||
# so it is not possible to limit per-destination concurrency here,
|
||||
# it is a job for filtermail-transport.
|
||||
# Total number of parallel deliveries is limited
|
||||
# by "maxproc" column in /etc/postfix/master.cf for lmtp-filtermail.
|
||||
# Settings below are to prevent Postfix queue manager
|
||||
# from limiting the number of LMTP connections to filtermail-transport.
|
||||
# Read <https://www.postfix.org/TUNING_README.html#rope> and
|
||||
# <https://www.postfix.org/SCHEDULER_README.html> for the details
|
||||
# of the Postfix algorithm that we effectively disable here.
|
||||
lmtp-filtermail_initial_destination_concurrency=10000
|
||||
lmtp-filtermail_destination_concurrency_limit=10000
|
||||
|
||||
# Do not try to deliver messages for more than 2 days.
|
||||
maximal_queue_lifetime = 2d
|
||||
|
||||
{% if not config.ipv4_relay %}
|
||||
# DKIM-sign locally generated mail (bounces, DSNs).
|
||||
# These bypass smtpd, so they need explicit milter configuration.
|
||||
non_smtpd_milters = unix:opendkim/opendkim.sock
|
||||
internal_mail_filter_classes = bounce
|
||||
milter_macro_daemon_name = ORIGINATING
|
||||
{% endif %}
|
||||
|
||||
@@ -17,7 +17,8 @@ smtp inet n - y - - smtpd
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
-o smtpd_tls_mandatory_protocols=>=TLSv1.2
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port_incoming }}
|
||||
submission inet n - y - 5000 smtpd
|
||||
-o smtpd_relay_restrictions=reject_unauth_destination
|
||||
submission inet n - y - {{ config.max_smtp_connections }} smtpd
|
||||
-o syslog_name=postfix/submission
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
-o smtpd_tls_mandatory_protocols=>=TLSv1.3
|
||||
@@ -31,9 +32,9 @@ submission inet n - y - 5000 smtpd
|
||||
-o smtpd_sender_restrictions=$mua_sender_restrictions
|
||||
-o smtpd_recipient_restrictions=
|
||||
-o smtpd_relay_restrictions=permit_sasl_authenticated,reject
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o smtpd_client_connection_count_limit={{ config.max_smtp_connections // 5 }}
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
smtps inet n - y - 5000 smtpd
|
||||
smtps inet n - y - {{ config.max_smtp_connections }} smtpd
|
||||
-o syslog_name=postfix/smtps
|
||||
-o smtpd_tls_wrappermode=yes
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
@@ -47,7 +48,7 @@ smtps inet n - y - 5000 smtpd
|
||||
-o smtpd_sender_restrictions=$mua_sender_restrictions
|
||||
-o smtpd_recipient_restrictions=
|
||||
-o smtpd_relay_restrictions=permit_sasl_authenticated,reject
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o smtpd_client_connection_count_limit={{ config.max_smtp_connections // 5 }}
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
#628 inet n - y - - qmqpd
|
||||
pickup unix n - y 60 1 pickup
|
||||
@@ -75,17 +76,19 @@ lmtp unix - - y - - lmtp
|
||||
anvil unix - - y - 1 anvil
|
||||
scache unix - - y - 1 scache
|
||||
postlog unix-dgram n - n - 1 postlogd
|
||||
filter unix - n n - - lmtp
|
||||
# Local SMTP server for reinjecting outgoing filtered mail.
|
||||
127.0.0.1:{{ config.postfix_reinject_port }} inet n - n - 100 smtpd
|
||||
-o syslog_name=postfix/reinject
|
||||
-o milter_macro_daemon_name=ORIGINATING
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
-o cleanup_service_name=authclean
|
||||
-o smtpd_relay_restrictions=permit_mynetworks,reject
|
||||
{% if not config.ipv4_relay %} -o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
{% endif %}
|
||||
|
||||
# Local SMTP server for reinjecting incoming filtered mail
|
||||
127.0.0.1:{{ config.postfix_reinject_port_incoming }} inet n - n - 100 smtpd
|
||||
-o syslog_name=postfix/reinject_incoming
|
||||
-o smtpd_relay_restrictions=reject_unauth_destination
|
||||
|
||||
# Cleanup `Received` headers for authenticated mail
|
||||
# to avoid leaking client IP.
|
||||
@@ -100,3 +103,16 @@ filter unix - n n - - lmtp
|
||||
# cannot send unprotected Subject.
|
||||
authclean unix n - - - 0 cleanup
|
||||
-o header_checks=regexp:/etc/postfix/submission_header_cleanup
|
||||
|
||||
# Reducing `maxproc` here may result in a head of line blocking
|
||||
# when there are many messages sent to unreachable destinations
|
||||
# at the same time.
|
||||
# LMTP clients here talk to filtermail-transport.
|
||||
# LMTP has no pipelining,
|
||||
# so while filtermail-transport tries to deliver the message,
|
||||
# possibly waiting for a long connection timeout
|
||||
# or talking to a slow server, LMTP client cannot be reused.
|
||||
lmtp-filtermail unix - - y - 500 lmtp
|
||||
-o syslog_name=postfix/lmtp-filtermail
|
||||
-o lmtp_header_checks=
|
||||
-o lmtp_tls_security_level=none
|
||||
|
||||
@@ -57,27 +57,32 @@ def get_dkim_entry(mail_domain, pre_command, dkim_selector):
|
||||
dkim_value_raw = f"v=DKIM1;k=rsa;p={dkim_pubkey};s=email;t=s"
|
||||
dkim_value = '" "'.join(re.findall(".{1,255}", dkim_value_raw))
|
||||
web_dkim_value = "".join(re.findall(".{1,255}", dkim_value_raw))
|
||||
name = f"{dkim_selector}._domainkey.{mail_domain}."
|
||||
return (
|
||||
f'{dkim_selector}._domainkey.{mail_domain}. 3600 IN TXT "{dkim_value}"',
|
||||
f'{dkim_selector}._domainkey.{mail_domain}. 3600 IN TXT "{web_dkim_value}"',
|
||||
f'{name:<40} 3600 IN TXT "{dkim_value}"',
|
||||
f'{name:<40} 3600 IN TXT "{web_dkim_value}"',
|
||||
)
|
||||
|
||||
|
||||
def query_dns(typ, domain):
|
||||
# Get autoritative nameserver from the SOA record.
|
||||
soa_answers = [
|
||||
def get_authoritative_ns(domain):
|
||||
ns_replies = [
|
||||
x.split()
|
||||
for x in shell(
|
||||
f"dig -r -q {domain} -t SOA +noall +authority +answer", print=log_progress
|
||||
f"dig -r -q {domain} -t NS +noall +authority +answer", print=log_progress
|
||||
).split("\n")
|
||||
]
|
||||
soa = [a for a in soa_answers if len(a) >= 3 and a[3] == "SOA"]
|
||||
if not soa:
|
||||
filtered_replies = [a for a in ns_replies if len(a) >= 5 and a[3] == "NS"]
|
||||
if not filtered_replies:
|
||||
return
|
||||
ns = soa[0][4]
|
||||
return filtered_replies[0][4]
|
||||
|
||||
|
||||
def query_dns(typ, domain):
|
||||
ns = get_authoritative_ns(domain)
|
||||
|
||||
# Query authoritative nameserver directly to bypass DNS cache.
|
||||
res = shell(f"dig @{ns} -r -q {domain} -t {typ} +short", print=log_progress)
|
||||
direct_ns = f"@{ns}" if ns else ""
|
||||
res = shell(f"dig {direct_ns} -r -q {domain} -t {typ} +short", print=log_progress)
|
||||
return next((line for line in res.split("\n") if not line.startswith(";")), "")
|
||||
|
||||
|
||||
@@ -94,11 +99,9 @@ def check_zonefile(zonefile, verbose=True):
|
||||
if not zf_line.strip() or zf_line.startswith(";"):
|
||||
continue
|
||||
print(f"dns-checking {zf_line!r}") if verbose else log_progress("")
|
||||
parts = zf_line.split(None, 4)
|
||||
zf_domain = parts[0].rstrip(".")
|
||||
# parts[1]=TTL, parts[2]=IN, parts[3]=type, parts[4]=rdata
|
||||
zf_typ = parts[3]
|
||||
zf_value = parts[4].strip()
|
||||
zf_domain, _ttl, _in, zf_typ, zf_value = zf_line.split(None, 4)
|
||||
zf_domain = zf_domain.rstrip(".")
|
||||
zf_value = zf_value.strip()
|
||||
query_value = query_dns(zf_typ, zf_domain)
|
||||
if zf_value != query_value:
|
||||
assert zf_typ in ("A", "AAAA", "CNAME", "CAA", "SRV", "MX", "TXT"), zf_line
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import shlex
|
||||
|
||||
from pyinfra.operations import apt, server
|
||||
from pyinfra.operations import server
|
||||
|
||||
from cmdeploy.basedeploy import Deployer
|
||||
from ..basedeploy import Deployer
|
||||
|
||||
|
||||
def openssl_selfsigned_args(domain, cert_path, key_path, days=36500):
|
||||
@@ -12,27 +12,15 @@ def openssl_selfsigned_args(domain, cert_path, key_path, days=36500):
|
||||
``www.<domain>`` and ``mta-sts.<domain>``.
|
||||
"""
|
||||
return [
|
||||
"openssl",
|
||||
"req",
|
||||
"-x509",
|
||||
"-newkey",
|
||||
"ec",
|
||||
"-pkeyopt",
|
||||
"ec_paramgen_curve:P-256",
|
||||
"-noenc",
|
||||
"-days",
|
||||
str(days),
|
||||
"-keyout",
|
||||
str(key_path),
|
||||
"-out",
|
||||
str(cert_path),
|
||||
"-subj",
|
||||
f"/CN={domain}",
|
||||
"openssl", "req", "-x509",
|
||||
"-newkey", "ec", "-pkeyopt", "ec_paramgen_curve:P-256",
|
||||
"-noenc", "-days", str(days),
|
||||
"-keyout", str(key_path),
|
||||
"-out", str(cert_path),
|
||||
"-subj", f"/CN={domain}",
|
||||
# Mark as end-entity cert so it cannot be used as a CA to sign others.
|
||||
"-addext",
|
||||
"basicConstraints=critical,CA:FALSE",
|
||||
"-addext",
|
||||
"extendedKeyUsage=serverAuth,clientAuth",
|
||||
"-addext", "basicConstraints=critical,CA:FALSE",
|
||||
"-addext", "extendedKeyUsage=serverAuth,clientAuth",
|
||||
"-addext",
|
||||
f"subjectAltName=DNS:{domain},DNS:www.{domain},DNS:mta-sts.{domain}",
|
||||
]
|
||||
@@ -46,17 +34,11 @@ class SelfSignedTlsDeployer(Deployer):
|
||||
self.cert_path = "/etc/ssl/certs/mailserver.pem"
|
||||
self.key_path = "/etc/ssl/private/mailserver.key"
|
||||
|
||||
def install(self):
|
||||
apt.packages(
|
||||
name="Install openssl",
|
||||
packages=["openssl"],
|
||||
)
|
||||
|
||||
|
||||
def configure(self):
|
||||
args = openssl_selfsigned_args(
|
||||
self.mail_domain,
|
||||
self.cert_path,
|
||||
self.key_path,
|
||||
self.mail_domain, self.cert_path, self.key_path,
|
||||
)
|
||||
cmd = shlex.join(args)
|
||||
server.shell(
|
||||
@@ -66,3 +48,5 @@ class SelfSignedTlsDeployer(Deployer):
|
||||
|
||||
def activate(self):
|
||||
pass
|
||||
|
||||
|
||||
|
||||
@@ -49,13 +49,8 @@ class SSHExec:
|
||||
RemoteError = execnet.RemoteError
|
||||
FuncError = FuncError
|
||||
|
||||
def __init__(
|
||||
self, host, verbose=False, python="python3", timeout=60, ssh_config=None
|
||||
):
|
||||
spec = f"ssh=root@{host}//python={python}"
|
||||
if ssh_config:
|
||||
spec += f"//ssh_config={ssh_config}"
|
||||
self.gateway = execnet.makegateway(spec)
|
||||
def __init__(self, host, verbose=False, python="python3", timeout=60):
|
||||
self.gateway = execnet.makegateway(f"ssh=root@{host}//python={python}")
|
||||
self._remote_cmdloop_channel = bootstrap_remote(self.gateway, remote)
|
||||
self.timeout = timeout
|
||||
self.verbose = verbose
|
||||
@@ -92,9 +87,8 @@ class SSHExec:
|
||||
class LocalExec:
|
||||
FuncError = FuncError
|
||||
|
||||
def __init__(self, verbose=False, docker=False):
|
||||
def __init__(self, verbose=False):
|
||||
self.verbose = verbose
|
||||
self.docker = docker
|
||||
|
||||
def __call__(self, call, kwargs=None, log_callback=None):
|
||||
if kwargs is None:
|
||||
@@ -106,10 +100,6 @@ class LocalExec:
|
||||
if not title:
|
||||
title = call.__name__
|
||||
where = "locally"
|
||||
if self.docker:
|
||||
if call == remote.rdns.perform_initial_checks:
|
||||
kwargs["pre_command"] = "docker exec chatmail "
|
||||
where = "in docker"
|
||||
if self.verbose:
|
||||
print_stderr(f"Running {where}: {title}(**{kwargs})")
|
||||
return self(call, kwargs, log_callback=print_stderr)
|
||||
@@ -118,46 +108,3 @@ class LocalExec:
|
||||
res = self(call, kwargs, log_callback=remote.rshell.log_progress)
|
||||
print_stderr()
|
||||
return res
|
||||
|
||||
|
||||
# pyinfra exposes a ``ssh_config_file`` data key that *should* let
|
||||
# paramiko parse an SSH config file directly. In practice it silently
|
||||
# fails to connect (zero hosts / zero operations), so we resolve the
|
||||
# hostname and identity-file ourselves and pass them via
|
||||
# ``--data ssh_hostname`` / ``--data ssh_key`` instead.
|
||||
# Execnet uses ssh natively (and not paramiko) and doesn't have this problem.
|
||||
|
||||
|
||||
def _get_from_ssh_config(host, ssh_config_path, key):
|
||||
"""Internal helper to parse a value for a specific key from ssh-config."""
|
||||
current_hosts = []
|
||||
found_value = None
|
||||
with open(ssh_config_path) as f:
|
||||
for raw_line in f:
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
parts = line.split(None, 1)
|
||||
if not parts:
|
||||
continue
|
||||
directive = parts[0].lower()
|
||||
if directive == "host":
|
||||
if host in current_hosts and found_value:
|
||||
return found_value
|
||||
current_hosts = parts[1].split()
|
||||
found_value = None
|
||||
elif directive == key.lower():
|
||||
found_value = parts[1]
|
||||
if host in current_hosts and found_value:
|
||||
return found_value
|
||||
return None
|
||||
|
||||
|
||||
def resolve_host_from_ssh_config(host, ssh_config_path):
|
||||
"""Resolve a host alias to its IP from an ssh-config file."""
|
||||
return _get_from_ssh_config(host, ssh_config_path, "Hostname") or host
|
||||
|
||||
|
||||
def resolve_key_from_ssh_config(host, ssh_config_path):
|
||||
"""Resolve a host alias to its IdentityFile from an ssh-config file."""
|
||||
return _get_from_ssh_config(host, ssh_config_path, "IdentityFile")
|
||||
|
||||
@@ -1,18 +1,18 @@
|
||||
; Required DNS entries
|
||||
zftest.testrun.org. 3600 IN A 135.181.204.127
|
||||
zftest.testrun.org. 3600 IN AAAA 2a01:4f9:c012:52f4::1
|
||||
zftest.testrun.org. 3600 IN MX 10 zftest.testrun.org.
|
||||
_mta-sts.zftest.testrun.org. 3600 IN TXT "v=STSv1; id=202403211706"
|
||||
mta-sts.zftest.testrun.org. 3600 IN CNAME zftest.testrun.org.
|
||||
www.zftest.testrun.org. 3600 IN CNAME zftest.testrun.org.
|
||||
opendkim._domainkey.zftest.testrun.org. 3600 IN TXT "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoYt82CVUyz2ouaqjX2kB+5J80knAyoOU3MGU5aWppmwUwwTvj/oSTSpkc5JMtVTRmKKr8NUDWAL1Yw7dfGqqPHdHfwwjS3BIvDzYx+hzgtz62RnfNgV+/2MAoNpfX7cAFIHdRzEHNtwugc3RDLquqPoupAE3Y2YRw2T5zG5fILh4vwIcJZL5Uq6B92j8wwJqOex" "33n+vm1NKQ9rxo/UsHAmZlJzpooXcG/4igTBxJyJlamVSRR6N7Nul1v//YJb7J6v2o0iPHW6uE0StzKaPPNC2IVosSRFbD9H2oqppltptFSNPlI0E+t0JBWHem6YK7xcugiO3ImMCaaU8g6Jt/wIDAQAB;s=email;t=s"
|
||||
zftest.testrun.org. 3600 IN A 135.181.204.127
|
||||
zftest.testrun.org. 3600 IN AAAA 2a01:4f9:c012:52f4::1
|
||||
zftest.testrun.org. 3600 IN MX 10 zftest.testrun.org.
|
||||
_mta-sts.zftest.testrun.org. 3600 IN TXT "v=STSv1; id=202403211706"
|
||||
mta-sts.zftest.testrun.org. 3600 IN CNAME zftest.testrun.org.
|
||||
www.zftest.testrun.org. 3600 IN CNAME zftest.testrun.org.
|
||||
opendkim._domainkey.zftest.testrun.org. 3600 IN TXT "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoYt82CVUyz2ouaqjX2kB+5J80knAyoOU3MGU5aWppmwUwwTvj/oSTSpkc5JMtVTRmKKr8NUDWAL1Yw7dfGqqPHdHfwwjS3BIvDzYx+hzgtz62RnfNgV+/2MAoNpfX7cAFIHdRzEHNtwugc3RDLquqPoupAE3Y2YRw2T5zG5fILh4vwIcJZL5Uq6B92j8wwJqOex" "33n+vm1NKQ9rxo/UsHAmZlJzpooXcG/4igTBxJyJlamVSRR6N7Nul1v//YJb7J6v2o0iPHW6uE0StzKaPPNC2IVosSRFbD9H2oqppltptFSNPlI0E+t0JBWHem6YK7xcugiO3ImMCaaU8g6Jt/wIDAQAB;s=email;t=s"
|
||||
|
||||
; Recommended DNS entries
|
||||
zftest.testrun.org. 3600 IN TXT "v=spf1 a ~all"
|
||||
_dmarc.zftest.testrun.org. 3600 IN TXT "v=DMARC1;p=reject;adkim=s;aspf=s"
|
||||
zftest.testrun.org. 3600 IN CAA 0 issue "letsencrypt.org;accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/1371472956"
|
||||
_adsp._domainkey.zftest.testrun.org. 3600 IN TXT "dkim=discardable"
|
||||
_submission._tcp.zftest.testrun.org. 3600 IN SRV 0 1 587 zftest.testrun.org.
|
||||
_submissions._tcp.zftest.testrun.org. 3600 IN SRV 0 1 465 zftest.testrun.org.
|
||||
_imap._tcp.zftest.testrun.org. 3600 IN SRV 0 1 143 zftest.testrun.org.
|
||||
_imaps._tcp.zftest.testrun.org. 3600 IN SRV 0 1 993 zftest.testrun.org.
|
||||
zftest.testrun.org. 3600 IN TXT "v=spf1 a ~all"
|
||||
_dmarc.zftest.testrun.org. 3600 IN TXT "v=DMARC1;p=reject;adkim=s;aspf=s"
|
||||
zftest.testrun.org. 3600 IN CAA 0 issue "letsencrypt.org;accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/1371472956"
|
||||
_adsp._domainkey.zftest.testrun.org. 3600 IN TXT "dkim=discardable"
|
||||
_submission._tcp.zftest.testrun.org. 3600 IN SRV 0 1 587 zftest.testrun.org.
|
||||
_submissions._tcp.zftest.testrun.org. 3600 IN SRV 0 1 465 zftest.testrun.org.
|
||||
_imap._tcp.zftest.testrun.org. 3600 IN SRV 0 1 143 zftest.testrun.org.
|
||||
_imaps._tcp.zftest.testrun.org. 3600 IN SRV 0 1 993 zftest.testrun.org.
|
||||
|
||||
@@ -12,7 +12,7 @@ def test_init(tmp_path, maildomain):
|
||||
inipath = tmp_path.joinpath("chatmail.ini")
|
||||
main(["init", "--config", str(inipath), maildomain])
|
||||
config = read_config(inipath)
|
||||
assert config.mail_domain == maildomain
|
||||
assert config.mail_domain_bare == maildomain
|
||||
|
||||
|
||||
def test_capabilities(imap):
|
||||
@@ -89,12 +89,11 @@ def test_concurrent_logins_same_account(
|
||||
assert login_results.get()
|
||||
|
||||
|
||||
def test_no_vrfy(cmfactory, chatmail_config):
|
||||
def test_no_vrfy(cmfactory, chatmail_config, maildomain):
|
||||
ac = cmfactory.get_online_account()
|
||||
addr = ac.get_config("addr")
|
||||
domain = chatmail_config.mail_domain
|
||||
|
||||
s = smtplib.SMTP(domain)
|
||||
s = smtplib.SMTP(maildomain)
|
||||
s.starttls()
|
||||
|
||||
s.putcmd("vrfy", f"wrongaddress@{chatmail_config.mail_domain}")
|
||||
|
||||
@@ -20,7 +20,7 @@ def test_fastcgi_working(maildomain, chatmail_config):
|
||||
|
||||
|
||||
@pytest.mark.filterwarnings("ignore::urllib3.exceptions.InsecureRequestWarning")
|
||||
def test_newemail_configure(maildomain, maildomain_ip, rpc, chatmail_config):
|
||||
def test_newemail_configure(maildomain, rpc, chatmail_config):
|
||||
"""Test configuring accounts by scanning a QR code works."""
|
||||
url = f"DCACCOUNT:https://{maildomain}/new"
|
||||
for i in range(3):
|
||||
@@ -30,15 +30,12 @@ def test_newemail_configure(maildomain, maildomain_ip, rpc, chatmail_config):
|
||||
# set_config_from_qr, so fetch credentials via requests instead
|
||||
res = requests.post(f"https://{maildomain}/new", verify=False)
|
||||
data = res.json()
|
||||
rpc.add_or_update_transport(
|
||||
account_id,
|
||||
{
|
||||
"addr": data["email"],
|
||||
"password": data["password"],
|
||||
"imapServer": maildomain_ip,
|
||||
"smtpServer": maildomain_ip,
|
||||
"certificateChecks": "acceptInvalidCertificates",
|
||||
},
|
||||
)
|
||||
rpc.add_or_update_transport(account_id, {
|
||||
"addr": data["email"],
|
||||
"password": data["password"],
|
||||
"imapServer": maildomain,
|
||||
"smtpServer": maildomain,
|
||||
"certificateChecks": "acceptInvalidCertificates",
|
||||
})
|
||||
else:
|
||||
rpc.add_transport_from_qr(account_id, url)
|
||||
|
||||
@@ -5,6 +5,7 @@ import subprocess
|
||||
import time
|
||||
|
||||
import pytest
|
||||
from chatmaild.config import is_valid_ipv4
|
||||
|
||||
from cmdeploy import remote
|
||||
from cmdeploy.cmdeploy import get_sshexec
|
||||
@@ -12,9 +13,8 @@ from cmdeploy.cmdeploy import get_sshexec
|
||||
|
||||
class TestSSHExecutor:
|
||||
@pytest.fixture(scope="class")
|
||||
def sshexec(self, sshdomain, pytestconfig):
|
||||
ssh_config = pytestconfig.getoption("ssh_config")
|
||||
return get_sshexec(sshdomain, ssh_config=ssh_config)
|
||||
def sshexec(self, sshdomain):
|
||||
return get_sshexec(sshdomain)
|
||||
|
||||
def test_ls(self, sshexec):
|
||||
out = sshexec(call=remote.rdns.shell, kwargs=dict(command="ls"))
|
||||
@@ -22,6 +22,8 @@ class TestSSHExecutor:
|
||||
assert out == out2
|
||||
|
||||
def test_perform_initial(self, sshexec, maildomain):
|
||||
if is_valid_ipv4(maildomain):
|
||||
pytest.skip(f"{maildomain} is not a domain")
|
||||
res = sshexec(
|
||||
remote.rdns.perform_initial_checks, kwargs=dict(mail_domain=maildomain)
|
||||
)
|
||||
@@ -62,8 +64,10 @@ class TestSSHExecutor:
|
||||
else:
|
||||
pytest.fail("didn't raise exception")
|
||||
|
||||
def test_opendkim_restarted(self, sshexec):
|
||||
def test_opendkim_restarted(self, sshexec, maildomain):
|
||||
"""check that opendkim is not running for longer than a day."""
|
||||
if is_valid_ipv4(maildomain):
|
||||
pytest.skip(f"{maildomain} is an IPv4 relay, opendkim is not installed")
|
||||
cmd = "systemctl show opendkim --timestamp=utc --property=ActiveEnterTimestamp"
|
||||
out = sshexec(call=remote.rshell.shell, kwargs=dict(command=cmd))
|
||||
datestring = out.split("=")[1]
|
||||
@@ -72,6 +76,44 @@ class TestSSHExecutor:
|
||||
assert (now - since_date).total_seconds() < 60 * 60 * 51
|
||||
|
||||
|
||||
def test_dovecot_main_process_matches_installed_binary(sshdomain):
|
||||
sshexec = get_sshexec(sshdomain)
|
||||
main_pid = int(
|
||||
sshexec(
|
||||
call=remote.rshell.shell,
|
||||
kwargs=dict(
|
||||
command="timeout 10 systemctl show -p MainPID --value dovecot.service"
|
||||
),
|
||||
).strip()
|
||||
)
|
||||
assert main_pid != 0, "dovecot.service MainPID is 0 -- service not running?"
|
||||
|
||||
exe = sshexec(
|
||||
call=remote.rshell.shell,
|
||||
kwargs=dict(command=f"timeout 10 readlink /proc/{main_pid}/exe"),
|
||||
).strip()
|
||||
status_text = sshexec(
|
||||
call=remote.rshell.shell,
|
||||
kwargs=dict(
|
||||
command="timeout 10 systemctl show -p StatusText --value dovecot.service"
|
||||
),
|
||||
).strip()
|
||||
installed_version = sshexec(
|
||||
call=remote.rshell.shell, kwargs=dict(command="timeout 10 dovecot --version")
|
||||
).strip()
|
||||
|
||||
assert not exe.endswith("(deleted)"), (
|
||||
f"running dovecot binary was deleted (stale after upgrade): {exe}"
|
||||
)
|
||||
expected_status_text = f"v{installed_version}"
|
||||
assert status_text == expected_status_text or status_text.startswith(
|
||||
f"{expected_status_text} "
|
||||
), (
|
||||
f"dovecot status version mismatch: "
|
||||
f"StatusText={status_text!r}, installed={installed_version!r}"
|
||||
)
|
||||
|
||||
|
||||
def test_timezone_env(remote):
|
||||
for line in remote.iter_output("env"):
|
||||
print(line)
|
||||
@@ -133,10 +175,11 @@ def test_authenticated_from(cmsetup, maildata):
|
||||
@pytest.mark.parametrize("from_addr", ["fake@example.org", "fake@testrun.org"])
|
||||
def test_reject_missing_dkim(cmsetup, maildata, from_addr):
|
||||
domain = cmsetup.maildomain
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.settimeout(10)
|
||||
try:
|
||||
sock = socket.create_connection((domain, 25), timeout=10)
|
||||
sock.close()
|
||||
except (socket.timeout, OSError):
|
||||
sock.connect((domain, 25))
|
||||
except socket.timeout:
|
||||
pytest.skip(f"port 25 not reachable for {domain}")
|
||||
|
||||
recipient = cmsetup.gen_users(1)[0]
|
||||
@@ -151,6 +194,34 @@ def test_reject_missing_dkim(cmsetup, maildata, from_addr):
|
||||
s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg)
|
||||
|
||||
|
||||
def test_bounces_are_dkim_signed(cmsetup, cmsetup2, maildata, maildomain):
|
||||
# we send a message to non-existant user and expect a bounce message
|
||||
# which will only get through if the bounce message was DKIM-signed
|
||||
|
||||
if is_valid_ipv4(maildomain):
|
||||
pytest.skip("DKIM is not configured on IPv4-only relays")
|
||||
|
||||
sender = cmsetup2.gen_users(1)[0]
|
||||
nonexistent = f"nosuchuser_test42@{cmsetup.maildomain}"
|
||||
|
||||
msg = maildata(
|
||||
"encrypted.eml",
|
||||
from_addr=sender.addr,
|
||||
to_addr=nonexistent,
|
||||
).as_string()
|
||||
sender.smtp.sendmail(sender.addr, [nonexistent], msg)
|
||||
|
||||
def bounce_in_inbox():
|
||||
messages = sender.imap.fetch_all_messages()
|
||||
for m in messages:
|
||||
if "mail delivery" in m.lower() or "undelivered" in m.lower():
|
||||
return m
|
||||
raise ValueError("bounce not yet in inbox")
|
||||
|
||||
bounce = try_n_times(30, bounce_in_inbox)
|
||||
assert "nosuchuser_test42" in bounce
|
||||
|
||||
|
||||
def try_n_times(n, f):
|
||||
for _ in range(n - 1):
|
||||
try:
|
||||
@@ -183,7 +254,6 @@ def test_rewrite_subject(cmsetup, maildata):
|
||||
assert "Subject: Unencrypted subject" not in rcvd_msg
|
||||
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_exceed_rate_limit(cmsetup, gencreds, maildata, chatmail_config):
|
||||
"""Test that the per-account send-mail limit is exceeded."""
|
||||
user1, user2 = cmsetup.gen_users(2)
|
||||
@@ -206,7 +276,6 @@ def test_exceed_rate_limit(cmsetup, gencreds, maildata, chatmail_config):
|
||||
pytest.fail("Rate limit was not exceeded")
|
||||
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_expunged(remote, chatmail_config):
|
||||
outdated_days = int(chatmail_config.delete_mails_after) + 1
|
||||
find_cmds = [
|
||||
@@ -245,3 +314,15 @@ def test_deployed_state(remote):
|
||||
# assert len(git_status) == len(remote_version) # for some reason, we only get 11 lines from remote.iter_output()
|
||||
for i in range(len(remote_version)):
|
||||
assert git_status[i] == remote_version[i], "You have undeployed changes."
|
||||
|
||||
|
||||
def test_nginx_access_log_only_defined_once(sshdomain):
|
||||
sshexec = get_sshexec(sshdomain)
|
||||
conf = sshexec(
|
||||
call=remote.rshell.shell,
|
||||
kwargs=dict(command="nginx -T 2>/dev/null"),
|
||||
)
|
||||
access_logs = [l for l in conf.splitlines() if l.strip().startswith("access_log")]
|
||||
assert len(access_logs) == 1, (
|
||||
f"expected 1 access_log, found {len(access_logs)}: {access_logs}"
|
||||
)
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
import ipaddress
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
|
||||
import imap_tools
|
||||
import pytest
|
||||
import requests
|
||||
from chatmaild.tests.test_appversions import check_appversions
|
||||
|
||||
from cmdeploy.cmdeploy import get_sshexec
|
||||
from cmdeploy.remote import rshell
|
||||
@@ -15,7 +17,7 @@ def imap_mailbox(cmfactory, ssl_context):
|
||||
(ac1,) = cmfactory.get_online_accounts(1)
|
||||
user = ac1.get_config("addr")
|
||||
password = ac1.get_config("mail_pw")
|
||||
host = user.split("@")[1]
|
||||
host = user.split("@")[1].strip("[").strip("]")
|
||||
mailbox = imap_tools.MailBox(host, ssl_context=ssl_context)
|
||||
mailbox.login(user, password)
|
||||
mailbox.dc_ac = ac1
|
||||
@@ -51,6 +53,17 @@ class TestMetadataTokens:
|
||||
assert res == b"1111 2222"
|
||||
assert b"Getmetadata completed" in client.readline()
|
||||
|
||||
def test_get_appversions(self, imap_mailbox):
|
||||
"get app version information shipped with the relay"
|
||||
client = imap_mailbox.client
|
||||
client.send(b'a01 GETMETADATA "" /shared/vendor/deltachat/appversions\n')
|
||||
res = client.readline()
|
||||
assert res[:1] == b"*"
|
||||
res = client.readline().strip().rstrip(b")")
|
||||
# the served value is a single line and passes the shipped file's schema
|
||||
check_appversions(json.loads(res))
|
||||
assert b"Getmetadata completed" in client.readline()
|
||||
|
||||
|
||||
class TestEndToEndDeltaChat:
|
||||
"Tests that use Delta Chat accounts on the chat mail instance."
|
||||
@@ -67,7 +80,7 @@ class TestEndToEndDeltaChat:
|
||||
assert msg2.get_snapshot().text == "message0"
|
||||
|
||||
def test_exceed_quota(
|
||||
self, cmfactory, lp, tmpdir, remote, chatmail_config, sshdomain, pytestconfig
|
||||
self, cmfactory, lp, tmpdir, remote, chatmail_config, sshdomain
|
||||
):
|
||||
"""This is a very slow test as it needs to upload >100MB of mail data
|
||||
before quota is exceeded, and thus depends on the speed of the upload.
|
||||
@@ -92,9 +105,7 @@ class TestEndToEndDeltaChat:
|
||||
lp.sec(f"filling remote inbox for {user}")
|
||||
fn = f"7743102289.M843172P2484002.c20,S={quota},W=2398:2,"
|
||||
path = chatmail_config.mailboxes_dir.joinpath(user, "cur", fn)
|
||||
sshexec = get_sshexec(
|
||||
sshdomain, ssh_config=pytestconfig.getoption("ssh_config")
|
||||
)
|
||||
sshexec = get_sshexec(sshdomain)
|
||||
sshexec(call=rshell.write_numbytes, kwargs=dict(path=str(path), num=120))
|
||||
res = sshexec(call=rshell.dovecot_recalc_quota, kwargs=dict(user=user))
|
||||
assert res["percent"] >= 100
|
||||
@@ -180,7 +191,7 @@ def test_hide_senders_ip_address(cmfactory, ssl_context):
|
||||
chat.send_text("testing submission header cleanup")
|
||||
user2.wait_for_incoming_msg()
|
||||
addr = user2.get_config("addr")
|
||||
host = addr.split("@")[1]
|
||||
host = addr.split("@")[1].strip("[").strip("]")
|
||||
pw = user2.get_config("mail_pw")
|
||||
mailbox = imap_tools.MailBox(host, ssl_context=ssl_context)
|
||||
mailbox.login(addr, pw)
|
||||
|
||||
@@ -3,15 +3,12 @@ import os
|
||||
from cmdeploy.cmdeploy import main
|
||||
|
||||
|
||||
def test_status_cmd(chatmail_config, capsys, request, pytestconfig):
|
||||
def test_status_cmd(chatmail_config, capsys, request):
|
||||
os.chdir(request.config.invocation_params.dir)
|
||||
command = ["status"]
|
||||
ssh_host = pytestconfig.getoption("ssh_host")
|
||||
if ssh_host:
|
||||
command.extend(["--ssh-host", ssh_host])
|
||||
ssh_config = pytestconfig.getoption("ssh_config")
|
||||
if ssh_config:
|
||||
command.extend(["--ssh-config", ssh_config])
|
||||
if os.getenv("CHATMAIL_SSH"):
|
||||
command.append("--ssh-host")
|
||||
command.append(os.getenv("CHATMAIL_SSH"))
|
||||
assert main(command) == 0
|
||||
status_out = capsys.readouterr()
|
||||
print(status_out.out)
|
||||
|
||||
@@ -2,96 +2,25 @@ import imaplib
|
||||
import itertools
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
import smtplib
|
||||
import socket
|
||||
import ssl
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from chatmaild.config import read_config
|
||||
from chatmaild.config import is_valid_ipv4, read_config
|
||||
|
||||
|
||||
def format_mail_domain(raw_domain: str) -> str:
|
||||
if is_valid_ipv4(raw_domain):
|
||||
return f"[{raw_domain}]"
|
||||
return raw_domain
|
||||
|
||||
|
||||
conftestdir = Path(__file__).parent
|
||||
|
||||
|
||||
def pytest_addoption(parser):
|
||||
parser.addoption(
|
||||
"--slow", action="store_true", default=False, help="also run slow tests"
|
||||
)
|
||||
parser.addoption(
|
||||
"--ssh-host",
|
||||
dest="ssh_host",
|
||||
default=None,
|
||||
help="SSH host (overrides mail_domain for SSH operations).",
|
||||
)
|
||||
parser.addoption(
|
||||
"--ssh-config",
|
||||
dest="ssh_config",
|
||||
default=None,
|
||||
help="Path to an SSH config file (e.g. lxconfigs/ssh-config).",
|
||||
)
|
||||
|
||||
|
||||
def _parse_ssh_config_hosts(path):
|
||||
"""Parse an OpenSSH config file and return a dict of hostname -> IP."""
|
||||
mapping = {}
|
||||
current_names = []
|
||||
for ln in Path(path).read_text().splitlines():
|
||||
line = ln.strip()
|
||||
m = re.match(r"^Host\s+(.+)", line)
|
||||
if m:
|
||||
current_names = m.group(1).split()
|
||||
continue
|
||||
m = re.match(r"^Hostname\s+(\S+)", line)
|
||||
if m and current_names:
|
||||
ip = m.group(1)
|
||||
for name in current_names:
|
||||
mapping[name] = ip
|
||||
current_names = []
|
||||
return mapping
|
||||
|
||||
|
||||
_original_getaddrinfo = socket.getaddrinfo
|
||||
|
||||
|
||||
def _make_patched_getaddrinfo(host_map):
|
||||
"""Return a getaddrinfo that resolves hosts in host_map to their IPs."""
|
||||
|
||||
def patched_getaddrinfo(host, port, family=0, type=0, proto=0, flags=0):
|
||||
if host in host_map:
|
||||
ip = host_map[host]
|
||||
return _original_getaddrinfo(ip, port, family, type, proto, flags)
|
||||
return _original_getaddrinfo(host, port, family, type, proto, flags)
|
||||
|
||||
return patched_getaddrinfo
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True, scope="session")
|
||||
def _setup_localchat_dns(pytestconfig):
|
||||
"""Monkey-patch socket.getaddrinfo to resolve .localchat via ssh-config."""
|
||||
ssh_config = pytestconfig.getoption("ssh_config")
|
||||
if not ssh_config or not Path(ssh_config).exists():
|
||||
yield {}
|
||||
return
|
||||
host_map = _parse_ssh_config_hosts(ssh_config)
|
||||
if not host_map:
|
||||
yield {}
|
||||
return
|
||||
socket.getaddrinfo = _make_patched_getaddrinfo(host_map)
|
||||
try:
|
||||
yield host_map
|
||||
finally:
|
||||
socket.getaddrinfo = _original_getaddrinfo
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def ssh_config_host_map(_setup_localchat_dns):
|
||||
"""Return the host-name → IP map parsed from ssh-config."""
|
||||
return _setup_localchat_dns
|
||||
|
||||
|
||||
def pytest_configure(config):
|
||||
config._benchresults = {}
|
||||
config.addinivalue_line(
|
||||
@@ -99,19 +28,12 @@ def pytest_configure(config):
|
||||
)
|
||||
|
||||
|
||||
def pytest_runtest_setup(item):
|
||||
markers = list(item.iter_markers(name="slow"))
|
||||
if markers:
|
||||
if not item.config.getoption("--slow"):
|
||||
pytest.skip("skipping slow test, use --slow to run")
|
||||
|
||||
|
||||
def _get_chatmail_config():
|
||||
ini = os.environ.get("CHATMAIL_INI")
|
||||
if ini:
|
||||
path = Path(ini).resolve()
|
||||
if path.exists():
|
||||
return read_config(path), path
|
||||
inipath = os.environ.get("CHATMAIL_INI")
|
||||
if inipath:
|
||||
path = Path(inipath).resolve()
|
||||
return read_config(path), path
|
||||
|
||||
current = Path().resolve()
|
||||
while 1:
|
||||
path = current.joinpath("chatmail.ini").resolve()
|
||||
@@ -134,18 +56,12 @@ def chatmail_config(pytestconfig):
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def maildomain(chatmail_config):
|
||||
return chatmail_config.mail_domain
|
||||
return chatmail_config.mail_domain_bare
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def sshdomain(maildomain, pytestconfig):
|
||||
return pytestconfig.getoption("ssh_host") or maildomain
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def maildomain_ip(maildomain, ssh_config_host_map):
|
||||
"""Return the IP for maildomain from ssh-config, or maildomain itself."""
|
||||
return ssh_config_host_map.get(maildomain, maildomain)
|
||||
def sshdomain(maildomain):
|
||||
return os.environ.get("CHATMAIL_SSH", maildomain)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -389,37 +305,26 @@ from deltachat_rpc_client import DeltaChat, Rpc
|
||||
class ChatmailACFactory:
|
||||
"""RPC-based account factory for chatmail testing."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
rpc,
|
||||
maildomain,
|
||||
maildomain_ip,
|
||||
gencreds,
|
||||
chatmail_config,
|
||||
ssh_config_host_map,
|
||||
):
|
||||
def __init__(self, rpc, maildomain, gencreds, chatmail_config):
|
||||
self.dc = DeltaChat(rpc)
|
||||
self.rpc = rpc
|
||||
self._maildomain = maildomain
|
||||
self._maildomain_ip = maildomain_ip
|
||||
self.gencreds = gencreds
|
||||
self.chatmail_config = chatmail_config
|
||||
self._ssh_config_host_map = ssh_config_host_map
|
||||
|
||||
def _make_transport(self, domain):
|
||||
"""Build a transport config dict for the given domain."""
|
||||
addr, password = self.gencreds(domain)
|
||||
domain_deliverable = format_mail_domain(domain)
|
||||
addr, password = self.gencreds(domain_deliverable)
|
||||
transport = {
|
||||
"addr": addr,
|
||||
"password": password,
|
||||
# Setting server explicitly skips requesting autoconfig XML,
|
||||
# see https://datatracker.ietf.org/doc/draft-ietf-mailmaint-autoconfig/
|
||||
"imapServer": domain,
|
||||
"smtpServer": domain,
|
||||
}
|
||||
# To support running against local relays without host DNS resolution
|
||||
# we attempt resolving the domain via ssh-config
|
||||
# because otherwise core fails to find the address
|
||||
server = self._ssh_config_host_map.get(domain)
|
||||
if server is not None:
|
||||
transport.update({"imapServer": server, "smtpServer": server})
|
||||
if self.chatmail_config.tls_cert_mode == "self":
|
||||
if domain.startswith("_") or is_valid_ipv4(domain):
|
||||
transport["certificateChecks"] = "acceptInvalidCertificates"
|
||||
return transport
|
||||
|
||||
@@ -434,14 +339,28 @@ class ChatmailACFactory:
|
||||
accounts = []
|
||||
for _ in range(num):
|
||||
account = self.dc.add_account()
|
||||
future = account.add_or_update_transport.future(
|
||||
self._make_transport(domain)
|
||||
)
|
||||
domain_deliverable = format_mail_domain(domain)
|
||||
addr, password = self.gencreds(domain_deliverable)
|
||||
if is_valid_ipv4(domain):
|
||||
# Use DCLOGIN scheme with explicit server hosts,
|
||||
# matching how madmail presents its addresses to users.
|
||||
qr = (
|
||||
f"dclogin:{addr}"
|
||||
f"?p={password}&v=1"
|
||||
f"&ih={domain}&ip=993&is=ssl"
|
||||
f"&sh={domain}&sp=465&ss=ssl"
|
||||
f"&ic=3"
|
||||
)
|
||||
future = account.add_transport_from_qr.future(qr)
|
||||
else:
|
||||
future = account.add_or_update_transport.future(
|
||||
self._make_transport(domain)
|
||||
)
|
||||
futures.append(future)
|
||||
|
||||
# ensure messages stay in INBOX so that they can be
|
||||
# concurrently fetched via extra IMAP connections during tests
|
||||
account.set_config("delete_server_after", "10")
|
||||
account.set_config("bcc_self", "1")
|
||||
accounts.append(account)
|
||||
|
||||
for future in futures:
|
||||
@@ -471,46 +390,35 @@ def rpc(tmp_path_factory):
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def cmfactory(
|
||||
rpc, gencreds, maildomain, maildomain_ip, chatmail_config, ssh_config_host_map
|
||||
):
|
||||
def cmfactory(rpc, gencreds, maildomain, chatmail_config):
|
||||
"""Return a ChatmailACFactory for creating online Delta Chat accounts."""
|
||||
return ChatmailACFactory(
|
||||
rpc=rpc,
|
||||
maildomain=maildomain,
|
||||
maildomain_ip=maildomain_ip,
|
||||
gencreds=gencreds,
|
||||
chatmail_config=chatmail_config,
|
||||
ssh_config_host_map=ssh_config_host_map,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def remote(sshdomain, pytestconfig):
|
||||
r = Remote(sshdomain, ssh_config=pytestconfig.getoption("ssh_config"))
|
||||
def remote(sshdomain):
|
||||
r = Remote(sshdomain)
|
||||
yield r
|
||||
r.close()
|
||||
|
||||
|
||||
class Remote:
|
||||
def __init__(self, sshdomain, ssh_config=None):
|
||||
def __init__(self, sshdomain):
|
||||
self.sshdomain = sshdomain
|
||||
self.ssh_config = ssh_config
|
||||
self._procs = []
|
||||
|
||||
def iter_output(self, logcmd="", ready=None):
|
||||
getjournal = "journalctl -f" if not logcmd else logcmd
|
||||
print(self.sshdomain)
|
||||
match self.sshdomain:
|
||||
case "@local":
|
||||
command = []
|
||||
case "localhost":
|
||||
command = []
|
||||
case _:
|
||||
command = ["ssh"]
|
||||
if self.ssh_config:
|
||||
command.extend(["-F", self.ssh_config])
|
||||
command.append(f"root@{self.sshdomain}")
|
||||
if self.sshdomain in ("@local", "localhost"):
|
||||
command = []
|
||||
else:
|
||||
command = ["ssh", f"root@{self.sshdomain}"]
|
||||
[command.append(arg) for arg in getjournal.split()]
|
||||
popen = subprocess.Popen(
|
||||
command,
|
||||
@@ -519,15 +427,19 @@ class Remote:
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
self._procs.append(popen)
|
||||
while 1:
|
||||
line = popen.stdout.readline()
|
||||
res = line.decode().strip().lower()
|
||||
if not res:
|
||||
break
|
||||
if ready is not None:
|
||||
ready()
|
||||
ready = None
|
||||
yield res
|
||||
try:
|
||||
while 1:
|
||||
line = popen.stdout.readline()
|
||||
res = line.decode().strip().lower()
|
||||
if not res:
|
||||
break
|
||||
if ready is not None:
|
||||
ready()
|
||||
ready = None
|
||||
yield res
|
||||
finally:
|
||||
popen.terminate()
|
||||
popen.wait()
|
||||
|
||||
def close(self):
|
||||
while self._procs:
|
||||
@@ -553,6 +465,11 @@ def cmsetup(maildomain, gencreds, ssl_context):
|
||||
return CMSetup(maildomain, gencreds, ssl_context)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def cmsetup2(maildomain2, gencreds, ssl_context):
|
||||
return CMSetup(maildomain2, gencreds, ssl_context)
|
||||
|
||||
|
||||
class CMSetup:
|
||||
def __init__(self, maildomain, gencreds, ssl_context):
|
||||
self.maildomain = maildomain
|
||||
@@ -563,7 +480,7 @@ class CMSetup:
|
||||
print(f"Creating {num} online users")
|
||||
users = []
|
||||
for i in range(num):
|
||||
addr, password = self.gencreds()
|
||||
addr, password = self.gencreds(format_mail_domain(self.maildomain))
|
||||
user = CMUser(self.maildomain, addr, password, self.ssl_context)
|
||||
assert user.smtp
|
||||
users.append(user)
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from cmdeploy.basedeploy import Deployer
|
||||
|
||||
|
||||
def test_put_file_restart_and_reload():
|
||||
deployer = Deployer()
|
||||
mock_res = MagicMock()
|
||||
mock_res.changed = True
|
||||
|
||||
with patch("cmdeploy.basedeploy.files.put", return_value=mock_res):
|
||||
deployer.put_file("foo.conf", "/etc/foo.conf")
|
||||
assert deployer.need_restart is True
|
||||
assert deployer.daemon_reload is False
|
||||
|
||||
deployer = Deployer()
|
||||
|
||||
deployer.put_file("test.service", "/etc/systemd/system/test.service")
|
||||
assert deployer.need_restart is True
|
||||
assert deployer.daemon_reload is True
|
||||
|
||||
|
||||
def test_remove_file():
|
||||
deployer = Deployer()
|
||||
mock_res = MagicMock()
|
||||
mock_res.changed = True
|
||||
|
||||
with patch("cmdeploy.basedeploy.files.file", return_value=mock_res) as mock_file:
|
||||
deployer.remove_file("/etc/foo.conf")
|
||||
mock_file.assert_called_once_with(
|
||||
name="Remove /etc/foo.conf", path="/etc/foo.conf", present=False
|
||||
)
|
||||
assert deployer.need_restart is True
|
||||
|
||||
|
||||
def test_ensure_systemd_unit():
|
||||
deployer = Deployer()
|
||||
mock_res = MagicMock()
|
||||
mock_res.changed = True
|
||||
|
||||
# Plain service file
|
||||
with patch("cmdeploy.basedeploy.files.put", return_value=mock_res) as mock_put:
|
||||
deployer.ensure_systemd_unit("iroh-relay.service")
|
||||
assert (
|
||||
mock_put.call_args.kwargs["dest"]
|
||||
== "/etc/systemd/system/iroh-relay.service"
|
||||
)
|
||||
assert deployer.need_restart is True
|
||||
assert deployer.daemon_reload is True
|
||||
|
||||
deployer = Deployer()
|
||||
|
||||
# Template (.j2) dispatches to put_template and strips .j2 suffix
|
||||
with patch("cmdeploy.basedeploy.files.template", return_value=mock_res) as mock_tpl:
|
||||
deployer.ensure_systemd_unit(
|
||||
"filtermail/chatmaild.service.j2",
|
||||
bin_path="/usr/local/bin/filtermail",
|
||||
)
|
||||
assert (
|
||||
mock_tpl.call_args.kwargs["dest"] == "/etc/systemd/system/chatmaild.service"
|
||||
)
|
||||
|
||||
deployer = Deployer()
|
||||
|
||||
# Explicit dest_name override
|
||||
with patch("cmdeploy.basedeploy.files.put", return_value=mock_res) as mock_put:
|
||||
deployer.ensure_systemd_unit(
|
||||
"acmetool/acmetool-reconcile.timer",
|
||||
dest_name="acmetool-reconcile.timer",
|
||||
)
|
||||
assert (
|
||||
mock_put.call_args.kwargs["dest"]
|
||||
== "/etc/systemd/system/acmetool-reconcile.timer"
|
||||
)
|
||||
|
||||
|
||||
def test_ensure_service():
|
||||
with patch("cmdeploy.basedeploy.systemd.service") as mock_svc:
|
||||
deployer = Deployer()
|
||||
deployer.need_restart = True
|
||||
deployer.daemon_reload = True
|
||||
deployer.ensure_service("nginx.service")
|
||||
mock_svc.assert_called_once_with(
|
||||
name="Start and enable nginx.service",
|
||||
service="nginx.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=True,
|
||||
daemon_reload=True,
|
||||
)
|
||||
# daemon_reload is cleared to avoid multiple systemctl daemon-reload calls
|
||||
# need_restart is kept to ensure all subsequent services also restart
|
||||
assert deployer.need_restart is True
|
||||
assert deployer.daemon_reload is False
|
||||
|
||||
with patch("cmdeploy.basedeploy.systemd.service") as mock_svc:
|
||||
# Stopping suppresses restarted even when need_restart is True
|
||||
deployer = Deployer()
|
||||
deployer.need_restart = True
|
||||
deployer.daemon_reload = True
|
||||
deployer.ensure_service(
|
||||
"mta-sts-daemon.service",
|
||||
running=False,
|
||||
enabled=False,
|
||||
)
|
||||
assert mock_svc.call_args.kwargs["restarted"] is False
|
||||
assert deployer.need_restart is True
|
||||
|
||||
with patch("cmdeploy.basedeploy.systemd.service") as mock_svc:
|
||||
# Multiple calls: daemon_reload resets after first, need_restart persists
|
||||
deployer = Deployer()
|
||||
deployer.need_restart = True
|
||||
deployer.daemon_reload = True
|
||||
deployer.ensure_service("chatmaild.service")
|
||||
deployer.ensure_service("chatmaild-metadata.service")
|
||||
second_call = mock_svc.call_args_list[1]
|
||||
assert second_call.kwargs["restarted"] is True
|
||||
assert second_call.kwargs["daemon_reload"] is False
|
||||
@@ -23,21 +23,30 @@ class TestCmdline:
|
||||
run = parser.parse_args(["run"])
|
||||
assert init and run
|
||||
|
||||
def test_init_not_overwrite(self, tmp_path, capsys, monkeypatch):
|
||||
def test_init_not_overwrite(self, capsys, tmp_path, monkeypatch):
|
||||
monkeypatch.delenv("CHATMAIL_INI", raising=False)
|
||||
monkeypatch.chdir(tmp_path)
|
||||
|
||||
assert main(["init", "chat.example.org"]) == 0
|
||||
inipath = tmp_path / "chatmail.ini"
|
||||
args = ["init", "--config", str(inipath), "chat.example.org"]
|
||||
assert main(args) == 0
|
||||
capsys.readouterr()
|
||||
|
||||
assert main(["init", "chat.example.org"]) == 1
|
||||
assert main(args) == 1
|
||||
out, err = capsys.readouterr()
|
||||
assert "path exists" in out.lower()
|
||||
|
||||
assert main(["init", "chat.example.org", "--force"]) == 0
|
||||
args.insert(1, "--force")
|
||||
assert main(args) == 0
|
||||
out, err = capsys.readouterr()
|
||||
assert "deleting config file" in out.lower()
|
||||
|
||||
def test_dns_skip_on_ip(self, capsys, tmp_path, monkeypatch):
|
||||
monkeypatch.delenv("CHATMAIL_INI", raising=False)
|
||||
inipath = tmp_path / "chatmail.ini"
|
||||
assert main(["init", "--config", str(inipath), "1.3.3.7"]) == 0
|
||||
assert main(["dns", "--config", str(inipath)]) == 0
|
||||
out, err = capsys.readouterr()
|
||||
assert out == "[WARNING] 1.3.3.7 is not a domain, skipping DNS checks.\n"
|
||||
|
||||
|
||||
def test_www_folder(example_config, tmp_path):
|
||||
reporoot = importlib.resources.files(__package__).joinpath("../../../../").resolve()
|
||||
|
||||
@@ -4,6 +4,7 @@ import pytest
|
||||
|
||||
from cmdeploy import remote
|
||||
from cmdeploy.dns import check_full_zone, check_initial_remote_data, parse_zone_records
|
||||
from cmdeploy.remote.rdns import get_authoritative_ns
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -14,11 +15,15 @@ def mockdns_base(monkeypatch):
|
||||
if command.startswith("dig"):
|
||||
if command == "dig":
|
||||
return "."
|
||||
if "SOA" in command:
|
||||
if "with.public.soa" in command and "NS" in command:
|
||||
return "domain.with.public.soa. 2419 IN NS ns1.first-ns.de."
|
||||
if "with.hidden.soa" in command and "NS" in command:
|
||||
return (
|
||||
"delta.chat. 21600 IN SOA ns1.first-ns.de. dns.hetzner.com."
|
||||
" 2025102800 14400 1800 604800 3600"
|
||||
"domain.with.hidden.soa. 2137 IN NS ns1.desec.io.\n"
|
||||
"domain.with.hidden.soa. 2137 IN NS ns2.desec.org."
|
||||
)
|
||||
if "NS" in command:
|
||||
return "delta.chat. 21600 IN NS ns1.first-ns.de."
|
||||
command_chunks = command.split()
|
||||
domain, typ = command_chunks[4], command_chunks[6]
|
||||
try:
|
||||
@@ -125,6 +130,17 @@ class TestPerformInitialChecks:
|
||||
assert not l
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("domain", "ns"),
|
||||
[
|
||||
("domain.with.public.soa", "ns1.first-ns.de."),
|
||||
("domain.with.hidden.soa", "ns1.desec.io."),
|
||||
],
|
||||
)
|
||||
def test_get_authoritative_ns(domain, ns, mockdns):
|
||||
assert get_authoritative_ns(domain) == ns
|
||||
|
||||
|
||||
def test_parse_zone_records():
|
||||
text = """
|
||||
; This is a comment
|
||||
@@ -132,11 +148,28 @@ def test_parse_zone_records():
|
||||
|
||||
; Another comment
|
||||
www.some.domain. 3600 IN CNAME some.domain.
|
||||
|
||||
; Multi-word rdata
|
||||
some.domain. 3600 IN MX 10 mail.some.domain.
|
||||
|
||||
; DKIM record (single line, multi-word TXT rdata)
|
||||
dkim._domainkey.some.domain. 3600 IN TXT "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG" "9w0BAQEFAAOCAQ8AMIIBCgKCAQEA"
|
||||
|
||||
; Another TXT record
|
||||
_dmarc.some.domain. 3600 IN TXT "v=DMARC1;p=reject"
|
||||
"""
|
||||
records = list(parse_zone_records(text))
|
||||
assert records == [
|
||||
("some.domain", "3600", "A", "1.1.1.1"),
|
||||
("www.some.domain", "3600", "CNAME", "some.domain."),
|
||||
("some.domain", "3600", "MX", "10 mail.some.domain."),
|
||||
(
|
||||
"dkim._domainkey.some.domain",
|
||||
"3600",
|
||||
"TXT",
|
||||
'"v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG" "9w0BAQEFAAOCAQ8AMIIBCgKCAQEA"',
|
||||
),
|
||||
("_dmarc.some.domain", "3600", "TXT", '"v=DMARC1;p=reject"'),
|
||||
]
|
||||
|
||||
|
||||
@@ -148,7 +181,6 @@ def test_parse_zone_records_invalid_line():
|
||||
|
||||
def parse_zonefile_into_dict(zonefile, mockdns_base, only_required=False):
|
||||
if only_required:
|
||||
# Only take records before the "; Recommended" section
|
||||
zonefile = zonefile.split("; Recommended")[0]
|
||||
for name, ttl, rtype, rdata in parse_zone_records(zonefile):
|
||||
mockdns_base.setdefault(rtype, {})[name] = rdata
|
||||
|
||||
@@ -0,0 +1,237 @@
|
||||
from contextlib import nullcontext
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from pyinfra.facts.deb import DebPackages
|
||||
|
||||
from cmdeploy.dovecot import deployer as dovecot_deployer
|
||||
|
||||
|
||||
def make_host(*fact_pairs):
|
||||
"""Build a mock host; get_fact(cls) dispatches to the provided facts mapping.
|
||||
|
||||
Args:
|
||||
*fact_pairs: tuples of (fact_class, fact_value) to register
|
||||
|
||||
Returns:
|
||||
SimpleNamespace with get_fact that raises a clear error if an
|
||||
unexpected fact type is requested.
|
||||
"""
|
||||
facts = dict(fact_pairs)
|
||||
|
||||
def get_fact(cls):
|
||||
if cls not in facts:
|
||||
registered = ", ".join(c.__name__ for c in facts)
|
||||
raise LookupError(
|
||||
f"unexpected get_fact({cls.__name__}); only registered: {registered}"
|
||||
)
|
||||
return facts[cls]
|
||||
|
||||
return SimpleNamespace(get_fact=get_fact)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def deployer():
|
||||
return dovecot_deployer.DovecotDeployer(
|
||||
SimpleNamespace(mail_domain="chat.example.org"),
|
||||
disable_mail=False,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def patch_blocked(monkeypatch):
|
||||
monkeypatch.setattr(dovecot_deployer, "blocked_service_startup", nullcontext)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_files_put(monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer.files,
|
||||
"put",
|
||||
lambda **kwargs: SimpleNamespace(changed=False),
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def track_shell(monkeypatch):
|
||||
calls = []
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer.server,
|
||||
"shell",
|
||||
lambda **kwargs: calls.append(kwargs) or SimpleNamespace(changed=False),
|
||||
)
|
||||
return calls
|
||||
|
||||
|
||||
def test_download_dovecot_package_skips_epoch_matched_install(monkeypatch):
|
||||
epoch_version = dovecot_deployer.DOVECOT_PACKAGE_VERSION
|
||||
downloads = []
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer,
|
||||
"host",
|
||||
make_host((DebPackages, {"dovecot-core": [epoch_version]})),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer,
|
||||
"_pick_url",
|
||||
lambda primary, fallback: primary,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer.files,
|
||||
"download",
|
||||
lambda **kwargs: downloads.append(kwargs),
|
||||
)
|
||||
|
||||
deb, changed = dovecot_deployer._download_dovecot_package("core", "amd64")
|
||||
|
||||
assert deb is None, f"expected no deb path when version matches, got {deb!r}"
|
||||
assert changed is False, "should not flag changed when version already installed"
|
||||
assert downloads == [], "should not download when version already installed"
|
||||
|
||||
|
||||
def test_download_dovecot_package_uses_archive_version_for_url_and_filename(
|
||||
monkeypatch,
|
||||
):
|
||||
downloads = []
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer,
|
||||
"host",
|
||||
make_host((DebPackages, {})),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer,
|
||||
"_pick_url",
|
||||
lambda primary, fallback: primary,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer.files,
|
||||
"download",
|
||||
lambda **kwargs: downloads.append(kwargs),
|
||||
)
|
||||
|
||||
deb, changed = dovecot_deployer._download_dovecot_package("core", "amd64")
|
||||
|
||||
archive_version = dovecot_deployer.DOVECOT_ARCHIVE_VERSION.replace("+", "%2B")
|
||||
expected_deb = f"/root/dovecot-core_{archive_version}_amd64.deb"
|
||||
|
||||
# Verify the returned path uses archive version, not package version (with epoch)
|
||||
assert changed is True, "should flag changed when package not yet installed"
|
||||
assert deb == expected_deb, f"deb path mismatch: {deb!r} != {expected_deb!r}"
|
||||
assert dovecot_deployer.DOVECOT_PACKAGE_VERSION not in deb, (
|
||||
f"deb path should use archive version (no epoch), got {deb!r}"
|
||||
)
|
||||
assert len(downloads) == 1, "files.download should be called exactly once"
|
||||
|
||||
|
||||
def test_install_skips_dpkg_path_when_epoch_matched_packages_present(
|
||||
deployer, patch_blocked, mock_files_put, track_shell, monkeypatch
|
||||
):
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer,
|
||||
"host",
|
||||
make_host(
|
||||
(
|
||||
dovecot_deployer.DebPackages,
|
||||
{
|
||||
"dovecot-core": [dovecot_deployer.DOVECOT_PACKAGE_VERSION],
|
||||
"dovecot-imapd": [dovecot_deployer.DOVECOT_PACKAGE_VERSION],
|
||||
"dovecot-lmtpd": [dovecot_deployer.DOVECOT_PACKAGE_VERSION],
|
||||
},
|
||||
),
|
||||
(dovecot_deployer.Arch, "x86_64"),
|
||||
),
|
||||
)
|
||||
downloads = []
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer.files,
|
||||
"download",
|
||||
lambda **kwargs: downloads.append(kwargs),
|
||||
)
|
||||
|
||||
deployer.install()
|
||||
|
||||
assert downloads == [], "should not download when all packages epoch-matched"
|
||||
assert track_shell == [], "should not run dpkg when all packages epoch-matched"
|
||||
assert deployer.need_restart is False, (
|
||||
"need_restart should be False when nothing changed"
|
||||
)
|
||||
|
||||
|
||||
def test_install_unsupported_arch_falls_back_to_apt(
|
||||
deployer, patch_blocked, mock_files_put, track_shell, monkeypatch
|
||||
):
|
||||
# For unsupported architectures, all fact lookups return the arch string.
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer,
|
||||
"host",
|
||||
SimpleNamespace(get_fact=lambda cls: "riscv64"),
|
||||
)
|
||||
apt_calls = []
|
||||
|
||||
# Mirrors apt.packages() return value: OperationMeta with .changed property.
|
||||
# Only lmtpd triggers a change to verify |= accumulation of changed flags.
|
||||
def fake_apt(**kwargs):
|
||||
apt_calls.append(kwargs)
|
||||
changed = "lmtpd" in kwargs["packages"][0]
|
||||
return SimpleNamespace(changed=changed)
|
||||
|
||||
monkeypatch.setattr(dovecot_deployer.apt, "packages", fake_apt)
|
||||
|
||||
deployer.install()
|
||||
|
||||
actual_pkgs = [c["packages"] for c in apt_calls]
|
||||
assert actual_pkgs == [["dovecot-core"], ["dovecot-imapd"], ["dovecot-lmtpd"]], (
|
||||
f"expected apt install of core/imapd/lmtpd, got {actual_pkgs}"
|
||||
)
|
||||
assert track_shell == [], "should not run dpkg for unsupported arch"
|
||||
assert deployer.need_restart is True, (
|
||||
"need_restart should be True when apt installed a package"
|
||||
)
|
||||
|
||||
|
||||
def test_install_runs_dpkg_when_packages_need_download(
|
||||
deployer, patch_blocked, mock_files_put, track_shell, monkeypatch
|
||||
):
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer,
|
||||
"host",
|
||||
make_host(
|
||||
(dovecot_deployer.DebPackages, {}),
|
||||
(dovecot_deployer.Arch, "x86_64"),
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer,
|
||||
"_pick_url",
|
||||
lambda primary, fallback: primary,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
dovecot_deployer.files,
|
||||
"download",
|
||||
lambda **kwargs: SimpleNamespace(changed=True),
|
||||
)
|
||||
|
||||
deployer.install()
|
||||
|
||||
assert len(track_shell) == 1, (
|
||||
f"expected one server.shell() call for dpkg install, got {len(track_shell)}"
|
||||
)
|
||||
cmds = track_shell[0]["commands"]
|
||||
assert len(cmds) == 3, f"expected 3 dpkg/apt commands, got: {cmds}"
|
||||
assert cmds[0].startswith("dpkg --force-confdef --force-confold -i ")
|
||||
assert "apt-get -y --fix-broken install" in cmds[1]
|
||||
assert cmds[2].startswith("dpkg --force-confdef --force-confold -i ")
|
||||
assert deployer.need_restart is True, (
|
||||
"need_restart should be True after dpkg install"
|
||||
)
|
||||
|
||||
|
||||
def test_pick_url_falls_back_on_primary_error(monkeypatch):
|
||||
def raise_error(req, timeout):
|
||||
raise OSError("connection timeout")
|
||||
|
||||
monkeypatch.setattr(dovecot_deployer.urllib.request, "urlopen", raise_error)
|
||||
result = dovecot_deployer._pick_url("http://primary", "http://fallback")
|
||||
assert result == "http://fallback", (
|
||||
f"should fall back when primary fails, got {result!r}"
|
||||
)
|
||||
@@ -1,11 +1,10 @@
|
||||
import importlib.resources
|
||||
from pathlib import Path
|
||||
|
||||
from cmdeploy.www import build_webpages
|
||||
|
||||
|
||||
def test_build_webpages(tmp_path, make_config):
|
||||
pkgroot = importlib.resources.files("cmdeploy")
|
||||
src_dir = pkgroot.joinpath("../../../www/src").resolve()
|
||||
src_dir = (Path(__file__).resolve() / "../../../../../www/src").resolve()
|
||||
assert src_dir.exists(), src_dir
|
||||
config = make_config("chat.example.org")
|
||||
build_dir = tmp_path.joinpath("build")
|
||||
|
||||
@@ -1,174 +0,0 @@
|
||||
"""Tests for cmdeploy lxc-* subcommands."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from cmdeploy.lxc import cli
|
||||
from cmdeploy.lxc.incus import Incus
|
||||
from cmdeploy.util import Out
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
not shutil.which("incus"),
|
||||
reason="incus not installed",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ix():
|
||||
out = Out()
|
||||
return Incus(out)
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def lxc_setup():
|
||||
out = Out()
|
||||
ix = Incus(out)
|
||||
ix.get_dns_container().ensure()
|
||||
return ix.list_managed()
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def relay_container(lxc_setup):
|
||||
test_names = {f"{n}-localchat" for n in cli.RELAY_NAMES}
|
||||
relays = [c for c in lxc_setup if c["name"] in test_names and c.get("ip")]
|
||||
if not relays:
|
||||
pytest.skip("no test relay containers running")
|
||||
return relays[0]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def cmdeploy():
|
||||
def run(*args):
|
||||
return subprocess.run(
|
||||
[sys.executable, "-m", "cmdeploy.cmdeploy", *args],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
return run
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"subcmd, expected, absent",
|
||||
[
|
||||
(None, ["lxc-start", "lxc-stop", "lxc-test", "lxc-status"], ["lxc-destroy"]),
|
||||
("lxc-start", ["--ipv4-only", "--run"], ["--config"]),
|
||||
("lxc-stop", ["--destroy", "--destroy-all"], ["--config"]),
|
||||
("lxc-test", ["--one"], ["--config"]),
|
||||
("lxc-status", [], ["--config"]),
|
||||
("run", ["--ssh-config"], ["--lxc"]),
|
||||
("dns", ["--ssh-config"], []),
|
||||
("test", ["--ssh-config"], []),
|
||||
("status", ["--ssh-config"], []),
|
||||
],
|
||||
)
|
||||
def test_help_options(cmdeploy, subcmd, expected, absent):
|
||||
args = [subcmd, "--help"] if subcmd else ["--help"]
|
||||
result = cmdeploy(*args)
|
||||
output = result.stdout + result.stderr
|
||||
assert result.returncode == 0
|
||||
for flag in expected:
|
||||
assert flag in output
|
||||
for flag in absent:
|
||||
assert flag not in output
|
||||
|
||||
|
||||
class TestSSHConfig:
|
||||
def test_lxconfigs(self, ix, lxc_setup):
|
||||
d = ix.lxconfigs_dir
|
||||
assert d.name == "lxconfigs"
|
||||
assert d.exists()
|
||||
path = ix.ssh_config_path
|
||||
assert path.name == "ssh-config"
|
||||
assert path.parent.name == "lxconfigs"
|
||||
|
||||
def test_write_ssh_config(self, ix, lxc_setup):
|
||||
path = ix.write_ssh_config()
|
||||
assert path.exists()
|
||||
text = path.read_text()
|
||||
|
||||
for c in lxc_setup:
|
||||
if c.get("ip"):
|
||||
assert c["name"] in text
|
||||
assert f"Hostname {c['ip']}" in text
|
||||
|
||||
assert "User root" in text
|
||||
assert "IdentityFile" in text
|
||||
assert "StrictHostKeyChecking accept-new" in text
|
||||
|
||||
|
||||
def test_dns(ix, relay_container):
|
||||
def dig(qname, qtype):
|
||||
ct = ix.get_dns_container()
|
||||
return ct.bash(f"dig @127.0.0.1 {qname} {qtype} +short").strip()
|
||||
|
||||
domain = relay_container["domain"]
|
||||
assert dig(domain, "A") == relay_container["ip"]
|
||||
assert domain in dig(domain, "MX")
|
||||
assert "587" in dig(f"_submission._tcp.{domain}", "SRV")
|
||||
|
||||
|
||||
class TestLxcStatus:
|
||||
def test_cli_lxc_status_help(self, cmdeploy):
|
||||
result = cmdeploy("lxc-status", "--help")
|
||||
assert result.returncode == 0
|
||||
assert "status" in result.stdout.lower()
|
||||
|
||||
def test_shows_containers(self, lxc_setup, capsys):
|
||||
class QuietOut(Out):
|
||||
def red(self, msg, **kw):
|
||||
pass
|
||||
|
||||
def green(self, msg, **kw):
|
||||
pass
|
||||
|
||||
ret = cli.lxc_status_cmd(None, QuietOut())
|
||||
assert ret == 0
|
||||
captured = capsys.readouterr().out
|
||||
assert "ns-localchat" in captured
|
||||
assert "running" in captured
|
||||
|
||||
def test_deploy_freshness(self, ix, monkeypatch):
|
||||
ct = ix.get_container("x")
|
||||
|
||||
monkeypatch.setattr(
|
||||
"cmdeploy.lxc.incus.RelayContainer.deployed_version",
|
||||
lambda _self: "abc123def456",
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"cmdeploy.lxc.incus.RelayContainer.deployed_domain",
|
||||
lambda _self: ct.domain,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"cmdeploy.lxc.cli.get_version_string",
|
||||
lambda: "abc123def456",
|
||||
)
|
||||
assert "IN-SYNC" in cli._deploy_status(ct, "abc123def456", ix)
|
||||
assert "STALE" in cli._deploy_status(ct, "other_hash_here", ix)
|
||||
|
||||
# Hash matches but local has uncommitted changes
|
||||
monkeypatch.setattr(
|
||||
"cmdeploy.lxc.cli.get_version_string",
|
||||
lambda: "abc123def456\ndiff --git a/foo",
|
||||
)
|
||||
assert "DIRTY" in cli._deploy_status(ct, "abc123def456", ix)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"cmdeploy.lxc.incus.RelayContainer.deployed_version",
|
||||
lambda _self: None,
|
||||
)
|
||||
assert "NOT DEPLOYED" in cli._deploy_status(ct, "abc123", ix)
|
||||
@@ -1,120 +0,0 @@
|
||||
import sys
|
||||
|
||||
from cmdeploy.util import Out, collapse, get_git_hash, get_version_string, shell
|
||||
|
||||
|
||||
class TestOut:
|
||||
def test_prefix_default(self, capsys):
|
||||
out = Out()
|
||||
out.print("hello")
|
||||
assert capsys.readouterr().out == "hello\n"
|
||||
|
||||
def test_prefix_custom(self, capsys):
|
||||
out = Out(prefix=">> ")
|
||||
out.print("hello")
|
||||
assert capsys.readouterr().out == ">> hello\n"
|
||||
|
||||
def test_prefix_print_file(self):
|
||||
import io
|
||||
|
||||
buf = io.StringIO()
|
||||
out = Out(prefix=":: ")
|
||||
out.print("msg", file=buf)
|
||||
assert ":: msg" in buf.getvalue()
|
||||
|
||||
def test_new_prefixed_out(self, capsys):
|
||||
parent = Out(prefix="A")
|
||||
child = parent.new_prefixed_out("B")
|
||||
child.print("x")
|
||||
assert capsys.readouterr().out == "ABx\n"
|
||||
# shares section_timings
|
||||
assert child.section_timings is parent.section_timings
|
||||
|
||||
def test_section_no_auto_indent(self, capsys):
|
||||
out = Out(prefix="")
|
||||
with out.section("test"):
|
||||
out.print("inside")
|
||||
captured = capsys.readouterr().out
|
||||
# "inside" should NOT be indented by section()
|
||||
lines = captured.strip().splitlines()
|
||||
inside_line = [l for l in lines if "inside" in l][0]
|
||||
assert inside_line == "inside"
|
||||
|
||||
def test_section_records_timing(self):
|
||||
out = Out()
|
||||
with out.section("s1"):
|
||||
pass
|
||||
assert len(out.section_timings) == 1
|
||||
assert out.section_timings[0][0] == "s1"
|
||||
|
||||
def test_shell_failure_shows_output(self):
|
||||
"""When a shell command fails, its output and exit code are shown."""
|
||||
import subprocess
|
||||
|
||||
result = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-c",
|
||||
"from cmdeploy.util import Out; Out(prefix='').shell("
|
||||
"\"echo 'boom on stderr' >&2; exit 42\")",
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
# the command's stderr is merged into stdout by Popen
|
||||
assert "boom on stderr" in result.stdout
|
||||
# Out.red() prints the failure notice to stderr
|
||||
assert "exit code 42" in result.stderr
|
||||
|
||||
|
||||
def test_collapse():
|
||||
text = """
|
||||
line 1
|
||||
line 2
|
||||
"""
|
||||
assert collapse(text) == "line 1 line 2"
|
||||
assert collapse(" single line ") == "single line"
|
||||
|
||||
|
||||
def test_git_helpers_no_git(tmp_path):
|
||||
# Not a git repo
|
||||
assert get_git_hash(root=tmp_path) is None
|
||||
assert get_version_string(root=tmp_path) == "unknown"
|
||||
|
||||
|
||||
def test_git_helpers_empty_repo(tmp_path):
|
||||
shell("git init", cwd=tmp_path, check=True)
|
||||
# No commits yet
|
||||
assert get_git_hash(root=tmp_path) is None
|
||||
assert get_version_string(root=tmp_path) == "unknown"
|
||||
|
||||
|
||||
def test_git_helpers_with_commits_and_diffs(tmp_path):
|
||||
shell("git init", cwd=tmp_path, check=True)
|
||||
shell("git config user.email you@example.com", cwd=tmp_path, check=True)
|
||||
shell("git config user.name 'Your Name'", cwd=tmp_path, check=True)
|
||||
|
||||
# First commit
|
||||
path = tmp_path / "file.txt"
|
||||
path.write_text("content")
|
||||
shell("git add file.txt", cwd=tmp_path, check=True)
|
||||
shell("git commit -m initial", cwd=tmp_path, check=True)
|
||||
|
||||
git_hash = get_git_hash(root=tmp_path)
|
||||
assert len(git_hash) >= 7 # usually 40, but git is git
|
||||
assert get_version_string(root=tmp_path) == git_hash
|
||||
|
||||
# Create a diff
|
||||
path.write_text("new content")
|
||||
v = get_version_string(root=tmp_path)
|
||||
assert v.startswith(git_hash + "\n")
|
||||
assert "new content" in v
|
||||
assert not v.endswith("\n")
|
||||
|
||||
# Commit again -> no diff
|
||||
shell("git add file.txt", cwd=tmp_path, check=True)
|
||||
shell("git commit -m second", cwd=tmp_path, check=True)
|
||||
new_hash = get_git_hash(root=tmp_path)
|
||||
assert new_hash != git_hash
|
||||
assert get_version_string(root=tmp_path) == new_hash
|
||||
@@ -1,4 +1,7 @@
|
||||
# Managed by cmdeploy: disable IPv6 in unbound.
|
||||
# Managed by cmdeploy
|
||||
server:
|
||||
{% if disable_ipv6 %}
|
||||
interface: 127.0.0.1
|
||||
do-ip6: no
|
||||
{% endif %}
|
||||
cache-max-negative-ttl: 0
|
||||
|
||||
@@ -1,169 +0,0 @@
|
||||
"""Shared utility functions for cmdeploy."""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
import time
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
|
||||
from termcolor import colored
|
||||
|
||||
|
||||
class Out:
|
||||
"""Convenience output printer providing coloring and section formatting."""
|
||||
|
||||
def __init__(self, prefix="", verbosity=0):
|
||||
self.section_timings = []
|
||||
self.prefix = prefix
|
||||
self.sepchar = "\u2501"
|
||||
self.verbosity = verbosity
|
||||
env_width = os.environ.get("_CMDEPLOY_WIDTH")
|
||||
if env_width:
|
||||
self.section_width = int(env_width)
|
||||
else:
|
||||
self.section_width = shutil.get_terminal_size((80, 24)).columns
|
||||
|
||||
def new_prefixed_out(self, newprefix=" "):
|
||||
"""Return a new Out with an extended prefix,
|
||||
sharing section_timings with the parent.
|
||||
"""
|
||||
out = Out(
|
||||
prefix=self.prefix + newprefix,
|
||||
verbosity=self.verbosity,
|
||||
)
|
||||
out.section_timings = self.section_timings
|
||||
return out
|
||||
|
||||
def red(self, msg, file=sys.stderr):
|
||||
print(colored(self.prefix + msg, "red"), file=file, flush=True)
|
||||
|
||||
def green(self, msg, file=sys.stderr):
|
||||
print(colored(self.prefix + msg, "green"), file=file, flush=True)
|
||||
|
||||
def print(self, msg="", **kwargs):
|
||||
"""Print to stdout with automatic flush."""
|
||||
if msg:
|
||||
msg = self.prefix + msg
|
||||
print(msg, flush=True, **kwargs)
|
||||
|
||||
def _format_header(self, title):
|
||||
"""Return a formatted section header string."""
|
||||
width = self.section_width - len(self.prefix)
|
||||
bar = self.sepchar * (width - len(title) - 5)
|
||||
return f"{self.sepchar * 3} {title} {bar}"
|
||||
|
||||
@contextmanager
|
||||
def section(self, title):
|
||||
"""Context manager that prints a section header and records elapsed time."""
|
||||
self.green(self._format_header(title))
|
||||
t0 = time.time()
|
||||
yield
|
||||
elapsed = time.time() - t0
|
||||
self.section_timings.append((title, elapsed))
|
||||
|
||||
def section_line(self, title):
|
||||
"""Print a section header without timing."""
|
||||
self.green(self._format_header(title))
|
||||
|
||||
def shell(self, cmd, quiet=False, **kwargs):
|
||||
"""Print *cmd*, run it, and re-print its output with the current prefix.
|
||||
|
||||
*cmd* is passed through :func:`collapse`, so callers
|
||||
can use triple-quoted f-strings freely.
|
||||
Stdout and stderr are merged, read line-by-line,
|
||||
and each line is printed with ``self.prefix`` prepended.
|
||||
When the command exits non-zero, a red error line is printed.
|
||||
"""
|
||||
cmd = collapse(cmd)
|
||||
if not quiet:
|
||||
self.print(f"$ {cmd}")
|
||||
indent = self.prefix + " "
|
||||
env = kwargs.pop("env", None)
|
||||
if env is None:
|
||||
env = os.environ.copy()
|
||||
env["_CMDEPLOY_WIDTH"] = str(self.section_width - len(indent))
|
||||
proc = subprocess.Popen(
|
||||
cmd,
|
||||
shell=True,
|
||||
text=True,
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
env=env,
|
||||
**kwargs,
|
||||
)
|
||||
for line in proc.stdout:
|
||||
sys.stdout.write(indent + line)
|
||||
sys.stdout.flush()
|
||||
ret = proc.wait()
|
||||
if ret:
|
||||
self.red(f"command failed with exit code {ret}: {cmd}")
|
||||
return ret
|
||||
|
||||
|
||||
def _project_root():
|
||||
"""Return the project root directory."""
|
||||
return Path(__file__).resolve().parent.parent.parent.parent
|
||||
|
||||
|
||||
def collapse(text):
|
||||
"""Dedent, join lines, and strip a (triple-quoted) string.
|
||||
|
||||
Handy for writing shell commands across multiple lines::
|
||||
|
||||
cmd = collapse(f\"""
|
||||
cmdeploy run
|
||||
--config {ct.ini}
|
||||
--ssh-host {ct.domain}
|
||||
\""")
|
||||
"""
|
||||
return textwrap.dedent(text).replace("\n", " ").strip()
|
||||
|
||||
|
||||
def shell(cmd, check=False, **kwargs):
|
||||
"""Run a shell command string with sensible defaults.
|
||||
|
||||
*cmd* is passed through :func:`collapse` first, so callers
|
||||
can use triple-quoted f-strings freely.
|
||||
Captures stdout/stderr by default; pass ``capture_output=False``
|
||||
to stream output to the terminal instead.
|
||||
"""
|
||||
if "capture_output" not in kwargs and "stdout" not in kwargs:
|
||||
kwargs["capture_output"] = True
|
||||
kwargs.setdefault("stdin", subprocess.DEVNULL)
|
||||
return subprocess.run(collapse(cmd), shell=True, text=True, check=check, **kwargs)
|
||||
|
||||
|
||||
def get_git_hash(root=None):
|
||||
"""Return the local HEAD commit hash, or None."""
|
||||
if root is None:
|
||||
root = _project_root()
|
||||
result = shell(
|
||||
"git rev-parse HEAD",
|
||||
cwd=str(root),
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return result.stdout.strip()
|
||||
return None
|
||||
|
||||
|
||||
def get_version_string(root=None):
|
||||
"""Return ``git_hash\\ngit_diff`` for the local working tree.
|
||||
|
||||
Used by :class:`~cmdeploy.deployers.GithashDeployer` to write
|
||||
``/etc/chatmail-version`` and by ``lxc-status`` to compare
|
||||
the deployed state against the local checkout.
|
||||
"""
|
||||
if root is None:
|
||||
root = _project_root()
|
||||
git_hash = get_git_hash(root=root) or "unknown"
|
||||
try:
|
||||
git_diff = shell("git diff", cwd=str(root)).stdout.strip()
|
||||
except Exception:
|
||||
git_diff = ""
|
||||
if git_diff:
|
||||
return f"{git_hash}\n{git_diff}"
|
||||
return git_hash
|
||||
@@ -1,5 +1,4 @@
|
||||
import hashlib
|
||||
import importlib.resources
|
||||
import re
|
||||
import time
|
||||
import traceback
|
||||
@@ -37,7 +36,7 @@ def prepare_template(source):
|
||||
|
||||
|
||||
def get_paths(config) -> (Path, Path, Path):
|
||||
reporoot = importlib.resources.files(__package__).joinpath("../../../").resolve()
|
||||
reporoot = (Path(__file__).resolve() / "../../../../").resolve()
|
||||
www_path = Path(config.www_folder)
|
||||
# if www_folder was not set, use default directory
|
||||
if config.www_folder == "":
|
||||
@@ -133,8 +132,7 @@ def find_merge_conflict(src_dir) -> Path:
|
||||
|
||||
|
||||
def main():
|
||||
path = importlib.resources.files(__package__)
|
||||
reporoot = path.joinpath("../../../").resolve()
|
||||
reporoot = (Path(__file__).resolve() / "../../../../").resolve()
|
||||
inipath = reporoot.joinpath("chatmail.ini")
|
||||
config = read_config(inipath)
|
||||
config.webdev = True
|
||||
|
||||
+5
-3
@@ -4,12 +4,14 @@
|
||||
|
||||
You can use the `make` command and `make html` to build web pages.
|
||||
|
||||
You need a Python environment where the following install was excuted:
|
||||
|
||||
pip install furo sphinx-autobuild
|
||||
You need a Python environment with `sphinx` and other
|
||||
dependencies, you can create it by running `scripts/initenv.sh`
|
||||
from the repository root.
|
||||
|
||||
To develop/change documentation, you can then do:
|
||||
|
||||
. venv/bin/activate
|
||||
cd doc
|
||||
make auto
|
||||
|
||||
A page will open at https://127.0.0.1:8000/ serving the docs and it will
|
||||
|
||||
+7
-1
@@ -15,13 +15,19 @@ author = 'chatmail collective'
|
||||
|
||||
extensions = [
|
||||
#'sphinx.ext.autodoc',
|
||||
#'sphinx.ext.viewcode',
|
||||
#'sphinx.ext.viewdoc',
|
||||
'sphinxcontrib.mermaid',
|
||||
]
|
||||
|
||||
templates_path = ['_templates']
|
||||
exclude_patterns = []
|
||||
|
||||
linkcheck_ignore = [
|
||||
# only resolves once the file is merged to main
|
||||
r"https://github\.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/defaults/appversions\.json",
|
||||
]
|
||||
|
||||
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
@@ -15,6 +15,7 @@ goes beyond what classic email servers offer:
|
||||
streaming, privacy-preserving Push Notifications for Apple, Google, and `Ubuntu Touch <https://docs.ubports.com/en/latest/appdev/guides/pushnotifications.html>`_;
|
||||
|
||||
- **Security Enforcement**: only strict TLS, DKIM and OpenPGP with minimized metadata accepted
|
||||
(DKIM is not enforced on :ref:`IP-only relays <iponly>`)
|
||||
|
||||
- **Reliable Federation and Decentralization:** No spam or IP reputation checks, federating
|
||||
depends on established IETF standards and protocols.
|
||||
@@ -47,6 +48,29 @@ Dovecot, and are configured to run unattended without much maintenance
|
||||
effort. Chatmail relays happily run on low-end hardware like a Raspberry
|
||||
Pi.
|
||||
|
||||
.. _upgrade:
|
||||
|
||||
How can I upgrade my chatmail relay?
|
||||
------------------------------------
|
||||
|
||||
To upgrade to the latest ``main`` branch,
|
||||
``cd`` into your local checkout of `https://github.com/chatmail/relay/`_
|
||||
and run the following commands:
|
||||
|
||||
::
|
||||
|
||||
git pull origin main --rebase --autostash
|
||||
scripts/initenv.sh
|
||||
scripts/cmdeploy run
|
||||
|
||||
If you don't want the latest development version,
|
||||
but a specific tagged release like `1.10.0 <https://github.com/chatmail/relay/releases/tag/1.10.0>`_,
|
||||
run ``git pull origin 1.10.0`` instead.
|
||||
|
||||
If you made local changes for your setup,
|
||||
they will be reapplied as long as they don't conflict with the upgrade.
|
||||
If a conflict arises, ``git status`` will tell you how to resolve it.
|
||||
|
||||
|
||||
How trustable are chatmail relays?
|
||||
----------------------------------
|
||||
|
||||
@@ -14,8 +14,6 @@ Minimal requirements and prerequisites
|
||||
|
||||
You will need the following:
|
||||
|
||||
- Control over a domain through a DNS provider of your choice.
|
||||
|
||||
- A Debian 12 **deployment server** with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
|
||||
IPv6 is encouraged if available. Chatmail relay servers only require
|
||||
1GB RAM, one CPU, and perhaps 10GB storage for a few thousand active
|
||||
@@ -28,6 +26,11 @@ You will need the following:
|
||||
(An ed25519 private key is required due to an `upstream bug in
|
||||
paramiko <https://github.com/paramiko/paramiko/issues/2191>`_)
|
||||
|
||||
- Control over a domain through a DNS provider of your choice
|
||||
(there is experimental support for :ref:`IP-only relays <iponly>`).
|
||||
|
||||
|
||||
.. _setup:
|
||||
|
||||
Setup with ``scripts/cmdeploy``
|
||||
-------------------------------------
|
||||
@@ -98,6 +101,15 @@ steps. Please substitute it with your own domain.
|
||||
configure at your DNS provider (it can take some time until they are
|
||||
public).
|
||||
|
||||
|
||||
|
||||
Docker installation
|
||||
-------------------
|
||||
|
||||
There is experimental support for running chatmail via Docker.
|
||||
A monolithic image based on the above cmdeploy method is available `through a separate repository <https://github.com/chatmail/docker/pkgs/container/docker>`_.
|
||||
See the `chatmail/docker README <https://github.com/chatmail/docker>`_ for full setup instructions.
|
||||
|
||||
Other helpful commands
|
||||
----------------------
|
||||
|
||||
@@ -187,6 +199,93 @@ creating addresses, login with ssh to the deployment machine and run:
|
||||
Chatmail address creation will be denied while this file is present.
|
||||
|
||||
|
||||
.. _system-limits:
|
||||
|
||||
Configurable System Limits
|
||||
--------------------------
|
||||
|
||||
Limits for auto-rejecting address creation
|
||||
..........................................
|
||||
|
||||
A relay refuses creation of new addresses
|
||||
when the machine runs low on resources,
|
||||
but existing addresses keep working.
|
||||
|
||||
Three ``chatmail.ini`` settings control this,
|
||||
shown here with their defaults::
|
||||
|
||||
max_load_1m = 5
|
||||
min_available_memory = 200M
|
||||
min_free_disk_space = 1G
|
||||
|
||||
- ``max_load_1m`` is the maximum 1-minute load average,
|
||||
as reported by ``uptime``;
|
||||
it counts processes waiting for disk I/O as well as for CPU.
|
||||
It is deliberately not scaled by the number of CPUs
|
||||
because I/O rather than CPU is what typically limits a relay.
|
||||
|
||||
- ``min_available_memory`` is the minimum memory available without swapping.
|
||||
|
||||
- ``min_free_disk_space`` is the minimum free disk space
|
||||
on the file system holding the mailboxes.
|
||||
|
||||
The defaults suit the small machine described in
|
||||
`Minimal requirements and prerequisites`_.
|
||||
|
||||
.. note::
|
||||
|
||||
If you run a bigger machine,
|
||||
raise ``max_load_1m`` after watching ``uptime`` under typical load.
|
||||
|
||||
Rejections are logged by the ``doveauth`` service,
|
||||
so you can check whether a limit is set too tightly::
|
||||
|
||||
journalctl -u doveauth --grep 'registration rejected'
|
||||
|
||||
If ``mtail_address`` is set, rejections are also counted
|
||||
in the ``rejected_registrations`` metric.
|
||||
|
||||
|
||||
Overall IMAP and SMTP connection limits
|
||||
.......................................
|
||||
|
||||
Two further settings bound how many connections
|
||||
the relay accepts at all, again shown with their defaults::
|
||||
|
||||
max_imap_connections = 10000
|
||||
max_smtp_connections = 1000
|
||||
|
||||
``max_imap_connections`` becomes the Dovecot imap process limit,
|
||||
and ``max_smtp_connections`` the Postfix process limit
|
||||
on each of the submission and smtps ports.
|
||||
A single client IP may use up to a fifth of ``max_smtp_connections``.
|
||||
Each connection costs memory,
|
||||
so these limits defend the relay against running out of RAM.
|
||||
|
||||
Unless ``imap_compress`` is enabled,
|
||||
an IMAP connection that is idle for ``imap_hibernate_timeout``
|
||||
is handed over to the ``imap-hibernate`` process
|
||||
and does not count towards ``max_imap_connections``,
|
||||
which is why a relay can serve far more IMAP clients
|
||||
than this setting suggests.
|
||||
|
||||
If you run a large relay with 10k or 100k's of addresses,
|
||||
check current connection counts before upgrading
|
||||
and set the limits accordingly.
|
||||
|
||||
To see how close a running relay is to these two limits,
|
||||
copy ``scripts/check-connections.sh`` from the relay repository
|
||||
onto the relay and run it there::
|
||||
|
||||
imap 5 ports 143,993 (max_imap_connections)
|
||||
5 dovecot sessions, 0 of them in an active imap process
|
||||
submission 0 ports 465,587 (max_smtp_connections per port)
|
||||
incoming 0 port 25 (from other relays, no chatmail.ini limit)
|
||||
|
||||
It counts established sockets with ``ss``
|
||||
and cross-checks the IMAP number against ``doveadm who``.
|
||||
|
||||
|
||||
Running a relay with self-signed certificates
|
||||
----------------------------------------------
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ Contributions and feedback welcome through the https://github.com/chatmail/relay
|
||||
proxy
|
||||
migrate
|
||||
overview
|
||||
lxc
|
||||
reverse_dns
|
||||
related
|
||||
faq
|
||||
iponly
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
.. _iponly:
|
||||
|
||||
Hosting without DNS records
|
||||
===========================
|
||||
|
||||
.. note::
|
||||
|
||||
This option is experimental and might change without notice.
|
||||
|
||||
In case you don't have a domain,
|
||||
for example in a local network,
|
||||
you can run a chatmail relay with only an IPv4 address as well.
|
||||
|
||||
To deploy a relay without a domain,
|
||||
run ``cmdeploy init`` with only the IPv4 address
|
||||
during the :ref:`installation steps <setup>`,
|
||||
for example ``cmdeploy init 13.12.23.42``.
|
||||
|
||||
Drawbacks
|
||||
---------
|
||||
|
||||
- your transport encryption will only use self-signed TLS certificates,
|
||||
which are vulnerable against MITM attacks.
|
||||
the chatmail core's end-to-end encryption should suffice in most scenarios though.
|
||||
|
||||
- your messages will not be DKIM-signed;
|
||||
experimentally, most chatmail relays accept non-DKIM-signed messages from IP-only relays,
|
||||
but some relays might not accept messages from yours.
|
||||
|
||||
|
||||
Email addresses
|
||||
---------------
|
||||
|
||||
When running without a domain,
|
||||
your chatmail addresses will use the IPv4 address
|
||||
in brackets as the domain part,
|
||||
for example ``user@[13.12.23.42]``.
|
||||
This is a valid email address format
|
||||
according to :rfc:`5321`.
|
||||
|
||||
@@ -1,271 +0,0 @@
|
||||
Local testing with LXC/Incus
|
||||
============================
|
||||
|
||||
The ``cmdeploy`` tool includes support for running
|
||||
chatmail relays inside local
|
||||
`Incus <https://linuxcontainers.org/incus/>`_ LXC containers.
|
||||
This is meant for development, testing, and CI
|
||||
without requiring a remote server.
|
||||
LXC system containers are lightweight virtual machines
|
||||
that share the host's kernel but run their own init system,
|
||||
package manager, and network stack,
|
||||
so the cmdeploy deployment scripts work pretty much
|
||||
as they would on a real Debian server or cloud VPS.
|
||||
|
||||
Prerequisites
|
||||
-------------
|
||||
|
||||
Install `Incus <https://linuxcontainers.org/incus/>`_
|
||||
(LXC container manager).
|
||||
See the `official installation guide
|
||||
<https://linuxcontainers.org/incus/docs/main/installing/>`_
|
||||
for full details.
|
||||
|
||||
After installing incus, initialise and grant yourself access::
|
||||
|
||||
sudo incus admin init --minimal
|
||||
sudo usermod -aG incus-admin $USER
|
||||
|
||||
|
||||
.. caution::
|
||||
|
||||
Adding yourself to ``incus-admin`` grants effective root access
|
||||
to the host: any member can mount host directories into a container
|
||||
and manipulate them as root.
|
||||
This is fine for local testing of your own relay branches,
|
||||
but do **not** use it for production setups
|
||||
or for testing untrusted relay branches from others.
|
||||
|
||||
.. warning::
|
||||
|
||||
You **must now log out and back in** (or run ``newgrp incus-admin``)
|
||||
after adding yourself to the group.
|
||||
Without this, all ``cmdeploy lxc-*`` commands
|
||||
will fail with permission errors.
|
||||
|
||||
Verify the installation works by running ``incus list``,
|
||||
which should print an empty table without errors.
|
||||
|
||||
|
||||
Quick start
|
||||
-----------
|
||||
|
||||
::
|
||||
|
||||
cd relay
|
||||
scripts/initenv.sh # bootstrap venv
|
||||
source venv/bin/activate # activate venv
|
||||
cmdeploy lxc-test # create containers, deploy, test
|
||||
|
||||
The ``lxc-test`` command provides an automated way
|
||||
to run the full deployment and test pipeline.
|
||||
It executes several ``cmdeploy`` subcommands in sequential steps.
|
||||
If a step fails, you can copy-paste the printed command
|
||||
and run it manually to debug.
|
||||
No host DNS delegation or ``~/.ssh/config`` changes are needed
|
||||
because ``lxc-test`` passes the required SSH and DNS options directly.
|
||||
|
||||
|
||||
CLI reference
|
||||
--------------
|
||||
|
||||
``lxc-start [--ipv4-only] [--run] [NAME ...]``
|
||||
Create and start containers.
|
||||
Without arguments, creates ``test0-localchat`` and ``ns-localchat`` (DNS).
|
||||
Pass one or more ``NAME`` arguments to create user relay containers instead
|
||||
(e.g. ``cmdeploy lxc-start myrelay``).
|
||||
Use ``--ipv4-only`` to set ``disable_ipv6 = True`` in the generated ``chatmail.ini``,
|
||||
producing an IPv4-only relay.
|
||||
Use ``--run`` to automatically run ``cmdeploy run`` on each container after starting it.
|
||||
Generates ``lxconfigs/ssh-config``.
|
||||
It reuses existing containers and resets DNS zones to minimal records.
|
||||
|
||||
``lxc-stop [--destroy] [--destroy-all] [NAME ...]``
|
||||
Stop relay containers.
|
||||
Without arguments, stops ``test0-localchat`` and ``test1-localchat``.
|
||||
Pass ``NAME`` to stop specific containers.
|
||||
Use ``--destroy`` to also delete the containers and their config files.
|
||||
Use ``--destroy-all`` to additionally destroy
|
||||
the ``ns-localchat`` DNS container **and** remove
|
||||
the cached ``localchat-base`` and ``localchat-relay``
|
||||
images, giving a fully clean slate for the next ``lxc-test``.
|
||||
User containers are **never** destroyed unless named explicitly.
|
||||
|
||||
``lxc-test [--one]``
|
||||
By default creates, deploys, and tests both ``test0`` and ``test1``
|
||||
for dual-domain federation testing (sets ``CHATMAIL_DOMAIN2=_test1.localchat``).
|
||||
test0 runs dual-stack (IPv4 + IPv6) while test1 runs IPv4-only (``disable_ipv6 = True``).
|
||||
Pass ``--one`` to only deploy and test against ``test0``
|
||||
(skips ``test1``, does not set ``CHATMAIL_DOMAIN2``).
|
||||
|
||||
``lxc-status``
|
||||
Show live status of all LXC containers (including the DNS container),
|
||||
deploy freshness (comparing ``/etc/chatmail-version``
|
||||
against local ``git rev-parse HEAD`` and ``git diff``),
|
||||
SSH config inclusion, and host DNS forwarding for ``.localchat``.
|
||||
Reports **IN-SYNC**, **DIRTY** (hash matches but uncommitted changes exist),
|
||||
**STALE** (different commit), or **NOT DEPLOYED**.
|
||||
|
||||
|
||||
Container types
|
||||
-----------------
|
||||
|
||||
**Test relay containers** (``test0-localchat``, ``test1-localchat``)
|
||||
Created automatically by ``lxc-test``.
|
||||
**test0** has IPv4 and IPv6 configured,
|
||||
**test1** is IPv4-only (``disable_ipv6 = True``).
|
||||
|
||||
**User relay containers** (``<name>-localchat``)
|
||||
Created by ``cmdeploy lxc-start <name>``
|
||||
where ``<name>`` does not start with ``test``.
|
||||
These are personal development instances,
|
||||
never touched by ``lxc-stop --destroy`` unless named explicitly.
|
||||
|
||||
**DNS container** (``ns-localchat``)
|
||||
Singleton container running PowerDNS.
|
||||
Created automatically when any relay is started.
|
||||
|
||||
|
||||
.. _lxc-ssh-config:
|
||||
|
||||
SSH configuration
|
||||
-----------------
|
||||
|
||||
``cmdeploy lxc-start`` generates ``lxconfigs/ssh-config``,
|
||||
a standard OpenSSH config file mapping every container name,
|
||||
its domain, and a short alias to the container's IP address::
|
||||
|
||||
Host test0-localchat _test0.localchat _test0
|
||||
Hostname 10.204.0.42
|
||||
User root
|
||||
IdentityFile /path/to/relay/lxconfigs/id_localchat
|
||||
IdentitiesOnly yes
|
||||
StrictHostKeyChecking accept-new
|
||||
UserKnownHostsFile /dev/null
|
||||
LogLevel ERROR
|
||||
|
||||
All ``cmdeploy`` commands (``run``, ``dns``, ``status``, ``test``)
|
||||
accept ``--ssh-config lxconfigs/ssh-config`` to use this file.
|
||||
``lxc-test`` passes it automatically.
|
||||
|
||||
**Using containers from the host shell:**
|
||||
|
||||
To make ``ssh _test0`` work from any terminal, add one line to ``~/.ssh/config``::
|
||||
|
||||
Include /absolute/path/to/relay/lxconfigs/ssh-config
|
||||
|
||||
|
||||
.. _lxc-dns-setup:
|
||||
.. _localchat-tld:
|
||||
|
||||
``.localchat`` DNS and name resolution
|
||||
---------------------------------------
|
||||
|
||||
All LXC-managed chatmail domains use the ``.localchat`` pseudo-TLD
|
||||
(e.g. ``_test0.localchat``, ``_test1.localchat``),
|
||||
a non-delegated suffix that exists only within the local PowerDNS infrastructure.
|
||||
A dedicated DNS container (``ns-localchat``)
|
||||
is created so that local test relays interact
|
||||
with DNS similar to a regular public Internet setup.
|
||||
On first start, ``cmdeploy lxc-start`` creates this container
|
||||
running two `PowerDNS <https://www.powerdns.com/>`_ services:
|
||||
|
||||
* **pdns-server** (authoritative) serves ``.localchat``
|
||||
zones from a local SQLite database.
|
||||
|
||||
* **pdns-recursor** (recursive) listens on the Incus
|
||||
bridge so all containers can use it.
|
||||
Forwards ``.localchat`` queries to the local
|
||||
authoritative server and resolves everything else recursively.
|
||||
|
||||
After the DNS container is up, ``lxc-start`` configures the Incus bridge
|
||||
to advertise its IP via DHCP and disables Incus's own DNS.
|
||||
DNS records are then created in two phases matching the "cmdeploy run" deployment flow:
|
||||
|
||||
1. **``lxc-start``** resets each relay zone to
|
||||
**SOA, NS, and A** records (plus **AAAA** for dual-stack containers).
|
||||
If host DNS resolution is configured, users can
|
||||
afterwards run ``cmdeploy run --config lxconfigs/chatmail-test0.ini
|
||||
--ssh-config lxconfigs/ssh-config --ssh-host _test0.localchat``.
|
||||
LXC subcommands do not depend on host DNS resolution
|
||||
and resolve addresses via ``lxconfigs/ssh-config``.
|
||||
|
||||
2. **``cmdeploy dns --zonefile``** generates a standard
|
||||
BIND-format zone file (MX, TXT/SPF, TXT/DMARC,
|
||||
TXT/MTA-STS, SRV, CNAME, DKIM) and loads it
|
||||
into PowerDNS.
|
||||
|
||||
This two-phase approach prevents premature configuration of mail records
|
||||
before the relay is actually deployed and running.
|
||||
Once ``cmdeploy run`` deploys `Unbound <https://nlnetlabs.nl/projects/unbound/>`_
|
||||
inside a relay container, Unbound has a configuration plugin snippet
|
||||
that forwards all ``.localchat`` queries to the PowerDNS recursor,
|
||||
and lets all other queries go through normal recursive resolution.
|
||||
|
||||
|
||||
State outside the repository
|
||||
-----------------------------
|
||||
|
||||
All generated configuration by lxc subcommands live in ``lxconfigs/``
|
||||
(git-ignored), including the SSH key pair (``id_localchat``),
|
||||
per-container ``chatmail-*.ini`` files, zone files, and ``ssh-config``.
|
||||
|
||||
The only state *outside* the repository is the Incus containers and images themselves
|
||||
(managed via the ``incus`` CLI, labelled with ``user.localchat-managed=true``).
|
||||
The Incus image store retains the following snapshot images:
|
||||
|
||||
* ``localchat-base``: Debian 12 with openssh-server and Python (built on first run)
|
||||
|
||||
* ``localchat-relay``: fully deployed relay snapshot,
|
||||
cached after the first successful ``cmdeploy run``.
|
||||
Subsequent relay containers launch from this image
|
||||
so the deploy step is mostly no-ops (roughly 3× faster than a fresh deploy).
|
||||
|
||||
|
||||
.. _lxc-tls:
|
||||
|
||||
TLS handling and underscore domains
|
||||
------------------------------------
|
||||
|
||||
Container domains start with ``_`` (e.g. ``_test0.localchat``).
|
||||
As described in :doc:`getting_started` ("Running a relay with self-signed certificates"),
|
||||
underscore domains automatically use self-signed TLS
|
||||
and ``smtp_tls_security_level = encrypt``.
|
||||
This permits cross-relay federation between LXC containers
|
||||
without any external certificate authority.
|
||||
Delta Chat clients connecting to these relays
|
||||
must be configured with
|
||||
``certificateChecks = acceptInvalidCertificates``
|
||||
(the test fixtures handle this automatically).
|
||||
`PR #7926 on chatmail-core <https://github.com/chatmail/core/pull/7926>`_
|
||||
is meant to make this special setting unnecessary for chatmail clients
|
||||
that are connecting to underscore domains.
|
||||
|
||||
|
||||
Known limitations
|
||||
------------------
|
||||
|
||||
The LXC environment differs from a production
|
||||
deployment in several ways:
|
||||
|
||||
**No ACME / Let's Encrypt**:
|
||||
Self-signed TLS only (see :ref:`lxc-tls`);
|
||||
ACME code paths are never exercised locally.
|
||||
|
||||
**No inbound connections from the internet**:
|
||||
Containers sit on a private Incus bridge and are not port-forwarded.
|
||||
Only the host and other containers on the same bridge can reach them.
|
||||
|
||||
**Local federation only**:
|
||||
Cross-relay mail delivery (e.g. test0 → test1) works between containers on the same host,
|
||||
but these relays are invisible to any external mail server.
|
||||
|
||||
**DNS is local only**:
|
||||
The ``.localchat`` pseudo-TLD is not resolvable from the wider internet
|
||||
(see :ref:`lxc-dns-setup`).
|
||||
|
||||
**IPv6 is ULA-only**:
|
||||
Containers receive IPv6 addresses from the ``fd42:...`` ULA range on the Incus bridge.
|
||||
These are not globally routable, but are sufficient for testing IPv6 service binding
|
||||
(Postfix, Dovecot, Nginx) and DNS AAAA records inside the local environment.
|
||||
test1 runs with ``disable_ipv6 = True`` to exercise the IPv4-only deployment path.
|
||||
+80
-21
@@ -102,8 +102,12 @@ short overview of ``chatmaild`` services:
|
||||
Apple/Google/Huawei.
|
||||
|
||||
- `chatmail-expire <https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/expire.py>`_
|
||||
deletes users if they have not logged in for a longer while.
|
||||
The timeframe can be configured in ``chatmail.ini``.
|
||||
deletes old messages, large messages, and entire mailboxes
|
||||
of users who have not logged in for longer than
|
||||
``delete_inactive_users_after`` days.
|
||||
|
||||
- ``chatmail-quota-expire`` is called by Dovecot's ``quota_warning`` mechanism
|
||||
and will automatically remove oldest messages to keep mailboxes well under ``max_mailbox_size``.
|
||||
|
||||
- `lastlogin <https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/lastlogin.py>`_
|
||||
is contacted by Dovecot when a user logs in and stores the date of
|
||||
@@ -149,13 +153,17 @@ Chatmail relay dependency diagram
|
||||
autoconfig.xml --- dovecot;
|
||||
postfix --- |10080|filtermail-outgoing;
|
||||
postfix --- |10081|filtermail-incoming;
|
||||
postfix --- |10083|filtermail-transport;
|
||||
filtermail-outgoing --- |10025 reinject|postfix;
|
||||
filtermail-incoming --- |10026 reinject|postfix;
|
||||
postfix --- |milter opendkim.sock|OpenDKIM
|
||||
dovecot --- |doveauth.socket|doveauth;
|
||||
dovecot --- |message delivery|maildir["maildir
|
||||
/home/vmail/.../user"];
|
||||
dovecot --- |lastlogin.socket|lastlogin;
|
||||
dovecot --- chatmail-metadata;
|
||||
dovecot --- |quota-warning|chatmail-quota-expire;
|
||||
chatmail-quota-expire --- maildir;
|
||||
lastlogin --- maildir;
|
||||
doveauth --- maildir;
|
||||
chatmail-expire-daily --- maildir;
|
||||
@@ -172,26 +180,51 @@ Chatmail relay dependency diagram
|
||||
style nginx-right fill:#f66;
|
||||
style postfix fill:#f66;
|
||||
style dovecot fill:#f66;
|
||||
style OpenDKIM fill:#f66;
|
||||
style notification-proxy fill:#f66;
|
||||
|
||||
Message between users on the same relay
|
||||
---------------------------------------
|
||||
Accepting and delivering mail
|
||||
-----------------------------
|
||||
|
||||
.. mermaid::
|
||||
:caption: This diagram shows the path a non-federated message takes.
|
||||
:caption: This diagram shows all the paths a message can take.
|
||||
|
||||
graph LR;
|
||||
sender --> |465|smtps/smtpd;
|
||||
sender --> |587|submission/smtpd;
|
||||
smtps/smtpd --> |10080|filtermail;
|
||||
submission/smtpd --> |10080|filtermail;
|
||||
filtermail --> |10025|smtpd_reinject;
|
||||
smtpd_reinject --> cleanup;
|
||||
cleanup --> qmgr;
|
||||
qmgr --> smtpd_accepts_message;
|
||||
qmgr --> |lmtp|dovecot;
|
||||
dovecot --> recipient;
|
||||
dovecot --> sender's_other_devices;
|
||||
flowchart LR
|
||||
subgraph chatmail relay
|
||||
subgraph postfix
|
||||
qmgr .-> lmtp-filtermail["lmtp/lmtp-filtermail (default_transport)"]
|
||||
qmgr .-> lmtp["lmtp (local_transport)"]
|
||||
lmtp --> cleanup["cleanup (lmtp_header_cleanup)"]
|
||||
bounce
|
||||
smtpd-submission["smtpd/submission"]
|
||||
smtpd-smtps["smtpd/smtps"]
|
||||
smtpd-reinject-outgoing["smtpd/reinject-outgoing"] --> authclean["cleanup/authclean (submission_header_cleanup)"]
|
||||
authclean --> qmgr
|
||||
smtpd-smtp["smtpd/smtp"]
|
||||
smtpd-reinject-incoming["smtpd/reinject-incoming"] --> qmgr
|
||||
end
|
||||
lmtp-filtermail --LMTP inet:10083--> filtermail-transport
|
||||
cleanup --LMTP unix:private/dovecot-lmtp --> dovecot
|
||||
dovecot --> maildir
|
||||
smtpd-submission --SMTP inet:10080--> filtermail-outgoing
|
||||
smtpd-smtps --SMTP inet:10080--> filtermail-outgoing
|
||||
filtermail-outgoing --SMTP inet:10025--> smtpd-reinject-outgoing
|
||||
open-dkim["OpenDKIM (signing only)"] <--milter unix:opendkim/opendkim.sock--> smtpd-reinject-outgoing
|
||||
bounce <--milter unix:opendkim/opendkim.sock--> open-dkim
|
||||
bounce --> qmgr
|
||||
nginx
|
||||
smtpd-smtp -.SMTP inet:10081.-> filtermail-incoming
|
||||
nginx -.HTTP inet:10082.-> filtermail-incoming
|
||||
filtermail-incoming --SMTP inet:10026--> smtpd-reinject-incoming
|
||||
end
|
||||
filtermail-transport -.SMTP inet:25.-> mta1[Remote relay]
|
||||
filtermail-transport -.HTTPS /mxdeliv.-> mta1
|
||||
client[Client] -.SMTP inet:587.-> smtpd-submission
|
||||
client -.SMTP inet:465.-> smtpd-smtps
|
||||
client -.SMTP inet:443.-> nginx
|
||||
nginx -.SMTP inet:465.-> smtpd-smtps
|
||||
mta2[Remote relay] -.SMTP inet:25.-> smtpd-smtp
|
||||
mta2 -.HTTPS /mxdeliv.-> nginx
|
||||
|
||||
Operational details of a chatmail relay
|
||||
----------------------------------------
|
||||
@@ -216,6 +249,33 @@ Fresh chatmail addresses have a mailbox directory that contains:
|
||||
directories will typically be empty unless the user of that address
|
||||
hasn’t been online for a while.
|
||||
|
||||
App version information (experimental)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
A chatmail relay ships the
|
||||
`appversions.json <https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/defaults/appversions.json>`_
|
||||
file of the ``chatmaild`` package
|
||||
and serves its content under the IMAP METADATA key
|
||||
``/shared/vendor/deltachat/appversions``.
|
||||
Chat apps installed outside of app stores read this key
|
||||
to learn about updates and where to download them.
|
||||
The mechanism is experimental and may change.
|
||||
|
||||
The file travels with the normal deploy:
|
||||
update the repository checkout and run ``cmdeploy run``.
|
||||
Local modifications of ``appversions.json`` are deployed as-is,
|
||||
so you can serve your own app version information,
|
||||
including links to app downloads.
|
||||
There is no automatic refresh:
|
||||
version information changes only when you deploy again.
|
||||
|
||||
.. note::
|
||||
|
||||
Note that as of August 2026, only Delta Chat Android Google Play version
|
||||
is beginning to support discovering app versions from relays.
|
||||
Generally, consumers of relay-provided app version information
|
||||
need to verify themselves that downloaded app files are valid.
|
||||
|
||||
Active ports
|
||||
~~~~~~~~~~~~
|
||||
|
||||
@@ -258,7 +318,8 @@ from the chatmail relay server.
|
||||
Email domain authentication (DKIM)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Chatmail relays enforce :rfc:`DKIM <6376>` to authenticate incoming emails.
|
||||
Chatmail relays enforce :rfc:`DKIM <6376>` to authenticate incoming emails
|
||||
(except for :ref:`IP-only relays <iponly>`).
|
||||
Incoming emails must have a valid DKIM signature with
|
||||
Signing Domain Identifier (SDID, ``d=`` parameter in the DKIM-Signature
|
||||
header) equal to the ``From:`` header domain. This property is checked
|
||||
@@ -289,9 +350,7 @@ ensured by ``filtermail`` proxy.
|
||||
TLS requirements
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
Postfix is configured to require valid TLS by setting
|
||||
`smtp_tls_security_level <https://www.postfix.org/postconf.5.html#smtp_tls_security_level>`_
|
||||
to ``verify``.
|
||||
Filtermail (used for delivery) requires a valid TLS.
|
||||
|
||||
You can test it by resolving ``MX`` records of your relay domain and
|
||||
then connecting to MX relays (e.g ``mx.example.org``) with
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
Configuring reverse DNS
|
||||
=======================
|
||||
|
||||
Some email servers reject the emails
|
||||
if they don't pass `FCrDNS`_ check, also known as `iprev`_ check.
|
||||
|
||||
.. _FCrDNS: https://en.wikipedia.org/wiki/Forward-confirmed_reverse_DNS
|
||||
.. _iprev: https://datatracker.ietf.org/doc/html/rfc8601#section-3
|
||||
|
||||
Passing the check requires that the IP address that email is sent from
|
||||
should have a ``PTR`` record pointing to the domain name of the server,
|
||||
and domain name record should have an ``A/AAAA`` record
|
||||
pointing to the IP address.
|
||||
|
||||
Modern email relies on DKIM and SPF for authentication,
|
||||
while iprev check exists for
|
||||
`historical reasons <https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-reverse-mapping-considerations-06#section-2.1>`_.
|
||||
Chatmail relays don't resolve ``PTR`` records,
|
||||
so you can ignore this section if configuring ``PTR`` records
|
||||
is difficult and federation with legacy email servers that don't accept
|
||||
valid DKIM signature for authentication is not important.
|
||||
|
||||
Multi-homed setups
|
||||
------------------
|
||||
|
||||
If you have a server with multiple IP addresses,
|
||||
also known as multi-homed setup,
|
||||
and don't publish all IP addresses in DNS,
|
||||
you need to make sure you are using
|
||||
the published address when making outgoing connections.
|
||||
|
||||
For example, your server may have a static IP
|
||||
address, and a so-called Floating IP or Virtual IP
|
||||
that can be moved between servers in case of
|
||||
migration or for failover.
|
||||
By using Floating IP you can avoid downtime
|
||||
and keep the IP address reputation
|
||||
for destinatinons that rely on IP reputation and IP blocklists.
|
||||
In this case you will only publish
|
||||
the Floating IP to DNS and only use the static IP
|
||||
to SSH into the server.
|
||||
|
||||
If you have such setup, make sure that
|
||||
you not only set ``PTR`` records for the Floating IP,
|
||||
but make outgoing connections using the Floating IP.
|
||||
Otherwise reverse DNS check succeed,
|
||||
but forward check making sure your domain name points
|
||||
to the IP address will fail.
|
||||
Such setup is indistinguishable from someone
|
||||
setting IP address ``PTR`` with the domain they don't own
|
||||
and as a result don't succeed.
|
||||
|
||||
On Linux you can configure source IP address with ``ip route`` command,
|
||||
for example:
|
||||
::
|
||||
|
||||
ip route change default via <default-gateway> dev eth0 src <source-address>
|
||||
|
||||
Make sure to persist the change after verifying it is working.
|
||||
You can check what your outgoing IP address is
|
||||
with ``curl icanhazip.com``.
|
||||
Check both the IPv4 and IPv6 addresses.
|
||||
For IPv4 address use ``curl ipv4.icanhazip.com`` or ``curl -4 icanhazip.com``
|
||||
and similarly for IPv6 if you have it.
|
||||
@@ -0,0 +1,291 @@
|
||||
## 0.7.4 - 2026-07-01
|
||||
|
||||
### Features
|
||||
|
||||
- *(logs)* Log incoming mailer-daemon message sources (#191)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- Update filtermail.mtail (#190)
|
||||
|
||||
### Testing
|
||||
|
||||
- Place #[tokio::test] after rstest case macros (#192)
|
||||
## 0.7.3 - 2026-06-27
|
||||
|
||||
### Features
|
||||
|
||||
- *(transport)* Worker eviction (#185)
|
||||
## 0.7.2 - 2026-06-26
|
||||
|
||||
### Documentation
|
||||
|
||||
- *(readme)* Disable colors in mermaid diagrams (#180)
|
||||
- *(readme)* Update Transport mode doc
|
||||
|
||||
### Features
|
||||
|
||||
- *(transport)* Destination worker pool
|
||||
|
||||
### Refactor
|
||||
|
||||
- Implement Display for AddressDomain
|
||||
|
||||
### Testing
|
||||
|
||||
- Test filtermail-transport
|
||||
## 0.7.1 - 2026-06-09
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Switch ratelimiter to MonotonicClock (#167)
|
||||
- *(smtp-server)* Correct error when EOF while reading DATA (#168)
|
||||
- *(resolver)* Disable negative caching (#170)
|
||||
- Ignore CNAME records when resolving TXT records (#177)
|
||||
|
||||
### Features
|
||||
|
||||
- Switch to aws-lc-rs cryptography provider (#178)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- Add filtermail.mtail so filtermail failures can be monitored (#169)
|
||||
## 0.7.0 - 2026-05-26
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Do not crash if accepting new connection fails
|
||||
|
||||
### Documentation
|
||||
|
||||
- *(readme)* Remove docs for options removed in da9a116 (#162)
|
||||
|
||||
### Features
|
||||
|
||||
- [**breaking**] Remove passthrough options that allowed unencrypted mail to pass
|
||||
## 0.6.6 - 2026-05-12
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Return HTTP 200 because madmail expects it, and make sure https is immediately retried when SMTP fails (#153)
|
||||
|
||||
### Features
|
||||
|
||||
- Improved SMTP error responses (#147)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- Remove mac and windows from matrix tests (#154)
|
||||
- Run cmlxc tests in all classic/classic-ipv4/madmail combinations
|
||||
## 0.6.5 - 2026-05-12
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(smtp-client)* Handle 421 on reused connection (#145)
|
||||
- Advertise 8BITMIME to prevent conversion after DKIM signing (#149)
|
||||
- Validate mail data (#150)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- Build and deploy relays with filtermail binary and run interop tests against madmail (#131)
|
||||
|
||||
### Performance
|
||||
|
||||
- *(smtp-client)* Use pipelining if server advertises support (#146)
|
||||
## 0.6.4 - 2026-05-01
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Implement missing "implicit MX" rule
|
||||
|
||||
### Features
|
||||
|
||||
- Https transport channel (#122)
|
||||
|
||||
### Refactor
|
||||
|
||||
- *(transport)* Explicitly handle RFC7505 null MX
|
||||
## 0.6.3 - 2026-04-20
|
||||
|
||||
### Features
|
||||
|
||||
- *(smtp-server)* Log malformed SMTP commands (#118)
|
||||
|
||||
### Performance
|
||||
|
||||
- *(smtp-client)* Cache connections (#117)
|
||||
## 0.6.2 - 2026-04-16
|
||||
|
||||
### Documentation
|
||||
|
||||
- *(readme)* Reformat README.md (#115)
|
||||
|
||||
### Features
|
||||
|
||||
- Log disabled recipients (#113)
|
||||
- *(transport)* Remote delivery over SMTP (#104)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- *(dependabot)* Update configuration (#114)
|
||||
## 0.6.1 - 2026-04-01
|
||||
|
||||
### Documentation
|
||||
|
||||
- *(readme)* Fix typo (#99)
|
||||
|
||||
### Features
|
||||
|
||||
- Configurable hosts for listen and reinject (#84)
|
||||
- *(resolver)* Enable DNSSEC (#94)
|
||||
- Add experimental option to disable mailboxes (#108)
|
||||
|
||||
### Refactor
|
||||
|
||||
- Derive Default for Envelope (#100)
|
||||
## 0.6.0 - 2026-03-13
|
||||
|
||||
### Documentation
|
||||
|
||||
- *(readme)* Fix license link
|
||||
- *(readme)* Clarify licensing of the binaries
|
||||
- *(readme)* Improve README.md (#88)
|
||||
|
||||
### Features
|
||||
|
||||
- [**breaking**] Remove IP verification for domain-literals (#90)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- *(ci)* Fix binary publish job (#79)
|
||||
|
||||
### Refactor
|
||||
|
||||
- Use enum for mode cli arg (#83)
|
||||
## 0.5.2 - 2026-02-27
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(logs)* Log correct address for outbound messages (#70)
|
||||
|
||||
### Features
|
||||
|
||||
- Check incoming email return address (#72)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- Bump cargo-dist (#73)
|
||||
- *(ci)* Build and upload binaries in CI (#76)
|
||||
- *(ci)* Add missing Zig dependency (#77)
|
||||
|
||||
### Refactor
|
||||
|
||||
- Check if email is encrypted before verifying DKIM (#71)
|
||||
## 0.5.1 - 2026-02-24
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(dkim)* Accept TXT records with no `v=` tag (#62)
|
||||
- *(smtp)* Properly handle bounce messages (#63)
|
||||
- *(dkim)* Accept TXT records with escaped quotes (#61)
|
||||
- *(logs)* Log `From` address instead of envelope `MAIL FROM`. (#66)
|
||||
## 0.5.0 - 2026-02-20
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(dkim)* Make simple header canonicalization work properly (#53)
|
||||
|
||||
### Features
|
||||
|
||||
- Save rejected messages to `/tmp` (#55)
|
||||
|
||||
### Refactor
|
||||
|
||||
- Do not copy the mail in memory for DKIM verification (#54)
|
||||
## 0.4.1 - 2026-02-17
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- *(tests)* Add a way to disable DKIM for tests (#50)
|
||||
## 0.4.0 - 2026-02-17
|
||||
|
||||
### Features
|
||||
|
||||
- [**breaking**] DKIM verifier (#35)
|
||||
- Support addresses using domain literals (#42)
|
||||
## 0.3.0 - 2026-02-14
|
||||
|
||||
### Features
|
||||
|
||||
- Support legacy, pre-OpenPGP packet format (#44)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- *(dist)* Switch to musl targets (#31)
|
||||
|
||||
### Refactor
|
||||
|
||||
- Remove unnecessary Arc (#36)
|
||||
- Use a custom, minimal SMTP client instead of lettre (#33)
|
||||
## 0.2.0 - 2026-01-28
|
||||
|
||||
### Features
|
||||
|
||||
- Configurable rate limiter max burst size (#28)
|
||||
|
||||
### Performance
|
||||
|
||||
- Disable Nagle's algorithm and do own buffering on server connections
|
||||
|
||||
### Refactor
|
||||
|
||||
- Remove Mutex around rate limiter
|
||||
## 0.1.2 - 2026-01-22
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Set logs required by grafana to INFO (#21)
|
||||
- Make inbound/outbound log messages consistent (#23)
|
||||
|
||||
### Performance
|
||||
|
||||
- Use governor for rate limiting (#20)
|
||||
## 0.1.1 - 2026-01-21
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Improve address extraction from SMTP commands (#14)
|
||||
- Correct a typo in SMTP answer (#11)
|
||||
- *(config)* Set default values for internal SMTP ports and max message size (#12)
|
||||
|
||||
### Features
|
||||
|
||||
- Improve logging (#13)
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- *(dist)* Configure cargo-dist (#10)
|
||||
- Configure git-cliff
|
||||
|
||||
### Refactor
|
||||
|
||||
- Get rid of indexing and slicing in check_armored_payload() (#15)
|
||||
- Apply more lints (#17)
|
||||
## 0.1.0 - 2026-01-19
|
||||
|
||||
### Documentation
|
||||
|
||||
- *(readme)* Add README.md
|
||||
- *(license)* Add LICENSE
|
||||
|
||||
### Features
|
||||
|
||||
- Initial implementation
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- Init repository
|
||||
- *(dependabot)* Setup dependabot
|
||||
- *(ci)* Setup CI
|
||||
- *(dependabot)* Add github-actions to dependabot
|
||||
- *(cargo)* Add metadata
|
||||
Generated
+3109
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user