mirror of
https://github.com/chatmail/relay.git
synced 2026-05-10 16:04:37 +00:00
Compare commits
1 Commits
docs-inter
...
link2xt/do
| Author | SHA1 | Date | |
|---|---|---|---|
|
d96c9221c4 |
33
.github/ISSUE_TEMPLATE/bug_report.md
vendored
33
.github/ISSUE_TEMPLATE/bug_report.md
vendored
@@ -1,33 +0,0 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Report something that isn't working.
|
||||
title: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
Please fill out as much of this form as you can (leaving out stuff that is not applicable is ok).
|
||||
-->
|
||||
|
||||
- Server OS (Operating System) - preferably Debian 12:
|
||||
- On which OS you run cmdeploy:
|
||||
- chatmail/relay version: `git rev-parse HEAD`
|
||||
|
||||
## Expected behavior
|
||||
|
||||
*What did you try to achieve?*
|
||||
|
||||
## Actual behavior
|
||||
|
||||
*What happened instead?*
|
||||
|
||||
### Steps to reproduce the problem:
|
||||
|
||||
1.
|
||||
2.
|
||||
|
||||
### Screenshots
|
||||
|
||||
### Logs
|
||||
|
||||
1
.github/ISSUE_TEMPLATE/config.yml
vendored
1
.github/ISSUE_TEMPLATE/config.yml
vendored
@@ -1 +0,0 @@
|
||||
blank_issues_enabled: true
|
||||
4
.github/workflows/ci.yaml
vendored
4
.github/workflows/ci.yaml
vendored
@@ -10,10 +10,6 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
# Checkout pull request HEAD commit instead of merge commit
|
||||
# Otherwise `test_deployed_state` will be unhappy.
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
|
||||
- name: run chatmaild tests
|
||||
working-directory: chatmaild
|
||||
|
||||
80
.github/workflows/docs-preview.yaml
vendored
80
.github/workflows/docs-preview.yaml
vendored
@@ -1,80 +0,0 @@
|
||||
name: documentation preview
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- 'doc/**'
|
||||
- 'scripts/build-docs.sh'
|
||||
- '.github/workflows/docs-preview.yaml'
|
||||
|
||||
jobs:
|
||||
scripts:
|
||||
name: build
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: initenv
|
||||
run: scripts/initenv.sh
|
||||
|
||||
- name: append venv/bin to PATH
|
||||
run: echo `pwd`/venv/bin >>$GITHUB_PATH
|
||||
|
||||
- name: build documentation
|
||||
working-directory: doc
|
||||
run: sphinx-build source build
|
||||
|
||||
- name: build documentation second time (for TOC)
|
||||
working-directory: doc
|
||||
run: sphinx-build source build
|
||||
|
||||
- name: Get Pullrequest ID
|
||||
id: prepare
|
||||
run: |
|
||||
export PULLREQUEST_ID=$(echo "${{ github.ref }}" | cut -d "/" -f3)
|
||||
echo "prid=$PULLREQUEST_ID" >> $GITHUB_OUTPUT
|
||||
if [ $(expr length "${{ secrets.USERNAME }}") -gt "1" ]; then echo "uploadtoserver=true" >> $GITHUB_OUTPUT; fi
|
||||
- run: |
|
||||
echo "baseurl: /${{ steps.prepare.outputs.prid }}" >> _config.yml
|
||||
|
||||
- name: Upload preview
|
||||
run: |
|
||||
mkdir -p "$HOME/.ssh"
|
||||
echo "${{ secrets.CHATMAIL_STAGING_SSHKEY }}" > "$HOME/.ssh/key"
|
||||
chmod 600 "$HOME/.ssh/key"
|
||||
rsync -rILvh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/doc/build/ "${{ secrets.USERNAME }}@chatmail.at:/var/www/html/staging.chatmail.at/doc/relay/${{ steps.prepare.outputs.prid }}/"
|
||||
|
||||
- name: "Post links to details"
|
||||
id: details
|
||||
if: steps.prepare.outputs.uploadtoserver
|
||||
run: |
|
||||
# URLs for API connection and uploads
|
||||
export GITHUB_API_URL="https://api.github.com/repos/chatmail/relay/statuses/${{ github.event.after }}"
|
||||
export PREVIEW_LINK="https://staging.chatmail.at/doc/relay/${{ steps.prepare.outputs.prid }}/"
|
||||
export STATUS_DATA="{\"state\": \"success\", \
|
||||
\"description\": \"Preview the changed documentation here:\", \
|
||||
\"context\": \"Documentation Preview\", \
|
||||
\"target_url\": \"${PREVIEW_LINK}\"}"
|
||||
curl -X POST --header "Accept: application/vnd.github+json" --header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" --url "$GITHUB_API_URL" --header "content-type: application/json" --data "$STATUS_DATA"
|
||||
|
||||
#check if comment already exists, if not post it
|
||||
export GITHUB_API_URL="https://api.github.com/repos/chatmail/relay/issues/${{ steps.prepare.outputs.prid }}/comments"
|
||||
export RESPONSE=$(curl -L --header "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" --url "$GITHUB_API_URL" --header "content-type: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28")
|
||||
echo $RESPONSE > response
|
||||
grep -v '"Check out the page preview at https://staging.chatmail.at/doc/relay' response && echo "comment=true" >> $GITHUB_OUTPUT || true
|
||||
- name: "Post link to comments"
|
||||
if: steps.details.outputs.comment
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: "Check out the page preview at https://staging.chatmail.at/doc/relay/${{ steps.prepare.outputs.prid }}/"
|
||||
})
|
||||
|
||||
- name: check links
|
||||
working-directory: doc
|
||||
run: sphinx-build --builder linkcheck source build
|
||||
|
||||
44
.github/workflows/docs.yaml
vendored
44
.github/workflows/docs.yaml
vendored
@@ -1,44 +0,0 @@
|
||||
name: build and upload documentation
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
- 'missytake/docs-ci'
|
||||
paths:
|
||||
- 'doc/**'
|
||||
- 'scripts/build-docs.sh'
|
||||
- '.github/workflows/docs.yaml'
|
||||
|
||||
jobs:
|
||||
scripts:
|
||||
name: build
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: initenv
|
||||
run: scripts/initenv.sh
|
||||
|
||||
- name: append venv/bin to PATH
|
||||
run: echo `pwd`/venv/bin >>$GITHUB_PATH
|
||||
|
||||
- name: build documentation
|
||||
working-directory: doc
|
||||
run: sphinx-build source build
|
||||
|
||||
- name: build documentation second time (for TOC)
|
||||
working-directory: doc
|
||||
run: sphinx-build source build
|
||||
|
||||
- name: check links
|
||||
working-directory: doc
|
||||
run: sphinx-build --builder linkcheck source build
|
||||
|
||||
- name: upload documentation
|
||||
run: |
|
||||
mkdir -p "$HOME/.ssh"
|
||||
echo "${{ secrets.CHATMAIL_STAGING_SSHKEY }}" > "$HOME/.ssh/key"
|
||||
chmod 600 "$HOME/.ssh/key"
|
||||
rsync -rILvh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/doc/build/ "${{ secrets.USERNAME }}@chatmail.at:/var/www/html/chatmail.at/doc/relay/"
|
||||
|
||||
25
.github/workflows/test-and-deploy-ipv4only.yaml
vendored
25
.github/workflows/test-and-deploy-ipv4only.yaml
vendored
@@ -20,7 +20,7 @@ jobs:
|
||||
group: ci-ipv4-${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: ${{ !contains(github.ref, '$GITHUB_REF') }}
|
||||
steps:
|
||||
- uses: jsok/serialize-workflow-action@515cd04c46d7ea7435c4a22a3b4419127afdefe9
|
||||
- uses: jsok/serialize-workflow-action@v1
|
||||
with:
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- uses: actions/checkout@v4
|
||||
@@ -38,9 +38,7 @@ jobs:
|
||||
if [ -f dkimkeys-ipv4/dkimkeys/opendkim.private ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" dkimkeys-ipv4 root@ns.testrun.org:/tmp/ || true; fi
|
||||
if [ "$(ls -A acme-ipv4/acme/certs)" ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" acme-ipv4 root@ns.testrun.org:/tmp/ || true; fi
|
||||
# make sure CAA record isn't set
|
||||
scp -o StrictHostKeyChecking=accept-new .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org nsd-checkzone staging-ipv4.testrun.org /etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh -o StrictHostKeyChecking=accept-new root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org systemctl reload nsd
|
||||
|
||||
- name: rebuild staging-ipv4.testrun.org to have a clean VPS
|
||||
@@ -49,7 +47,7 @@ jobs:
|
||||
-H "Authorization: Bearer ${{ secrets.HETZNER_API_TOKEN }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"image":"debian-12"}' \
|
||||
"https://api.hetzner.cloud/v1/servers/${{ secrets.STAGING_IPV4_SERVER_ID }}/actions/rebuild"
|
||||
"https://api.hetzner.cloud/v1/servers/${{ secrets.STAGING_SERVER_ID }}/actions/rebuild"
|
||||
|
||||
- run: scripts/initenv.sh
|
||||
|
||||
@@ -63,13 +61,16 @@ jobs:
|
||||
while ! ssh -o ConnectTimeout=180 -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org id -u ; do sleep 1 ; done
|
||||
ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org id -u
|
||||
# download acme & dkim state from ns.testrun.org
|
||||
rsync -e "ssh -o StrictHostKeyChecking=accept-new" -avz root@ns.testrun.org:/tmp/acme-ipv4/acme acme-restore || true
|
||||
rsync -avz root@ns.testrun.org:/tmp/dkimkeys-ipv4/dkimkeys dkimkeys-restore || true
|
||||
rsync -e "ssh -o StrictHostKeyChecking=accept-new" -avz root@ns.testrun.org:/tmp/acme-ipv4 acme-restore || true
|
||||
rsync -avz root@ns.testrun.org:/tmp/dkimkeys-ipv4 dkimkeys-restore || true
|
||||
# restore acme & dkim state to staging2.testrun.org
|
||||
rsync -avz acme-restore/acme root@staging-ipv4.testrun.org:/var/lib/ || true
|
||||
rsync -avz dkimkeys-restore/dkimkeys root@staging-ipv4.testrun.org:/etc/ || true
|
||||
rsync -avz acme-restore/acme-ipv4/acme root@staging-ipv4.testrun.org:/var/lib/acme || true
|
||||
rsync -avz dkimkeys-restore/dkimkeys-ipv4/dkimkeys root@staging-ipv4.testrun.org:/etc/dkimkeys || true
|
||||
ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org chown root:root -R /var/lib/acme || true
|
||||
|
||||
- name: run formatting checks
|
||||
run: cmdeploy fmt -v
|
||||
|
||||
- name: run deploy-chatmail offline tests
|
||||
run: pytest --pyargs cmdeploy
|
||||
|
||||
@@ -77,7 +78,7 @@ jobs:
|
||||
cmdeploy init staging-ipv4.testrun.org
|
||||
sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' chatmail.ini
|
||||
|
||||
- run: cmdeploy run --verbose --skip-dns-check
|
||||
- run: cmdeploy run
|
||||
|
||||
- name: set DNS entries
|
||||
run: |
|
||||
@@ -92,6 +93,6 @@ jobs:
|
||||
- name: cmdeploy test
|
||||
run: CHATMAIL_DOMAIN2=nine.testrun.org cmdeploy test --slow
|
||||
|
||||
- name: cmdeploy dns
|
||||
run: cmdeploy dns -v
|
||||
- name: cmdeploy dns (try 3 times)
|
||||
run: cmdeploy dns || cmdeploy dns || cmdeploy dns
|
||||
|
||||
|
||||
19
.github/workflows/test-and-deploy.yaml
vendored
19
.github/workflows/test-and-deploy.yaml
vendored
@@ -20,7 +20,7 @@ jobs:
|
||||
group: ci-${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: ${{ !contains(github.ref, '$GITHUB_REF') }}
|
||||
steps:
|
||||
- uses: jsok/serialize-workflow-action@515cd04c46d7ea7435c4a22a3b4419127afdefe9
|
||||
- uses: jsok/serialize-workflow-action@v1
|
||||
with:
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- uses: actions/checkout@v4
|
||||
@@ -38,9 +38,7 @@ jobs:
|
||||
if [ -f dkimkeys/opendkim.private ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" dkimkeys root@ns.testrun.org:/tmp/ || true; fi
|
||||
if [ "$(ls -A acme/certs)" ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" acme root@ns.testrun.org:/tmp/ || true; fi
|
||||
# make sure CAA record isn't set
|
||||
scp -o StrictHostKeyChecking=accept-new .github/workflows/staging.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging2.testrun.org.zone
|
||||
ssh root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging2.testrun.org.zone
|
||||
ssh root@ns.testrun.org nsd-checkzone staging2.testrun.org /etc/nsd/staging2.testrun.org.zone
|
||||
ssh -o StrictHostKeyChecking=accept-new root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging2.testrun.org.zone
|
||||
ssh root@ns.testrun.org systemctl reload nsd
|
||||
|
||||
- name: rebuild staging2.testrun.org to have a clean VPS
|
||||
@@ -66,16 +64,19 @@ jobs:
|
||||
rsync -e "ssh -o StrictHostKeyChecking=accept-new" -avz root@ns.testrun.org:/tmp/acme acme-restore || true
|
||||
rsync -avz root@ns.testrun.org:/tmp/dkimkeys dkimkeys-restore || true
|
||||
# restore acme & dkim state to staging2.testrun.org
|
||||
rsync -avz acme-restore/acme root@staging2.testrun.org:/var/lib/ || true
|
||||
rsync -avz dkimkeys-restore/dkimkeys root@staging2.testrun.org:/etc/ || true
|
||||
rsync -avz acme-restore/acme/ root@staging2.testrun.org:/var/lib/acme || true
|
||||
rsync -avz dkimkeys-restore/dkimkeys/ root@staging2.testrun.org:/etc/dkimkeys || true
|
||||
ssh -o StrictHostKeyChecking=accept-new -v root@staging2.testrun.org chown root:root -R /var/lib/acme || true
|
||||
|
||||
- name: run formatting checks
|
||||
run: cmdeploy fmt -v
|
||||
|
||||
- name: run deploy-chatmail offline tests
|
||||
run: pytest --pyargs cmdeploy
|
||||
|
||||
- run: cmdeploy init staging2.testrun.org
|
||||
|
||||
- run: cmdeploy run --verbose --skip-dns-check
|
||||
- run: cmdeploy run --verbose
|
||||
|
||||
- name: set DNS entries
|
||||
run: |
|
||||
@@ -90,6 +91,6 @@ jobs:
|
||||
- name: cmdeploy test
|
||||
run: CHATMAIL_DOMAIN2=nine.testrun.org cmdeploy test --slow
|
||||
|
||||
- name: cmdeploy dns
|
||||
run: cmdeploy dns -v
|
||||
- name: cmdeploy dns (try 3 times)
|
||||
run: cmdeploy dns -v || cmdeploy dns -v || cmdeploy dns -v
|
||||
|
||||
|
||||
389
CHANGELOG.md
389
CHANGELOG.md
@@ -2,449 +2,184 @@
|
||||
|
||||
## untagged
|
||||
|
||||
- Organized cmdeploy into install, configure, and activate stages
|
||||
([#695](https://github.com/chatmail/relay/pull/695))
|
||||
|
||||
- docs: move readme.md docs to sphinx documentation rendered at https://chatmail.at/doc/relay
|
||||
([#711](https://github.com/chatmail/relay/pull/711))
|
||||
|
||||
- acmetool: replace cronjob with a systemd timer
|
||||
([#719](https://github.com/chatmail/relay/pull/719))
|
||||
|
||||
- remove xstore@testrun.org from default passthrough recipients
|
||||
([#722](https://github.com/chatmail/relay/pull/722))
|
||||
|
||||
- don't deploy the website if there are merge conflicts in the www folder
|
||||
([#714](https://github.com/chatmail/relay/pull/714))
|
||||
|
||||
- acmetool: use ECDSA keys instead of RSA
|
||||
([#689](https://github.com/chatmail/relay/pull/689))
|
||||
|
||||
- Require TLS 1.2 for outgoing SMTP connections
|
||||
([#685](https://github.com/chatmail/relay/pull/685))
|
||||
|
||||
- require STARTTLS for incoming port 25 connections
|
||||
([#684](https://github.com/chatmail/relay/pull/684))
|
||||
|
||||
- filtermail: run CPU-intensive handle_DATA in a thread pool executor
|
||||
([#676](https://github.com/chatmail/relay/pull/676))
|
||||
|
||||
- don't use the complicated logging module in filtermail to exclude a potential source of errors.
|
||||
([#674](https://github.com/chatmail/relay/pull/674))
|
||||
|
||||
- Specify nginx.conf to only handle `mail_domain`, www, and mta-sts domains
|
||||
([#636](https://github.com/chatmail/relay/pull/636))
|
||||
|
||||
- Setup TURN server
|
||||
([#621](https://github.com/chatmail/relay/pull/621))
|
||||
|
||||
- cmdeploy: make --ssh-host work with localhost
|
||||
([#659](https://github.com/chatmail/relay/pull/659))
|
||||
|
||||
- Update iroh-relay to 0.35.0
|
||||
([#650](https://github.com/chatmail/relay/pull/650))
|
||||
|
||||
- filtermail: accept mails from Protonmail
|
||||
([#616](https://github.com/chatmail/relay/pull/616))
|
||||
|
||||
- Ignore all RCPT TO: parameters
|
||||
([#651](https://github.com/chatmail/relay/pull/651))
|
||||
|
||||
- Increase opendkim DNS Timeout from 5 to 60 seconds
|
||||
([#672](https://github.com/chatmail/relay/pull/672))
|
||||
|
||||
- Add config parameter for Let's Encrypt ACME email
|
||||
([#663](https://github.com/chatmail/relay/pull/663))
|
||||
|
||||
- Use max username length in newemail.py, not min
|
||||
([#648](https://github.com/chatmail/relay/pull/648))
|
||||
|
||||
- Add startup for `fcgiwrap.service` because sometimes it did not start automatically.
|
||||
([#657](https://github.com/chatmail/relay/pull/657))
|
||||
|
||||
- Add `cmdeploy init --force` command for recreating chatmail.ini
|
||||
([#656](https://github.com/chatmail/relay/pull/656))
|
||||
|
||||
- Increase maxproc for reinjecting ports from 10 to 100
|
||||
([#646](https://github.com/chatmail/relay/pull/646))
|
||||
|
||||
- Allow ports 143 and 993 to be used by `dovecot` process
|
||||
([#639](https://github.com/chatmail/relay/pull/639))
|
||||
|
||||
- Add `--skip-dns-check` argument to `cmdeploy run` command, which disables DNS record checking before installation.
|
||||
([#661](https://github.com/chatmail/relay/pull/661))
|
||||
|
||||
- Rework expiry of message files and mailboxes in Python
|
||||
to only do a single iteration over sometimes millions of messages
|
||||
instead of doing "find" commands that iterate 9 times over the messages.
|
||||
Provide an "fsreport" CLI for more fine grained analysis of message files.
|
||||
([#637](https://github.com/chatmail/relay/pull/637))
|
||||
|
||||
|
||||
## 1.7.0 2025-09-11
|
||||
|
||||
- Make www upload path configurable
|
||||
([#618](https://github.com/chatmail/relay/pull/618))
|
||||
|
||||
- Check whether GCC is installed in initenv.sh
|
||||
([#608](https://github.com/chatmail/relay/pull/608))
|
||||
|
||||
- Expire push notification tokens after 90 days
|
||||
([#583](https://github.com/chatmail/relay/pull/583))
|
||||
|
||||
- Use official `mtail` binary instead of `mtail` package
|
||||
([#581](https://github.com/chatmail/relay/pull/581))
|
||||
|
||||
- dovecot: install from download.delta.chat instead of openSUSE Build Service
|
||||
([#590](https://github.com/chatmail/relay/pull/590))
|
||||
|
||||
- Reconfigure Dovecot imap-login service to high-performance mode
|
||||
([#578](https://github.com/chatmail/relay/pull/578))
|
||||
|
||||
- Set timezone to improve dovecot performance
|
||||
([#584](https://github.com/chatmail/relay/pull/584))
|
||||
|
||||
- Increase nginx connection limits
|
||||
([#576](https://github.com/chatmail/relay/pull/576))
|
||||
|
||||
- If `dns-utils` needs to be installed before cmdeploy run, apt update to make sure it works
|
||||
([#560](https://github.com/chatmail/relay/pull/560))
|
||||
|
||||
- filtermail: respect config message size limit
|
||||
([#572](https://github.com/chatmail/relay/pull/572))
|
||||
|
||||
- Don't deploy if one of the ports used for chatmail relay services is occupied by an unexpected process
|
||||
([#568](https://github.com/chatmail/relay/pull/568))
|
||||
|
||||
- Add config value after how many days large files are deleted
|
||||
([#555](https://github.com/chatmail/relay/pull/555))
|
||||
|
||||
- cmdeploy: push relay version to /etc/chatmail-version
|
||||
([#573](https://github.com/chatmail/relay/pull/573))
|
||||
|
||||
- filtermail: allow partial body length in OpenPGP payloads
|
||||
([#570](https://github.com/chatmail/relay/pull/570))
|
||||
|
||||
- chatmaild: allow echobot to receive unencrypted messages by default
|
||||
([#556](https://github.com/chatmail/relay/pull/556))
|
||||
|
||||
|
||||
## 1.6.0 2025-04-11
|
||||
|
||||
- Handle Port-25 connect errors more gracefully (common with VPNs)
|
||||
([#552](https://github.com/chatmail/relay/pull/552))
|
||||
|
||||
- Avoid "acmetool not found" during initial run
|
||||
([#550](https://github.com/chatmail/relay/pull/550))
|
||||
|
||||
- Fix timezone handling such that client/servers do not need to use
|
||||
same timezone.
|
||||
([#553](https://github.com/chatmail/relay/pull/553))
|
||||
|
||||
- Enforce end-to-end encryption for incoming messages.
|
||||
New user address mailboxes now get a `enforceE2EEincoming` file
|
||||
which prohibits incoming cleartext messages from other domains.
|
||||
An outside MTA trying to submit a cleartext message will
|
||||
get a "523 Encryption Needed" response, see RFC5248.
|
||||
If the file does not exist (as it the case for all existing accounts)
|
||||
incoming cleartext messages are accepted.
|
||||
([#538](https://github.com/chatmail/server/pull/538))
|
||||
|
||||
- Enforce end-to-end encryption between local addresses
|
||||
([#535](https://github.com/chatmail/server/pull/535))
|
||||
|
||||
- unbound: check that port 53 is not occupied by a different process
|
||||
([#537](https://github.com/chatmail/server/pull/537))
|
||||
|
||||
- unbound: before unbound is there, use 9.9.9.9 for resolving
|
||||
([#518](https://github.com/chatmail/relay/pull/518))
|
||||
|
||||
- Limit the bind for the HTTPS server on 8443 to 127.0.0.1
|
||||
([#522](https://github.com/chatmail/server/pull/522))
|
||||
([#532](https://github.com/chatmail/server/pull/532))
|
||||
|
||||
- Send SNI when connecting to outside servers
|
||||
([#524](https://github.com/chatmail/server/pull/524))
|
||||
|
||||
- postfix master.cf: use 127.0.0.1 for consistency
|
||||
([#544](https://github.com/chatmail/relay/pull/544))
|
||||
|
||||
- Pass through `original_content` instead of `content` in filtermail
|
||||
([#509](https://github.com/chatmail/server/pull/509))
|
||||
|
||||
- Document TLS requirements in the readme
|
||||
([#514](https://github.com/chatmail/server/pull/514))
|
||||
|
||||
- Remove cleanup service from submission ports
|
||||
([#512](https://github.com/chatmail/server/pull/512))
|
||||
|
||||
- cmdeploy dovecot: delete big messages after 7 days
|
||||
([#504](https://github.com/chatmail/server/pull/504))
|
||||
|
||||
- mtail: fix getting logs from STDIN
|
||||
([#502](https://github.com/chatmail/server/pull/502))
|
||||
|
||||
- filtermail: don't require exactly 2 lines after openPGP payload
|
||||
([#497](https://github.com/chatmail/server/pull/497))
|
||||
|
||||
- cmdeploy dns: offer alternative DKIM record format for some web interfaces
|
||||
([#470](https://github.com/chatmail/server/pull/470))
|
||||
|
||||
- journald: remove old logs from disk
|
||||
([#490](https://github.com/chatmail/server/pull/490))
|
||||
|
||||
- opendkim: restart once every day to mend RAM leaks
|
||||
([#498](https://github.com/chatmail/server/pull/498)
|
||||
|
||||
- migration guide: let opendkim own the DKIM keys directory
|
||||
([#468](https://github.com/chatmail/server/pull/468))
|
||||
|
||||
- improve secure-join message detection
|
||||
([#473](https://github.com/chatmail/server/pull/473))
|
||||
|
||||
- use old crypt lib in python < 3.11
|
||||
([#483](https://github.com/chatmail/server/pull/483))
|
||||
|
||||
- chatmaild: set umask to 0700 for doveauth + metadata
|
||||
([#490](https://github.com/chatmail/server/pull/492))
|
||||
|
||||
- remove MTA-STS daemon
|
||||
([#488](https://github.com/chatmail/server/pull/488))
|
||||
|
||||
- replace `Subject` with `[...]` for all outgoing mails.
|
||||
([#481](https://github.com/chatmail/server/pull/481))
|
||||
|
||||
- opendkim: use su instead of sudo
|
||||
([#491](https://github.com/chatmail/server/pull/491))
|
||||
|
||||
## 1.5.0 2024-12-20
|
||||
|
||||
- cmdeploy dns: always show recommended DNS records
|
||||
([#463](https://github.com/chatmail/server/pull/463))
|
||||
|
||||
- add `--all` to `cmdeploy dns`
|
||||
([#462](https://github.com/chatmail/server/pull/462))
|
||||
|
||||
- fix `_mta-sts` TXT DNS record
|
||||
([#461](https://github.com/chatmail/server/pull/461)
|
||||
|
||||
- deploy `iroh-relay` and also update "realtime relay services" in privacy policy.
|
||||
([#434](https://github.com/chatmail/server/pull/434))
|
||||
([#451](https://github.com/chatmail/server/pull/451))
|
||||
|
||||
- add guide to migrate chatmail to a new server
|
||||
([#429](https://github.com/chatmail/server/pull/429))
|
||||
|
||||
- disable anvil authentication penalty
|
||||
([#414](https://github.com/chatmail/server/pull/444)
|
||||
|
||||
- increase `request_queue_size` for UNIX sockets to 1000.
|
||||
([#437](https://github.com/chatmail/server/pull/437))
|
||||
|
||||
- add argument to `cmdeploy run` for specifying
|
||||
a different SSH host than `mail_domain`
|
||||
([#439](https://github.com/chatmail/server/pull/439))
|
||||
|
||||
- query autoritative nameserver to bypass DNS cache
|
||||
([#424](https://github.com/chatmail/server/pull/424))
|
||||
|
||||
- add mtail support (new optional `mtail_address` ini value)
|
||||
This defines the address on which [`mtail`](https://google.github.io/mtail/)
|
||||
exposes its metrics collected from the logs.
|
||||
If you want to collect the metrics with Prometheus,
|
||||
setup a private network (e.g. WireGuard interface)
|
||||
and assign an IP address from this network to the host.
|
||||
If you do not plan to collect metrics,
|
||||
keep this setting unset.
|
||||
([#388](https://github.com/chatmail/server/pull/388))
|
||||
|
||||
- fix checking for required DNS records
|
||||
([#412](https://github.com/chatmail/server/pull/412))
|
||||
|
||||
- add support for specifying whole domains for recipient passthrough list
|
||||
([#408](https://github.com/chatmail/server/pull/408))
|
||||
([#412](https://github.com/deltachat/chatmail/pull/412))
|
||||
|
||||
- add a paragraph about "account deletion" to info page
|
||||
([#405](https://github.com/chatmail/server/pull/405))
|
||||
([#405](https://github.com/deltachat/chatmail/pull/405))
|
||||
|
||||
- avoid nginx listening on ipv6 if v6 is dsiabled
|
||||
([#402](https://github.com/chatmail/server/pull/402))
|
||||
([#402](https://github.com/deltachat/chatmail/pull/402))
|
||||
|
||||
- refactor ssh-based execution to allow organizing remote functions in
|
||||
modules.
|
||||
([#396](https://github.com/chatmail/server/pull/396))
|
||||
([#396](https://github.com/deltachat/chatmail/pull/396))
|
||||
|
||||
- trigger "apt upgrade" during "cmdeploy run"
|
||||
([#398](https://github.com/chatmail/server/pull/398))
|
||||
([#398](https://github.com/deltachat/chatmail/pull/398))
|
||||
|
||||
- drop hispanilandia passthrough address
|
||||
([#401](https://github.com/chatmail/server/pull/401))
|
||||
([#401](https://github.com/deltachat/chatmail/pull/401))
|
||||
|
||||
- set CAA record flags to 0
|
||||
|
||||
- add IMAP capabilities instead of overwriting them
|
||||
([#413](https://github.com/chatmail/server/pull/413))
|
||||
|
||||
- fix OpenPGP payload check
|
||||
([#435](https://github.com/chatmail/server/pull/435))
|
||||
|
||||
- fix Dovecot quota_max_mail_size to use max_message_size config value
|
||||
([#438](https://github.com/chatmail/server/pull/438))
|
||||
([#413](https://github.com/deltachat/chatmail/pull/413))
|
||||
|
||||
|
||||
## 1.4.1 2024-07-31
|
||||
|
||||
- fix metadata dictproxy which would confuse transactions
|
||||
resulting in missed notifications and other issues.
|
||||
([#393](https://github.com/chatmail/server/pull/393))
|
||||
([#394](https://github.com/chatmail/server/pull/394))
|
||||
([#393](https://github.com/deltachat/chatmail/pull/393))
|
||||
([#394](https://github.com/deltachat/chatmail/pull/394))
|
||||
|
||||
- add optional "imap_rawlog" config option. If true,
|
||||
.in/.out files are created in user home dirs
|
||||
containing the imap protocol messages.
|
||||
([#389](https://github.com/chatmail/server/pull/389))
|
||||
([#389](https://github.com/deltachat/chatmail/pull/389))
|
||||
|
||||
## 1.4.0 2024-07-28
|
||||
|
||||
- Add `disable_ipv6` config option to chatmail.ini.
|
||||
Required if the server doesn't have IPv6 connectivity.
|
||||
([#312](https://github.com/chatmail/server/pull/312))
|
||||
([#312](https://github.com/deltachat/chatmail/pull/312))
|
||||
|
||||
- allow current K9/Thunderbird-mail releases to send encrypted messages
|
||||
outside by accepting their localized "encrypted subject" strings.
|
||||
([#370](https://github.com/chatmail/server/pull/370))
|
||||
([#370](https://github.com/deltachat/chatmail/pull/370))
|
||||
|
||||
- Migrate and remove sqlite database in favor of password/lastlogin tracking
|
||||
in a user's maildir.
|
||||
([#379](https://github.com/chatmail/server/pull/379))
|
||||
([#379](https://github.com/deltachat/chatmail/pull/379))
|
||||
|
||||
- Require pyinfra V3 installed on the client side,
|
||||
run `./scripts/initenv.sh` to upgrade locally.
|
||||
([#378](https://github.com/chatmail/server/pull/378))
|
||||
([#378](https://github.com/deltachat/chatmail/pull/378))
|
||||
|
||||
- don't hardcode "/home/vmail" paths but rather set them
|
||||
once in the config object and use it everywhere else,
|
||||
thereby also improving testability.
|
||||
([#351](https://github.com/chatmail/server/pull/351))
|
||||
([#351](https://github.com/deltachat/chatmail/pull/351))
|
||||
temporarily introduced obligatory "passdb_path" and "mailboxes_dir"
|
||||
settings but they were removed/obsoleted in
|
||||
([#380](https://github.com/chatmail/server/pull/380))
|
||||
([#380](https://github.com/deltachat/chatmail/pull/380))
|
||||
|
||||
- BREAKING: new required chatmail.ini value 'delete_inactive_users_after = 100'
|
||||
which removes users from database and mails after 100 days without any login.
|
||||
([#350](https://github.com/chatmail/server/pull/350))
|
||||
([#350](https://github.com/deltachat/chatmail/pull/350))
|
||||
|
||||
- Refine DNS checking to distinguish between "required" and "recommended" settings
|
||||
([#372](https://github.com/chatmail/server/pull/372))
|
||||
([#372](https://github.com/deltachat/chatmail/pull/372))
|
||||
|
||||
- reload nginx in the acmetool cronjob
|
||||
([#360](https://github.com/chatmail/server/pull/360))
|
||||
([#360](https://github.com/deltachat/chatmail/pull/360))
|
||||
|
||||
- remove checking of reverse-DNS PTR records. Chatmail-servers don't
|
||||
depend on it and even in the wider e-mail system it's not common anymore.
|
||||
If it's an issue, a chatmail operator can still care to properly set reverse DNS.
|
||||
([#348](https://github.com/chatmail/server/pull/348))
|
||||
([#348](https://github.com/deltachat/chatmail/pull/348))
|
||||
|
||||
- Make DNS-checking faster and more interactive, run it fully during "cmdeploy run",
|
||||
also introducing a generic mechanism for rapid remote ssh-based python function execution.
|
||||
([#346](https://github.com/chatmail/server/pull/346))
|
||||
([#346](https://github.com/deltachat/chatmail/pull/346))
|
||||
|
||||
- Don't fix file owner ship of /home/vmail
|
||||
([#345](https://github.com/chatmail/server/pull/345))
|
||||
([#345](https://github.com/deltachat/chatmail/pull/345))
|
||||
|
||||
- Support iterating over all users with doveadm commands
|
||||
([#344](https://github.com/chatmail/server/pull/344))
|
||||
([#344](https://github.com/deltachat/chatmail/pull/344))
|
||||
|
||||
- Test and fix for attempts to create inadmissible accounts
|
||||
([#333](https://github.com/chatmail/server/pull/321))
|
||||
([#333](https://github.com/deltachat/chatmail/pull/321))
|
||||
|
||||
- check that OpenPGP has only PKESK, SKESK and SEIPD packets
|
||||
([#323](https://github.com/chatmail/server/pull/323),
|
||||
[#324](https://github.com/chatmail/server/pull/324))
|
||||
([#323](https://github.com/deltachat/chatmail/pull/323),
|
||||
[#324](https://github.com/deltachat/chatmail/pull/324))
|
||||
|
||||
- improve filtermail checks for encrypted messages and drop support for unencrypted MDNs
|
||||
([#320](https://github.com/chatmail/server/pull/320))
|
||||
([#320](https://github.com/deltachat/chatmail/pull/320))
|
||||
|
||||
- replace `bash` with `/bin/sh`
|
||||
([#334](https://github.com/chatmail/server/pull/334))
|
||||
([#334](https://github.com/deltachat/chatmail/pull/334))
|
||||
|
||||
- Increase number of logged in IMAP sessions to 50000
|
||||
([#335](https://github.com/chatmail/server/pull/335))
|
||||
([#335](https://github.com/deltachat/chatmail/pull/335))
|
||||
|
||||
- filtermail: do not allow ASCII armor without actual payload
|
||||
([#325](https://github.com/chatmail/server/pull/325))
|
||||
([#325](https://github.com/deltachat/chatmail/pull/325))
|
||||
|
||||
- Remove sieve to enable hardlink deduplication in LMTP
|
||||
([#343](https://github.com/chatmail/server/pull/343))
|
||||
([#343](https://github.com/deltachat/chatmail/pull/343))
|
||||
|
||||
- dovecot: enable gzip compression on disk
|
||||
([#341](https://github.com/chatmail/server/pull/341))
|
||||
([#341](https://github.com/deltachat/chatmail/pull/341))
|
||||
|
||||
- DKIM-sign Content-Type and oversign all signed headers
|
||||
([#296](https://github.com/chatmail/server/pull/296))
|
||||
([#296](https://github.com/deltachat/chatmail/pull/296))
|
||||
|
||||
- Add nonci_accounts metric
|
||||
([#347](https://github.com/chatmail/server/pull/347))
|
||||
([#347](https://github.com/deltachat/chatmail/pull/347))
|
||||
|
||||
- doveauth: log when a new account is created
|
||||
([#349](https://github.com/chatmail/server/pull/349))
|
||||
([#349](https://github.com/deltachat/chatmail/pull/349))
|
||||
|
||||
- Multiplex HTTPS, IMAP and SMTP on port 443
|
||||
([#357](https://github.com/chatmail/server/pull/357))
|
||||
([#357](https://github.com/deltachat/chatmail/pull/357))
|
||||
|
||||
## 1.3.0 - 2024-06-06
|
||||
|
||||
- don't check necessary DNS records on cmdeploy init anymore
|
||||
([#316](https://github.com/chatmail/server/pull/316))
|
||||
([#316](https://github.com/deltachat/chatmail/pull/316))
|
||||
|
||||
- ensure cron and acl are installed
|
||||
([#293](https://github.com/chatmail/server/pull/293),
|
||||
[#310](https://github.com/chatmail/server/pull/310))
|
||||
([#293](https://github.com/deltachat/chatmail/pull/293),
|
||||
[#310](https://github.com/deltachat/chatmail/pull/310))
|
||||
|
||||
- change default for delete_mails_after from 40 to 20 days
|
||||
([#300](https://github.com/chatmail/server/pull/300))
|
||||
([#300](https://github.com/deltachat/chatmail/pull/300))
|
||||
|
||||
- save journald logs only to memory and save nginx logs to journald instead of file
|
||||
([#299](https://github.com/chatmail/server/pull/299))
|
||||
([#299](https://github.com/deltachat/chatmail/pull/299))
|
||||
|
||||
- fix writing of multiple obs repositories in `/etc/apt/sources.list`
|
||||
([#290](https://github.com/chatmail/server/pull/290))
|
||||
([#290](https://github.com/deltachat/chatmail/pull/290))
|
||||
|
||||
- metadata: add support for `/shared/vendor/deltachat/irohrelay`
|
||||
([#284](https://github.com/chatmail/server/pull/284))
|
||||
([#284](https://github.com/deltachat/chatmail/pull/284))
|
||||
|
||||
- Emit "XCHATMAIL" capability from IMAP server
|
||||
([#278](https://github.com/chatmail/server/pull/278))
|
||||
([#278](https://github.com/deltachat/chatmail/pull/278))
|
||||
|
||||
- Move echobot `into /var/lib/echobot`
|
||||
([#281](https://github.com/chatmail/server/pull/281))
|
||||
([#281](https://github.com/deltachat/chatmail/pull/281))
|
||||
|
||||
- Accept Let's Encrypt's new Terms of Services
|
||||
([#275](https://github.com/chatmail/server/pull/276))
|
||||
([#275](https://github.com/deltachat/chatmail/pull/276))
|
||||
|
||||
- Reload Dovecot and Postfix when TLS certificate updates
|
||||
([#271](https://github.com/chatmail/server/pull/271))
|
||||
([#271](https://github.com/deltachat/chatmail/pull/271))
|
||||
|
||||
- Use forked version of dovecot without hardcoded delays
|
||||
([#270](https://github.com/chatmail/server/pull/270))
|
||||
([#270](https://github.com/deltachat/chatmail/pull/270))
|
||||
|
||||
## 1.2.0 - 2024-04-04
|
||||
|
||||
- Install dig on the server to resolve DNS records
|
||||
([#267](https://github.com/chatmail/server/pull/267))
|
||||
([#267](https://github.com/deltachat/chatmail/pull/267))
|
||||
|
||||
- preserve notification order and exponentially backoff with
|
||||
retries for tokens where we didn't get a successful return
|
||||
([#265](https://github.com/chatmail/server/pull/263))
|
||||
([#265](https://github.com/deltachat/chatmail/pull/263))
|
||||
|
||||
- Run chatmail-metadata and doveauth as vmail
|
||||
([#261](https://github.com/chatmail/server/pull/261))
|
||||
([#261](https://github.com/deltachat/chatmail/pull/261))
|
||||
|
||||
- Apply systemd restrictions to echobot
|
||||
([#259](https://github.com/chatmail/server/pull/259))
|
||||
([#259](https://github.com/deltachat/chatmail/pull/259))
|
||||
|
||||
- re-enable running the CI in pull requests, but not concurrently
|
||||
([#258](https://github.com/chatmail/server/pull/258))
|
||||
([#258](https://github.com/deltachat/chatmail/pull/258))
|
||||
|
||||
|
||||
## 1.1.0 - 2024-03-28
|
||||
@@ -452,27 +187,27 @@
|
||||
### The changelog starts to record changes from March 15th, 2024
|
||||
|
||||
- Move systemd unit templates to cmdeploy package
|
||||
([#255](https://github.com/chatmail/server/pull/255))
|
||||
([#255](https://github.com/deltachat/chatmail/pull/255))
|
||||
|
||||
- Persist push tokens and support multiple device per address
|
||||
([#254](https://github.com/chatmail/server/pull/254))
|
||||
([#254](https://github.com/deltachat/chatmail/pull/254))
|
||||
|
||||
- Avoid warning for regular doveauth protocol's hello message.
|
||||
([#250](https://github.com/chatmail/server/pull/250))
|
||||
([#250](https://github.com/deltachat/chatmail/pull/250))
|
||||
|
||||
- Fix various tests to pass again with "cmdeploy test".
|
||||
([#245](https://github.com/chatmail/server/pull/245),
|
||||
[#242](https://github.com/chatmail/server/pull/242)
|
||||
([#245](https://github.com/deltachat/chatmail/pull/245),
|
||||
[#242](https://github.com/deltachat/chatmail/pull/242)
|
||||
|
||||
- Ensure lets-encrypt certificates are reloaded after renewal
|
||||
([#244]) https://github.com/chatmail/server/pull/244
|
||||
([#244]) https://github.com/deltachat/chatmail/pull/244
|
||||
|
||||
- Persist tokens to avoid iOS users loosing push-notifications when the
|
||||
chatmail metadata service is restarted (happens regularly during deploys)
|
||||
([#238](https://github.com/chatmail/server/pull/239)
|
||||
([#238](https://github.com/deltachat/chatmail/pull/239)
|
||||
|
||||
- Fix failing sieve-script compile errors on incoming messages
|
||||
([#237](https://github.com/chatmail/server/pull/239)
|
||||
([#237](https://github.com/deltachat/chatmail/pull/239)
|
||||
|
||||
- Fix quota reporting after expunging of old mails
|
||||
([#233](https://github.com/chatmail/server/pull/239)
|
||||
([#233](https://github.com/deltachat/chatmail/pull/239)
|
||||
|
||||
301
README.md
301
README.md
@@ -1,20 +1,299 @@
|
||||
|
||||
# Chatmail relays for end-to-end encrypted email
|
||||
<img width="800px" src="www/src/collage-top.png"/>
|
||||
|
||||
Chatmail relay servers are interoperable Mail Transport Agents (MTAs) designed for:
|
||||
# Chatmail services optimized for Delta Chat apps
|
||||
|
||||
- **Zero State:** no private data or metadata collected, messages are auto-deleted, low disk usage
|
||||
This repository helps to setup a ready-to-use chatmail server
|
||||
comprised of a minimal setup of the battle-tested
|
||||
[postfix smtp](https://www.postfix.org) and [dovecot imap](https://www.dovecot.org) services.
|
||||
|
||||
- **Instant/Realtime:** sub-second message delivery, realtime P2P
|
||||
streaming, privacy-preserving Push Notifications for Apple, Google, and Huawei;
|
||||
The setup is designed and optimized for providing chatmail accounts
|
||||
for use by [Delta Chat apps](https://delta.chat).
|
||||
|
||||
- **Security Enforcement**: only strict TLS, DKIM and OpenPGP with minimized metadata accepted
|
||||
Chatmail accounts are automatically created by a first login,
|
||||
after which the initially specified password is required for using them.
|
||||
|
||||
- **Reliable Federation and Decentralization:** No spam or IP reputation checks, federating
|
||||
depends on established IETF standards and protocols.
|
||||
## Deploying your own chatmail server
|
||||
|
||||
This repository contains everything needed to setup a ready-to-use chatmail relay on an ssh-reachable host.
|
||||
For getting started and more information please refer to the web version of this repositories' documentation at
|
||||
To deploy chatmail on your own server, you must have set-up ssh authentication and need to use an ed25519 key, due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191). You also need to add your private key to the local ssh-agent, because you can't type in your password during deployment.
|
||||
|
||||
[https://chatmail.at/doc/relay](https://chatmail.at/doc/relay)
|
||||
We use `chat.example.org` as the chatmail domain in the following steps.
|
||||
Please substitute it with your own domain.
|
||||
|
||||
1. Install the `cmdeploy` command in a virtualenv
|
||||
|
||||
```
|
||||
git clone https://github.com/deltachat/chatmail
|
||||
cd chatmail
|
||||
scripts/initenv.sh
|
||||
```
|
||||
|
||||
2. Create chatmail configuration file `chatmail.ini`:
|
||||
|
||||
```
|
||||
scripts/cmdeploy init chat.example.org # <-- use your domain
|
||||
```
|
||||
|
||||
3. Point your domain to the server's IP address,
|
||||
if you haven't done so already.
|
||||
Verify that SSH root login works:
|
||||
|
||||
```
|
||||
ssh root@chat.example.org # <-- use your domain
|
||||
```
|
||||
|
||||
4. Deploy to the remote chatmail server:
|
||||
|
||||
```
|
||||
scripts/cmdeploy run
|
||||
```
|
||||
This script will check that you have all necessary DNS records.
|
||||
If DNS records are missing, it will recommend
|
||||
which you should configure at your DNS provider
|
||||
(it can take some time until they are public).
|
||||
|
||||
### Other helpful commands:
|
||||
|
||||
To check the status of your remotely running chatmail service:
|
||||
|
||||
```
|
||||
scripts/cmdeploy status
|
||||
```
|
||||
|
||||
To display and check all recommended DNS records:
|
||||
|
||||
```
|
||||
scripts/cmdeploy dns
|
||||
```
|
||||
|
||||
To test whether your chatmail service is working correctly:
|
||||
|
||||
```
|
||||
scripts/cmdeploy test
|
||||
```
|
||||
|
||||
To measure the performance of your chatmail service:
|
||||
|
||||
```
|
||||
scripts/cmdeploy bench
|
||||
```
|
||||
|
||||
## Overview of this repository
|
||||
|
||||
This repository drives the development of chatmail services,
|
||||
comprised of minimal setups of
|
||||
|
||||
- [postfix smtp server](https://www.postfix.org)
|
||||
- [dovecot imap server](https://www.dovecot.org)
|
||||
|
||||
as well as custom services that are integrated with these two:
|
||||
|
||||
- `chatmaild/src/chatmaild/doveauth.py` implements
|
||||
create-on-login account creation semantics and is used
|
||||
by Dovecot during login authentication and by Postfix
|
||||
which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)
|
||||
to authenticate users
|
||||
to send mails for them.
|
||||
|
||||
- `chatmaild/src/chatmaild/filtermail.py` prevents
|
||||
unencrypted e-mail from leaving the chatmail service
|
||||
and is integrated into postfix's outbound mail pipelines.
|
||||
|
||||
There is also the `cmdeploy/src/cmdeploy/cmdeploy.py` command line tool
|
||||
which helps with setting up and managing the chatmail service.
|
||||
`cmdeploy run` uses [pyinfra-based scripting](https://pyinfra.com/)
|
||||
in `cmdeploy/src/cmdeploy/__init__.py`
|
||||
to automatically install all chatmail components on a server.
|
||||
|
||||
|
||||
### Home page and getting started for users
|
||||
|
||||
`cmdeploy run` also creates default static Web pages and deploys them
|
||||
to a nginx web server with:
|
||||
|
||||
- a default `index.html` along with a QR code that users can click to
|
||||
create accounts on your chatmail provider,
|
||||
|
||||
- a default `info.html` that is linked from the home page,
|
||||
|
||||
- a default `policy.html` that is linked from the home page.
|
||||
|
||||
All `.html` files are generated
|
||||
by the according markdown `.md` file in the `www/src` directory.
|
||||
|
||||
|
||||
### Refining the web pages
|
||||
|
||||
|
||||
```
|
||||
scripts/cmdeploy webdev
|
||||
```
|
||||
|
||||
This starts a local live development cycle for chatmail Web pages:
|
||||
|
||||
- uses the `www/src/page-layout.html` file for producing static
|
||||
HTML pages from `www/src/*.md` files
|
||||
|
||||
- continously builds the web presence reading files from `www/src` directory
|
||||
and generating html files and copying assets to the `www/build` directory.
|
||||
|
||||
- Starts a browser window automatically where you can "refresh" as needed.
|
||||
|
||||
|
||||
## Emergency Commands to disable automatic account creation
|
||||
|
||||
If you need to stop account creation,
|
||||
e.g. because some script is wildly creating accounts,
|
||||
login to the server with ssh and run:
|
||||
|
||||
```
|
||||
touch /etc/chatmail-nocreate
|
||||
```
|
||||
|
||||
While this file is present, account creation will be blocked.
|
||||
|
||||
### Ports
|
||||
|
||||
[Postfix](http://www.postfix.org/) listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
|
||||
[Dovecot](https://www.dovecot.org/) listens on ports 143 (imap) and 993 (imaps).
|
||||
[nginx](https://www.nginx.com/) listens on port 8443 (https-alt) and 443 (https).
|
||||
Port 443 multiplexes HTTPS, IMAP and SMTP using ALPN to redirect connections to ports 8443, 465 or 993.
|
||||
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (http).
|
||||
|
||||
Delta Chat apps will, however, discover all ports and configurations
|
||||
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail service.
|
||||
|
||||
## Email authentication
|
||||
|
||||
chatmail servers rely on [DKIM](https://www.rfc-editor.org/rfc/rfc6376)
|
||||
to authenticate incoming emails.
|
||||
Incoming emails must have a valid DKIM signature with
|
||||
Signing Domain Identifier (SDID, `d=` parameter in the DKIM-Signature header)
|
||||
equal to the `From:` header domain.
|
||||
This property is checked by OpenDKIM screen policy script
|
||||
before validating the signatures.
|
||||
This correpsonds to strict [DMARC](https://www.rfc-editor.org/rfc/rfc7489) alignment (`adkim=s`),
|
||||
but chatmail does not rely on DMARC and does not consult the sender policy published in DMARC records.
|
||||
Other legacy authentication mechanisms such as [iprev](https://www.rfc-editor.org/rfc/rfc8601#section-2.7.3)
|
||||
and [SPF](https://www.rfc-editor.org/rfc/rfc7208) are also not taken into account.
|
||||
If there is no valid DKIM signature on the incoming email,
|
||||
the sender receives a "5.7.1 No valid DKIM signature found" error.
|
||||
|
||||
Outgoing emails must be sent over authenticated connection
|
||||
with envelope MAIL FROM (return path) corresponding to the login.
|
||||
This is ensured by Postfix which maps login username
|
||||
to MAIL FROM with
|
||||
[`smtpd_sender_login_maps`](https://www.postfix.org/postconf.5.html#smtpd_sender_login_maps)
|
||||
and rejects incorrectly authenticated emails with [`reject_sender_login_mismatch`](reject_sender_login_mismatch) policy.
|
||||
`From:` header must correspond to envelope MAIL FROM,
|
||||
this is ensured by `filtermail` proxy.
|
||||
|
||||
## Migrating chatmail server to a new host
|
||||
|
||||
If you want to migrate your chatmail server to a new host,
|
||||
follow these steps:
|
||||
|
||||
1. Block all ports except 80 and 22 with firewall on a new server.
|
||||
|
||||
To do this, add the following config to `/etc/nftables.conf`:
|
||||
```
|
||||
#!/usr/sbin/nft -f
|
||||
|
||||
flush ruleset
|
||||
|
||||
table inet filter {
|
||||
chain input {
|
||||
type filter hook input priority filter; policy drop;
|
||||
|
||||
# Accept ICMP.
|
||||
# It is especially important to accept ICMPv6 ND messages,
|
||||
# otherwise IPv6 connectivity breaks.
|
||||
icmp type { echo-request } accept
|
||||
icmpv6 type { echo-request, nd-neighbor-solicit, nd-router-advert, nd-neighbor-advert } accept
|
||||
|
||||
tcp dport { ssh, http } accept
|
||||
|
||||
ct state established accept
|
||||
}
|
||||
chain forward {
|
||||
type filter hook forward priority filter;
|
||||
}
|
||||
chain output {
|
||||
type filter hook output priority filter;
|
||||
}
|
||||
}
|
||||
```
|
||||
Then execute `nft -f /etc/nftables.conf` as root.
|
||||
|
||||
This will ensure users will not connect to the new server
|
||||
and mails will not be delivered to the new server
|
||||
before you finish the setup.
|
||||
|
||||
Port 22 is needed for SSH access
|
||||
and port 80 is needed to get a TLS certificate.
|
||||
They are not used by Delta Chat
|
||||
or by other email servers trying to deliver the messages.
|
||||
|
||||
2. Point DNS to the new IP addresses.
|
||||
|
||||
You can already remove the old IP addresses from DNS.
|
||||
Existing Delta Chat users will still be able to connect
|
||||
to the old server, send and receive messages,
|
||||
but new users will fail to create new profiles
|
||||
with your chatmail server.
|
||||
|
||||
3. Setup the new server with `cmdeploy`.
|
||||
|
||||
This step is similar to initial setup.
|
||||
However, because ports Delta Chat uses are blocked,
|
||||
new server will not become usable immediately.
|
||||
If other servers try to deliver messages to your new server they will fail,
|
||||
but normally email servers will retry delivering messages
|
||||
for at least a week, so messages will not be lost.
|
||||
|
||||
4. Firewall all ports except `ssh` (22) on the old server.
|
||||
Existing users will not be able to connect from now on
|
||||
and no more messages will be delivered to your old chatmail server.
|
||||
|
||||
Blocking users from connecting to the new server
|
||||
until mailboxes are migrated is needed to avoid UID validity change.
|
||||
If Delta Chat connects to the new server before it is fully set up,
|
||||
it will lose track of the IMAP message UID
|
||||
and miss messages that arrived during migration.
|
||||
|
||||
Same for SMTP port 25, you want it blocked during migration so no new mails arrive
|
||||
while the server is moving.
|
||||
|
||||
5. Use `rsync -avz` over SSH to copy /home/vmail/mail from the old server to the new one
|
||||
preserving file permissions and timestamps.
|
||||
|
||||
6. Unblock ports used by Delta Chat and SMTP message exchange.
|
||||
For that you can modify `/etc/nftables.conf` as follows:
|
||||
```
|
||||
#!/usr/sbin/nft -f
|
||||
|
||||
flush ruleset
|
||||
|
||||
table inet filter {
|
||||
chain input {
|
||||
type filter hook input priority filter; policy drop;
|
||||
|
||||
# Accept ICMP.
|
||||
# It is especially important to accept ICMPv6 ND messages,
|
||||
# otherwise IPv6 connectivity breaks.
|
||||
icmp type { echo-request } accept
|
||||
icmpv6 type { echo-request, nd-neighbor-solicit, nd-router-advert, nd-neighbor-advert } accept
|
||||
|
||||
tcp dport { ssh, smtp, http, https, imap, imaps, submission, submissions } accept
|
||||
|
||||
ct state established accept
|
||||
}
|
||||
chain forward {
|
||||
type filter hook forward priority filter;
|
||||
}
|
||||
chain output {
|
||||
type filter hook output priority filter;
|
||||
}
|
||||
}
|
||||
```
|
||||
Execute `nft -f /etc/nftables.conf` as root to apply the changes.
|
||||
|
||||
@@ -12,7 +12,6 @@ dependencies = [
|
||||
"deltachat-rpc-client",
|
||||
"filelock",
|
||||
"requests",
|
||||
"crypt-r >= 3.13.1 ; python_version >= '3.11'",
|
||||
]
|
||||
|
||||
[tool.setuptools]
|
||||
@@ -27,10 +26,8 @@ chatmail-metadata = "chatmaild.metadata:main"
|
||||
filtermail = "chatmaild.filtermail:main"
|
||||
echobot = "chatmaild.echo:main"
|
||||
chatmail-metrics = "chatmaild.metrics:main"
|
||||
chatmail-expire = "chatmaild.expire:main"
|
||||
chatmail-fsreport = "chatmaild.fsreport:main"
|
||||
delete_inactive_users = "chatmaild.delete_inactive_users:main"
|
||||
lastlogin = "chatmaild.lastlogin:main"
|
||||
turnserver = "chatmaild.turnserver:main"
|
||||
|
||||
[project.entry-points.pytest11]
|
||||
"chatmaild.testplugin" = "chatmaild.tests.plugin"
|
||||
@@ -50,9 +47,6 @@ lint.select = [
|
||||
"PLE", # Pylint Error
|
||||
"PLW", # Pylint Warning
|
||||
]
|
||||
lint.ignore = [
|
||||
"PLC0415" # import-outside-top-level
|
||||
]
|
||||
|
||||
[tool.tox]
|
||||
legacy_tox_ini = """
|
||||
@@ -72,6 +66,5 @@ commands =
|
||||
[testenv]
|
||||
deps = pytest
|
||||
pdbpp
|
||||
pytest-localserver
|
||||
commands = pytest -v -rsXx {posargs}
|
||||
"""
|
||||
|
||||
59
chatmaild/src/chatmaild/common_encrypted_subjects.py
Normal file
59
chatmaild/src/chatmaild/common_encrypted_subjects.py
Normal file
@@ -0,0 +1,59 @@
|
||||
"""Generated from deltachat, draft-ietf-lamps-header-protection, and
|
||||
encrypted_subject localizations in
|
||||
https://github.com/thunderbird/thunderbird-android/
|
||||
|
||||
"""
|
||||
|
||||
common_encrypted_subjects = {
|
||||
"...",
|
||||
"[...]",
|
||||
"암호화된 메시지",
|
||||
"Ĉifrita mesaĝo",
|
||||
"Courriel chiffré",
|
||||
"Dulrituð skilaboð",
|
||||
"Encrypted Message",
|
||||
"Fersifere berjocht",
|
||||
"Kemennadenn enrineget",
|
||||
"Krüptitud kiri",
|
||||
"Krypterat meddelande",
|
||||
"Krypteret besked",
|
||||
"Kryptert melding",
|
||||
"Mensagem criptografada",
|
||||
"Mensagem encriptada",
|
||||
"Mensaje cifrado",
|
||||
"Mensaxe cifrada",
|
||||
"Mesaj Criptat",
|
||||
"Mesazh i Fshehtëzuar",
|
||||
"Messaggio criptato",
|
||||
"Messaghju cifratu",
|
||||
"Missatge encriptat",
|
||||
"Neges wedi'i Hamgryptio",
|
||||
"Pesan terenkripsi",
|
||||
"Salattu viesti",
|
||||
"Şifreli İleti",
|
||||
"Šifrēta ziņa",
|
||||
"Šifrirana poruka",
|
||||
"Šifrirano sporočilo",
|
||||
"Šifruotas laiškas",
|
||||
"Tin nhắn được mã hóa",
|
||||
"Titkosított üzenet",
|
||||
"Verschlüsselte Nachricht",
|
||||
"Versleuteld bericht",
|
||||
"Zašifrovaná zpráva",
|
||||
"Zaszyfrowana wiadomość",
|
||||
"Zifratu mezua",
|
||||
"Κρυπτογραφημένο μήνυμα",
|
||||
"Зашифроване повідомлення",
|
||||
"Зашифрованное сообщение",
|
||||
"Зашыфраваны ліст",
|
||||
"Криптирано съобщение",
|
||||
"Шифрована порука",
|
||||
"დაშიფრული წერილი",
|
||||
"הודעה מוצפנת",
|
||||
"پیام رمزنگاریشده",
|
||||
"رسالة مشفّرة",
|
||||
"എൻക്രിപ്റ്റുചെയ്ത സന്ദേശം",
|
||||
"加密邮件",
|
||||
"已加密的訊息",
|
||||
"暗号化されたメッセージ",
|
||||
}
|
||||
@@ -11,11 +11,7 @@ def read_config(inipath):
|
||||
assert Path(inipath).exists(), inipath
|
||||
cfg = iniconfig.IniConfig(inipath)
|
||||
params = cfg.sections["params"]
|
||||
default_config_content = get_default_config_content(params["mail_domain"])
|
||||
df_params = iniconfig.IniConfig("ini", data=default_config_content)["params"]
|
||||
new_params = dict(df_params.items())
|
||||
new_params.update(params)
|
||||
return Config(inipath, params=new_params)
|
||||
return Config(inipath, params=params)
|
||||
|
||||
|
||||
class Config:
|
||||
@@ -26,32 +22,17 @@ class Config:
|
||||
self.max_mailbox_size = params["max_mailbox_size"]
|
||||
self.max_message_size = int(params.get("max_message_size", "31457280"))
|
||||
self.delete_mails_after = params["delete_mails_after"]
|
||||
self.delete_large_after = params["delete_large_after"]
|
||||
self.delete_inactive_users_after = int(params["delete_inactive_users_after"])
|
||||
self.username_min_length = int(params["username_min_length"])
|
||||
self.username_max_length = int(params["username_max_length"])
|
||||
self.password_min_length = int(params["password_min_length"])
|
||||
self.passthrough_senders = params["passthrough_senders"].split()
|
||||
self.passthrough_recipients = params["passthrough_recipients"].split()
|
||||
self.www_folder = params.get("www_folder", "")
|
||||
self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
|
||||
self.filtermail_smtp_port_incoming = int(
|
||||
params["filtermail_smtp_port_incoming"]
|
||||
)
|
||||
self.postfix_reinject_port = int(params["postfix_reinject_port"])
|
||||
self.postfix_reinject_port_incoming = int(
|
||||
params["postfix_reinject_port_incoming"]
|
||||
)
|
||||
self.mtail_address = params.get("mtail_address")
|
||||
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
|
||||
self.acme_email = params.get("acme_email", "")
|
||||
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
|
||||
if "iroh_relay" not in params:
|
||||
self.iroh_relay = "https://" + params["mail_domain"]
|
||||
self.enable_iroh_relay = True
|
||||
else:
|
||||
self.iroh_relay = params["iroh_relay"].strip()
|
||||
self.enable_iroh_relay = False
|
||||
self.iroh_relay = params.get("iroh_relay")
|
||||
self.privacy_postal = params.get("privacy_postal")
|
||||
self.privacy_mail = params.get("privacy_mail")
|
||||
self.privacy_pdo = params.get("privacy_pdo")
|
||||
@@ -67,7 +48,7 @@ class Config:
|
||||
def _getbytefile(self):
|
||||
return open(self._inipath, "rb")
|
||||
|
||||
def get_user(self, addr) -> User:
|
||||
def get_user(self, addr):
|
||||
if not addr or "@" not in addr or "/" in addr:
|
||||
raise ValueError(f"invalid address {addr!r}")
|
||||
|
||||
@@ -82,11 +63,6 @@ class Config:
|
||||
|
||||
def write_initial_config(inipath, mail_domain, overrides):
|
||||
"""Write out default config file, using the specified config value overrides."""
|
||||
content = get_default_config_content(mail_domain, **overrides)
|
||||
inipath.write_text(content)
|
||||
|
||||
|
||||
def get_default_config_content(mail_domain, **overrides):
|
||||
from importlib.resources import files
|
||||
|
||||
inidir = files(__package__).joinpath("ini")
|
||||
@@ -118,7 +94,7 @@ def get_default_config_content(mail_domain, **overrides):
|
||||
lines = []
|
||||
for line in content.split("\n"):
|
||||
for key, value in privacy.items():
|
||||
value_lines = value.format(mail_domain=mail_domain).strip().split("\n")
|
||||
value_lines = value.strip().split("\n")
|
||||
if not line.startswith(f"{key} =") or not value_lines:
|
||||
continue
|
||||
if len(value_lines) == 1:
|
||||
@@ -131,4 +107,5 @@ def get_default_config_content(mail_domain, **overrides):
|
||||
else:
|
||||
lines.append(line)
|
||||
content = "\n".join(lines)
|
||||
return content
|
||||
|
||||
inipath.write_text(content)
|
||||
|
||||
31
chatmaild/src/chatmaild/delete_inactive_users.py
Normal file
31
chatmaild/src/chatmaild/delete_inactive_users.py
Normal file
@@ -0,0 +1,31 @@
|
||||
"""
|
||||
Remove inactive users
|
||||
"""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
|
||||
from .config import read_config
|
||||
|
||||
|
||||
def delete_inactive_users(config):
|
||||
cutoff_date = time.time() - config.delete_inactive_users_after * 86400
|
||||
for addr in os.listdir(config.mailboxes_dir):
|
||||
try:
|
||||
user = config.get_user(addr)
|
||||
except ValueError:
|
||||
continue
|
||||
|
||||
read_timestamp = user.get_last_login_timestamp()
|
||||
if read_timestamp and read_timestamp < cutoff_date:
|
||||
path = config.mailboxes_dir.joinpath(addr)
|
||||
assert path == user.maildir
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
|
||||
|
||||
def main():
|
||||
(cfgpath,) = sys.argv[1:]
|
||||
config = read_config(cfgpath)
|
||||
delete_inactive_users(config)
|
||||
@@ -87,12 +87,8 @@ class DictProxy:
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
|
||||
with CustomThreadingUnixStreamServer(socket, Handler) as server:
|
||||
with ThreadingUnixStreamServer(socket, Handler) as server:
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
|
||||
|
||||
class CustomThreadingUnixStreamServer(ThreadingUnixStreamServer):
|
||||
request_queue_size = 1000
|
||||
|
||||
@@ -1,13 +1,9 @@
|
||||
import crypt
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
|
||||
try:
|
||||
import crypt_r
|
||||
except ImportError:
|
||||
import crypt as crypt_r
|
||||
|
||||
from .config import Config, read_config
|
||||
from .dictproxy import DictProxy
|
||||
from .migrate_db import migrate_from_db_to_maildir
|
||||
@@ -17,7 +13,7 @@ NOCREATE_FILE = "/etc/chatmail-nocreate"
|
||||
|
||||
def encrypt_password(password: str):
|
||||
# https://doc.dovecot.org/configuration_manual/authentication/password_schemes/
|
||||
passhash = crypt_r.crypt(password, crypt_r.METHOD_SHA512)
|
||||
passhash = crypt.crypt(password, crypt.METHOD_SHA512)
|
||||
return "{SHA512-CRYPT}" + passhash
|
||||
|
||||
|
||||
|
||||
@@ -8,7 +8,6 @@ import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from deltachat_rpc_client import Bot, DeltaChat, EventType, Rpc, events
|
||||
|
||||
@@ -98,10 +97,6 @@ def main():
|
||||
if not bot.is_configured():
|
||||
bot.configure(addr, password)
|
||||
|
||||
# write invite link to working directory
|
||||
invitelink = bot.account.get_qr_code()
|
||||
Path("invite-link.txt").write_text(invitelink)
|
||||
|
||||
bot.run_forever()
|
||||
|
||||
|
||||
|
||||
@@ -1,218 +0,0 @@
|
||||
"""
|
||||
Expire old messages and addresses.
|
||||
|
||||
"""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
from argparse import ArgumentParser
|
||||
from collections import namedtuple
|
||||
from datetime import datetime
|
||||
from stat import S_ISREG
|
||||
|
||||
from chatmaild.config import read_config
|
||||
|
||||
FileEntry = namedtuple("FileEntry", ("relpath", "mtime", "size"))
|
||||
|
||||
|
||||
def iter_mailboxes(basedir, maxnum):
|
||||
if not os.path.exists(basedir):
|
||||
print_info(f"no mailboxes found at: {basedir}")
|
||||
return
|
||||
|
||||
for name in os_listdir_if_exists(basedir)[:maxnum]:
|
||||
if "@" in name:
|
||||
yield MailboxStat(basedir + "/" + name)
|
||||
|
||||
|
||||
def get_file_entry(path):
|
||||
"""return a FileEntry or None if the path does not exist or is not a regular file."""
|
||||
try:
|
||||
st = os.stat(path)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
if not S_ISREG(st.st_mode):
|
||||
return None
|
||||
return FileEntry(path, st.st_mtime, st.st_size)
|
||||
|
||||
|
||||
def os_listdir_if_exists(path):
|
||||
"""return a list of names obtained from os.listdir or an empty list if the path does not exist."""
|
||||
try:
|
||||
return os.listdir(path)
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
|
||||
|
||||
class MailboxStat:
|
||||
last_login = None
|
||||
|
||||
def __init__(self, basedir):
|
||||
self.basedir = str(basedir)
|
||||
# all detected messages in cur/new/tmp folders
|
||||
self.messages = []
|
||||
|
||||
# all detected files in mailbox top dir
|
||||
self.extrafiles = []
|
||||
|
||||
# scan all relevant files (without recursion)
|
||||
old_cwd = os.getcwd()
|
||||
try:
|
||||
os.chdir(self.basedir)
|
||||
except FileNotFoundError:
|
||||
return
|
||||
for name in os_listdir_if_exists("."):
|
||||
if name in ("cur", "new", "tmp"):
|
||||
for msg_name in os_listdir_if_exists(name):
|
||||
entry = get_file_entry(f"{name}/{msg_name}")
|
||||
if entry is not None:
|
||||
self.messages.append(entry)
|
||||
|
||||
else:
|
||||
entry = get_file_entry(name)
|
||||
if entry is not None:
|
||||
self.extrafiles.append(entry)
|
||||
if name == "password":
|
||||
self.last_login = entry.mtime
|
||||
self.extrafiles.sort(key=lambda x: -x.size)
|
||||
os.chdir(old_cwd)
|
||||
|
||||
|
||||
def print_info(msg):
|
||||
print(msg, file=sys.stderr)
|
||||
|
||||
|
||||
class Expiry:
|
||||
def __init__(self, config, dry, now, verbose):
|
||||
self.config = config
|
||||
self.dry = dry
|
||||
self.now = now
|
||||
self.verbose = verbose
|
||||
self.del_mboxes = 0
|
||||
self.all_mboxes = 0
|
||||
self.del_files = 0
|
||||
self.all_files = 0
|
||||
self.start = time.time()
|
||||
|
||||
def remove_mailbox(self, mboxdir):
|
||||
if self.verbose:
|
||||
print_info(f"removing {mboxdir}")
|
||||
if not self.dry:
|
||||
shutil.rmtree(mboxdir)
|
||||
self.del_mboxes += 1
|
||||
|
||||
def remove_file(self, path, mtime=None):
|
||||
if self.verbose:
|
||||
if mtime is not None:
|
||||
date = datetime.fromtimestamp(mtime).strftime("%b %d")
|
||||
print_info(f"removing {date} {path}")
|
||||
else:
|
||||
print_info(f"removing {path}")
|
||||
if not self.dry:
|
||||
try:
|
||||
os.unlink(path)
|
||||
except FileNotFoundError:
|
||||
print_info(f"file not found/vanished {path}")
|
||||
self.del_files += 1
|
||||
|
||||
def process_mailbox_stat(self, mbox):
|
||||
cutoff_without_login = (
|
||||
self.now - int(self.config.delete_inactive_users_after) * 86400
|
||||
)
|
||||
cutoff_mails = self.now - int(self.config.delete_mails_after) * 86400
|
||||
cutoff_large_mails = self.now - int(self.config.delete_large_after) * 86400
|
||||
|
||||
self.all_mboxes += 1
|
||||
changed = False
|
||||
if mbox.last_login and mbox.last_login < cutoff_without_login:
|
||||
self.remove_mailbox(mbox.basedir)
|
||||
return
|
||||
|
||||
# all to-be-removed files are relative to the mailbox basedir
|
||||
try:
|
||||
os.chdir(mbox.basedir)
|
||||
except FileNotFoundError:
|
||||
print_info(f"mailbox not found/vanished {mbox.basedir}")
|
||||
return
|
||||
|
||||
mboxname = os.path.basename(mbox.basedir)
|
||||
if self.verbose:
|
||||
date = datetime.fromtimestamp(mbox.last_login) if mbox.last_login else None
|
||||
if date:
|
||||
print_info(f"checking mailbox {date.strftime('%b %d')} {mboxname}")
|
||||
else:
|
||||
print_info(f"checking mailbox (no last_login) {mboxname}")
|
||||
self.all_files += len(mbox.messages)
|
||||
for message in mbox.messages:
|
||||
if message.mtime < cutoff_mails:
|
||||
self.remove_file(message.relpath, mtime=message.mtime)
|
||||
elif message.size > 200000 and message.mtime < cutoff_large_mails:
|
||||
# we only remove noticed large files (not unnoticed ones in new/)
|
||||
if message.relpath.startswith("cur/"):
|
||||
self.remove_file(message.relpath, mtime=message.mtime)
|
||||
else:
|
||||
continue
|
||||
changed = True
|
||||
if changed:
|
||||
self.remove_file("maildirsize")
|
||||
|
||||
def get_summary(self):
|
||||
return (
|
||||
f"Removed {self.del_mboxes} out of {self.all_mboxes} mailboxes "
|
||||
f"and {self.del_files} out of {self.all_files} files in existing mailboxes "
|
||||
f"in {time.time() - self.start:2.2f} seconds"
|
||||
)
|
||||
|
||||
|
||||
def main(args=None):
|
||||
"""Expire mailboxes and messages according to chatmail config"""
|
||||
parser = ArgumentParser(description=main.__doc__)
|
||||
ini = "/usr/local/lib/chatmaild/chatmail.ini"
|
||||
parser.add_argument(
|
||||
"chatmail_ini",
|
||||
action="store",
|
||||
nargs="?",
|
||||
help=f"path pointing to chatmail.ini file, default: {ini}",
|
||||
default=ini,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--days", action="store", help="assume date to be days older than now"
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--maxnum",
|
||||
default=None,
|
||||
action="store",
|
||||
help="maximum number of mailboxes to iterate on",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-v",
|
||||
dest="verbose",
|
||||
action="store_true",
|
||||
help="print out removed files and mailboxes",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--remove",
|
||||
dest="remove",
|
||||
action="store_true",
|
||||
help="actually remove all expired files and dirs",
|
||||
)
|
||||
args = parser.parse_args(args)
|
||||
|
||||
config = read_config(args.chatmail_ini)
|
||||
now = datetime.utcnow().timestamp()
|
||||
if args.days:
|
||||
now = now - 86400 * int(args.days)
|
||||
|
||||
maxnum = int(args.maxnum) if args.maxnum else None
|
||||
exp = Expiry(config, dry=not args.remove, now=now, verbose=args.verbose)
|
||||
for mailbox in iter_mailboxes(str(config.mailboxes_dir), maxnum=maxnum):
|
||||
exp.process_mailbox_stat(mailbox)
|
||||
print(exp.get_summary())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv[1:])
|
||||
@@ -2,6 +2,7 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import binascii
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from email import policy
|
||||
@@ -10,12 +11,10 @@ from email.utils import parseaddr
|
||||
from smtplib import SMTP as SMTPClient
|
||||
|
||||
from aiosmtpd.controller import Controller
|
||||
from aiosmtpd.smtp import SMTP
|
||||
|
||||
from .common_encrypted_subjects import common_encrypted_subjects
|
||||
from .config import read_config
|
||||
|
||||
ENCRYPTION_NEEDED_523 = "523 Encryption Needed: Invalid Unencrypted Mail"
|
||||
|
||||
|
||||
def check_openpgp_payload(payload: bytes):
|
||||
"""Checks the OpenPGP payload.
|
||||
@@ -37,12 +36,6 @@ def check_openpgp_payload(payload: bytes):
|
||||
|
||||
packet_type_id = payload[i] & 0x3F
|
||||
i += 1
|
||||
|
||||
while payload[i] >= 224 and payload[i] < 255:
|
||||
# Partial body length.
|
||||
partial_length = 1 << (payload[i] & 0x1F)
|
||||
i += 1 + partial_length
|
||||
|
||||
if payload[i] < 192:
|
||||
# One-octet length.
|
||||
body_len = payload[i]
|
||||
@@ -61,17 +54,16 @@ def check_openpgp_payload(payload: bytes):
|
||||
)
|
||||
i += 5
|
||||
else:
|
||||
# Impossible, partial body length was processed above.
|
||||
# Partial body length is not allowed.
|
||||
return False
|
||||
|
||||
i += body_len
|
||||
|
||||
if i == len(payload):
|
||||
# Last packet should be
|
||||
# Symmetrically Encrypted and Integrity Protected Data Packet (SEIPD)
|
||||
#
|
||||
# This is the only place where this function may return `True`.
|
||||
return packet_type_id == 18
|
||||
if packet_type_id == 18:
|
||||
# Last packet should be
|
||||
# Symmetrically Encrypted and Integrity Protected Data Packet (SEIPD)
|
||||
return True
|
||||
elif packet_type_id not in [1, 3]:
|
||||
# All packets except the last one must be either
|
||||
# Public-Key Encrypted Session Key Packet (PKESK)
|
||||
@@ -79,38 +71,26 @@ def check_openpgp_payload(payload: bytes):
|
||||
# Symmetric-Key Encrypted Session Key Packet (SKESK)
|
||||
return False
|
||||
|
||||
return False
|
||||
if i == 0:
|
||||
return False
|
||||
|
||||
if i > len(payload):
|
||||
# Payload is truncated.
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def check_armored_payload(payload: str, outgoing: bool):
|
||||
"""Check the armored PGP message for invalid content.
|
||||
|
||||
:param payload: the armored PGP message
|
||||
:param outgoing: whether the message is outgoing or incoming
|
||||
:return: whether the message is a valid PGP message
|
||||
"""
|
||||
prefix = "-----BEGIN PGP MESSAGE-----\r\n"
|
||||
def check_armored_payload(payload: str):
|
||||
prefix = "-----BEGIN PGP MESSAGE-----\r\n\r\n"
|
||||
if not payload.startswith(prefix):
|
||||
return False
|
||||
payload = payload.removeprefix(prefix)
|
||||
|
||||
while payload.endswith("\r\n"):
|
||||
payload = payload.removesuffix("\r\n")
|
||||
suffix = "-----END PGP MESSAGE-----"
|
||||
suffix = "-----END PGP MESSAGE-----\r\n\r\n"
|
||||
if not payload.endswith(suffix):
|
||||
return False
|
||||
payload = payload.removesuffix(suffix)
|
||||
|
||||
version_comment = "Version: "
|
||||
if payload.startswith(version_comment):
|
||||
if outgoing: # Disallow comments in outgoing messages
|
||||
return False
|
||||
# Remove comments from incoming messages
|
||||
payload = payload.partition("\r\n")[2]
|
||||
|
||||
while payload.startswith("\r\n"):
|
||||
payload = payload.removeprefix("\r\n")
|
||||
|
||||
# Remove CRC24.
|
||||
payload = payload.rpartition("=")[0]
|
||||
|
||||
@@ -125,34 +105,15 @@ def check_armored_payload(payload: str, outgoing: bool):
|
||||
return False
|
||||
|
||||
|
||||
def is_securejoin(message):
|
||||
if message.get("secure-join") not in ["vc-request", "vg-request"]:
|
||||
return False
|
||||
if not message.is_multipart():
|
||||
return False
|
||||
parts_count = 0
|
||||
for part in message.iter_parts():
|
||||
parts_count += 1
|
||||
if parts_count > 1:
|
||||
return False
|
||||
if part.is_multipart():
|
||||
return False
|
||||
if part.get_content_type() != "text/plain":
|
||||
return False
|
||||
|
||||
payload = part.get_payload().strip().lower()
|
||||
if payload not in ("secure-join: vc-request", "secure-join: vg-request"):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def check_encrypted(message, outgoing=True):
|
||||
def check_encrypted(message):
|
||||
"""Check that the message is an OpenPGP-encrypted message.
|
||||
|
||||
MIME structure of the message must correspond to <https://www.rfc-editor.org/rfc/rfc3156>.
|
||||
"""
|
||||
if not message.is_multipart():
|
||||
return False
|
||||
if message.get("subject") not in common_encrypted_subjects:
|
||||
return False
|
||||
if message.get_content_type() != "multipart/encrypted":
|
||||
return False
|
||||
parts_count = 0
|
||||
@@ -173,7 +134,7 @@ def check_encrypted(message, outgoing=True):
|
||||
if part.get_content_type() != "application/octet-stream":
|
||||
return False
|
||||
|
||||
if not check_armored_payload(part.get_payload(), outgoing=outgoing):
|
||||
if not check_armored_payload(part.get_payload()):
|
||||
return False
|
||||
else:
|
||||
return False
|
||||
@@ -181,53 +142,18 @@ def check_encrypted(message, outgoing=True):
|
||||
return True
|
||||
|
||||
|
||||
async def asyncmain_beforequeue(config, mode):
|
||||
if mode == "outgoing":
|
||||
port = config.filtermail_smtp_port
|
||||
handler = OutgoingBeforeQueueHandler(config)
|
||||
else:
|
||||
port = config.filtermail_smtp_port_incoming
|
||||
handler = IncomingBeforeQueueHandler(config)
|
||||
HackedController(
|
||||
handler,
|
||||
hostname="127.0.0.1",
|
||||
port=port,
|
||||
data_size_limit=config.max_message_size,
|
||||
).start()
|
||||
async def asyncmain_beforequeue(config):
|
||||
port = config.filtermail_smtp_port
|
||||
Controller(BeforeQueueHandler(config), hostname="127.0.0.1", port=port).start()
|
||||
|
||||
|
||||
def recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
for addr in passthrough_recipients:
|
||||
if recipient == addr:
|
||||
return True
|
||||
if addr[0] == "@" and recipient.endswith(addr):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
class HackedController(Controller):
|
||||
def factory(self):
|
||||
return SMTPDiscardRCPTO_options(self.handler, **self.SMTP_kwargs)
|
||||
|
||||
|
||||
class SMTPDiscardRCPTO_options(SMTP):
|
||||
def _getparams(self, params):
|
||||
# Ignore RCPT TO parameters.
|
||||
#
|
||||
# Otherwise parameters such as `ORCPT=...`
|
||||
# or `NOTIFY=DELAY,FAILURE` (generated by Stalwart)
|
||||
# make aiosmtpd reject the message here:
|
||||
# <https://github.com/aio-libs/aiosmtpd/blob/98f578389ae86e5345cc343fa4e5a17b21d9c96d/aiosmtpd/smtp.py#L1379-L1384>
|
||||
return {}
|
||||
|
||||
|
||||
class OutgoingBeforeQueueHandler:
|
||||
class BeforeQueueHandler:
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
self.send_rate_limiter = SendRateLimiter()
|
||||
|
||||
async def handle_MAIL(self, server, session, envelope, address, mail_options):
|
||||
log_info(f"handle_MAIL from {address}")
|
||||
logging.info(f"handle_MAIL from {address}")
|
||||
envelope.mail_from = address
|
||||
max_sent = self.config.max_user_send_per_minute
|
||||
if not self.send_rate_limiter.is_sending_allowed(address, max_sent):
|
||||
@@ -240,113 +166,51 @@ class OutgoingBeforeQueueHandler:
|
||||
return "250 OK"
|
||||
|
||||
async def handle_DATA(self, server, session, envelope):
|
||||
loop = asyncio.get_running_loop()
|
||||
return await loop.run_in_executor(None, self.sync_handle_DATA, envelope)
|
||||
|
||||
def sync_handle_DATA(self, envelope):
|
||||
log_info("handle_DATA before-queue")
|
||||
logging.info("handle_DATA before-queue")
|
||||
error = self.check_DATA(envelope)
|
||||
if error:
|
||||
return error
|
||||
log_info("re-injecting the mail that passed checks")
|
||||
logging.info("re-injecting the mail that passed checks")
|
||||
client = SMTPClient("localhost", self.config.postfix_reinject_port)
|
||||
client.sendmail(
|
||||
envelope.mail_from, envelope.rcpt_tos, envelope.original_content
|
||||
)
|
||||
client.sendmail(envelope.mail_from, envelope.rcpt_tos, envelope.content)
|
||||
return "250 OK"
|
||||
|
||||
def check_DATA(self, envelope):
|
||||
"""the central filtering function for e-mails."""
|
||||
log_info(f"Processing DATA message from {envelope.mail_from}")
|
||||
logging.info(f"Processing DATA message from {envelope.mail_from}")
|
||||
|
||||
message = BytesParser(policy=policy.default).parsebytes(envelope.content)
|
||||
mail_encrypted = check_encrypted(message, outgoing=True)
|
||||
mail_encrypted = check_encrypted(message)
|
||||
|
||||
_, from_addr = parseaddr(message.get("from").strip())
|
||||
|
||||
logging.info(f"mime-from: {from_addr} envelope-from: {envelope.mail_from!r}")
|
||||
if envelope.mail_from.lower() != from_addr.lower():
|
||||
return f"500 Invalid FROM <{from_addr!r}> for <{envelope.mail_from!r}>"
|
||||
|
||||
if mail_encrypted or is_securejoin(message):
|
||||
print("Outgoing: Filtering encrypted mail.", file=sys.stderr)
|
||||
return
|
||||
|
||||
print("Outgoing: Filtering unencrypted mail.", file=sys.stderr)
|
||||
|
||||
if envelope.mail_from in self.config.passthrough_senders:
|
||||
return
|
||||
|
||||
# allow self-sent Autocrypt Setup Message
|
||||
if envelope.rcpt_tos == [from_addr]:
|
||||
if message.get("subject") == "Autocrypt Setup Message":
|
||||
if message.get_content_type() == "multipart/mixed":
|
||||
return
|
||||
|
||||
passthrough_recipients = self.config.passthrough_recipients
|
||||
|
||||
envelope_from_domain = from_addr.split("@").pop()
|
||||
for recipient in envelope.rcpt_tos:
|
||||
if recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
if envelope.mail_from == recipient:
|
||||
# Always allow sending emails to self.
|
||||
continue
|
||||
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return ENCRYPTION_NEEDED_523
|
||||
|
||||
|
||||
class IncomingBeforeQueueHandler:
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
|
||||
async def handle_DATA(self, server, session, envelope):
|
||||
loop = asyncio.get_running_loop()
|
||||
return await loop.run_in_executor(None, self.sync_handle_DATA, envelope)
|
||||
|
||||
def sync_handle_DATA(self, envelope):
|
||||
log_info("handle_DATA before-queue")
|
||||
error = self.check_DATA(envelope)
|
||||
if error:
|
||||
return error
|
||||
log_info("re-injecting the mail that passed checks")
|
||||
|
||||
# the smtp daemon on reinject_port_incoming gives it to dkim milter
|
||||
# which looks at source address to determine whether to verify or sign
|
||||
client = SMTPClient(
|
||||
"localhost",
|
||||
self.config.postfix_reinject_port_incoming,
|
||||
source_address=("127.0.0.2", 0),
|
||||
)
|
||||
client.sendmail(
|
||||
envelope.mail_from, envelope.rcpt_tos, envelope.original_content
|
||||
)
|
||||
return "250 OK"
|
||||
|
||||
def check_DATA(self, envelope):
|
||||
"""the central filtering function for e-mails."""
|
||||
log_info(f"Processing DATA message from {envelope.mail_from}")
|
||||
|
||||
message = BytesParser(policy=policy.default).parsebytes(envelope.content)
|
||||
mail_encrypted = check_encrypted(message, outgoing=False)
|
||||
|
||||
if mail_encrypted or is_securejoin(message):
|
||||
print("Incoming: Filtering encrypted mail.", file=sys.stderr)
|
||||
return
|
||||
|
||||
print("Incoming: Filtering unencrypted mail.", file=sys.stderr)
|
||||
|
||||
# we want cleartext mailer-daemon messages to pass through
|
||||
# chatmail core will typically not display them as normal messages
|
||||
if message.get("auto-submitted"):
|
||||
_, from_addr = parseaddr(message.get("from").strip())
|
||||
if from_addr.lower().startswith("mailer-daemon@"):
|
||||
if message.get_content_type() == "multipart/report":
|
||||
return
|
||||
|
||||
for recipient in envelope.rcpt_tos:
|
||||
user = self.config.get_user(recipient)
|
||||
if user is None or user.is_incoming_cleartext_ok():
|
||||
if recipient in passthrough_recipients:
|
||||
continue
|
||||
res = recipient.split("@")
|
||||
if len(res) != 2:
|
||||
return f"500 Invalid address <{recipient}>"
|
||||
_recipient_addr, recipient_domain = res
|
||||
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return ENCRYPTION_NEEDED_523
|
||||
is_outgoing = recipient_domain != envelope_from_domain
|
||||
if is_outgoing and not mail_encrypted:
|
||||
is_securejoin = message.get("secure-join") in [
|
||||
"vc-request",
|
||||
"vg-request",
|
||||
]
|
||||
if not is_securejoin:
|
||||
return f"500 Invalid unencrypted mail to <{recipient}>"
|
||||
|
||||
|
||||
class SendRateLimiter:
|
||||
@@ -363,19 +227,13 @@ class SendRateLimiter:
|
||||
return False
|
||||
|
||||
|
||||
def log_info(msg):
|
||||
print(msg, file=sys.stderr)
|
||||
|
||||
|
||||
def main():
|
||||
args = sys.argv[1:]
|
||||
assert len(args) == 2
|
||||
assert len(args) == 1
|
||||
config = read_config(args[0])
|
||||
mode = args[1]
|
||||
logging.basicConfig(level=logging.WARN)
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
assert mode in ["incoming", "outgoing"]
|
||||
task = asyncmain_beforequeue(config, mode)
|
||||
task = asyncmain_beforequeue(config)
|
||||
loop.create_task(task)
|
||||
log_info("entering serving loop")
|
||||
loop.run_forever()
|
||||
|
||||
@@ -1,168 +0,0 @@
|
||||
"""
|
||||
command line tool to analyze mailbox message storage
|
||||
|
||||
example invocation:
|
||||
|
||||
python -m chatmaild.fsreport /path/to/chatmail.ini
|
||||
|
||||
to show storage summaries for all "cur" folders
|
||||
|
||||
python -m chatmaild.fsreport /path/to/chatmail.ini --mdir cur
|
||||
|
||||
to show storage summaries only for first 1000 mailboxes
|
||||
|
||||
python -m chatmaild.fsreport /path/to/chatmail.ini --maxnum 1000
|
||||
|
||||
"""
|
||||
|
||||
import os
|
||||
from argparse import ArgumentParser
|
||||
from datetime import datetime
|
||||
|
||||
from chatmaild.config import read_config
|
||||
from chatmaild.expire import iter_mailboxes
|
||||
|
||||
DAYSECONDS = 24 * 60 * 60
|
||||
MONTHSECONDS = DAYSECONDS * 30
|
||||
|
||||
|
||||
def HSize(size: int):
|
||||
"""Format a size integer as a Human-readable string Kilobyte, Megabyte or Gigabyte"""
|
||||
if size < 10000:
|
||||
return f"{size / 1000:5.2f}K"
|
||||
if size < 1000 * 1000:
|
||||
return f"{size / 1000:5.0f}K"
|
||||
if size < 1000 * 1000 * 1000:
|
||||
return f"{int(size / 1000000):5.0f}M"
|
||||
return f"{size / 1000000000:5.2f}G"
|
||||
|
||||
|
||||
class Report:
|
||||
def __init__(self, now, min_login_age, mdir):
|
||||
self.size_extra = 0
|
||||
self.size_messages = 0
|
||||
self.now = now
|
||||
self.min_login_age = min_login_age
|
||||
self.mdir = mdir
|
||||
|
||||
self.num_ci_logins = self.num_all_logins = 0
|
||||
self.login_buckets = {x: 0 for x in (1, 10, 30, 40, 80, 100, 150)}
|
||||
|
||||
self.message_buckets = {x: 0 for x in (0, 160000, 500000, 2000000)}
|
||||
|
||||
def process_mailbox_stat(self, mailbox):
|
||||
# categorize login times
|
||||
last_login = mailbox.last_login
|
||||
if last_login:
|
||||
self.num_all_logins += 1
|
||||
if os.path.basename(mailbox.basedir)[:3] == "ci-":
|
||||
self.num_ci_logins += 1
|
||||
else:
|
||||
for days in self.login_buckets:
|
||||
if last_login >= self.now - days * DAYSECONDS:
|
||||
self.login_buckets[days] += 1
|
||||
|
||||
cutoff_login_date = self.now - self.min_login_age * DAYSECONDS
|
||||
if last_login and last_login <= cutoff_login_date:
|
||||
# categorize message sizes
|
||||
for size in self.message_buckets:
|
||||
for msg in mailbox.messages:
|
||||
if msg.size >= size:
|
||||
if self.mdir and not msg.relpath.startswith(self.mdir):
|
||||
continue
|
||||
self.message_buckets[size] += msg.size
|
||||
|
||||
self.size_messages += sum(entry.size for entry in mailbox.messages)
|
||||
self.size_extra += sum(entry.size for entry in mailbox.extrafiles)
|
||||
|
||||
def dump_summary(self):
|
||||
all_messages = self.size_messages
|
||||
print()
|
||||
print("## Mailbox storage use analysis")
|
||||
print(f"Mailbox data total size: {HSize(self.size_extra + all_messages)}")
|
||||
print(f"Messages total size : {HSize(all_messages)}")
|
||||
try:
|
||||
percent = self.size_extra / (self.size_extra + all_messages) * 100
|
||||
except ZeroDivisionError:
|
||||
percent = 100
|
||||
print(f"Extra files : {HSize(self.size_extra)} ({percent:.2f}%)")
|
||||
|
||||
print()
|
||||
if self.min_login_age:
|
||||
print(f"### Message storage for {self.min_login_age} days old logins")
|
||||
|
||||
pref = f"[{self.mdir}] " if self.mdir else ""
|
||||
for minsize, sumsize in self.message_buckets.items():
|
||||
percent = (sumsize / all_messages * 100) if all_messages else 0
|
||||
print(
|
||||
f"{pref}larger than {HSize(minsize)}: {HSize(sumsize)} ({percent:.2f}%)"
|
||||
)
|
||||
|
||||
user_logins = self.num_all_logins - self.num_ci_logins
|
||||
|
||||
def p(num):
|
||||
return f"({num / user_logins * 100:2.2f}%)" if user_logins else "100%"
|
||||
|
||||
print()
|
||||
print(f"## Login stats, from date reference {datetime.fromtimestamp(self.now)}")
|
||||
print(f"all: {HSize(self.num_all_logins)}")
|
||||
print(f"non-ci: {HSize(user_logins)}")
|
||||
print(f"ci: {HSize(self.num_ci_logins)}")
|
||||
for days, active in self.login_buckets.items():
|
||||
print(f"last {days:3} days: {HSize(active)} {p(active)}")
|
||||
|
||||
|
||||
def main(args=None):
|
||||
"""Report about filesystem storage usage of all mailboxes and messages"""
|
||||
parser = ArgumentParser(description=main.__doc__)
|
||||
ini = "/usr/local/lib/chatmaild/chatmail.ini"
|
||||
parser.add_argument(
|
||||
"chatmail_ini",
|
||||
action="store",
|
||||
nargs="?",
|
||||
help=f"path pointing to chatmail.ini file, default: {ini}",
|
||||
default=ini,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--days",
|
||||
default=0,
|
||||
action="store",
|
||||
help="assume date to be days older than now",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--min-login-age",
|
||||
default=0,
|
||||
dest="min_login_age",
|
||||
action="store",
|
||||
help="only sum up message size if last login is at least min-login-age days old",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--mdir",
|
||||
action="store",
|
||||
help="only consider 'cur' or 'new' or 'tmp' messages for summary",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--maxnum",
|
||||
default=None,
|
||||
action="store",
|
||||
help="maximum number of mailboxes to iterate on",
|
||||
)
|
||||
|
||||
args = parser.parse_args(args)
|
||||
|
||||
config = read_config(args.chatmail_ini)
|
||||
|
||||
now = datetime.utcnow().timestamp()
|
||||
if args.days:
|
||||
now = now - 86400 * int(args.days)
|
||||
|
||||
maxnum = int(args.maxnum) if args.maxnum else None
|
||||
rep = Report(now=now, min_login_age=int(args.min_login_age), mdir=args.mdir)
|
||||
for mbox in iter_mailboxes(str(config.mailboxes_dir), maxnum=maxnum):
|
||||
rep.process_mailbox_stat(mbox)
|
||||
rep.dump_summary()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -23,9 +23,6 @@ max_message_size = 31457280
|
||||
# days after which mails are unconditionally deleted
|
||||
delete_mails_after = 20
|
||||
|
||||
# days after which large messages (>200k) are unconditionally deleted
|
||||
delete_large_after = 7
|
||||
|
||||
# days after which users without a successful login are deleted (database and mails)
|
||||
delete_inactive_users_after = 90
|
||||
|
||||
@@ -42,53 +39,22 @@ password_min_length = 9
|
||||
passthrough_senders =
|
||||
|
||||
# list of e-mail recipients for which to accept outbound un-encrypted mails
|
||||
# (space-separated, item may start with "@" to whitelist whole recipient domains)
|
||||
passthrough_recipients = echo@{mail_domain}
|
||||
|
||||
# path to www directory - documented here: https://github.com/chatmail/relay/#custom-web-pages
|
||||
#www_folder = www
|
||||
# (space-separated)
|
||||
passthrough_recipients = xstore@testrun.org
|
||||
|
||||
#
|
||||
# Deployment Details
|
||||
#
|
||||
|
||||
# SMTP outgoing filtermail and reinjection
|
||||
# where the filtermail SMTP service listens
|
||||
filtermail_smtp_port = 10080
|
||||
postfix_reinject_port = 10025
|
||||
|
||||
# SMTP incoming filtermail and reinjection
|
||||
filtermail_smtp_port_incoming = 10081
|
||||
postfix_reinject_port_incoming = 10026
|
||||
# postfix accepts on the localhost reinject SMTP port
|
||||
postfix_reinject_port = 10025
|
||||
|
||||
# if set to "True" IPv6 is disabled
|
||||
disable_ipv6 = False
|
||||
|
||||
# Your email adress, which will be used in acmetool to manage Let's Encrypt SSL certificates
|
||||
acme_email =
|
||||
|
||||
# Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail
|
||||
# service.
|
||||
# If you set it to anything else, the service will be disabled
|
||||
# and users will be directed to use the given iroh relay URL.
|
||||
# Set it to empty string if you want users to use their default iroh relay.
|
||||
# iroh_relay =
|
||||
|
||||
# Address on which `mtail` listens,
|
||||
# e.g. 127.0.0.1 or some private network
|
||||
# address like 192.168.10.1.
|
||||
# You can point Prometheus
|
||||
# or some other OpenMetrics-compatible
|
||||
# collector to
|
||||
# http://{{mtail_address}}:3903/metrics
|
||||
# and display collected metrics with Grafana.
|
||||
#
|
||||
# WARNING: do not expose this service
|
||||
# to the public IP address.
|
||||
#
|
||||
# `mtail is not running if the setting is not set.
|
||||
|
||||
# mtail_address = 127.0.0.1
|
||||
|
||||
#
|
||||
# Debugging options
|
||||
#
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
|
||||
[privacy]
|
||||
|
||||
passthrough_recipients = privacy@testrun.org echo@{mail_domain}
|
||||
passthrough_recipients = privacy@testrun.org xstore@testrun.org
|
||||
|
||||
privacy_postal =
|
||||
Merlinux GmbH, Represented by the managing director H. Krekel,
|
||||
|
||||
@@ -1,24 +1,14 @@
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from contextlib import contextmanager
|
||||
|
||||
from .config import read_config
|
||||
from .dictproxy import DictProxy
|
||||
from .filedict import FileDict
|
||||
from .notifier import Notifier
|
||||
from .turnserver import turn_credentials
|
||||
|
||||
|
||||
def _is_valid_token_timestamp(timestamp, now):
|
||||
# Token if invalid after 90 days
|
||||
# or if the timestamp is in the future.
|
||||
return timestamp > now - 3600 * 24 * 90 and timestamp < now + 60
|
||||
|
||||
|
||||
class Metadata:
|
||||
# each SETMETADATA on this key appends to dictionary
|
||||
# mapping of unique device tokens
|
||||
# each SETMETADATA on this key appends to a list of unique device tokens
|
||||
# which only ever get removed if the upstream indicates the token is invalid
|
||||
DEVICETOKEN_KEY = "devicetoken"
|
||||
|
||||
@@ -28,60 +18,29 @@ class Metadata:
|
||||
def get_metadata_dict(self, addr):
|
||||
return FileDict(self.vmail_dir / addr / "metadata.json")
|
||||
|
||||
@contextmanager
|
||||
def _modify_tokens(self, addr):
|
||||
with self.get_metadata_dict(addr).modify() as data:
|
||||
tokens = data.setdefault(self.DEVICETOKEN_KEY, {})
|
||||
now = int(time.time())
|
||||
if isinstance(tokens, list):
|
||||
data[self.DEVICETOKEN_KEY] = tokens = {t: now for t in tokens}
|
||||
|
||||
expired_tokens = [
|
||||
token
|
||||
for token, timestamp in tokens.items()
|
||||
if not _is_valid_token_timestamp(tokens[token], now)
|
||||
]
|
||||
for expired_token in expired_tokens:
|
||||
del tokens[expired_token]
|
||||
|
||||
yield tokens
|
||||
|
||||
def add_token_to_addr(self, addr, token):
|
||||
with self._modify_tokens(addr) as tokens:
|
||||
tokens[token] = int(time.time())
|
||||
with self.get_metadata_dict(addr).modify() as data:
|
||||
tokens = data.setdefault(self.DEVICETOKEN_KEY, [])
|
||||
if token not in tokens:
|
||||
tokens.append(token)
|
||||
|
||||
def remove_token_from_addr(self, addr, token):
|
||||
with self._modify_tokens(addr) as tokens:
|
||||
with self.get_metadata_dict(addr).modify() as data:
|
||||
tokens = data.get(self.DEVICETOKEN_KEY, [])
|
||||
if token in tokens:
|
||||
del tokens[token]
|
||||
tokens.remove(token)
|
||||
|
||||
def get_tokens_for_addr(self, addr):
|
||||
mdict = self.get_metadata_dict(addr).read()
|
||||
tokens = mdict.get(self.DEVICETOKEN_KEY, {})
|
||||
|
||||
now = int(time.time())
|
||||
if isinstance(tokens, dict):
|
||||
token_list = [
|
||||
token
|
||||
for token, timestamp in tokens.items()
|
||||
if _is_valid_token_timestamp(timestamp, now)
|
||||
]
|
||||
if len(token_list) < len(tokens):
|
||||
# Some tokens have expired, remove them.
|
||||
with self._modify_tokens(addr) as _tokens:
|
||||
pass
|
||||
else:
|
||||
token_list = []
|
||||
return token_list
|
||||
return mdict.get(self.DEVICETOKEN_KEY, [])
|
||||
|
||||
|
||||
class MetadataDictProxy(DictProxy):
|
||||
def __init__(self, notifier, metadata, iroh_relay=None, turn_hostname=None):
|
||||
def __init__(self, notifier, metadata, iroh_relay=None):
|
||||
super().__init__()
|
||||
self.notifier = notifier
|
||||
self.metadata = metadata
|
||||
self.iroh_relay = iroh_relay
|
||||
self.turn_hostname = turn_hostname
|
||||
|
||||
def handle_lookup(self, parts):
|
||||
# Lpriv/43f5f508a7ea0366dff30200c15250e3/devicetoken\tlkj123poi@c2.testrun.org
|
||||
@@ -100,11 +59,6 @@ class MetadataDictProxy(DictProxy):
|
||||
):
|
||||
# Handle `GETMETADATA "" /shared/vendor/deltachat/irohrelay`
|
||||
return f"O{self.iroh_relay}\n"
|
||||
elif keyname == "vendor/vendor.dovecot/pvt/server/vendor/deltachat/turn":
|
||||
res = turn_credentials()
|
||||
port = 3478
|
||||
return f"O{self.turn_hostname}:{port}:{res}\n"
|
||||
|
||||
logging.warning(f"lookup ignored: {parts!r}")
|
||||
return "N\n"
|
||||
|
||||
@@ -128,7 +82,6 @@ def main():
|
||||
|
||||
config = read_config(config_path)
|
||||
iroh_relay = config.iroh_relay
|
||||
mail_domain = config.mail_domain
|
||||
|
||||
vmail_dir = config.mailboxes_dir
|
||||
if not vmail_dir.exists():
|
||||
@@ -142,10 +95,7 @@ def main():
|
||||
notifier.start_notification_threads(metadata.remove_token_from_addr)
|
||||
|
||||
dictproxy = MetadataDictProxy(
|
||||
notifier=notifier,
|
||||
metadata=metadata,
|
||||
iroh_relay=iroh_relay,
|
||||
turn_hostname=mail_domain,
|
||||
notifier=notifier, metadata=metadata, iroh_relay=iroh_relay
|
||||
)
|
||||
|
||||
dictproxy.serve_forever_from_socket(socket)
|
||||
|
||||
@@ -11,8 +11,6 @@ def main(vmail_dir=None):
|
||||
ci_accounts = 0
|
||||
|
||||
for path in Path(vmail_dir).iterdir():
|
||||
if not path.joinpath("cur").is_dir():
|
||||
continue
|
||||
accounts += 1
|
||||
if path.name[:3] in ("ci-", "ac_"):
|
||||
ci_accounts += 1
|
||||
|
||||
@@ -32,7 +32,7 @@ def migrate_from_db_to_maildir(config, chunking=10000):
|
||||
# don't transfer special/CI accounts
|
||||
rows = [row for row in all_rows if row[0][:3] not in ("ci-", "ac_")]
|
||||
|
||||
logging.info(f"ignoring {len(all_rows) - len(rows)} CI accounts")
|
||||
logging.info(f"ignoring {len(all_rows)-len(rows)} CI accounts")
|
||||
logging.info(f"migrating {len(rows)} sqlite database passwords to user dirs")
|
||||
|
||||
for i, row in enumerate(rows):
|
||||
|
||||
@@ -15,7 +15,7 @@ ALPHANUMERIC_PUNCT = string.ascii_letters + string.digits + string.punctuation
|
||||
|
||||
|
||||
def create_newemail_dict(config: Config):
|
||||
user = "".join(random.choices(ALPHANUMERIC, k=config.username_max_length))
|
||||
user = "".join(random.choices(ALPHANUMERIC, k=config.username_min_length))
|
||||
password = "".join(
|
||||
secrets.choice(ALPHANUMERIC_PUNCT)
|
||||
for _ in range(config.password_min_length + 3)
|
||||
|
||||
@@ -17,11 +17,11 @@ and which are scheduled for retry using exponential back-off timing.
|
||||
If a token notification would be scheduled more than DROP_DEADLINE seconds
|
||||
after its first attempt, it is dropped with a log error.
|
||||
|
||||
Note that tokens are opaque to the notification machinery here
|
||||
and are encrypted foreclosing all ability to distinguish
|
||||
Note that tokens are completely opaque to the notification machinery here
|
||||
and will in the future be encrypted foreclosing all ability to distinguish
|
||||
which device token ultimately goes to which phone-provider notification service,
|
||||
or to understand the relation of "device tokens" and chatmail addresses.
|
||||
The meaning and format of tokens is basically a matter of chatmail Core and
|
||||
The meaning and format of tokens is basically a matter of Delta-Chat Core and
|
||||
the `notification.delta.chat` service.
|
||||
"""
|
||||
|
||||
@@ -95,12 +95,7 @@ class Notifier:
|
||||
logging.warning(f"removing spurious queue item: {queue_path!r}")
|
||||
queue_path.unlink()
|
||||
continue
|
||||
try:
|
||||
queue_item = PersistentQueueItem.read_from_path(queue_path)
|
||||
except ValueError:
|
||||
logging.warning(f"removing spurious queue item: {queue_path!r}")
|
||||
queue_path.unlink()
|
||||
continue
|
||||
queue_item = PersistentQueueItem.read_from_path(queue_path)
|
||||
self.queue_for_retry(queue_item)
|
||||
|
||||
def queue_for_retry(self, queue_item, retry_num=0):
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
From: {from_addr}
|
||||
To: {to_addr}
|
||||
Autocrypt-Setup-Message: v1
|
||||
Subject: Autocrypt Setup Message
|
||||
Date: Tue, 22 Jan 2019 12:56:29 +0100
|
||||
Content-type: multipart/mixed; boundary="Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ"
|
||||
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
|
||||
Content-Type: text/plain
|
||||
|
||||
This message contains all information to transfer your Autocrypt
|
||||
settings along with your secret key securely from your original
|
||||
device.
|
||||
|
||||
To set up your new device for Autocrypt, please follow the
|
||||
instuctions that should be presented by your new device.
|
||||
|
||||
You can keep this message and use it as a backup for your secret
|
||||
key. If you want to do this, you should write down the Setup Code
|
||||
and store it securely.
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
|
||||
Content-Type: application/autocrypt-setup
|
||||
Content-Disposition: attachment; filename="autocrypt-setup-message.html"
|
||||
|
||||
<html><body>
|
||||
<p>
|
||||
This is the Autocrypt setup file used to transfer settings and
|
||||
keys between clients. You can decrypt it using the Setup Code
|
||||
presented on your old device, and then import the contained key
|
||||
into your keyring.
|
||||
</p>
|
||||
|
||||
<pre>
|
||||
-----BEGIN PGP MESSAGE-----
|
||||
Passphrase-Format: numeric9x4
|
||||
Passphrase-Begin: 17
|
||||
|
||||
jA0EBwMCFAxADoCdzeX/0ukBlqI5+pfpKb751qd/7nLNbkpy3gVcaf1QwRPZYt40
|
||||
Ynp08UqRQ2g48ZlnzHLSwlTGOPTuv2Jt8ka+pgZ45xzvJSG2gau03xP4VsC271kR
|
||||
VmCjdb0Y6Rk96mAwfGzrkbaRQ9Z7fIoL866GOv6h9neiVIkp+JYlTV6ISD0ZQJ4Q
|
||||
I6dOQkB/TWZyVjtiJDOQHdfNWliA6NtqaLq19wlu9L5xXjuNpY95KwR8EJXWe0+o
|
||||
Y3d2U/KxOAkXKghP2Qg1GtlPVeGC5T4p03TGI6pzKT+kHX6Rrm9wK6sM9aTquMmF
|
||||
Vok84Jg1DFnwivWC2RILR81rXi7k/+Y6MUbveFgJ9cQduqpxnmD7TjOblYu7M6zp
|
||||
YGAUxh8DRKlIMn2QsA++DBYQ6ACZvwuY8qTDLkqPDo4WqM313dsMJbyGjDdVE7EM
|
||||
PESS+RlABETpZXz8g/ycr6DIUNdlbPcmYlsBfHWDOuR2GFFTwmlv5slWS39dJv38
|
||||
E0eIe1CwdxI801Se7t7dUUS/ZF8wb6GlmxOcqGbF8eko1Z0S64IAm7/h13MRQCxI
|
||||
geQnHfGYVJ2FOimoCMEKwfa9x++RFTDW0u7spDC2uWvK/1viV8OfRppFhLr/kmKb
|
||||
18lWXuAz80DAjUDUsVqEq2MvJBJGoCJUEyjuRsLkHYRM5jYk4v50LyyR0Om73nWF
|
||||
nZBqmqNzdr7Xb9PHHdFhnEc0VvoYbrcM0RVYcEMW3YbmejM891j1d6Iv+/n/qND/
|
||||
NdebGrfWJMmFLf/iEkzTZ3/v5inW9LpWoRc94ioCjJTaEo8Rib6ARRFaJVIsmNXi
|
||||
YicFGO98D+zX+a2t9Yz6IpPajVslnOp6ScpmXgts/2XWD7oE+JgxSAqo/dLVsHgP
|
||||
Ufo=
|
||||
=pulM
|
||||
-----END PGP MESSAGE-----
|
||||
</pre></body></html>
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ--
|
||||
@@ -1,44 +1,44 @@
|
||||
From: {from_addr}
|
||||
To: {to_addr}
|
||||
Subject: ...
|
||||
Date: Sun, 15 Oct 2023 16:43:21 +0000
|
||||
Message-ID: <Mr.UVyJWZmkCKM.hGzNc6glBE_@c2.testrun.org>
|
||||
In-Reply-To: <Mr.MvmCz-GQbi_.6FGRkhDf05c@c2.testrun.org>
|
||||
References: <Mr.3gckbNy5bch.uK3Hd2Ws6-w@c2.testrun.org>
|
||||
<Mr.MvmCz-GQbi_.6FGRkhDf05c@c2.testrun.org>
|
||||
Chat-Version: 1.0
|
||||
Autocrypt: addr={from_addr}; prefer-encrypt=mutual;
|
||||
keydata=xjMEZSwWjhYJKwYBBAHaRw8BAQdAQBEhqeJh0GueHB6kF/DUQqYCxARNBVokg/AzT+7LqH
|
||||
rNFzxiYXJiYXpAYzIudGVzdHJ1bi5vcmc+wosEEBYIADMCGQEFAmUsFo4CGwMECwkIBwYVCAkKCwID
|
||||
FgIBFiEEFTfUNvVnY3b9F7yHnmme1PfUhX8ACgkQnmme1PfUhX9A4AEAnHWHp49eBCMHK5t66gYPiW
|
||||
XQuB1mwUjzGfYWB+0RXUoA/0xcQ3FbUNlGKW7Blp6eMFfViv6Mv2d3kNSXACB6nmcMzjgEZSwWjhIK
|
||||
KwYBBAGXVQEFAQEHQBpY5L2M1XHo0uxf8SX1wNLBp/OVvidoWHQF2Jz+kJsUAwEIB8J4BBgWCAAgBQ
|
||||
JlLBaOAhsMFiEEFTfUNvVnY3b9F7yHnmme1PfUhX8ACgkQnmme1PfUhX/INgEA37AJaNvruYsJVanP
|
||||
IXnYw4CKd55UAwl8Zcy+M2diAbkA/0fHHcGV4r78hpbbL1Os52DPOdqYQRauIeJUeG+G6bQO
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/encrypted; protocol="application/pgp-encrypted";
|
||||
boundary="YFrteb74qSXmggbOxZL9dRnhymywAi"
|
||||
|
||||
|
||||
--YFrteb74qSXmggbOxZL9dRnhymywAi
|
||||
Content-Description: PGP/MIME version identification
|
||||
Content-Type: application/pgp-encrypted
|
||||
|
||||
Version: 1
|
||||
|
||||
|
||||
--YFrteb74qSXmggbOxZL9dRnhymywAi
|
||||
Content-Description: OpenPGP encrypted message
|
||||
Content-Disposition: inline; filename="encrypted.asc";
|
||||
Content-Type: application/octet-stream; name="encrypted.asc"
|
||||
|
||||
-----BEGIN PGP MESSAGE-----
|
||||
|
||||
yxJiAAAAAABIZWxsbyB3b3JsZCE=
|
||||
=1I/B
|
||||
-----END PGP MESSAGE-----
|
||||
|
||||
|
||||
--YFrteb74qSXmggbOxZL9dRnhymywAi--
|
||||
|
||||
From: {from_addr}
|
||||
|
||||
To: {to_addr}
|
||||
|
||||
Subject: ...
|
||||
|
||||
Date: Sun, 15 Oct 2023 16:43:21 +0000
|
||||
|
||||
Message-ID: <Mr.UVyJWZmkCKM.hGzNc6glBE_@c2.testrun.org>
|
||||
|
||||
In-Reply-To: <Mr.MvmCz-GQbi_.6FGRkhDf05c@c2.testrun.org>
|
||||
|
||||
References: <Mr.3gckbNy5bch.uK3Hd2Ws6-w@c2.testrun.org>
|
||||
|
||||
<Mr.MvmCz-GQbi_.6FGRkhDf05c@c2.testrun.org>
|
||||
|
||||
Chat-Version: 1.0
|
||||
|
||||
Autocrypt: addr={from_addr}; prefer-encrypt=mutual;
|
||||
|
||||
keydata=xjMEZSwWjhYJKwYBBAHaRw8BAQdAQBEhqeJh0GueHB6kF/DUQqYCxARNBVokg/AzT+7LqH
|
||||
|
||||
rNFzxiYXJiYXpAYzIudGVzdHJ1bi5vcmc+wosEEBYIADMCGQEFAmUsFo4CGwMECwkIBwYVCAkKCwID
|
||||
|
||||
FgIBFiEEFTfUNvVnY3b9F7yHnmme1PfUhX8ACgkQnmme1PfUhX9A4AEAnHWHp49eBCMHK5t66gYPiW
|
||||
|
||||
XQuB1mwUjzGfYWB+0RXUoA/0xcQ3FbUNlGKW7Blp6eMFfViv6Mv2d3kNSXACB6nmcMzjgEZSwWjhIK
|
||||
|
||||
KwYBBAGXVQEFAQEHQBpY5L2M1XHo0uxf8SX1wNLBp/OVvidoWHQF2Jz+kJsUAwEIB8J4BBgWCAAgBQ
|
||||
|
||||
JlLBaOAhsMFiEEFTfUNvVnY3b9F7yHnmme1PfUhX8ACgkQnmme1PfUhX/INgEA37AJaNvruYsJVanP
|
||||
|
||||
IXnYw4CKd55UAwl8Zcy+M2diAbkA/0fHHcGV4r78hpbbL1Os52DPOdqYQRauIeJUeG+G6bQO
|
||||
|
||||
MIME-Version: 1.0
|
||||
|
||||
Content-Type: multipart/encrypted; protocol="application/pgp-encrypted";
|
||||
|
||||
boundary="YFrteb74qSXmggbOxZL9dRnhymywAi"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
Date: Fri, 8 Jul 1994 09:21:47 -0400
|
||||
From: Mail Delivery Subsystem <MAILER-DAEMON@example.org>
|
||||
Subject: Returned mail: User unknown
|
||||
To: <owner-ups-mib@CS.UTK.EDU>
|
||||
Auto-Submitted: auto-replied
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/report; report-type=delivery-status;
|
||||
boundary="JAA13167.773673707/CS.UTK.EDU"
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: text/plain; charset=us-ascii
|
||||
|
||||
----- The following addresses had delivery problems -----
|
||||
<arathib@vnet.ibm.com> (unrecoverable error)
|
||||
<wsnell@sdcc13.ucsd.edu> (unrecoverable error)
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: message/delivery-status
|
||||
|
||||
Reporting-MTA: dns; cs.utk.edu
|
||||
|
||||
Original-Recipient: rfc822;arathib@vnet.ibm.com
|
||||
Final-Recipient: rfc822;arathib@vnet.ibm.com
|
||||
Action: failed
|
||||
Status: 5.0.0 (permanent failure)
|
||||
Diagnostic-Code: smtp;
|
||||
550 'arathib@vnet.IBM.COM' is not a registered gateway user
|
||||
Remote-MTA: dns; vnet.ibm.com
|
||||
|
||||
Original-Recipient: rfc822;johnh@hpnjld.njd.hp.com
|
||||
Final-Recipient: rfc822;johnh@hpnjld.njd.hp.com
|
||||
Action: delayed
|
||||
Status: 4.0.0 (hpnjld.njd.jp.com: host name lookup failure)
|
||||
|
||||
Original-Recipient: rfc822;wsnell@sdcc13.ucsd.edu
|
||||
Final-Recipient: rfc822;wsnell@sdcc13.ucsd.edu
|
||||
Action: failed
|
||||
Status: 5.0.0
|
||||
Diagnostic-Code: smtp; 550 user unknown
|
||||
Remote-MTA: dns; sdcc13.ucsd.edu
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: message/rfc822
|
||||
|
||||
[original message goes here]
|
||||
--JAA13167.773673707/CS.UTK.EDU--
|
||||
@@ -1,21 +0,0 @@
|
||||
Subject: Message from {from_addr}
|
||||
From: <{from_addr}>
|
||||
To: <{to_addr}>
|
||||
Date: Sun, 15 Oct 2023 16:43:25 +0000
|
||||
Message-ID: <Mr.78MWtlV7RAi.goCFzBhCYfy@c2.testrun.org>
|
||||
Chat-Version: 1.0
|
||||
Secure-Join: vc-request
|
||||
Secure-Join-Invitenumber: RANDOM-TOKEN
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/mixed; boundary="Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi"
|
||||
|
||||
|
||||
--Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi
|
||||
Content-Type: text/plain; charset=utf-8
|
||||
|
||||
Buy viagra!
|
||||
|
||||
|
||||
--Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi--
|
||||
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
Subject: Message from {from_addr}
|
||||
From: <{from_addr}>
|
||||
To: <{to_addr}>
|
||||
Date: Sun, 15 Oct 2023 16:43:25 +0000
|
||||
Message-ID: <Mr.78MWtlV7RAi.goCFzBhCYfy@c2.testrun.org>
|
||||
Chat-Version: 1.0
|
||||
Secure-Join: vc-request
|
||||
Secure-Join-Invitenumber: RANDOM-TOKEN
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/mixed; boundary="Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi"
|
||||
|
||||
|
||||
--Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi
|
||||
Content-Type: text/plain; charset=utf-8
|
||||
|
||||
Secure-Join: vc-request
|
||||
|
||||
|
||||
--Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi--
|
||||
|
||||
|
||||
@@ -69,11 +69,12 @@ def maildata(request):
|
||||
|
||||
assert datadir.exists(), datadir
|
||||
|
||||
def maildata(name, from_addr, to_addr, subject="[...]"):
|
||||
def maildata(name, from_addr, to_addr, subject="..."):
|
||||
# Using `.read_bytes().decode()` instead of `.read_text()` to preserve newlines.
|
||||
data = datadir.joinpath(name).read_bytes().decode()
|
||||
|
||||
text = data.format(from_addr=from_addr, to_addr=to_addr, subject=subject)
|
||||
return BytesParser(policy=policy.SMTP).parsebytes(text.encode())
|
||||
return BytesParser(policy=policy.default).parsebytes(text.encode())
|
||||
|
||||
return maildata
|
||||
|
||||
|
||||
@@ -15,14 +15,6 @@ def test_read_config_basic(example_config):
|
||||
assert example_config.mail_domain == "chat.example.org"
|
||||
|
||||
|
||||
def test_read_config_basic_using_defaults(tmp_path, maildomain):
|
||||
inipath = tmp_path.joinpath("chatmail.ini")
|
||||
inipath.write_text(f"[params]\nmail_domain = {maildomain}")
|
||||
example_config = read_config(inipath)
|
||||
assert example_config.max_user_send_per_minute == 60
|
||||
assert example_config.filtermail_smtp_port_incoming == 10081
|
||||
|
||||
|
||||
def test_read_config_testrun(make_config):
|
||||
config = make_config("something.testrun.org")
|
||||
assert config.mail_domain == "something.testrun.org"
|
||||
@@ -35,7 +27,6 @@ def test_read_config_testrun(make_config):
|
||||
assert config.max_user_send_per_minute == 60
|
||||
assert config.max_mailbox_size == "100M"
|
||||
assert config.delete_mails_after == "20"
|
||||
assert config.delete_large_after == "7"
|
||||
assert config.username_min_length == 9
|
||||
assert config.username_max_length == 9
|
||||
assert config.password_min_length == 9
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import time
|
||||
|
||||
from chatmaild.delete_inactive_users import delete_inactive_users
|
||||
from chatmaild.doveauth import AuthDictProxy
|
||||
from chatmaild.expire import main as main_expire
|
||||
|
||||
|
||||
def test_login_timestamps(example_config):
|
||||
@@ -45,12 +45,7 @@ def test_delete_inactive_users(example_config):
|
||||
for addr in to_remove:
|
||||
assert example_config.get_user(addr).maildir.exists()
|
||||
|
||||
main_expire(
|
||||
args=[
|
||||
"--remove",
|
||||
str(example_config._inipath),
|
||||
]
|
||||
)
|
||||
delete_inactive_users(example_config)
|
||||
|
||||
for p in example_config.mailboxes_dir.iterdir():
|
||||
assert not p.name.startswith("old")
|
||||
|
||||
@@ -1,150 +0,0 @@
|
||||
import os
|
||||
import random
|
||||
from datetime import datetime
|
||||
from fnmatch import fnmatch
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from chatmaild.expire import (
|
||||
FileEntry,
|
||||
MailboxStat,
|
||||
get_file_entry,
|
||||
iter_mailboxes,
|
||||
os_listdir_if_exists,
|
||||
)
|
||||
from chatmaild.expire import main as expiry_main
|
||||
from chatmaild.fsreport import main as report_main
|
||||
|
||||
|
||||
def fill_mbox(basedir):
|
||||
basedir1 = basedir.joinpath("mailbox1@example.org")
|
||||
basedir1.mkdir()
|
||||
password = basedir1.joinpath("password")
|
||||
password.write_text("xxx")
|
||||
basedir1.joinpath("maildirsize").write_text("xxx")
|
||||
|
||||
garbagedir = basedir1.joinpath("garbagedir")
|
||||
garbagedir.mkdir()
|
||||
|
||||
create_new_messages(basedir1, ["cur/msg1"], size=500)
|
||||
create_new_messages(basedir1, ["new/msg2"], size=600)
|
||||
return basedir1
|
||||
|
||||
|
||||
def create_new_messages(basedir, relpaths, size=1000, days=0):
|
||||
now = datetime.utcnow().timestamp()
|
||||
|
||||
for relpath in relpaths:
|
||||
msg_path = Path(basedir).joinpath(relpath)
|
||||
msg_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
msg_path.write_text("x" * size)
|
||||
# accessed now, modified N days ago
|
||||
os.utime(msg_path, (now, now - days * 86400))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mbox1(example_config):
|
||||
basedir1 = fill_mbox(example_config.mailboxes_dir)
|
||||
return MailboxStat(basedir1)
|
||||
|
||||
|
||||
def test_filentry_ordering(tmp_path):
|
||||
l = [FileEntry(f"x{i}", size=i + 10, mtime=1000 - i) for i in range(10)]
|
||||
sorted = list(l)
|
||||
random.shuffle(l)
|
||||
l.sort(key=lambda x: x.size)
|
||||
assert l == sorted
|
||||
|
||||
|
||||
def test_no_mailbxoes(tmp_path, capsys):
|
||||
assert [] == list(iter_mailboxes(str(tmp_path.joinpath("notexists")), maxnum=10))
|
||||
out, err = capsys.readouterr()
|
||||
assert "no mailboxes" in err
|
||||
|
||||
|
||||
def test_stats_mailbox(mbox1):
|
||||
password = Path(mbox1.basedir).joinpath("password")
|
||||
assert mbox1.last_login == password.stat().st_mtime
|
||||
assert len(mbox1.messages) == 2
|
||||
|
||||
msgs = list(sorted(mbox1.messages, key=lambda x: x.size))
|
||||
assert len(msgs) == 2
|
||||
assert msgs[0].size == 500 # cur
|
||||
assert msgs[1].size == 600 # new
|
||||
|
||||
create_new_messages(mbox1.basedir, ["large-extra"], size=1000)
|
||||
create_new_messages(mbox1.basedir, ["index-something"], size=3)
|
||||
mbox2 = MailboxStat(mbox1.basedir)
|
||||
assert len(mbox2.extrafiles) == 4
|
||||
assert mbox2.extrafiles[0].size == 1000
|
||||
|
||||
# cope well with mailbox dirs that have no password (for whatever reason)
|
||||
Path(mbox1.basedir).joinpath("password").unlink()
|
||||
mbox3 = MailboxStat(mbox1.basedir)
|
||||
assert mbox3.last_login is None
|
||||
|
||||
|
||||
def test_report_no_mailboxes(example_config):
|
||||
args = (str(example_config._inipath),)
|
||||
report_main(args)
|
||||
|
||||
|
||||
def test_report(mbox1, example_config):
|
||||
args = (str(example_config._inipath),)
|
||||
report_main(args)
|
||||
args = list(args) + "--days 1".split()
|
||||
report_main(args)
|
||||
args = list(args) + "--min-login-age 1".split()
|
||||
report_main(args)
|
||||
args = list(args) + "--mdir cur".split()
|
||||
report_main(args)
|
||||
|
||||
|
||||
def test_expiry_cli_basic(example_config, mbox1):
|
||||
args = (str(example_config._inipath),)
|
||||
expiry_main(args)
|
||||
|
||||
|
||||
def test_expiry_cli_old_files(capsys, example_config, mbox1):
|
||||
relpaths_old = ["cur/msg_old1", "cur/msg_old1"]
|
||||
cutoff_days = int(example_config.delete_mails_after) + 1
|
||||
create_new_messages(mbox1.basedir, relpaths_old, size=1000, days=cutoff_days)
|
||||
|
||||
relpaths_large = ["cur/msg_old_large1", "new/msg_old_large2"]
|
||||
cutoff_days = int(example_config.delete_large_after) + 1
|
||||
create_new_messages(
|
||||
mbox1.basedir, relpaths_large, size=1000 * 300, days=cutoff_days
|
||||
)
|
||||
|
||||
create_new_messages(mbox1.basedir, ["cur/shouldstay"], size=1000 * 300, days=1)
|
||||
|
||||
args = str(example_config._inipath), "--remove", "-v"
|
||||
expiry_main(args)
|
||||
out, err = capsys.readouterr()
|
||||
|
||||
allpaths = relpaths_old + relpaths_large + ["maildirsize"]
|
||||
for path in allpaths:
|
||||
for line in err.split("\n"):
|
||||
if fnmatch(line, f"removing*{path}"):
|
||||
break
|
||||
else:
|
||||
if path != "new/msg_old_large2":
|
||||
pytest.fail(f"failed to remove {path}\n{err}")
|
||||
|
||||
assert "shouldstay" not in err
|
||||
|
||||
|
||||
def test_get_file_entry(tmp_path):
|
||||
assert get_file_entry(str(tmp_path.joinpath("123123"))) is None
|
||||
p = tmp_path.joinpath("x")
|
||||
p.write_text("hello")
|
||||
entry = get_file_entry(str(p))
|
||||
assert entry.size == 5
|
||||
assert entry.mtime
|
||||
|
||||
|
||||
def test_os_listdir_if_exists(tmp_path):
|
||||
tmp_path.joinpath("x").write_text("hello")
|
||||
assert len(os_listdir_if_exists(str(tmp_path))) == 1
|
||||
assert len(os_listdir_if_exists(str(tmp_path.joinpath("123123")))) == 0
|
||||
@@ -1,12 +1,11 @@
|
||||
import pytest
|
||||
|
||||
from chatmaild.filtermail import (
|
||||
IncomingBeforeQueueHandler,
|
||||
OutgoingBeforeQueueHandler,
|
||||
BeforeQueueHandler,
|
||||
SendRateLimiter,
|
||||
check_armored_payload,
|
||||
check_encrypted,
|
||||
is_securejoin,
|
||||
common_encrypted_subjects,
|
||||
)
|
||||
|
||||
|
||||
@@ -19,13 +18,7 @@ def maildomain():
|
||||
@pytest.fixture
|
||||
def handler(make_config, maildomain):
|
||||
config = make_config(maildomain)
|
||||
return OutgoingBeforeQueueHandler(config)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def inhandler(make_config, maildomain):
|
||||
config = make_config(maildomain)
|
||||
return IncomingBeforeQueueHandler(config)
|
||||
return BeforeQueueHandler(config)
|
||||
|
||||
|
||||
def test_reject_forged_from(maildata, gencreds, handler):
|
||||
@@ -36,14 +29,14 @@ def test_reject_forged_from(maildata, gencreds, handler):
|
||||
# test that the filter lets good mail through
|
||||
to_addr = gencreds()[0]
|
||||
env.content = maildata(
|
||||
"encrypted.eml", from_addr=env.mail_from, to_addr=to_addr
|
||||
"plain.eml", from_addr=env.mail_from, to_addr=to_addr
|
||||
).as_bytes()
|
||||
|
||||
assert not handler.check_DATA(envelope=env)
|
||||
|
||||
# test that the filter rejects forged mail
|
||||
env.content = maildata(
|
||||
"encrypted.eml", from_addr="forged@c3.testrun.org", to_addr=to_addr
|
||||
"plain.eml", from_addr="forged@c3.testrun.org", to_addr=to_addr
|
||||
).as_bytes()
|
||||
error = handler.check_DATA(envelope=env)
|
||||
assert "500" in error
|
||||
@@ -62,28 +55,19 @@ def test_filtermail_no_encryption_detection(maildata):
|
||||
assert not check_encrypted(msg)
|
||||
|
||||
|
||||
def test_filtermail_securejoin_detection(maildata):
|
||||
msg = maildata(
|
||||
"securejoin-vc.eml", from_addr="some@example.org", to_addr="other@example.org"
|
||||
)
|
||||
assert is_securejoin(msg)
|
||||
|
||||
msg = maildata(
|
||||
"securejoin-vc-fake.eml",
|
||||
from_addr="some@example.org",
|
||||
to_addr="other@example.org",
|
||||
)
|
||||
assert not is_securejoin(msg)
|
||||
|
||||
|
||||
def test_filtermail_encryption_detection(maildata):
|
||||
msg = maildata(
|
||||
"encrypted.eml",
|
||||
from_addr="1@example.org",
|
||||
to_addr="2@example.org",
|
||||
subject="Subject does not matter, will be replaced anyway",
|
||||
)
|
||||
assert check_encrypted(msg)
|
||||
for subject in common_encrypted_subjects:
|
||||
msg = maildata(
|
||||
"encrypted.eml",
|
||||
from_addr="1@example.org",
|
||||
to_addr="2@example.org",
|
||||
subject=subject,
|
||||
)
|
||||
assert check_encrypted(msg)
|
||||
|
||||
# if the subject is not a known encrypted subject value, it is not considered ac-encrypted
|
||||
msg.replace_header("Subject", "Click this link")
|
||||
assert not check_encrypted(msg)
|
||||
|
||||
|
||||
def test_filtermail_no_literal_packets(maildata):
|
||||
@@ -113,7 +97,7 @@ def test_send_rate_limiter():
|
||||
break
|
||||
|
||||
|
||||
def test_cleartext_excempt_privacy(maildata, gencreds, handler):
|
||||
def test_excempt_privacy(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = "privacy@testrun.org"
|
||||
handler.config.passthrough_recipients = [to_addr]
|
||||
@@ -134,97 +118,10 @@ def test_cleartext_excempt_privacy(maildata, gencreds, handler):
|
||||
rcpt_tos = [to_addr, false_to]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert "523" in handler.check_DATA(envelope=env2)
|
||||
assert "500" in handler.check_DATA(envelope=env2)
|
||||
|
||||
|
||||
def test_cleartext_self_send_autocrypt_setup_message(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = from_addr
|
||||
|
||||
msg = maildata("asm.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert not handler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_send_fails(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = gencreds()[0]
|
||||
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
res = handler.check_DATA(envelope=env)
|
||||
assert "523 Encryption Needed" in res
|
||||
|
||||
|
||||
def test_cleartext_incoming_fails(maildata, gencreds, inhandler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr, password = gencreds()
|
||||
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
user = inhandler.config.get_user(to_addr)
|
||||
user.set_password(password)
|
||||
res = inhandler.check_DATA(envelope=env)
|
||||
assert "523 Encryption Needed" in res
|
||||
|
||||
user.allow_incoming_cleartext()
|
||||
assert not inhandler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_incoming_mailer_daemon(maildata, gencreds, inhandler):
|
||||
from_addr = "mailer-daemon@example.org"
|
||||
to_addr = gencreds()[0]
|
||||
|
||||
msg = maildata("mailer-daemon.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert not inhandler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_passthrough_domains(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = "privacy@x.y.z"
|
||||
handler.config.passthrough_recipients = ["@x.y.z"]
|
||||
false_to = "something@x.y"
|
||||
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
# assert that None/no error is returned
|
||||
assert not handler.check_DATA(envelope=env)
|
||||
|
||||
class env2:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr, false_to]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert "523" in handler.check_DATA(envelope=env2)
|
||||
|
||||
|
||||
def test_cleartext_passthrough_senders(gencreds, handler, maildata):
|
||||
def test_passthrough_senders(gencreds, handler, maildata):
|
||||
acc1 = gencreds()[0]
|
||||
to_addr = "recipient@something.org"
|
||||
handler.config.passthrough_senders = [acc1]
|
||||
@@ -241,9 +138,8 @@ def test_cleartext_passthrough_senders(gencreds, handler, maildata):
|
||||
|
||||
|
||||
def test_check_armored_payload():
|
||||
prefix = "-----BEGIN PGP MESSAGE-----\r\n"
|
||||
comment = "Version: ProtonMail\r\n"
|
||||
payload = """\r
|
||||
payload = """-----BEGIN PGP MESSAGE-----\r
|
||||
\r
|
||||
wU4DSqFx0d1yqAoSAQdAYkX/ZN/Az4B0k7X47zKyWrXxlDEdS3WOy0Yf2+GJTFgg\r
|
||||
Zk5ql0mLG8Ze+ZifCS0XMO4otlemSyJ0K1ZPdFMGzUDBTgNqzkFabxXoXRIBB0AM\r
|
||||
755wlX41X6Ay3KhnwBq7yEqSykVH6F3x11iHPKraLCAGZoaS8bKKNy/zg5slda1X\r
|
||||
@@ -276,28 +172,9 @@ UN4fiB0KR9JyG2ayUdNJVkXZSZLnHyRgiaadlpUo16LVvw==\r
|
||||
=b5Kp\r
|
||||
-----END PGP MESSAGE-----\r
|
||||
\r
|
||||
\r
|
||||
"""
|
||||
|
||||
commented_payload = prefix + comment + payload
|
||||
assert check_armored_payload(commented_payload, outgoing=False) == True
|
||||
assert check_armored_payload(commented_payload, outgoing=True) == False
|
||||
|
||||
payload = prefix + payload
|
||||
assert check_armored_payload(payload, outgoing=False) == True
|
||||
assert check_armored_payload(payload, outgoing=True) == True
|
||||
|
||||
payload = payload.removesuffix("\r\n")
|
||||
assert check_armored_payload(payload, outgoing=False) == True
|
||||
assert check_armored_payload(payload, outgoing=True) == True
|
||||
|
||||
payload = payload.removesuffix("\r\n")
|
||||
assert check_armored_payload(payload, outgoing=False) == True
|
||||
assert check_armored_payload(payload, outgoing=True) == True
|
||||
|
||||
payload = payload.removesuffix("\r\n")
|
||||
assert check_armored_payload(payload, outgoing=False) == True
|
||||
assert check_armored_payload(payload, outgoing=True) == True
|
||||
assert check_armored_payload(payload) == True
|
||||
|
||||
payload = """-----BEGIN PGP MESSAGE-----\r
|
||||
\r
|
||||
@@ -305,8 +182,7 @@ HELLOWORLD
|
||||
-----END PGP MESSAGE-----\r
|
||||
\r
|
||||
"""
|
||||
assert check_armored_payload(payload, outgoing=False) == False
|
||||
assert check_armored_payload(payload, outgoing=True) == False
|
||||
assert check_armored_payload(payload) == False
|
||||
|
||||
payload = """-----BEGIN PGP MESSAGE-----\r
|
||||
\r
|
||||
@@ -314,48 +190,4 @@ HELLOWORLD
|
||||
-----END PGP MESSAGE-----\r
|
||||
\r
|
||||
"""
|
||||
assert check_armored_payload(payload, outgoing=False) == False
|
||||
assert check_armored_payload(payload, outgoing=True) == False
|
||||
|
||||
# Test payload using partial body length
|
||||
# as generated by GopenPGP.
|
||||
payload = """-----BEGIN PGP MESSAGE-----\r
|
||||
\r
|
||||
wV4DdCVjRfOT3TQSAQdAY5+pjT6mlCxPGdR3be4w7oJJRUGIPI/Vnh+mJxGSm34w\r
|
||||
LNlVc89S1g22uQYFif2sUJsQWbpoHpNkuWpkSgOaHmNvrZiY/YU5iv+cZ3LbmtUG\r
|
||||
0uoBisSHh9O1c+5sYZSbrvYZ1NOwlD7Fv/U5/Mw4E5+CjxfdgNGp5o3DDddzPK78\r
|
||||
jseDhdSXxnaiIJC93hxNX6R1RPt3G2gukyzx69wciPQShcF8zf3W3o75Ed7B8etV\r
|
||||
QEeB16xzdFhKa9JxdjTu3osgCs21IO7wpcFkjc7nZzlW6jPnELJJaNmv4yOOCjMp\r
|
||||
6YAkaN/BkL+jHTznHDuDsT5ilnTXpwHDU1Cm9PIx/KFcNCQnIB+2DcdIHPHUH1ci\r
|
||||
jvqoeXAVWjKXEjS7PqPFuP/xGbrWG2ugs+toXJOKbgRkExvKs1dwPFKrgghvCVbW\r
|
||||
AcKejQKAPArLwpkA7aD875TZQShvGt74fNs45XBlGOYOnNOAJ1KAmzrXLIDViyyB\r
|
||||
kDsmTBk785xofuCkjBpXSe6vsMprPzCteDfaUibh8FHeJjucxPerwuOPEmnogNaf\r
|
||||
YyL4+iy8H8I9/p7pmUqILprxTG0jTOtlk0bTVzeiF56W1xbtSEMuOo4oFbQTyOM2\r
|
||||
bKXaYo774Jm+rRtKAnnI2dtf9RpK19cog6YNzfYjesLKbXDsPZbN5rmwyFiCvvxC\r
|
||||
kQ6JLob+B2fPdY2gzy7LypxktS8Zi1HJcWDHJGVmQodaDLqKUObb4M26bXDe6oxI\r
|
||||
NS8PJz5exVbM3KhZnUOEn6PJRBBf5a/ZqxlhZPcQo/oBuhKpBRpO5kSDwPIUByu3\r
|
||||
UlXLSkpMqe9pUarAOEuQjfl2RVY7U+RrQYp4YP5keMO+i8NCefAFbowTTufO1JIq\r
|
||||
2nVgCi/QVnxZyEc9OYt/8AE3g4cdojE+vsSDifZLSWYIetpfrohHv3dT3StD1QRG\r
|
||||
0QE6qq6oKpg/IL0cjvuX4c7a7bslv2fXp8t75y37RU6253qdIebhxc/cRhPbc/yu\r
|
||||
p0YLyD4SrvKTLP2ZV95jT4IPEpqm4AN3QmiOzdtqR2gLyb62L8QfqI/FdwsIiRiM\r
|
||||
hqydwoqt/lfSqG1WKPh+6EkMkH+TDiCC1BQdbN1MNcyUtcjb35PR2c8Ld2TF3guA\r
|
||||
jLIqMt/Vb7hBoMb2FcsOYY25ka9oV62OwgKWLXnFzk+modMR5fzb4kxVVAYEqP+D\r
|
||||
T5KO1Vs76v1fyPGOq6BbBCvLwTqe/e6IZInJles4v5jrhnLcGKmNGivCUDe6X6NY\r
|
||||
UKNt5RsZllwDQpaAb5dMNhyrk8SgIE7TBI7rvqIdUCE52Vy+0JDxFg5olRpFUfO6\r
|
||||
/MyTW3Yo/ekk/npHr7iYYqJTCc21bDGLWQcIo/XO7WPxrKNWGBNPFnkRdw0MaKr4\r
|
||||
+cEM3V8NFnSEpC12xA+RX/CezuJtwXZK5MpG76eYqMO6qyC+c25YcFecEufDZDxx\r
|
||||
ZLqRszVRyxyWPtk/oIeQK2v9wOqY6N9/ff01gHz69vqYqN5bUw/QKZsmx1zW+gPw\r
|
||||
6x2tDK2BHeYl182gCbhlKISRFwCtbjqZSkiKWao/VtygHkw0fK34avJuyQ/X9YaN\r
|
||||
BRy+7Lf3VA53pnB5WJ1xwRXN8VDvmZeXzv2krHveCMemj0OjnRoCLu117xN0A5m9\r
|
||||
Fm/RoDix5PolDHtWTtr2m1n2hp2LHnj8at9lFEd0SKhAYHVL9KjzycwWODZRXt+x\r
|
||||
zGDDuooEeTvdY5NLyKcl4gETz1ZP4Ez5jGGjhPSwSpq1mU7UaJ9ZXXdr4KHyifW6\r
|
||||
ggNzNsGhXTap7IWZpTtqXABydfiBshmH2NjqtNDwBweJVSgP10+r0WhMWlaZs6xl\r
|
||||
V3o5yskJt6GlkwpJxZrTvN6Tiww/eW7HFV6NGf7IRSWY5tJc/iA7/92tOmkdvJ1q\r
|
||||
myLbG7cJB787QjplEyVe2P/JBO6xYvbkJLf9Q+HaviTO25rugRSrYsoKMDfO8VlQ\r
|
||||
1CcnTPVtApPZJEQzAWJEgVAM8uIlkqWJJMgyWT34sTkdBeCUFGloXQFs9Yxd0AGf\r
|
||||
/zHEkYZSTKpVSvAIGu4=\r
|
||||
=6iHb\r
|
||||
-----END PGP MESSAGE-----\r
|
||||
"""
|
||||
assert check_armored_payload(payload, outgoing=False) == True
|
||||
assert check_armored_payload(payload, outgoing=True) == True
|
||||
assert check_armored_payload(payload) == False
|
||||
|
||||
@@ -1,78 +0,0 @@
|
||||
import smtplib
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def smtpserver():
|
||||
from pytest_localserver import smtp
|
||||
|
||||
server = smtp.Server("127.0.0.1")
|
||||
server.start()
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def make_popen(request):
|
||||
def popen(cmdargs, stdout=subprocess.PIPE, stderr=subprocess.PIPE, **kw):
|
||||
p = subprocess.Popen(
|
||||
cmdargs,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
|
||||
def fin():
|
||||
p.terminate()
|
||||
out, err = p.communicate()
|
||||
print(out.decode("ascii"))
|
||||
print(err.decode("ascii"), file=sys.stderr)
|
||||
|
||||
request.addfinalizer(fin)
|
||||
return p
|
||||
|
||||
return popen
|
||||
|
||||
|
||||
@pytest.mark.parametrize("filtermail_mode", ["outgoing", "incoming"])
|
||||
def test_one_mail(
|
||||
make_config, make_popen, smtpserver, maildata, filtermail_mode, monkeypatch
|
||||
):
|
||||
monkeypatch.setenv("PYTHONUNBUFFERED", "1")
|
||||
smtp_inject_port = 20025
|
||||
if filtermail_mode == "outgoing":
|
||||
settings = dict(
|
||||
postfix_reinject_port=smtpserver.port,
|
||||
filtermail_smtp_port=smtp_inject_port,
|
||||
)
|
||||
else:
|
||||
settings = dict(
|
||||
postfix_reinject_port_incoming=smtpserver.port,
|
||||
filtermail_smtp_port_incoming=smtp_inject_port,
|
||||
)
|
||||
|
||||
config = make_config("example.org", settings=settings)
|
||||
path = str(config._inipath)
|
||||
|
||||
popen = make_popen(["filtermail", path, filtermail_mode])
|
||||
line = popen.stderr.readline().strip()
|
||||
if b"loop" not in line:
|
||||
print(line.decode("ascii"), file=sys.stderr)
|
||||
pytest.fail("starting filtermail failed")
|
||||
|
||||
addr = f"user1@{config.mail_domain}"
|
||||
config.get_user(addr).set_password("l1k2j3l1k2j3l")
|
||||
|
||||
# send encrypted mail
|
||||
data = str(maildata("encrypted.eml", from_addr=addr, to_addr=addr))
|
||||
client = smtplib.SMTP("localhost", smtp_inject_port)
|
||||
client.sendmail(addr, [addr], data)
|
||||
assert len(smtpserver.outbox) == 1
|
||||
|
||||
# send un-encrypted mail that errors
|
||||
data = str(maildata("fake-encrypted.eml", from_addr=addr, to_addr=addr))
|
||||
with pytest.raises(smtplib.SMTPDataError) as e:
|
||||
client.sendmail(addr, [addr], data)
|
||||
assert e.value.smtp_code == 523
|
||||
@@ -242,22 +242,6 @@ def test_requeue_removes_tmp_files(notifier, metadata, testaddr, caplog):
|
||||
assert queue_item.addr == testaddr
|
||||
|
||||
|
||||
def test_requeue_removes_invalid_files(notifier, metadata, testaddr, caplog):
|
||||
metadata.add_token_to_addr(testaddr, "01234")
|
||||
notifier.new_message_for_addr(testaddr, metadata)
|
||||
# empty/invalid files should be ignored
|
||||
p = notifier.queue_dir.joinpath("1203981203")
|
||||
p.touch()
|
||||
notifier2 = notifier.__class__(notifier.queue_dir)
|
||||
notifier2.requeue_persistent_queue_items()
|
||||
assert "spurious" in caplog.records[0].msg
|
||||
assert not p.exists()
|
||||
assert notifier2.retry_queues[0].qsize() == 1
|
||||
when, queue_item = notifier2.retry_queues[0].get()
|
||||
assert when <= int(time.time())
|
||||
assert queue_item.addr == testaddr
|
||||
|
||||
|
||||
def test_start_and_stop_notification_threads(notifier, testaddr):
|
||||
threads = notifier.start_notification_threads(None)
|
||||
for retry_num, threadlist in threads.items():
|
||||
|
||||
@@ -2,15 +2,8 @@ from chatmaild.metrics import main
|
||||
|
||||
|
||||
def test_main(tmp_path, capsys):
|
||||
paths = []
|
||||
for x in ("ci-asllkj", "ac_12l3kj", "qweqwe", "ci-l1k2j31l2k3"):
|
||||
p = tmp_path.joinpath(x)
|
||||
p.mkdir()
|
||||
p.joinpath("cur").mkdir()
|
||||
paths.append(p)
|
||||
|
||||
tmp_path.joinpath("nomailbox").mkdir()
|
||||
|
||||
tmp_path.joinpath(x).mkdir()
|
||||
main(tmp_path)
|
||||
out, _ = capsys.readouterr()
|
||||
d = {}
|
||||
|
||||
@@ -40,17 +40,3 @@ def test_no_mailboxes_dir(testaddr, example_config, tmp_path):
|
||||
user.set_password("someeqkjwelkqwjleqwe")
|
||||
user.set_last_login_timestamp(100000)
|
||||
assert user.get_last_login_timestamp() == 86400
|
||||
|
||||
|
||||
def test_set_get_cleartext_flag(testaddr, example_config, tmp_path):
|
||||
p = tmp_path.joinpath("a", "mailboxes")
|
||||
example_config.mailboxes_dir = p
|
||||
|
||||
user = example_config.get_user(testaddr)
|
||||
user.set_password("someeqkjwelkqwjleqwe")
|
||||
user.set_last_login_timestamp(100000)
|
||||
assert user.get_last_login_timestamp() == 86400
|
||||
|
||||
assert not user.is_incoming_cleartext_ok()
|
||||
user.allow_incoming_cleartext()
|
||||
assert user.is_incoming_cleartext_ok()
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
import socket
|
||||
|
||||
|
||||
def turn_credentials() -> str:
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client_socket:
|
||||
client_socket.connect("/run/chatmail-turn/turn.socket")
|
||||
with client_socket.makefile("rb") as file:
|
||||
return file.readline().decode("utf-8").strip()
|
||||
@@ -13,7 +13,6 @@ class User:
|
||||
self.maildir = maildir
|
||||
self.addr = addr
|
||||
self.password_path = password_path
|
||||
self.enforce_E2EE_path = maildir.joinpath("enforceE2EEincoming")
|
||||
self.uid = uid
|
||||
self.gid = gid
|
||||
|
||||
@@ -36,13 +35,6 @@ class User:
|
||||
home = str(self.maildir)
|
||||
return dict(addr=self.addr, home=home, uid=self.uid, gid=self.gid, password=pw)
|
||||
|
||||
def is_incoming_cleartext_ok(self):
|
||||
return not self.enforce_E2EE_path.exists()
|
||||
|
||||
def allow_incoming_cleartext(self):
|
||||
if self.enforce_E2EE_path.exists():
|
||||
self.enforce_E2EE_path.unlink()
|
||||
|
||||
def set_password(self, enc_password):
|
||||
"""Set the specified password for this user.
|
||||
|
||||
@@ -58,8 +50,6 @@ class User:
|
||||
if not self.addr.startswith("echo@"):
|
||||
logging.error(f"could not write password for: {self.addr}")
|
||||
raise
|
||||
if not self.addr.startswith("echo@"):
|
||||
self.enforce_E2EE_path.touch()
|
||||
|
||||
def set_last_login_timestamp(self, timestamp):
|
||||
"""Track login time with daily granularity
|
||||
|
||||
@@ -41,6 +41,3 @@ lint.select = [
|
||||
"PLE", # Pylint Error
|
||||
"PLW", # Pylint Warning
|
||||
]
|
||||
lint.ignore = [
|
||||
"PLC0415" # import-outside-top-level
|
||||
]
|
||||
|
||||
638
cmdeploy/src/cmdeploy/__init__.py
Normal file
638
cmdeploy/src/cmdeploy/__init__.py
Normal file
@@ -0,0 +1,638 @@
|
||||
"""
|
||||
Chat Mail pyinfra deploy.
|
||||
"""
|
||||
|
||||
import importlib.resources
|
||||
import io
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from chatmaild.config import Config, read_config
|
||||
from pyinfra import host
|
||||
from pyinfra.facts.files import File
|
||||
from pyinfra.facts.systemd import SystemdEnabled
|
||||
from pyinfra.operations import apt, files, pip, server, systemd
|
||||
|
||||
from .acmetool import deploy_acmetool
|
||||
|
||||
|
||||
def _build_chatmaild(dist_dir) -> None:
|
||||
dist_dir = Path(dist_dir).resolve()
|
||||
if dist_dir.exists():
|
||||
shutil.rmtree(dist_dir)
|
||||
dist_dir.mkdir()
|
||||
subprocess.check_output(
|
||||
[sys.executable, "-m", "build", "-n"]
|
||||
+ ["--sdist", "chatmaild", "--outdir", str(dist_dir)]
|
||||
)
|
||||
entries = list(dist_dir.iterdir())
|
||||
assert len(entries) == 1
|
||||
return entries[0]
|
||||
|
||||
|
||||
def remove_legacy_artifacts():
|
||||
# disable legacy doveauth-dictproxy.service
|
||||
if host.get_fact(SystemdEnabled).get("doveauth-dictproxy.service"):
|
||||
systemd.service(
|
||||
name="Disable legacy doveauth-dictproxy.service",
|
||||
service="doveauth-dictproxy.service",
|
||||
running=False,
|
||||
enabled=False,
|
||||
)
|
||||
|
||||
|
||||
def _install_remote_venv_with_chatmaild(config) -> None:
|
||||
remove_legacy_artifacts()
|
||||
dist_file = _build_chatmaild(dist_dir=Path("chatmaild/dist"))
|
||||
remote_base_dir = "/usr/local/lib/chatmaild"
|
||||
remote_dist_file = f"{remote_base_dir}/dist/{dist_file.name}"
|
||||
remote_venv_dir = f"{remote_base_dir}/venv"
|
||||
remote_chatmail_inipath = f"{remote_base_dir}/chatmail.ini"
|
||||
root_owned = dict(user="root", group="root", mode="644")
|
||||
|
||||
apt.packages(
|
||||
name="apt install python3-virtualenv",
|
||||
packages=["python3-virtualenv"],
|
||||
)
|
||||
|
||||
files.put(
|
||||
name="Upload chatmaild source package",
|
||||
src=dist_file.open("rb"),
|
||||
dest=remote_dist_file,
|
||||
create_remote_dir=True,
|
||||
**root_owned,
|
||||
)
|
||||
|
||||
files.put(
|
||||
name=f"Upload {remote_chatmail_inipath}",
|
||||
src=config._getbytefile(),
|
||||
dest=remote_chatmail_inipath,
|
||||
**root_owned,
|
||||
)
|
||||
|
||||
pip.virtualenv(
|
||||
name=f"chatmaild virtualenv {remote_venv_dir}",
|
||||
path=remote_venv_dir,
|
||||
always_copy=True,
|
||||
)
|
||||
|
||||
server.shell(
|
||||
name=f"forced pip-install {dist_file.name}",
|
||||
commands=[
|
||||
f"{remote_venv_dir}/bin/pip install --force-reinstall {remote_dist_file}"
|
||||
],
|
||||
)
|
||||
|
||||
files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("metrics.cron.j2"),
|
||||
dest="/etc/cron.d/chatmail-metrics",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
config={
|
||||
"mailboxes_dir": config.mailboxes_dir,
|
||||
"execpath": f"{remote_venv_dir}/bin/chatmail-metrics",
|
||||
},
|
||||
)
|
||||
|
||||
# install systemd units
|
||||
for fn in (
|
||||
"doveauth",
|
||||
"filtermail",
|
||||
"echobot",
|
||||
"chatmail-metadata",
|
||||
"lastlogin",
|
||||
):
|
||||
params = dict(
|
||||
execpath=f"{remote_venv_dir}/bin/{fn}",
|
||||
config_path=remote_chatmail_inipath,
|
||||
remote_venv_dir=remote_venv_dir,
|
||||
mail_domain=config.mail_domain,
|
||||
)
|
||||
source_path = importlib.resources.files(__package__).joinpath(
|
||||
"service", f"{fn}.service.f"
|
||||
)
|
||||
content = source_path.read_text().format(**params).encode()
|
||||
|
||||
files.put(
|
||||
name=f"Upload {fn}.service",
|
||||
src=io.BytesIO(content),
|
||||
dest=f"/etc/systemd/system/{fn}.service",
|
||||
**root_owned,
|
||||
)
|
||||
systemd.service(
|
||||
name=f"Setup {fn} service",
|
||||
service=f"{fn}.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=True,
|
||||
daemon_reload=True,
|
||||
)
|
||||
|
||||
|
||||
def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
|
||||
"""Configures OpenDKIM"""
|
||||
need_restart = False
|
||||
|
||||
main_config = files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("opendkim/opendkim.conf"),
|
||||
dest="/etc/opendkim.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
config={"domain_name": domain, "opendkim_selector": dkim_selector},
|
||||
)
|
||||
need_restart |= main_config.changed
|
||||
|
||||
screen_script = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath("opendkim/screen.lua"),
|
||||
dest="/etc/opendkim/screen.lua",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= screen_script.changed
|
||||
|
||||
final_script = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath("opendkim/final.lua"),
|
||||
dest="/etc/opendkim/final.lua",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= final_script.changed
|
||||
|
||||
files.directory(
|
||||
name="Add opendkim directory to /etc",
|
||||
path="/etc/opendkim",
|
||||
user="opendkim",
|
||||
group="opendkim",
|
||||
mode="750",
|
||||
present=True,
|
||||
)
|
||||
|
||||
keytable = files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("opendkim/KeyTable"),
|
||||
dest="/etc/dkimkeys/KeyTable",
|
||||
user="opendkim",
|
||||
group="opendkim",
|
||||
mode="644",
|
||||
config={"domain_name": domain, "opendkim_selector": dkim_selector},
|
||||
)
|
||||
need_restart |= keytable.changed
|
||||
|
||||
signing_table = files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("opendkim/SigningTable"),
|
||||
dest="/etc/dkimkeys/SigningTable",
|
||||
user="opendkim",
|
||||
group="opendkim",
|
||||
mode="644",
|
||||
config={"domain_name": domain, "opendkim_selector": dkim_selector},
|
||||
)
|
||||
need_restart |= signing_table.changed
|
||||
files.directory(
|
||||
name="Add opendkim socket directory to /var/spool/postfix",
|
||||
path="/var/spool/postfix/opendkim",
|
||||
user="opendkim",
|
||||
group="opendkim",
|
||||
mode="750",
|
||||
present=True,
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
name="apt install opendkim opendkim-tools",
|
||||
packages=["opendkim", "opendkim-tools"],
|
||||
)
|
||||
|
||||
if not host.get_fact(File, f"/etc/dkimkeys/{dkim_selector}.private"):
|
||||
server.shell(
|
||||
name="Generate OpenDKIM domain keys",
|
||||
commands=[
|
||||
f"opendkim-genkey -D /etc/dkimkeys -d {domain} -s {dkim_selector}"
|
||||
],
|
||||
_sudo=True,
|
||||
_sudo_user="opendkim",
|
||||
)
|
||||
|
||||
return need_restart
|
||||
|
||||
|
||||
def _install_mta_sts_daemon() -> bool:
|
||||
need_restart = False
|
||||
|
||||
config = files.put(
|
||||
name="upload postfix-mta-sts-resolver config",
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"postfix/mta-sts-daemon.yml"
|
||||
),
|
||||
dest="/etc/mta-sts-daemon.yml",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= config.changed
|
||||
|
||||
server.shell(
|
||||
name="install postfix-mta-sts-resolver with pip",
|
||||
commands=[
|
||||
"python3 -m virtualenv /usr/local/lib/postfix-mta-sts-resolver",
|
||||
"/usr/local/lib/postfix-mta-sts-resolver/bin/pip install postfix-mta-sts-resolver",
|
||||
],
|
||||
)
|
||||
|
||||
systemd_unit = files.put(
|
||||
name="upload mta-sts-daemon systemd unit",
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"postfix/mta-sts-daemon.service"
|
||||
),
|
||||
dest="/etc/systemd/system/mta-sts-daemon.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= systemd_unit.changed
|
||||
|
||||
return need_restart
|
||||
|
||||
|
||||
def _configure_postfix(config: Config, debug: bool = False) -> bool:
|
||||
"""Configures Postfix SMTP server."""
|
||||
need_restart = False
|
||||
|
||||
main_config = files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("postfix/main.cf.j2"),
|
||||
dest="/etc/postfix/main.cf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
config=config,
|
||||
disable_ipv6=config.disable_ipv6,
|
||||
)
|
||||
need_restart |= main_config.changed
|
||||
|
||||
master_config = files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("postfix/master.cf.j2"),
|
||||
dest="/etc/postfix/master.cf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
debug=debug,
|
||||
config=config,
|
||||
)
|
||||
need_restart |= master_config.changed
|
||||
|
||||
header_cleanup = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"postfix/submission_header_cleanup"
|
||||
),
|
||||
dest="/etc/postfix/submission_header_cleanup",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= header_cleanup.changed
|
||||
|
||||
# Login map that 1:1 maps email address to login.
|
||||
login_map = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath("postfix/login_map"),
|
||||
dest="/etc/postfix/login_map",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= login_map.changed
|
||||
|
||||
return need_restart
|
||||
|
||||
|
||||
def _configure_dovecot(config: Config, debug: bool = False) -> bool:
|
||||
"""Configures Dovecot IMAP server."""
|
||||
need_restart = False
|
||||
|
||||
main_config = files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("dovecot/dovecot.conf.j2"),
|
||||
dest="/etc/dovecot/dovecot.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
config=config,
|
||||
debug=debug,
|
||||
disable_ipv6=config.disable_ipv6,
|
||||
)
|
||||
need_restart |= main_config.changed
|
||||
auth_config = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath("dovecot/auth.conf"),
|
||||
dest="/etc/dovecot/auth.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= auth_config.changed
|
||||
lua_push_notification_script = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"dovecot/push_notification.lua"
|
||||
),
|
||||
dest="/etc/dovecot/push_notification.lua",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= lua_push_notification_script.changed
|
||||
|
||||
files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("dovecot/expunge.cron.j2"),
|
||||
dest="/etc/cron.d/expunge",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
config=config,
|
||||
)
|
||||
|
||||
# as per https://doc.dovecot.org/configuration_manual/os/
|
||||
# it is recommended to set the following inotify limits
|
||||
for name in ("max_user_instances", "max_user_watches"):
|
||||
key = f"fs.inotify.{name}"
|
||||
server.sysctl(
|
||||
name=f"Change {key}",
|
||||
key=key,
|
||||
value=65535,
|
||||
persist=True,
|
||||
)
|
||||
|
||||
return need_restart
|
||||
|
||||
|
||||
def _configure_nginx(config: Config, debug: bool = False) -> bool:
|
||||
"""Configures nginx HTTP server."""
|
||||
need_restart = False
|
||||
|
||||
main_config = files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("nginx/nginx.conf.j2"),
|
||||
dest="/etc/nginx/nginx.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
config={"domain_name": config.mail_domain},
|
||||
disable_ipv6=config.disable_ipv6,
|
||||
)
|
||||
need_restart |= main_config.changed
|
||||
|
||||
autoconfig = files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("nginx/autoconfig.xml.j2"),
|
||||
dest="/var/www/html/.well-known/autoconfig/mail/config-v1.1.xml",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
config={"domain_name": config.mail_domain},
|
||||
)
|
||||
need_restart |= autoconfig.changed
|
||||
|
||||
mta_sts_config = files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("nginx/mta-sts.txt.j2"),
|
||||
dest="/var/www/html/.well-known/mta-sts.txt",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
config={"domain_name": config.mail_domain},
|
||||
)
|
||||
need_restart |= mta_sts_config.changed
|
||||
|
||||
# install CGI newemail script
|
||||
#
|
||||
cgi_dir = "/usr/lib/cgi-bin"
|
||||
files.directory(
|
||||
name=f"Ensure {cgi_dir} exists",
|
||||
path=cgi_dir,
|
||||
user="root",
|
||||
group="root",
|
||||
)
|
||||
|
||||
files.put(
|
||||
name="Upload cgi newemail.py script",
|
||||
src=importlib.resources.files("chatmaild").joinpath("newemail.py").open("rb"),
|
||||
dest=f"{cgi_dir}/newemail.py",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="755",
|
||||
)
|
||||
|
||||
return need_restart
|
||||
|
||||
|
||||
def _remove_rspamd() -> None:
|
||||
"""Remove rspamd"""
|
||||
apt.packages(name="Remove rspamd", packages="rspamd", present=False)
|
||||
|
||||
|
||||
def check_config(config):
|
||||
mail_domain = config.mail_domain
|
||||
if mail_domain != "testrun.org" and not mail_domain.endswith(".testrun.org"):
|
||||
blocked_words = "merlinux schmieder testrun.org".split()
|
||||
for key in config.__dict__:
|
||||
value = config.__dict__[key]
|
||||
if key.startswith("privacy") and any(
|
||||
x in str(value) for x in blocked_words
|
||||
):
|
||||
raise ValueError(
|
||||
f"please set your own privacy contacts/addresses in {config._inipath}"
|
||||
)
|
||||
return config
|
||||
|
||||
|
||||
def deploy_chatmail(config_path: Path) -> None:
|
||||
"""Deploy a chat-mail instance.
|
||||
|
||||
:param config_path: path to chatmail.ini
|
||||
"""
|
||||
config = read_config(config_path)
|
||||
check_config(config)
|
||||
mail_domain = config.mail_domain
|
||||
|
||||
from .www import build_webpages
|
||||
|
||||
server.group(name="Create vmail group", group="vmail", system=True)
|
||||
server.user(name="Create vmail user", user="vmail", group="vmail", system=True)
|
||||
server.user(name="Create filtermail user", user="filtermail", system=True)
|
||||
server.group(name="Create opendkim group", group="opendkim", system=True)
|
||||
server.user(
|
||||
name="Create opendkim user",
|
||||
user="opendkim",
|
||||
groups=["opendkim"],
|
||||
system=True,
|
||||
)
|
||||
server.user(
|
||||
name="Add postfix user to opendkim group for socket access",
|
||||
user="postfix",
|
||||
groups=["opendkim"],
|
||||
system=True,
|
||||
)
|
||||
server.user(name="Create echobot user", user="echobot", system=True)
|
||||
|
||||
# Add our OBS repository for dovecot_no_delay
|
||||
files.put(
|
||||
name="Add Deltachat OBS GPG key to apt keyring",
|
||||
src=importlib.resources.files(__package__).joinpath("obs-home-deltachat.gpg"),
|
||||
dest="/etc/apt/keyrings/obs-home-deltachat.gpg",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
|
||||
files.line(
|
||||
name="Add DeltaChat OBS home repository to sources.list",
|
||||
path="/etc/apt/sources.list",
|
||||
line="deb [signed-by=/etc/apt/keyrings/obs-home-deltachat.gpg] https://download.opensuse.org/repositories/home:/deltachat/Debian_12/ ./",
|
||||
escape_regex_characters=True,
|
||||
ensure_newline=True,
|
||||
)
|
||||
|
||||
apt.update(name="apt update", cache_time=24 * 3600)
|
||||
apt.upgrade(name="upgrade apt packages", auto_remove=True)
|
||||
|
||||
apt.packages(
|
||||
name="Install rsync",
|
||||
packages=["rsync"],
|
||||
)
|
||||
|
||||
# Run local DNS resolver `unbound`.
|
||||
# `resolvconf` takes care of setting up /etc/resolv.conf
|
||||
# to use 127.0.0.1 as the resolver.
|
||||
apt.packages(
|
||||
name="Install unbound",
|
||||
packages=["unbound", "unbound-anchor", "dnsutils"],
|
||||
)
|
||||
server.shell(
|
||||
name="Generate root keys for validating DNSSEC",
|
||||
commands=[
|
||||
"unbound-anchor -a /var/lib/unbound/root.key || true",
|
||||
"systemctl reset-failed unbound.service",
|
||||
],
|
||||
)
|
||||
systemd.service(
|
||||
name="Start and enable unbound",
|
||||
service="unbound.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
)
|
||||
|
||||
# Deploy acmetool to have TLS certificates.
|
||||
deploy_acmetool(
|
||||
domains=[mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"],
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
# required for setfacl for echobot
|
||||
name="Install acl",
|
||||
packages="acl",
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
name="Install Postfix",
|
||||
packages="postfix",
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
name="Install Dovecot",
|
||||
packages=["dovecot-imapd", "dovecot-lmtpd"],
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
name="Install nginx",
|
||||
packages=["nginx", "libnginx-mod-stream"],
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
name="Install fcgiwrap",
|
||||
packages=["fcgiwrap"],
|
||||
)
|
||||
|
||||
www_path = importlib.resources.files(__package__).joinpath("../../../www").resolve()
|
||||
|
||||
build_dir = www_path.joinpath("build")
|
||||
src_dir = www_path.joinpath("src")
|
||||
build_webpages(src_dir, build_dir, config)
|
||||
files.rsync(f"{build_dir}/", "/var/www/html", flags=["-avz"])
|
||||
|
||||
_install_remote_venv_with_chatmaild(config)
|
||||
debug = False
|
||||
dovecot_need_restart = _configure_dovecot(config, debug=debug)
|
||||
postfix_need_restart = _configure_postfix(config, debug=debug)
|
||||
mta_sts_need_restart = _install_mta_sts_daemon()
|
||||
nginx_need_restart = _configure_nginx(config)
|
||||
|
||||
_remove_rspamd()
|
||||
opendkim_need_restart = _configure_opendkim(mail_domain, "opendkim")
|
||||
|
||||
systemd.service(
|
||||
name="Start and enable OpenDKIM",
|
||||
service="opendkim.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=opendkim_need_restart,
|
||||
)
|
||||
|
||||
systemd.service(
|
||||
name="Start and enable MTA-STS daemon",
|
||||
service="mta-sts-daemon.service",
|
||||
daemon_reload=True,
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=mta_sts_need_restart,
|
||||
)
|
||||
|
||||
# Dovecot should be started before Postfix
|
||||
# because it creates authentication socket
|
||||
# required by Postfix.
|
||||
systemd.service(
|
||||
name="Start and enable Dovecot",
|
||||
service="dovecot.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=dovecot_need_restart,
|
||||
)
|
||||
|
||||
systemd.service(
|
||||
name="Start and enable Postfix",
|
||||
service="postfix.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=postfix_need_restart,
|
||||
)
|
||||
|
||||
systemd.service(
|
||||
name="Start and enable nginx",
|
||||
service="nginx.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=nginx_need_restart,
|
||||
)
|
||||
|
||||
# This file is used by auth proxy.
|
||||
# https://wiki.debian.org/EtcMailName
|
||||
server.shell(
|
||||
name="Setup /etc/mailname",
|
||||
commands=[f"echo {mail_domain} >/etc/mailname; chmod 644 /etc/mailname"],
|
||||
)
|
||||
|
||||
journald_conf = files.put(
|
||||
name="Configure journald",
|
||||
src=importlib.resources.files(__package__).joinpath("journald.conf"),
|
||||
dest="/etc/systemd/journald.conf",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
systemd.service(
|
||||
name="Start and enable journald",
|
||||
service="systemd-journald.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=journald_conf.changed,
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
name="Ensure cron is installed",
|
||||
packages=["cron"],
|
||||
)
|
||||
@@ -1,128 +1,75 @@
|
||||
import importlib.resources
|
||||
|
||||
from pyinfra import host
|
||||
from pyinfra.facts.systemd import SystemdStatus
|
||||
from pyinfra.operations import apt, files, server, systemd
|
||||
|
||||
from ..basedeploy import Deployer
|
||||
|
||||
def deploy_acmetool(email="", domains=[]):
|
||||
"""Deploy acmetool."""
|
||||
apt.packages(
|
||||
name="Install acmetool",
|
||||
packages=["acmetool"],
|
||||
)
|
||||
|
||||
class AcmetoolDeployer(Deployer):
|
||||
def __init__(self, email, domains):
|
||||
self.domains = domains
|
||||
self.email = email
|
||||
self.need_restart_redirector = False
|
||||
self.need_restart_reconcile_service = False
|
||||
self.need_restart_reconcile_timer = False
|
||||
files.put(
|
||||
src=importlib.resources.files(__package__).joinpath("acmetool.cron").open("rb"),
|
||||
dest="/etc/cron.d/acmetool",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
|
||||
def install(self):
|
||||
apt.packages(
|
||||
name="Install acmetool",
|
||||
packages=["acmetool"],
|
||||
)
|
||||
files.put(
|
||||
src=importlib.resources.files(__package__).joinpath("acmetool.hook").open("rb"),
|
||||
dest="/usr/lib/acme/hooks/nginx",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="744",
|
||||
)
|
||||
|
||||
files.file(
|
||||
name="Remove old acmetool cronjob, it is replaced with systemd timer.",
|
||||
path="/etc/cron.d/acmetool",
|
||||
present=False,
|
||||
)
|
||||
files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("response-file.yaml.j2"),
|
||||
dest="/var/lib/acme/conf/responses",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
email=email,
|
||||
)
|
||||
|
||||
files.put(
|
||||
name="Install acmetool hook.",
|
||||
src=importlib.resources.files(__package__)
|
||||
.joinpath("acmetool.hook")
|
||||
.open("rb"),
|
||||
dest="/etc/acme/hooks/nginx",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="755",
|
||||
)
|
||||
files.file(
|
||||
name="Remove acmetool hook from the wrong location where it was previously installed.",
|
||||
path="/usr/lib/acme/hooks/nginx",
|
||||
present=False,
|
||||
)
|
||||
files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("target.yaml.j2"),
|
||||
dest="/var/lib/acme/conf/target",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
|
||||
def configure(self):
|
||||
files.template(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"response-file.yaml.j2"
|
||||
),
|
||||
dest="/var/lib/acme/conf/responses",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
email=self.email,
|
||||
)
|
||||
|
||||
files.template(
|
||||
src=importlib.resources.files(__package__).joinpath("target.yaml.j2"),
|
||||
dest="/var/lib/acme/conf/target",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
|
||||
service_file = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"acmetool-redirector.service"
|
||||
),
|
||||
dest="/etc/systemd/system/acmetool-redirector.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart_redirector = service_file.changed
|
||||
|
||||
reconcile_service_file = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"acmetool-reconcile.service"
|
||||
),
|
||||
dest="/etc/systemd/system/acmetool-reconcile.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart_reconcile_service = reconcile_service_file.changed
|
||||
|
||||
reconcile_timer_file = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"acmetool-reconcile.timer"
|
||||
),
|
||||
dest="/etc/systemd/system/acmetool-reconcile.timer",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
self.need_restart_reconcile_timer = reconcile_timer_file.changed
|
||||
|
||||
def activate(self):
|
||||
service_file = files.put(
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"acmetool-redirector.service"
|
||||
),
|
||||
dest="/etc/systemd/system/acmetool-redirector.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
if host.get_fact(SystemdStatus).get("nginx.service"):
|
||||
systemd.service(
|
||||
name="Setup acmetool-redirector service",
|
||||
service="acmetool-redirector.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=self.need_restart_redirector,
|
||||
)
|
||||
self.need_restart_redirector = False
|
||||
|
||||
systemd.service(
|
||||
name="Setup acmetool-reconcile service",
|
||||
service="acmetool-reconcile.service",
|
||||
name="Stop nginx service to free port 80",
|
||||
service="nginx",
|
||||
running=False,
|
||||
enabled=False,
|
||||
daemon_reload=self.need_restart_reconcile_service,
|
||||
)
|
||||
self.need_restart_reconcile_service = False
|
||||
|
||||
systemd.service(
|
||||
name="Setup acmetool-reconcile timer",
|
||||
service="acmetool-reconcile.timer",
|
||||
running=True,
|
||||
enabled=True,
|
||||
daemon_reload=self.need_restart_reconcile_timer,
|
||||
)
|
||||
self.need_restart_reconcile_timer = False
|
||||
systemd.service(
|
||||
name="Setup acmetool-redirector service",
|
||||
service="acmetool-redirector.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=service_file.changed,
|
||||
)
|
||||
|
||||
server.shell(
|
||||
name=f"Request certificate for: {', '.join(self.domains)}",
|
||||
commands=[f"acmetool want --xlog.severity=debug {' '.join(self.domains)}"],
|
||||
)
|
||||
server.shell(
|
||||
name=f"Request certificate for: { ', '.join(domains) }",
|
||||
commands=[f"acmetool want --xlog.severity=debug { ' '.join(domains)}"],
|
||||
)
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
[Unit]
|
||||
Description=Renew TLS certificates with acmetool
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/bin/acmetool --batch reconcile
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
[Unit]
|
||||
Description=Renew TLS certificates with acmetool
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*-*-* 16:20:00
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
4
cmdeploy/src/cmdeploy/acmetool/acmetool.cron
Normal file
4
cmdeploy/src/cmdeploy/acmetool/acmetool.cron
Normal file
@@ -0,0 +1,4 @@
|
||||
SHELL=/bin/sh
|
||||
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
|
||||
MAILTO=root
|
||||
20 16 * * * root /usr/bin/acmetool --batch reconcile && systemctl reload dovecot && systemctl reload postfix && systemctl reload nginx
|
||||
@@ -1,2 +1,2 @@
|
||||
"acme-enter-email": "{{ email }}"
|
||||
"acme-agreement:https://letsencrypt.org/documents/LE-SA-v1.5-February-24-2025.pdf": true
|
||||
"acme-agreement:https://letsencrypt.org/documents/LE-SA-v1.4-April-3-2024.pdf": true
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
request:
|
||||
provider: https://acme-v02.api.letsencrypt.org/directory
|
||||
key:
|
||||
type: ecdsa
|
||||
ecdsa-curve: nistp256
|
||||
type: rsa
|
||||
challenge:
|
||||
webroot-paths:
|
||||
- /var/www/html/.well-known/acme-challenge
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
import os
|
||||
|
||||
from pyinfra.operations import server
|
||||
|
||||
|
||||
class Deployment:
|
||||
def install(self, deployer):
|
||||
# optional 'required_users' contains a list of (user, group, secondary-group-list) tuples.
|
||||
# If the group is None, no group is created corresponding to that user.
|
||||
# If the secondary group list is not None, all listed groups are created as well.
|
||||
required_users = getattr(deployer, "required_users", [])
|
||||
for user, group, groups in required_users:
|
||||
if group is not None:
|
||||
server.group(
|
||||
name="Create {} group".format(group), group=group, system=True
|
||||
)
|
||||
if groups is not None:
|
||||
for group2 in groups:
|
||||
server.group(
|
||||
name="Create {} group".format(group2), group=group2, system=True
|
||||
)
|
||||
server.user(
|
||||
name="Create {} user".format(user),
|
||||
user=user,
|
||||
group=group,
|
||||
groups=groups,
|
||||
system=True,
|
||||
)
|
||||
|
||||
deployer.install()
|
||||
|
||||
def configure(self, deployer):
|
||||
deployer.configure()
|
||||
|
||||
def activate(self, deployer):
|
||||
deployer.activate()
|
||||
|
||||
def perform_stages(self, deployers):
|
||||
default_stages = "install,configure,activate"
|
||||
stages = os.getenv("CMDEPLOY_STAGES", default_stages).split(",")
|
||||
|
||||
for stage in stages:
|
||||
for deployer in deployers:
|
||||
getattr(self, stage)(deployer)
|
||||
|
||||
|
||||
class Deployer:
|
||||
need_restart = False
|
||||
|
||||
def install(self):
|
||||
pass
|
||||
|
||||
def configure(self):
|
||||
pass
|
||||
|
||||
def activate(self):
|
||||
pass
|
||||
@@ -16,7 +16,7 @@ www.{{ mail_domain }}. CNAME {{ mail_domain }}.
|
||||
;
|
||||
; Recommended DNS entries for interoperability and security-hardening
|
||||
;
|
||||
{{ mail_domain }}. TXT "v=spf1 a ~all"
|
||||
{{ mail_domain }}. TXT "v=spf1 a:{{ mail_domain }} ~all"
|
||||
_dmarc.{{ mail_domain }}. TXT "v=DMARC1;p=reject;adkim=s;aspf=s"
|
||||
|
||||
{% if acme_account_url %}
|
||||
|
||||
@@ -19,7 +19,7 @@ from packaging import version
|
||||
from termcolor import colored
|
||||
|
||||
from . import dns, remote
|
||||
from .sshexec import LocalExec, SSHExec
|
||||
from .sshexec import SSHExec
|
||||
|
||||
#
|
||||
# cmdeploy sub commands and options
|
||||
@@ -32,30 +32,17 @@ def init_cmd_options(parser):
|
||||
action="store",
|
||||
help="fully qualified DNS domain name for your chatmail instance",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--force",
|
||||
dest="recreate_ini",
|
||||
action="store_true",
|
||||
help="force reacreate ini file",
|
||||
)
|
||||
|
||||
|
||||
def init_cmd(args, out):
|
||||
"""Initialize chatmail config file."""
|
||||
mail_domain = args.chatmail_domain
|
||||
inipath = args.inipath
|
||||
if args.inipath.exists():
|
||||
if not args.recreate_ini:
|
||||
print(f"[WARNING] Path exists, not modifying: {inipath}")
|
||||
return 1
|
||||
else:
|
||||
print(
|
||||
f"[WARNING] Force argument was provided, deleting config file: {inipath}"
|
||||
)
|
||||
inipath.unlink()
|
||||
|
||||
write_initial_config(inipath, mail_domain, overrides={})
|
||||
out.green(f"created config file for {mail_domain} in {inipath}")
|
||||
print(f"Path exists, not modifying: {args.inipath}")
|
||||
return 1
|
||||
else:
|
||||
write_initial_config(args.inipath, mail_domain, overrides={})
|
||||
out.green(f"created config file for {mail_domain} in {args.inipath}")
|
||||
|
||||
|
||||
def run_cmd_options(parser):
|
||||
@@ -65,69 +52,34 @@ def run_cmd_options(parser):
|
||||
action="store_true",
|
||||
help="don't actually modify the server",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--disable-mail",
|
||||
dest="disable_mail",
|
||||
action="store_true",
|
||||
help="install/upgrade the server, but disable postfix & dovecot for now",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--skip-dns-check",
|
||||
dest="dns_check_disabled",
|
||||
action="store_true",
|
||||
help="disable checks nslookup for dns",
|
||||
)
|
||||
add_ssh_host_option(parser)
|
||||
|
||||
|
||||
def run_cmd(args, out):
|
||||
"""Deploy chatmail services on the remote server."""
|
||||
|
||||
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
|
||||
sshexec = get_sshexec(ssh_host)
|
||||
require_iroh = args.config.enable_iroh_relay
|
||||
if not args.dns_check_disabled:
|
||||
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
|
||||
if not dns.check_initial_remote_data(remote_data, print=out.red):
|
||||
return 1
|
||||
sshexec = args.get_sshexec()
|
||||
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
|
||||
if not dns.check_initial_remote_data(remote_data, print=out.red):
|
||||
return 1
|
||||
|
||||
env = os.environ.copy()
|
||||
env["CHATMAIL_INI"] = args.inipath
|
||||
env["CHATMAIL_DISABLE_MAIL"] = "True" if args.disable_mail else ""
|
||||
env["CHATMAIL_REQUIRE_IROH"] = "True" if require_iroh else ""
|
||||
deploy_path = importlib.resources.files(__package__).joinpath("run.py").resolve()
|
||||
deploy_path = importlib.resources.files(__package__).joinpath("deploy.py").resolve()
|
||||
pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
|
||||
|
||||
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
|
||||
if ssh_host in ["localhost", "@docker"]:
|
||||
cmd = f"{pyinf} @local {deploy_path} -y"
|
||||
|
||||
cmd = f"{pyinf} --ssh-user root {args.config.mail_domain} {deploy_path} -y"
|
||||
if version.parse(pyinfra.__version__) < version.parse("3"):
|
||||
out.red("Please re-run scripts/initenv.sh to update pyinfra to version 3.")
|
||||
return 1
|
||||
|
||||
try:
|
||||
retcode = out.check_call(cmd, env=env)
|
||||
if retcode == 0:
|
||||
if not args.disable_mail:
|
||||
print("\nYou can try out the relay by talking to this echo bot: ")
|
||||
sshexec = SSHExec(args.config.mail_domain, verbose=args.verbose)
|
||||
print(
|
||||
sshexec(
|
||||
call=remote.rshell.shell,
|
||||
kwargs=dict(command="cat /var/lib/echobot/invite-link.txt"),
|
||||
)
|
||||
)
|
||||
out.green("Deploy completed, call `cmdeploy dns` next.")
|
||||
elif not remote_data["acme_account_url"]:
|
||||
out.red("Deploy completed but letsencrypt not configured")
|
||||
out.red("Run 'cmdeploy run' again")
|
||||
retcode = 0
|
||||
else:
|
||||
out.red("Deploy failed")
|
||||
except subprocess.CalledProcessError:
|
||||
retcode = out.check_call(cmd, env=env)
|
||||
if retcode == 0:
|
||||
out.green("Deploy completed, call `cmdeploy dns` next.")
|
||||
elif not remote_data["acme_account_url"]:
|
||||
out.red("Deploy completed but letsencrypt not configured")
|
||||
out.red("Run 'cmdeploy run' again")
|
||||
retcode = 0
|
||||
else:
|
||||
out.red("Deploy failed")
|
||||
retcode = 1
|
||||
return retcode
|
||||
|
||||
|
||||
@@ -139,13 +91,11 @@ def dns_cmd_options(parser):
|
||||
default=None,
|
||||
help="write out a zonefile",
|
||||
)
|
||||
add_ssh_host_option(parser)
|
||||
|
||||
|
||||
def dns_cmd(args, out):
|
||||
"""Check DNS entries and optionally generate dns zone file."""
|
||||
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
|
||||
sshexec = get_sshexec(ssh_host, verbose=args.verbose)
|
||||
sshexec = args.get_sshexec()
|
||||
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
|
||||
if not remote_data:
|
||||
return 1
|
||||
@@ -171,15 +121,10 @@ def dns_cmd(args, out):
|
||||
return retcode
|
||||
|
||||
|
||||
def status_cmd_options(parser):
|
||||
add_ssh_host_option(parser)
|
||||
|
||||
|
||||
def status_cmd(args, out):
|
||||
"""Display status for online chatmail instance."""
|
||||
|
||||
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
|
||||
sshexec = get_sshexec(ssh_host, verbose=args.verbose)
|
||||
sshexec = args.get_sshexec()
|
||||
|
||||
out.green(f"chatmail domain: {args.config.mail_domain}")
|
||||
if args.config.privacy_mail:
|
||||
@@ -238,12 +183,7 @@ def fmt_cmd_options(parser):
|
||||
def fmt_cmd(args, out):
|
||||
"""Run formattting fixes on all chatmail source code."""
|
||||
|
||||
chatmaild_dir = importlib.resources.files("chatmaild").resolve()
|
||||
cmdeploy_dir = chatmaild_dir.joinpath(
|
||||
"..", "..", "..", "cmdeploy", "src", "cmdeploy"
|
||||
).resolve()
|
||||
sources = [str(chatmaild_dir), str(cmdeploy_dir)]
|
||||
|
||||
sources = [str(importlib.resources.files(x)) for x in ("chatmaild", "cmdeploy")]
|
||||
format_args = [shutil.which("ruff"), "format"]
|
||||
check_args = [shutil.which("ruff"), "check"]
|
||||
|
||||
@@ -309,15 +249,6 @@ class Out:
|
||||
return proc.returncode
|
||||
|
||||
|
||||
def add_ssh_host_option(parser):
|
||||
parser.add_argument(
|
||||
"--ssh-host",
|
||||
dest="ssh_host",
|
||||
help="Run commands on 'localhost', via '@docker', or on a specific SSH host "
|
||||
"instead of chatmail.ini's mail_domain.",
|
||||
)
|
||||
|
||||
|
||||
def add_config_option(parser):
|
||||
parser.add_argument(
|
||||
"--config",
|
||||
@@ -373,16 +304,6 @@ def get_parser():
|
||||
return parser
|
||||
|
||||
|
||||
def get_sshexec(ssh_host: str, verbose=True):
|
||||
if ssh_host in ["localhost", "@local"]:
|
||||
return LocalExec(verbose, docker=False)
|
||||
elif ssh_host == "@docker":
|
||||
return LocalExec(verbose, docker=True)
|
||||
if verbose:
|
||||
print(f"[ssh] login to {ssh_host}")
|
||||
return SSHExec(ssh_host, verbose=verbose)
|
||||
|
||||
|
||||
def main(args=None):
|
||||
"""Provide main entry point for 'cmdeploy' CLI invocation."""
|
||||
parser = get_parser()
|
||||
@@ -390,6 +311,12 @@ def main(args=None):
|
||||
if not hasattr(args, "func"):
|
||||
return parser.parse_args(["-h"])
|
||||
|
||||
def get_sshexec():
|
||||
print(f"[ssh] login to {args.config.mail_domain}")
|
||||
return SSHExec(args.config.mail_domain, verbose=args.verbose)
|
||||
|
||||
args.get_sshexec = get_sshexec
|
||||
|
||||
out = Out()
|
||||
kwargs = {}
|
||||
if args.func.__name__ not in ("init_cmd", "fmt_cmd"):
|
||||
|
||||
19
cmdeploy/src/cmdeploy/deploy.py
Normal file
19
cmdeploy/src/cmdeploy/deploy.py
Normal file
@@ -0,0 +1,19 @@
|
||||
import importlib.resources
|
||||
import os
|
||||
|
||||
import pyinfra
|
||||
|
||||
from cmdeploy import deploy_chatmail
|
||||
|
||||
|
||||
def main():
|
||||
config_path = os.getenv(
|
||||
"CHATMAIL_INI",
|
||||
importlib.resources.files("cmdeploy").joinpath("../../../chatmail.ini"),
|
||||
)
|
||||
|
||||
deploy_chatmail(config_path)
|
||||
|
||||
|
||||
if pyinfra.is_cli:
|
||||
main()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -12,7 +12,7 @@ def get_initial_remote_data(sshexec, mail_domain):
|
||||
)
|
||||
|
||||
|
||||
def check_initial_remote_data(remote_data, *, print=print):
|
||||
def check_initial_remote_data(remote_data, print=print):
|
||||
mail_domain = remote_data["mail_domain"]
|
||||
if not remote_data["A"] and not remote_data["AAAA"]:
|
||||
print(f"Missing A and/or AAAA DNS records for {mail_domain}!")
|
||||
@@ -29,7 +29,7 @@ def check_initial_remote_data(remote_data, *, print=print):
|
||||
def get_filled_zone_file(remote_data):
|
||||
sts_id = remote_data.get("sts_id")
|
||||
if not sts_id:
|
||||
remote_data["sts_id"] = datetime.datetime.now().strftime("%Y%m%d%H%M")
|
||||
sts_id = datetime.datetime.now().strftime("%Y%m%d%H%M")
|
||||
|
||||
template = importlib.resources.files(__package__).joinpath("chatmail.zone.j2")
|
||||
content = template.read_text()
|
||||
@@ -46,26 +46,19 @@ def check_full_zone(sshexec, remote_data, out, zonefile) -> int:
|
||||
|
||||
required_diff, recommended_diff = sshexec.logged(
|
||||
remote.rdns.check_zonefile,
|
||||
kwargs=dict(zonefile=zonefile, verbose=False),
|
||||
kwargs=dict(zonefile=zonefile, mail_domain=remote_data["mail_domain"]),
|
||||
)
|
||||
|
||||
returncode = 0
|
||||
if required_diff:
|
||||
out.red("Please set required DNS entries at your DNS provider:\n")
|
||||
for line in required_diff:
|
||||
out(line)
|
||||
out("")
|
||||
returncode = 1
|
||||
if remote_data.get("dkim_entry") in required_diff:
|
||||
out(
|
||||
"If the DKIM entry above does not work with your DNS provider, you can try this one:\n"
|
||||
)
|
||||
out(remote_data.get("web_dkim_entry") + "\n")
|
||||
if recommended_diff:
|
||||
return 1
|
||||
elif recommended_diff:
|
||||
out("WARNING: these recommended DNS entries are not set:\n")
|
||||
for line in recommended_diff:
|
||||
out(line)
|
||||
return 0
|
||||
|
||||
if not (recommended_diff or required_diff):
|
||||
out.green("Great! All your DNS entries are verified and correct.")
|
||||
return returncode
|
||||
out.green("Great! All your DNS entries are verified and correct.")
|
||||
return 0
|
||||
|
||||
@@ -70,12 +70,6 @@ userdb {
|
||||
# Mailboxes are stored in the "mail" directory of the vmail user home.
|
||||
mail_location = maildir:{{ config.mailboxes_dir }}/%u
|
||||
|
||||
# index/cache files are not very useful for chatmail relay operations
|
||||
# but it's not clear how to disable them completely.
|
||||
# According to https://doc.dovecot.org/2.3/settings/advanced/#core_setting-mail_cache_max_size
|
||||
# if the cache file becomes larger than the specified size, it is truncated by dovecot
|
||||
mail_cache_max_size = 500K
|
||||
|
||||
namespace inbox {
|
||||
inbox = yes
|
||||
|
||||
@@ -147,7 +141,7 @@ plugin {
|
||||
# for now we define static quota-rules for all users
|
||||
quota = maildir:User quota
|
||||
quota_rule = *:storage={{ config.max_mailbox_size }}
|
||||
quota_max_mail_size={{ config.max_message_size }}
|
||||
quota_max_mail_size=30M
|
||||
quota_grace = 0
|
||||
# quota_over_flag_value = TRUE
|
||||
}
|
||||
@@ -183,53 +177,28 @@ service auth-worker {
|
||||
}
|
||||
|
||||
service imap-login {
|
||||
# High-performance mode as described in
|
||||
# <https://doc.dovecot.org/2.3/admin_manual/login_processes/#high-performance-mode>
|
||||
#
|
||||
# So-called high-security mode described in
|
||||
# <https://doc.dovecot.org/2.3/admin_manual/login_processes/#high-security-mode>
|
||||
# and enabled by default with `service_count = 1` starts one process per connection
|
||||
# and has problems logging in thousands of users after Dovecot restart.
|
||||
service_count = 0
|
||||
# High-security mode.
|
||||
# Each process serves a single connection and exits afterwards.
|
||||
# This is the default, but we set it explicitly to be sure.
|
||||
# See <https://doc.dovecot.org/admin_manual/login_processes/#high-security-mode> for details.
|
||||
service_count = 1
|
||||
|
||||
# Increase virtual memory size limit.
|
||||
# Since imap-login processes handle TLS connections
|
||||
# even after logging users in
|
||||
# and many connections are handled by each process,
|
||||
# memory size limit should be increased.
|
||||
# Inrease the number of simultaneous connections.
|
||||
#
|
||||
# Otherwise the whole process eventually dies
|
||||
# with an error similar to
|
||||
# imap-login: Fatal: master: service(imap-login):
|
||||
# child 1422951 returned error 83
|
||||
# (Out of memory (service imap-login { vsz_limit=256 MB },
|
||||
# you may need to increase it)
|
||||
# and takes down all its TLS connections at once.
|
||||
vsz_limit = 1G
|
||||
# As of Dovecot 2.3.19.1 the default is 100 processes.
|
||||
# Combined with `service_count = 1` it means only 100 connections
|
||||
# can be handled simultaneously.
|
||||
process_limit = 10000
|
||||
|
||||
# Avoid startup latency for new connections.
|
||||
#
|
||||
# Should be set to at least the number of CPU cores
|
||||
# according to the documentation.
|
||||
process_min_avail = 10
|
||||
}
|
||||
|
||||
service anvil {
|
||||
# We are disabling anvil penalty on failed login attempts
|
||||
# because it can only detect brute forcing by IP address
|
||||
# not by username. As the correct IP address is not handed
|
||||
# to dovecot anyway, it is more of hindrance than of use.
|
||||
# See <https://www.dovecot.org/list/dovecot/2012-May/135485.html> for details.
|
||||
unix_listener anvil-auth-penalty {
|
||||
mode = 0
|
||||
}
|
||||
}
|
||||
|
||||
ssl = required
|
||||
ssl_cert = </var/lib/acme/live/{{ config.mail_domain }}/fullchain
|
||||
ssl_key = </var/lib/acme/live/{{ config.mail_domain }}/privkey
|
||||
ssl_dh = </usr/share/dovecot/dh.pem
|
||||
ssl_min_protocol = TLSv1.3
|
||||
ssl_min_protocol = TLSv1.2
|
||||
ssl_prefer_server_ciphers = yes
|
||||
|
||||
|
||||
|
||||
12
cmdeploy/src/cmdeploy/dovecot/expunge.cron.j2
Normal file
12
cmdeploy/src/cmdeploy/dovecot/expunge.cron.j2
Normal file
@@ -0,0 +1,12 @@
|
||||
# delete all mails after {{ config.delete_mails_after }} days, in the Inbox
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/cur/*' -mtime +{{ config.delete_mails_after }} -type f -delete
|
||||
# or in any IMAP subfolder
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/.*/cur/*' -mtime +{{ config.delete_mails_after }} -type f -delete
|
||||
# even if they are unseen
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/new/*' -mtime +{{ config.delete_mails_after }} -type f -delete
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/.*/new/*' -mtime +{{ config.delete_mails_after }} -type f -delete
|
||||
# or only temporary (but then they shouldn't be around after {{ config.delete_mails_after }} days anyway).
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/tmp/*' -mtime +{{ config.delete_mails_after }} -type f -delete
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/.*/tmp/*' -mtime +{{ config.delete_mails_after }} -type f -delete
|
||||
3 0 * * * vmail find {{ config.mailboxes_dir }} -name 'maildirsize' -type f -delete
|
||||
4 0 * * * vmail /usr/local/lib/chatmaild/venv/bin/delete_inactive_users /usr/local/lib/chatmaild/chatmail.ini
|
||||
@@ -2,6 +2,15 @@ function dovecot_lua_notify_begin_txn(user)
|
||||
return user
|
||||
end
|
||||
|
||||
function contains(v, needle)
|
||||
for _, keyword in ipairs(v) do
|
||||
if keyword == needle then
|
||||
return true
|
||||
end
|
||||
end
|
||||
return false
|
||||
end
|
||||
|
||||
function dovecot_lua_notify_event_message_new(user, event)
|
||||
local mbox = user:mailbox(event.mailbox)
|
||||
mbox:sync()
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
[Unit]
|
||||
Description=Iroh relay
|
||||
|
||||
[Service]
|
||||
ExecStart=/usr/local/bin/iroh-relay --config-path /etc/iroh-relay.toml
|
||||
Restart=on-failure
|
||||
RestartSec=5s
|
||||
User=iroh
|
||||
Group=iroh
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,11 +0,0 @@
|
||||
enable_relay = true
|
||||
http_bind_addr = "[::]:3340"
|
||||
|
||||
# Disable built-in STUN server in iroh-relay 0.35
|
||||
# as we deploy our own TURN server instead.
|
||||
# STUN server is going to be removed in iroh-relay 1.0
|
||||
# and this line can be removed after upgrade.
|
||||
enable_stun = false
|
||||
|
||||
enable_metrics = false
|
||||
metrics_bind_addr = "127.0.0.1:9092"
|
||||
@@ -1,64 +0,0 @@
|
||||
counter delivered_mail
|
||||
/saved mail to INBOX$/ {
|
||||
delivered_mail++
|
||||
}
|
||||
|
||||
counter quota_exceeded
|
||||
/Quota exceeded \(mailbox for user is full\)$/ {
|
||||
quota_exceeded++
|
||||
}
|
||||
|
||||
# Essentially the number of outgoing messages.
|
||||
counter dkim_signed
|
||||
/DKIM-Signature field added/ {
|
||||
dkim_signed++
|
||||
}
|
||||
|
||||
counter created_accounts
|
||||
counter created_ci_accounts
|
||||
counter created_nonci_accounts
|
||||
|
||||
/: Created address: (?P<addr>.*)$/ {
|
||||
created_accounts++
|
||||
|
||||
$addr =~ /ci-/ {
|
||||
created_ci_accounts++
|
||||
} else {
|
||||
created_nonci_accounts++
|
||||
}
|
||||
}
|
||||
|
||||
counter postfix_timeouts
|
||||
/timeout after DATA/ {
|
||||
postfix_timeouts++
|
||||
}
|
||||
|
||||
counter postfix_noqueue
|
||||
/postfix\/.*NOQUEUE/ {
|
||||
postfix_noqueue++
|
||||
}
|
||||
|
||||
counter warning_count
|
||||
/warning/ {
|
||||
warning_count++
|
||||
}
|
||||
|
||||
|
||||
counter filtered_mail_count
|
||||
|
||||
counter encrypted_mail_count
|
||||
/Filtering encrypted mail\./ {
|
||||
encrypted_mail_count++
|
||||
filtered_mail_count++
|
||||
}
|
||||
|
||||
counter unencrypted_mail_count
|
||||
/Filtering unencrypted mail\./ {
|
||||
unencrypted_mail_count++
|
||||
filtered_mail_count++
|
||||
}
|
||||
|
||||
counter rejected_unencrypted_mail_count
|
||||
/Rejected unencrypted mail\./ {
|
||||
rejected_unencrypted_mail_count++
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
[Unit]
|
||||
Description=mtail
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/bin/sh -c "journalctl -f -o short-iso -n 0 | /usr/local/bin/mtail --address={{ address }} --port={{ port }} --progs /etc/mtail --logtostderr --logs -"
|
||||
Restart=on-failure
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -2,25 +2,11 @@ load_module modules/ngx_stream_module.so;
|
||||
|
||||
user www-data;
|
||||
worker_processes auto;
|
||||
|
||||
# Increase the number of connections
|
||||
# that a worker process can open
|
||||
# to avoid errors such as
|
||||
# accept4() failed (24: Too many open files)
|
||||
# and
|
||||
# socket() failed (24: Too many open files) while connecting to upstream
|
||||
# in the logs.
|
||||
# <https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile>
|
||||
worker_rlimit_nofile 2048;
|
||||
pid /run/nginx.pid;
|
||||
error_log syslog:server=unix:/dev/log,facility=local3;
|
||||
|
||||
events {
|
||||
# Increase to avoid errors such as
|
||||
# 768 worker_connections are not enough while connecting to upstream
|
||||
# in the logs.
|
||||
# <https://nginx.org/en/docs/ngx_core_module.html#worker_connections>
|
||||
worker_connections 2048;
|
||||
worker_connections 768;
|
||||
# multi_accept on;
|
||||
}
|
||||
|
||||
@@ -60,13 +46,16 @@ http {
|
||||
|
||||
server {
|
||||
|
||||
listen 127.0.0.1:8443 ssl default_server;
|
||||
listen 8443 ssl default_server;
|
||||
{% if not disable_ipv6 %}
|
||||
listen [::]:8443 ssl default_server;
|
||||
{% endif %}
|
||||
|
||||
root /var/www/html;
|
||||
|
||||
index index.html index.htm;
|
||||
|
||||
server_name {{ config.domain_name }} www.{{ config.domain_name }} mta-sts.{{ config.domain_name }};
|
||||
server_name _;
|
||||
|
||||
access_log syslog:server=unix:/dev/log,facility=local7;
|
||||
|
||||
@@ -107,31 +96,14 @@ http {
|
||||
include /etc/nginx/fastcgi_params;
|
||||
fastcgi_param SCRIPT_FILENAME /usr/lib/cgi-bin/newemail.py;
|
||||
}
|
||||
|
||||
# Proxy to iroh-relay service.
|
||||
location /relay {
|
||||
proxy_pass http://127.0.0.1:3340;
|
||||
proxy_http_version 1.1;
|
||||
|
||||
# Upgrade header is normally set to "iroh derp http" or "websocket".
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
}
|
||||
|
||||
location /relay/probe {
|
||||
proxy_pass http://127.0.0.1:3340;
|
||||
proxy_http_version 1.1;
|
||||
}
|
||||
|
||||
location /generate_204 {
|
||||
proxy_pass http://127.0.0.1:3340;
|
||||
proxy_http_version 1.1;
|
||||
}
|
||||
}
|
||||
|
||||
# Redirect www. to non-www
|
||||
server {
|
||||
listen 127.0.0.1:8443 ssl;
|
||||
listen 8443 ssl;
|
||||
{% if not disable_ipv6 %}
|
||||
listen [::]:8443 ssl;
|
||||
{% endif %}
|
||||
server_name www.{{ config.domain_name }};
|
||||
return 301 $scheme://{{ config.domain_name }}$request_uri;
|
||||
access_log syslog:server=unix:/dev/log,facility=local7;
|
||||
|
||||
BIN
cmdeploy/src/cmdeploy/obs-home-deltachat.gpg
Normal file
BIN
cmdeploy/src/cmdeploy/obs-home-deltachat.gpg
Normal file
Binary file not shown.
@@ -13,7 +13,6 @@ OversignHeaders From
|
||||
On-BadSignature reject
|
||||
On-KeyNotFound reject
|
||||
On-NoSignature reject
|
||||
DNSTimeout 60
|
||||
|
||||
# Signing domain, selector, and key (required). For example, perform signing
|
||||
# for domain "example.com" with selector "2020" (2020._domainkey.example.com),
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
[Service]
|
||||
Restart=always
|
||||
RuntimeMaxSec=1d
|
||||
@@ -1,3 +0,0 @@
|
||||
#!/bin/sh
|
||||
echo "All runlevel operations denied by policy" >&2
|
||||
exit 101
|
||||
@@ -20,13 +20,9 @@ smtpd_tls_key_file=/var/lib/acme/live/{{ config.mail_domain }}/privkey
|
||||
smtpd_tls_security_level=may
|
||||
|
||||
smtp_tls_CApath=/etc/ssl/certs
|
||||
smtp_tls_security_level=verify
|
||||
# Send SNI extension when connecting to other servers.
|
||||
# <https://www.postfix.org/postconf.5.html#smtp_tls_servername>
|
||||
smtp_tls_servername = hostname
|
||||
smtp_tls_security_level=may
|
||||
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
|
||||
smtp_tls_policy_maps = inline:{nauta.cu=may}
|
||||
smtp_tls_protocols = >=TLSv1.2
|
||||
smtp_tls_policy_maps = socketmap:inet:127.0.0.1:8461:postfix
|
||||
smtpd_tls_protocols = >=TLSv1.2
|
||||
|
||||
# Disable anonymous cipher suites
|
||||
|
||||
@@ -14,12 +14,10 @@ smtp inet n - y - - smtpd -v
|
||||
{%- else %}
|
||||
smtp inet n - y - - smtpd
|
||||
{%- endif %}
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port_incoming }}
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
submission inet n - y - 5000 smtpd
|
||||
-o syslog_name=postfix/submission
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
-o smtpd_tls_mandatory_protocols=>=TLSv1.3
|
||||
-o smtpd_sasl_auth_enable=yes
|
||||
-o smtpd_sasl_type=dovecot
|
||||
-o smtpd_sasl_path=private/auth
|
||||
@@ -33,11 +31,11 @@ submission inet n - y - 5000 smtpd
|
||||
-o milter_macro_daemon_name=ORIGINATING
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
-o cleanup_service_name=authclean
|
||||
smtps inet n - y - 5000 smtpd
|
||||
-o syslog_name=postfix/smtps
|
||||
-o smtpd_tls_wrappermode=yes
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
-o smtpd_tls_mandatory_protocols=>=TLSv1.3
|
||||
-o smtpd_sasl_auth_enable=yes
|
||||
-o smtpd_sasl_type=dovecot
|
||||
-o smtpd_sasl_path=private/auth
|
||||
@@ -50,6 +48,7 @@ smtps inet n - y - 5000 smtpd
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o milter_macro_daemon_name=ORIGINATING
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
-o cleanup_service_name=authclean
|
||||
#628 inet n - y - - qmqpd
|
||||
pickup unix n - y 60 1 pickup
|
||||
cleanup unix n - y - 0 cleanup
|
||||
@@ -77,27 +76,17 @@ anvil unix - - y - 1 anvil
|
||||
scache unix - - y - 1 scache
|
||||
postlog unix-dgram n - n - 1 postlogd
|
||||
filter unix - n n - - lmtp
|
||||
# Local SMTP server for reinjecting outgoing filtered mail.
|
||||
127.0.0.1:{{ config.postfix_reinject_port }} inet n - n - 100 smtpd
|
||||
# Local SMTP server for reinjecting filered mail.
|
||||
localhost:{{ config.postfix_reinject_port }} inet n - n - 10 smtpd
|
||||
-o syslog_name=postfix/reinject
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
-o cleanup_service_name=authclean
|
||||
|
||||
# Local SMTP server for reinjecting incoming filtered mail
|
||||
127.0.0.1:{{ config.postfix_reinject_port_incoming }} inet n - n - 100 smtpd
|
||||
-o syslog_name=postfix/reinject_incoming
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
|
||||
# Cleanup `Received` headers for authenticated mail
|
||||
# to avoid leaking client IP.
|
||||
#
|
||||
# We do not do this for received mails
|
||||
# as this will break DKIM signatures
|
||||
# if `Received` header is signed.
|
||||
#
|
||||
# This service also rewrites
|
||||
# Subject with `[...]`
|
||||
# to make sure the users
|
||||
# cannot send unprotected Subject.
|
||||
authclean unix n - - - 0 cleanup
|
||||
-o header_checks=regexp:/etc/postfix/submission_header_cleanup
|
||||
|
||||
@@ -2,4 +2,3 @@
|
||||
/^X-Originating-IP:/ IGNORE
|
||||
/^X-Mailer:/ IGNORE
|
||||
/^User-Agent:/ IGNORE
|
||||
/^Subject:/ REPLACE Subject: [...]
|
||||
|
||||
@@ -12,74 +12,57 @@ All functions of this module
|
||||
|
||||
import re
|
||||
|
||||
from .rshell import CalledProcessError, log_progress, shell
|
||||
from .rshell import CalledProcessError, shell
|
||||
|
||||
|
||||
def perform_initial_checks(mail_domain, pre_command=""):
|
||||
def perform_initial_checks(mail_domain):
|
||||
"""Collecting initial DNS settings."""
|
||||
assert mail_domain
|
||||
if not shell("dig", fail_ok=True, print=log_progress):
|
||||
shell("apt-get update && apt-get install -y dnsutils", print=log_progress)
|
||||
if not shell("dig", fail_ok=True):
|
||||
shell("apt-get install -y dnsutils")
|
||||
shell(f"unbound-control flush_zone {mail_domain}", fail_ok=True)
|
||||
A = query_dns("A", mail_domain)
|
||||
AAAA = query_dns("AAAA", mail_domain)
|
||||
MTA_STS = query_dns("CNAME", f"mta-sts.{mail_domain}")
|
||||
WWW = query_dns("CNAME", f"www.{mail_domain}")
|
||||
|
||||
res = dict(mail_domain=mail_domain, A=A, AAAA=AAAA, MTA_STS=MTA_STS, WWW=WWW)
|
||||
res["acme_account_url"] = shell(
|
||||
pre_command + "acmetool account-url", fail_ok=True, print=log_progress
|
||||
)
|
||||
res["dkim_entry"], res["web_dkim_entry"] = get_dkim_entry(
|
||||
mail_domain, pre_command, dkim_selector="opendkim"
|
||||
)
|
||||
|
||||
if not MTA_STS or not WWW or (not A and not AAAA):
|
||||
return res
|
||||
|
||||
res["acme_account_url"] = shell("acmetool account-url", fail_ok=True)
|
||||
res["dkim_entry"] = get_dkim_entry(mail_domain, dkim_selector="opendkim")
|
||||
|
||||
# parse out sts-id if exists, example: "v=STSv1; id=2090123"
|
||||
parts = query_dns("TXT", f"_mta-sts.{mail_domain}").split("id=")
|
||||
res["sts_id"] = parts[1].rstrip('"') if len(parts) == 2 else ""
|
||||
return res
|
||||
|
||||
|
||||
def get_dkim_entry(mail_domain, pre_command, dkim_selector):
|
||||
def get_dkim_entry(mail_domain, dkim_selector):
|
||||
try:
|
||||
dkim_pubkey = shell(
|
||||
f"{pre_command}openssl rsa -in /etc/dkimkeys/{dkim_selector}.private "
|
||||
"-pubout 2>/dev/null | awk '/-/{next}{printf(\"%s\",$0)}'",
|
||||
print=log_progress,
|
||||
f"openssl rsa -in /etc/dkimkeys/{dkim_selector}.private "
|
||||
"-pubout 2>/dev/null | awk '/-/{next}{printf(\"%s\",$0)}'"
|
||||
)
|
||||
except CalledProcessError:
|
||||
return
|
||||
dkim_value_raw = f"v=DKIM1;k=rsa;p={dkim_pubkey};s=email;t=s"
|
||||
dkim_value = '" "'.join(re.findall(".{1,255}", dkim_value_raw))
|
||||
web_dkim_value = "".join(re.findall(".{1,255}", dkim_value_raw))
|
||||
return (
|
||||
f'{dkim_selector}._domainkey.{mail_domain}. TXT "{dkim_value}"',
|
||||
f'{dkim_selector}._domainkey.{mail_domain}. TXT "{web_dkim_value}"',
|
||||
)
|
||||
return f'{dkim_selector}._domainkey.{mail_domain}. TXT "{dkim_value}"'
|
||||
|
||||
|
||||
def query_dns(typ, domain):
|
||||
# Get autoritative nameserver from the SOA record.
|
||||
soa_answers = [
|
||||
x.split()
|
||||
for x in shell(
|
||||
f"dig -r -q {domain} -t SOA +noall +authority +answer", print=log_progress
|
||||
).split("\n")
|
||||
]
|
||||
soa = [a for a in soa_answers if len(a) >= 3 and a[3] == "SOA"]
|
||||
if not soa:
|
||||
return
|
||||
ns = soa[0][4]
|
||||
|
||||
# Query authoritative nameserver directly to bypass DNS cache.
|
||||
res = shell(f"dig @{ns} -r -q {domain} -t {typ} +short", print=log_progress)
|
||||
return next((line for line in res.split("\n") if not line.startswith(";")), "")
|
||||
res = shell(f"dig -r -q {domain} -t {typ} +short")
|
||||
print(res)
|
||||
if res:
|
||||
return res.split("\n")[0]
|
||||
return ""
|
||||
|
||||
|
||||
def check_zonefile(zonefile, verbose=True):
|
||||
def check_zonefile(zonefile, mail_domain):
|
||||
"""Check expected zone file entries."""
|
||||
shell(f"unbound-control flush_zone {mail_domain}", fail_ok=True)
|
||||
required = True
|
||||
required_diff = []
|
||||
recommended_diff = []
|
||||
@@ -90,7 +73,7 @@ def check_zonefile(zonefile, verbose=True):
|
||||
continue
|
||||
if not zf_line.strip() or zf_line.startswith(";"):
|
||||
continue
|
||||
print(f"dns-checking {zf_line!r}") if verbose else log_progress("")
|
||||
print(f"dns-checking {zf_line!r}")
|
||||
zf_domain, zf_typ, zf_value = zf_line.split(maxsplit=2)
|
||||
zf_domain = zf_domain.rstrip(".")
|
||||
zf_value = zf_value.strip()
|
||||
|
||||
@@ -1,19 +1,10 @@
|
||||
import sys
|
||||
from subprocess import DEVNULL, CalledProcessError, check_output
|
||||
from subprocess import CalledProcessError, check_output
|
||||
|
||||
|
||||
def log_progress(data):
|
||||
sys.stderr.write(".")
|
||||
sys.stderr.flush()
|
||||
|
||||
|
||||
def shell(command, fail_ok=False, print=print):
|
||||
def shell(command, fail_ok=False):
|
||||
print(f"$ {command}")
|
||||
args = dict(shell=True)
|
||||
if fail_ok:
|
||||
args["stderr"] = DEVNULL
|
||||
try:
|
||||
return check_output(command, **args).decode().rstrip()
|
||||
return check_output(command, shell=True).decode().rstrip()
|
||||
except CalledProcessError:
|
||||
if not fail_ok:
|
||||
raise
|
||||
@@ -23,22 +14,3 @@ def shell(command, fail_ok=False, print=print):
|
||||
def get_systemd_running():
|
||||
lines = shell("systemctl --type=service --state=running").split("\n")
|
||||
return [line for line in lines if line.startswith(" ")]
|
||||
|
||||
|
||||
def write_numbytes(path, num):
|
||||
with open(path, "w") as f:
|
||||
f.write("x" * num)
|
||||
|
||||
|
||||
def dovecot_recalc_quota(user):
|
||||
shell(f"doveadm quota recalc -u {user}")
|
||||
output = shell(f"doveadm quota get -u {user}")
|
||||
#
|
||||
# Quota name Type Value Limit %
|
||||
# User quota STORAGE 5 102400 0
|
||||
# User quota MESSAGE 2 - 0
|
||||
#
|
||||
for line in output.split("\n"):
|
||||
parts = line.split()
|
||||
if parts[2] == "STORAGE":
|
||||
return dict(value=int(parts[3]), limit=int(parts[4]), percent=int(parts[5]))
|
||||
|
||||
@@ -1,22 +0,0 @@
|
||||
import importlib.resources
|
||||
import os
|
||||
|
||||
import pyinfra
|
||||
|
||||
# pyinfra runs this module as a python file and not as a module so
|
||||
# import paths must be absolute
|
||||
from cmdeploy.deployers import deploy_chatmail
|
||||
|
||||
|
||||
def main():
|
||||
config_path = os.getenv(
|
||||
"CHATMAIL_INI",
|
||||
importlib.resources.files("cmdeploy").joinpath("../../../chatmail.ini"),
|
||||
)
|
||||
disable_mail = bool(os.environ.get("CHATMAIL_DISABLE_MAIL"))
|
||||
|
||||
deploy_chatmail(config_path, disable_mail)
|
||||
|
||||
|
||||
if pyinfra.is_cli:
|
||||
main()
|
||||
@@ -1,9 +0,0 @@
|
||||
[Unit]
|
||||
Description=chatmail mail storage expiration job
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=vmail
|
||||
ExecStart=/usr/local/lib/chatmaild/venv/bin/chatmail-expire /usr/local/lib/chatmaild/chatmail.ini -v --remove
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
[Unit]
|
||||
Description=Run Daily chatmail-expire job
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*-*-* 00:02:00
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -1,9 +0,0 @@
|
||||
[Unit]
|
||||
Description=chatmail file system storage reporting job
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=vmail
|
||||
ExecStart=/usr/local/lib/chatmaild/venv/bin/chatmail-fsreport /usr/local/lib/chatmaild/chatmail.ini
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
[Unit]
|
||||
Description=Run Daily Chatmail fsreport Job
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*-*-* 08:02:00
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -7,7 +7,6 @@ Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
RuntimeDirectory=chatmail-metadata
|
||||
UMask=0077
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -7,7 +7,6 @@ Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
RuntimeDirectory=doveauth
|
||||
UMask=0077
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
[Unit]
|
||||
Description=Incoming Chatmail Postfix before queue filter
|
||||
|
||||
[Service]
|
||||
ExecStart={execpath} {config_path} incoming
|
||||
Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
[Unit]
|
||||
Description=Outgoing Chatmail Postfix before queue filter
|
||||
Description=Chatmail Postfix before queue filter
|
||||
|
||||
[Service]
|
||||
ExecStart={execpath} {config_path} outgoing
|
||||
ExecStart={execpath} {config_path}
|
||||
Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
User=filtermail
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
[Unit]
|
||||
Description=A wrapper for the TURN server
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
Restart=always
|
||||
ExecStart=/usr/local/bin/chatmail-turn --realm {mail_domain} --socket /run/chatmail-turn/turn.socket
|
||||
|
||||
# Create /run/chatmail-turn
|
||||
RuntimeDirectory=chatmail-turn
|
||||
User=vmail
|
||||
Group=vmail
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -42,7 +42,6 @@ def bootstrap_remote(gateway, remote=remote):
|
||||
|
||||
def print_stderr(item="", end="\n"):
|
||||
print(item, file=sys.stderr, end=end)
|
||||
sys.stderr.flush()
|
||||
|
||||
|
||||
class SSHExec:
|
||||
@@ -71,6 +70,10 @@ class SSHExec:
|
||||
raise self.FuncError(data)
|
||||
|
||||
def logged(self, call, kwargs):
|
||||
def log_progress(data):
|
||||
sys.stderr.write(".")
|
||||
sys.stderr.flush()
|
||||
|
||||
title = call.__doc__
|
||||
if not title:
|
||||
title = call.__name__
|
||||
@@ -79,22 +82,6 @@ class SSHExec:
|
||||
return self(call, kwargs, log_callback=print_stderr)
|
||||
else:
|
||||
print_stderr(title, end="")
|
||||
res = self(call, kwargs, log_callback=remote.rshell.log_progress)
|
||||
res = self(call, kwargs, log_callback=log_progress)
|
||||
print_stderr()
|
||||
return res
|
||||
|
||||
|
||||
class LocalExec:
|
||||
def __init__(self, verbose=False, docker=False):
|
||||
self.verbose = verbose
|
||||
self.docker = docker
|
||||
|
||||
def logged(self, call, kwargs: dict):
|
||||
where = "locally"
|
||||
if self.docker:
|
||||
if call == remote.rdns.perform_initial_checks:
|
||||
kwargs["pre_command"] = "docker exec chatmail "
|
||||
where = "in docker"
|
||||
if self.verbose:
|
||||
print(f"Running {where}: {call.__name__}(**{kwargs})")
|
||||
return call(**kwargs)
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import queue
|
||||
import smtplib
|
||||
import socket
|
||||
import threading
|
||||
|
||||
import pytest
|
||||
@@ -90,24 +90,21 @@ def test_concurrent_logins_same_account(
|
||||
|
||||
|
||||
def test_no_vrfy(chatmail_config):
|
||||
domain = chatmail_config.mail_domain
|
||||
|
||||
s = smtplib.SMTP(domain)
|
||||
s.starttls()
|
||||
|
||||
s.putcmd("vrfy", f"wrongaddress@{chatmail_config.mail_domain}")
|
||||
result = s.getreply()
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.connect((chatmail_config.mail_domain, 25))
|
||||
banner = sock.recv(1024)
|
||||
print(banner)
|
||||
sock.send(b"VRFY wrongaddress@%s\r\n" % (chatmail_config.mail_domain.encode(),))
|
||||
result = sock.recv(1024)
|
||||
print(result)
|
||||
s.putcmd("vrfy", f"echo@{chatmail_config.mail_domain}")
|
||||
result2 = s.getreply()
|
||||
sock.send(b"VRFY echo@%s\r\n" % (chatmail_config.mail_domain.encode(),))
|
||||
result2 = sock.recv(1024)
|
||||
print(result2)
|
||||
assert result[0] == result2[0] == 252
|
||||
assert result[1][0:6] == result2[1][0:6] == b"2.0.0 "
|
||||
s.putcmd("vrfy", "wrongaddress")
|
||||
result = s.getreply()
|
||||
assert result[0:10] == result2[0:10]
|
||||
sock.send(b"VRFY wrongaddress\r\n")
|
||||
result = sock.recv(1024)
|
||||
print(result)
|
||||
s.putcmd("vrfy", "echo")
|
||||
result2 = s.getreply()
|
||||
sock.send(b"VRFY echo\r\n")
|
||||
result2 = sock.recv(1024)
|
||||
print(result2)
|
||||
assert result[0] == result2[0] == 252
|
||||
assert result[1][0:6] == result2[1][0:6] == b"2.0.0 "
|
||||
assert result[0:10] == result2[0:10] == b"252 2.0.0 "
|
||||
|
||||
@@ -1,14 +1,8 @@
|
||||
import datetime
|
||||
import os
|
||||
import smtplib
|
||||
import socket
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from cmdeploy import remote
|
||||
from cmdeploy.cmdeploy import main
|
||||
from cmdeploy.sshexec import SSHExec
|
||||
|
||||
|
||||
@@ -34,8 +28,7 @@ class TestSSHExecutor:
|
||||
)
|
||||
out, err = capsys.readouterr()
|
||||
assert err.startswith("Collecting")
|
||||
# XXX could not figure out how capturing can be made to work properly
|
||||
# assert err.endswith("....\n")
|
||||
assert err.endswith("....\n")
|
||||
assert err.count("\n") == 1
|
||||
|
||||
sshexec.verbose = True
|
||||
@@ -44,8 +37,7 @@ class TestSSHExecutor:
|
||||
)
|
||||
out, err = capsys.readouterr()
|
||||
lines = err.split("\n")
|
||||
# XXX could not figure out how capturing can be made to work properly
|
||||
# assert len(lines) > 4
|
||||
assert len(lines) > 4
|
||||
assert remote.rdns.perform_initial_checks.__doc__ in lines[0]
|
||||
|
||||
def test_exception(self, sshexec, capsys):
|
||||
@@ -60,64 +52,6 @@ class TestSSHExecutor:
|
||||
else:
|
||||
pytest.fail("didn't raise exception")
|
||||
|
||||
def test_opendkim_restarted(self, sshexec):
|
||||
"""check that opendkim is not running for longer than a day."""
|
||||
cmd = "systemctl show opendkim --timestamp=utc --property=ActiveEnterTimestamp"
|
||||
out = sshexec(call=remote.rshell.shell, kwargs=dict(command=cmd))
|
||||
datestring = out.split("=")[1]
|
||||
since_date = datetime.datetime.strptime(datestring, "%a %Y-%m-%d %H:%M:%S %Z")
|
||||
now = datetime.datetime.now(since_date.tzinfo)
|
||||
assert (now - since_date).total_seconds() < 60 * 60 * 51
|
||||
|
||||
|
||||
def test_status_cmd(chatmail_config, capsys, request):
|
||||
os.chdir(request.config.invocation_params.dir)
|
||||
assert main(["status"]) == 0
|
||||
status_out = capsys.readouterr()
|
||||
print(status_out.out)
|
||||
|
||||
services = [
|
||||
"acmetool-redirector",
|
||||
"chatmail-metadata",
|
||||
"doveauth",
|
||||
"dovecot",
|
||||
"echobot",
|
||||
"fcgiwrap",
|
||||
"filtermail-incoming",
|
||||
"filtermail",
|
||||
"lastlogin",
|
||||
"nginx",
|
||||
"opendkim",
|
||||
"postfix@-",
|
||||
"systemd-journald",
|
||||
"turnserver",
|
||||
"unbound",
|
||||
]
|
||||
not_running = []
|
||||
for service in services:
|
||||
active = False
|
||||
for line in status_out:
|
||||
if service in line:
|
||||
active = True
|
||||
if not "loaded" in line:
|
||||
active = False
|
||||
if not "active" in line:
|
||||
active = False
|
||||
if not "running" in line:
|
||||
active = False
|
||||
break
|
||||
if not active:
|
||||
not_running.append(service)
|
||||
assert not_running == []
|
||||
|
||||
|
||||
def test_timezone_env(remote):
|
||||
for line in remote.iter_output("env"):
|
||||
print(line)
|
||||
if line == "tz=:/etc/localtime":
|
||||
return
|
||||
pytest.fail("TZ is not set")
|
||||
|
||||
|
||||
def test_remote(remote, imap_or_smtp):
|
||||
lineproducer = remote.iter_output(imap_or_smtp.logcmd)
|
||||
@@ -173,58 +107,14 @@ def test_authenticated_from(cmsetup, maildata):
|
||||
|
||||
@pytest.mark.parametrize("from_addr", ["fake@example.org", "fake@testrun.org"])
|
||||
def test_reject_missing_dkim(cmsetup, maildata, from_addr):
|
||||
domain = cmsetup.maildomain
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.settimeout(10)
|
||||
try:
|
||||
sock.connect((domain, 25))
|
||||
except socket.timeout:
|
||||
pytest.skip(f"port 25 not reachable for {domain}")
|
||||
|
||||
"""Test that emails with missing or wrong DMARC, DKIM, and SPF entries are rejected."""
|
||||
recipient = cmsetup.gen_users(1)[0]
|
||||
msg = maildata(
|
||||
"encrypted.eml", from_addr=from_addr, to_addr=recipient.addr
|
||||
).as_string()
|
||||
conn = smtplib.SMTP(cmsetup.maildomain, 25, timeout=10)
|
||||
conn.starttls()
|
||||
|
||||
with conn as s:
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=recipient.addr).as_string()
|
||||
with smtplib.SMTP(cmsetup.maildomain, 25) as s:
|
||||
with pytest.raises(smtplib.SMTPDataError, match="No valid DKIM signature"):
|
||||
s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg)
|
||||
|
||||
|
||||
def try_n_times(n, f):
|
||||
for _ in range(n - 1):
|
||||
try:
|
||||
return f()
|
||||
except Exception:
|
||||
time.sleep(1)
|
||||
|
||||
return f()
|
||||
|
||||
|
||||
def test_rewrite_subject(cmsetup, maildata):
|
||||
"""Test that subject gets replaced with [...]."""
|
||||
user1, user2 = cmsetup.gen_users(2)
|
||||
|
||||
sent_msg = maildata(
|
||||
"encrypted.eml",
|
||||
from_addr=user1.addr,
|
||||
to_addr=user2.addr,
|
||||
subject="Unencrypted subject",
|
||||
).as_string()
|
||||
user1.smtp.sendmail(from_addr=user1.addr, to_addrs=[user2.addr], msg=sent_msg)
|
||||
|
||||
# The message may need some time to get delivered by postfix.
|
||||
messages = try_n_times(5, user2.imap.fetch_all_messages)
|
||||
assert len(messages) == 1
|
||||
rcvd_msg = messages[0]
|
||||
assert "Subject: [...]" not in sent_msg
|
||||
assert "Subject: [...]" in rcvd_msg
|
||||
assert "Subject: Unencrypted subject" in sent_msg
|
||||
assert "Subject: Unencrypted subject" not in rcvd_msg
|
||||
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_exceed_rate_limit(cmsetup, gencreds, maildata, chatmail_config):
|
||||
"""Test that the per-account send-mail limit is exceeded."""
|
||||
@@ -257,31 +147,6 @@ def test_expunged(remote, chatmail_config):
|
||||
f"find {chatmail_config.mailboxes_dir} -path '*/tmp/*' -mtime +{outdated_days} -type f",
|
||||
f"find {chatmail_config.mailboxes_dir} -path '*/.*/tmp/*' -mtime +{outdated_days} -type f",
|
||||
]
|
||||
outdated_days = int(chatmail_config.delete_large_after) + 1
|
||||
find_cmds.append(
|
||||
"find {chatmail_config.mailboxes_dir} -path '*/cur/*' -mtime +{outdated_days} -size +200k -type f"
|
||||
)
|
||||
for cmd in find_cmds:
|
||||
for line in remote.iter_output(cmd):
|
||||
assert not line
|
||||
|
||||
|
||||
def test_deployed_state(remote):
|
||||
try:
|
||||
git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode()
|
||||
except Exception:
|
||||
git_hash = "unknown\n"
|
||||
try:
|
||||
git_diff = subprocess.check_output(["git", "diff"]).decode()
|
||||
except Exception:
|
||||
git_diff = ""
|
||||
git_status = [git_hash.strip()]
|
||||
for line in git_diff.splitlines():
|
||||
git_status.append(line.strip().lower())
|
||||
remote_version = []
|
||||
for line in remote.iter_output("cat /etc/chatmail-version"):
|
||||
print(line)
|
||||
remote_version.append(line)
|
||||
# assert len(git_status) == len(remote_version) # for some reason, we only get 11 lines from remote.iter_output()
|
||||
for i in range(len(remote_version)):
|
||||
assert git_status[i] == remote_version[i], "You have undeployed changes."
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import ipaddress
|
||||
import random
|
||||
import re
|
||||
import time
|
||||
|
||||
@@ -6,9 +7,6 @@ import imap_tools
|
||||
import pytest
|
||||
import requests
|
||||
|
||||
from cmdeploy.remote import rshell
|
||||
from cmdeploy.sshexec import SSHExec
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def imap_mailbox(cmfactory):
|
||||
@@ -57,23 +55,22 @@ class TestEndToEndDeltaChat:
|
||||
on the same chat-mail instance."""
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
lp.sec("ac1: prepare and send text message to ac2")
|
||||
chat.send_text("message0")
|
||||
|
||||
lp.sec("wait for ac2 to receive message")
|
||||
msg2 = ac2._evtracker.wait_next_incoming_message()
|
||||
assert msg2.text == "message0"
|
||||
|
||||
def test_exceed_quota(
|
||||
self, cmfactory, lp, tmpdir, remote, chatmail_config, sshdomain
|
||||
):
|
||||
@pytest.mark.slow
|
||||
def test_exceed_quota(self, cmfactory, lp, tmpdir, remote, chatmail_config):
|
||||
"""This is a very slow test as it needs to upload >100MB of mail data
|
||||
before quota is exceeded, and thus depends on the speed of the upload.
|
||||
"""
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
user = ac2.get_config("configured_addr")
|
||||
|
||||
def parse_size_limit(limit: str) -> int:
|
||||
"""Parse a size limit and return the number of bytes as integer.
|
||||
|
||||
@@ -85,27 +82,49 @@ class TestEndToEndDeltaChat:
|
||||
return int(float(number) * units[unit])
|
||||
|
||||
quota = parse_size_limit(chatmail_config.max_mailbox_size)
|
||||
attachsize = 1 * 1024 * 1024
|
||||
num_to_send = quota // attachsize + 2
|
||||
lp.sec(f"ac1: send {num_to_send} large files to ac2")
|
||||
lp.indent(f"per-user quota is assumed to be: {quota/(1024*1024)}MB")
|
||||
alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
|
||||
msgs = []
|
||||
for i in range(num_to_send):
|
||||
attachment = tmpdir / f"attachment{i}"
|
||||
data = "".join(random.choice(alphanumeric) for i in range(1024))
|
||||
with open(attachment, "w+") as f:
|
||||
for j in range(attachsize // len(data)):
|
||||
f.write(data)
|
||||
|
||||
lp.sec(f"filling remote inbox for {user}")
|
||||
fn = f"7743102289.M843172P2484002.c20,S={quota},W=2398:2,"
|
||||
path = chatmail_config.mailboxes_dir.joinpath(user, "cur", fn)
|
||||
sshexec = SSHExec(sshdomain)
|
||||
sshexec(call=rshell.write_numbytes, kwargs=dict(path=str(path), num=120))
|
||||
res = sshexec(call=rshell.dovecot_recalc_quota, kwargs=dict(user=user))
|
||||
assert res["percent"] >= 100
|
||||
msg = chat.send_file(str(attachment))
|
||||
msgs.append(msg)
|
||||
lp.indent(f"Sent out msg {i}, size {attachsize/(1024*1024)}MB")
|
||||
|
||||
lp.sec("ac2: check quota is triggered")
|
||||
lp.sec("ac2: check messages are arriving until quota is reached")
|
||||
|
||||
starting = True
|
||||
addr = ac2.get_config("addr").lower()
|
||||
saved_ok = 0
|
||||
for line in remote.iter_output("journalctl -n0 -f -u dovecot"):
|
||||
if starting:
|
||||
chat.send_text("hello")
|
||||
starting = False
|
||||
if user not in line:
|
||||
if addr not in line:
|
||||
# print(line)
|
||||
continue
|
||||
if "quota exceeded" in line:
|
||||
return
|
||||
if "quota" in line:
|
||||
if "quota exceeded" in line:
|
||||
if saved_ok < num_to_send // 2:
|
||||
pytest.fail(
|
||||
f"quota exceeded too early: after {saved_ok} messages already"
|
||||
)
|
||||
lp.indent("good, message sending failed because quota was exceeded")
|
||||
return
|
||||
if (
|
||||
"stored mail into mailbox 'inbox'" in line
|
||||
or "saved mail to inbox" in line
|
||||
):
|
||||
saved_ok += 1
|
||||
print(f"{saved_ok}: {line}")
|
||||
if saved_ok >= num_to_send:
|
||||
break
|
||||
|
||||
pytest.fail("sending succeeded although messages should exceed quota")
|
||||
|
||||
def test_securejoin(self, cmfactory, lp, maildomain2):
|
||||
ac1 = cmfactory.new_online_configuring_account(cache=False)
|
||||
@@ -162,18 +181,11 @@ def test_hide_senders_ip_address(cmfactory):
|
||||
assert public_ip not in msg.obj.as_string()
|
||||
|
||||
|
||||
def test_echobot(cmfactory, chatmail_config, lp, sshdomain):
|
||||
def test_echobot(cmfactory, chatmail_config, lp):
|
||||
ac = cmfactory.get_online_accounts(1)[0]
|
||||
|
||||
# establish contact with echobot
|
||||
sshexec = SSHExec(sshdomain)
|
||||
command = "cat /var/lib/echobot/invite-link.txt"
|
||||
echo_invite_link = sshexec(call=rshell.shell, kwargs=dict(command=command))
|
||||
chat = ac.qr_setup_contact(echo_invite_link)
|
||||
ac._evtracker.wait_securejoin_joiner_progress(1000)
|
||||
|
||||
# send message and check it gets replied back
|
||||
lp.sec("Send message to echobot")
|
||||
lp.sec(f"Send message to echo@{chatmail_config.mail_domain}")
|
||||
chat = ac.create_chat(f"echo@{chatmail_config.mail_domain}")
|
||||
text = "hi, I hope you text me back"
|
||||
chat.send_text(text)
|
||||
lp.sec("Wait for reply from echobot")
|
||||
|
||||
@@ -62,7 +62,7 @@ def sshdomain(maildomain):
|
||||
def maildomain2():
|
||||
domain = os.environ.get("CHATMAIL_DOMAIN2")
|
||||
if not domain:
|
||||
pytest.skip("set CHATMAIL_DOMAIN2 to a second chatmail server")
|
||||
pytest.skip("set CHATMAIL_DOMAIN2 to a ssh-reachable chatmail instance")
|
||||
return domain
|
||||
|
||||
|
||||
@@ -302,13 +302,10 @@ def cmfactory(request, gencreds, tmpdir, maildomain):
|
||||
pytest.importorskip("deltachat")
|
||||
from deltachat.testplugin import ACFactory
|
||||
|
||||
data = request.getfixturevalue("data")
|
||||
|
||||
testproc = ChatmailTestProcess(request.config, maildomain, gencreds)
|
||||
|
||||
class Data:
|
||||
def read_path(self, path):
|
||||
return
|
||||
|
||||
am = ACFactory(request=request, tmpdir=tmpdir, testprocess=testproc, data=Data())
|
||||
am = ACFactory(request=request, tmpdir=tmpdir, testprocess=testproc, data=data)
|
||||
|
||||
# nb. a bit hacky
|
||||
# would probably be better if deltachat's test machinery grows native support
|
||||
|
||||
@@ -1,10 +1,8 @@
|
||||
import importlib
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from cmdeploy.cmdeploy import get_parser, main
|
||||
from cmdeploy.www import get_paths
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
@@ -26,36 +24,6 @@ class TestCmdline:
|
||||
def test_init_not_overwrite(self, capsys):
|
||||
assert main(["init", "chat.example.org"]) == 0
|
||||
capsys.readouterr()
|
||||
|
||||
assert main(["init", "chat.example.org"]) == 1
|
||||
out, err = capsys.readouterr()
|
||||
assert "path exists" in out.lower()
|
||||
|
||||
assert main(["init", "chat.example.org", "--force"]) == 0
|
||||
out, err = capsys.readouterr()
|
||||
assert "deleting config file" in out.lower()
|
||||
|
||||
|
||||
def test_www_folder(example_config, tmp_path):
|
||||
reporoot = importlib.resources.files(__package__).joinpath("../../../../").resolve()
|
||||
assert not example_config.www_folder
|
||||
www_path, src_dir, build_dir = get_paths(example_config)
|
||||
assert www_path.absolute() == reporoot.joinpath("www").absolute()
|
||||
assert src_dir == reporoot.joinpath("www").joinpath("src")
|
||||
assert build_dir == reporoot.joinpath("www").joinpath("build")
|
||||
example_config.www_folder = "disabled"
|
||||
www_path, _, _ = get_paths(example_config)
|
||||
assert not www_path.is_dir()
|
||||
example_config.www_folder = str(tmp_path)
|
||||
www_path, src_dir, build_dir = get_paths(example_config)
|
||||
assert www_path == tmp_path
|
||||
assert not src_dir.exists()
|
||||
assert not build_dir
|
||||
src_path = tmp_path.joinpath("src")
|
||||
os.mkdir(src_path)
|
||||
with open(src_path / "index.md", "w") as f:
|
||||
f.write("# Test")
|
||||
www_path, src_dir, build_dir = get_paths(example_config)
|
||||
assert www_path == tmp_path
|
||||
assert src_dir == src_path
|
||||
assert build_dir == tmp_path.joinpath("build")
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
from copy import deepcopy
|
||||
|
||||
import pytest
|
||||
|
||||
from cmdeploy import remote
|
||||
@@ -10,65 +8,38 @@ from cmdeploy.dns import check_full_zone, check_initial_remote_data
|
||||
def mockdns_base(monkeypatch):
|
||||
qdict = {}
|
||||
|
||||
def shell(command, fail_ok=False, print=print):
|
||||
if command.startswith("dig"):
|
||||
if command == "dig":
|
||||
return "."
|
||||
if "SOA" in command:
|
||||
return (
|
||||
"delta.chat. 21600 IN SOA ns1.first-ns.de. dns.hetzner.com."
|
||||
" 2025102800 14400 1800 604800 3600"
|
||||
)
|
||||
command_chunks = command.split()
|
||||
domain, typ = command_chunks[4], command_chunks[6]
|
||||
try:
|
||||
return qdict[typ][domain]
|
||||
except KeyError:
|
||||
return ""
|
||||
return remote.rshell.shell(command=command, fail_ok=fail_ok, print=print)
|
||||
def query_dns(typ, domain):
|
||||
try:
|
||||
return qdict[typ][domain]
|
||||
except KeyError:
|
||||
return ""
|
||||
|
||||
monkeypatch.setattr(remote.rdns, shell.__name__, shell)
|
||||
monkeypatch.setattr(remote.rdns, query_dns.__name__, query_dns)
|
||||
return qdict
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mockdns_expected():
|
||||
return {
|
||||
"A": {"some.domain": "1.1.1.1"},
|
||||
"AAAA": {"some.domain": "fde5:cd7a:9e1c:3240:5a99:936f:cdac:53ae"},
|
||||
"CNAME": {
|
||||
"mta-sts.some.domain": "some.domain.",
|
||||
"www.some.domain": "some.domain.",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture(params=["plain", "with-dns-comments"])
|
||||
def mockdns(request, mockdns_base, mockdns_expected):
|
||||
mockdns_base.update(deepcopy(mockdns_expected))
|
||||
match request.param:
|
||||
case "plain":
|
||||
pass
|
||||
case "with-dns-comments":
|
||||
for typ, data in mockdns_base.items():
|
||||
for host, result in data.items():
|
||||
mockdns_base[typ][host] = (
|
||||
";; some unsuccessful attempt result\n"
|
||||
"; and another with a single semicolon\n"
|
||||
f"{result}"
|
||||
)
|
||||
def mockdns(mockdns_base):
|
||||
mockdns_base.update(
|
||||
{
|
||||
"A": {"some.domain": "1.1.1.1"},
|
||||
"AAAA": {"some.domain": "fde5:cd7a:9e1c:3240:5a99:936f:cdac:53ae"},
|
||||
"CNAME": {
|
||||
"mta-sts.some.domain": "some.domain.",
|
||||
"www.some.domain": "some.domain.",
|
||||
},
|
||||
}
|
||||
)
|
||||
return mockdns_base
|
||||
|
||||
|
||||
class TestPerformInitialChecks:
|
||||
def test_perform_initial_checks_ok1(self, mockdns, mockdns_expected):
|
||||
def test_perform_initial_checks_ok1(self, mockdns):
|
||||
remote_data = remote.rdns.perform_initial_checks("some.domain")
|
||||
assert remote_data["A"] == mockdns_expected["A"]["some.domain"]
|
||||
assert remote_data["AAAA"] == mockdns_expected["AAAA"]["some.domain"]
|
||||
assert (
|
||||
remote_data["MTA_STS"] == mockdns_expected["CNAME"]["mta-sts.some.domain"]
|
||||
)
|
||||
assert remote_data["WWW"] == mockdns_expected["CNAME"]["www.some.domain"]
|
||||
assert remote_data["A"] == mockdns["A"]["some.domain"]
|
||||
assert remote_data["AAAA"] == mockdns["AAAA"]["some.domain"]
|
||||
assert remote_data["MTA_STS"] == mockdns["CNAME"]["mta-sts.some.domain"]
|
||||
assert remote_data["WWW"] == mockdns["CNAME"]["www.some.domain"]
|
||||
|
||||
@pytest.mark.parametrize("drop", ["A", "AAAA"])
|
||||
def test_perform_initial_checks_with_one_of_A_AAAA(self, mockdns, drop):
|
||||
@@ -118,14 +89,18 @@ class TestZonefileChecks:
|
||||
def test_check_zonefile_all_ok(self, cm_data, mockdns_base):
|
||||
zonefile = cm_data.get("zftest.zone")
|
||||
parse_zonefile_into_dict(zonefile, mockdns_base)
|
||||
required_diff, recommended_diff = remote.rdns.check_zonefile(zonefile)
|
||||
required_diff, recommended_diff = remote.rdns.check_zonefile(
|
||||
zonefile, "some.domain"
|
||||
)
|
||||
assert not required_diff and not recommended_diff
|
||||
|
||||
def test_check_zonefile_recommended_not_set(self, cm_data, mockdns_base):
|
||||
zonefile = cm_data.get("zftest.zone")
|
||||
zonefile_mocked = zonefile.split("; Recommended")[0]
|
||||
parse_zonefile_into_dict(zonefile_mocked, mockdns_base)
|
||||
required_diff, recommended_diff = remote.rdns.check_zonefile(zonefile)
|
||||
required_diff, recommended_diff = remote.rdns.check_zonefile(
|
||||
zonefile, "some.domain"
|
||||
)
|
||||
assert not required_diff
|
||||
assert len(recommended_diff) == 8
|
||||
|
||||
|
||||
@@ -1,10 +1,8 @@
|
||||
import hashlib
|
||||
import importlib.resources
|
||||
import re
|
||||
import time
|
||||
import traceback
|
||||
import webbrowser
|
||||
from pathlib import Path
|
||||
|
||||
import markdown
|
||||
from chatmaild.config import read_config
|
||||
@@ -12,10 +10,6 @@ from jinja2 import Template
|
||||
|
||||
from .genqr import gen_qr_png_data
|
||||
|
||||
_MERGE_CONFLICT_RE = re.compile(
|
||||
r"^<<<<<<<.+^=======.+^>>>>>>>", re.DOTALL | re.MULTILINE
|
||||
)
|
||||
|
||||
|
||||
def snapshot_dir_stats(somedir):
|
||||
d = {}
|
||||
@@ -36,25 +30,9 @@ def prepare_template(source):
|
||||
return render_vars, page_layout
|
||||
|
||||
|
||||
def get_paths(config) -> (Path, Path, Path):
|
||||
reporoot = importlib.resources.files(__package__).joinpath("../../../").resolve()
|
||||
www_path = Path(config.www_folder)
|
||||
# if www_folder was not set, use default directory
|
||||
if config.www_folder == "":
|
||||
www_path = reporoot.joinpath("www")
|
||||
src_dir = www_path.joinpath("src")
|
||||
# if www_folder is a hugo page, build it
|
||||
if src_dir.joinpath("index.md").is_file():
|
||||
build_dir = www_path.joinpath("build")
|
||||
# if it is not a hugo page, upload it as is
|
||||
else:
|
||||
build_dir = None
|
||||
return www_path, src_dir, build_dir
|
||||
|
||||
|
||||
def build_webpages(src_dir, build_dir, config) -> Path:
|
||||
def build_webpages(src_dir, build_dir, config):
|
||||
try:
|
||||
return _build_webpages(src_dir, build_dir, config)
|
||||
_build_webpages(src_dir, build_dir, config)
|
||||
except Exception:
|
||||
print(traceback.format_exc())
|
||||
|
||||
@@ -121,17 +99,6 @@ def _build_webpages(src_dir, build_dir, config):
|
||||
return build_dir
|
||||
|
||||
|
||||
def find_merge_conflict(src_dir) -> Path:
|
||||
assert src_dir.exists(), src_dir
|
||||
result = None
|
||||
for path in src_dir.iterdir():
|
||||
if path.suffix in [".css", ".html", ".md"]:
|
||||
if _MERGE_CONFLICT_RE.search(path.read_text()):
|
||||
result = path
|
||||
break
|
||||
return result
|
||||
|
||||
|
||||
def main():
|
||||
path = importlib.resources.files(__package__)
|
||||
reporoot = path.joinpath("../../../").resolve()
|
||||
@@ -139,11 +106,15 @@ def main():
|
||||
config = read_config(inipath)
|
||||
config.webdev = True
|
||||
assert config.mail_domain
|
||||
www_path = reporoot.joinpath("www")
|
||||
src_path = www_path.joinpath("src")
|
||||
stats = None
|
||||
build_dir = www_path.joinpath("build")
|
||||
src_dir = www_path.joinpath("src")
|
||||
index_path = build_dir.joinpath("index.html")
|
||||
|
||||
# start web page generation, open a browser and wait for changes
|
||||
www_path, src_path, build_dir = get_paths(config)
|
||||
build_dir = build_webpages(src_path, build_dir, config)
|
||||
index_path = build_dir.joinpath("index.html")
|
||||
build_webpages(src_dir, build_dir, config)
|
||||
webbrowser.open(str(index_path))
|
||||
stats = snapshot_dir_stats(src_path)
|
||||
print(f"\nOpened URL: file://{index_path.resolve()}\n")
|
||||
@@ -164,7 +135,7 @@ def main():
|
||||
changenum += 1
|
||||
|
||||
stats = newstats
|
||||
build_webpages(src_path, build_dir, config)
|
||||
build_webpages(src_dir, build_dir, config)
|
||||
print(f"[{changenum}] regenerated web pages at: {index_path}")
|
||||
print(f"URL: file://{index_path.resolve()}\n\n")
|
||||
count = 0
|
||||
|
||||
24
doc/Makefile
24
doc/Makefile
@@ -1,24 +0,0 @@
|
||||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line, and also
|
||||
# from the environment for the first two.
|
||||
SPHINXOPTS ?=
|
||||
SPHINXBUILD ?= sphinx-build
|
||||
SOURCEDIR = source
|
||||
BUILDDIR = build
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
auto:
|
||||
sphinx-autobuild "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
.PHONY: help Makefile auto
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
@@ -1,17 +0,0 @@
|
||||
|
||||
|
||||
## Building the documentation
|
||||
|
||||
You can use the `make` command and `make html` to build web pages.
|
||||
|
||||
You need a Python environment where the following install was excuted:
|
||||
|
||||
pip install sphinx-build furo sphinx-autobuild
|
||||
|
||||
To develop/change documentation, you can then do:
|
||||
|
||||
make auto
|
||||
|
||||
A page will open at https://127.0.0.1:8000/ serving the docs and it will
|
||||
react to changes to source files pretty fast.
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:svg="http://www.w3.org/2000/svg" id="svg4" width="145" height="145" version="1.1"><g id="text2" aria-label="@" style="font-size:144px;font-family:Arial" transform="matrix(1.0934997,0,0,1.0934997,-6.7787266,-6.7787281)"><path id="path347" d="m 79.927878,94.422406 c -2.704286,3.120332 -5.741407,5.637394 -9.111364,7.551194 -3.328352,1.87221 -6.677506,2.80831 -10.047463,2.80831 -3.702792,0 -7.301573,-1.08172 -10.796342,-3.24515 -3.49477,-2.163426 -6.344671,-5.491779 -8.549704,-9.985058 -2.163429,-4.493275 -3.245144,-9.423397 -3.245144,-14.790365 0,-6.615099 1.684978,-13.230199 5.054935,-19.845299 3.411561,-6.656705 7.634407,-11.649233 12.66854,-14.977585 5.034133,-3.328352 9.92265,-4.992528 14.665552,-4.992528 3.619583,0 7.072748,0.956901 10.359496,2.870704 3.286748,1.872198 6.115847,4.742902 8.487297,8.612111 l 2.121825,-9.673023 h 11.170784 l -8.986557,41.87483 c -1.248129,5.824616 -1.872194,9.048957 -1.872194,9.673023 0,1.123319 0.416044,2.101022 1.248132,2.93311 0.873692,0.790484 1.913802,1.185726 3.120332,1.185726 2.20503,0 5.096537,-1.268934 8.674517,-3.806803 4.7429,-3.328352 8.4873,-7.780023 11.23319,-13.355013 2.78749,-5.616594 4.18124,-11.399606 4.18124,-17.349035 0,-6.947935 -1.78899,-13.438222 -5.36697,-19.47086 -3.53637,-6.032638 -8.84094,-10.858749 -15.913687,-14.478332 -7.03114,-3.619583 -14.811161,-5.429374 -23.340064,-5.429374 -9.73543,0 -18.638772,2.288242 -26.710026,6.864726 -8.029649,4.534879 -14.27031,11.06677 -18.721981,19.595673 -4.410066,8.487298 -6.615099,17.598662 -6.615099,27.334092 0,10.193078 2.205033,18.971607 6.615099,26.33559 2.290454,3.78888 -7.136335,18.96983 -3.810585,21.73443 3.138096,2.60861 18.971963,-7.14297 23.031819,-5.44631 8.404089,3.53637 17.702673,5.30456 27.895752,5.30456 10.90035,0 20.032515,-1.83059 27.396492,-5.49178 7.36399,-3.66119 12.87657,-8.11286 16.53776,-13.35501 l 9.29559,4 c -2.12183,4.36846 -3.76221,4.82013 -8.92116,9.35501 -5.15895,4.53488 -11.2956,8.11286 -18.40995,10.73393 -7.114346,2.66268 -15.684851,3.99402 -25.711512,3.99402 -9.236177,0 -17.76508,-1.18572 -25.586707,-3.55717 -7.780023,-2.37145 -29.296198,9.26152 -34.78798,4.47701 -5.49178,-4.7429 5.248856,-25.42482 2.461361,-31.62388 -3.49477,-7.863231 -5.242155,-16.350531 -5.242155,-25.461894 0,-10.151474 2.08022,-19.824498 6.240661,-29.019071 5.075736,-11.274793 12.273297,-19.907706 21.592683,-25.898739 9.360991,-5.991034 20.69819,-8.986551 34.011599,-8.986551 10.317891,0 19.574873,2.121824 27.77093,6.365473 8.23767,4.202045 14.72796,10.484309 19.47086,18.846794 4.03563,7.197561 6.05344,15.019189 6.05344,23.464883 0,12.065277 -4.24365,22.77841 -12.73094,32.1394 -7.572,8.404095 -15.85128,12.606135 -24.837827,12.606135 -2.870704,0 -5.200551,-0.43684 -6.98954,-1.31053 -1.747385,-0.8737 -3.037121,-2.12183 -3.869209,-3.744402 -0.540857,-1.040114 -0.936099,-2.829105 -1.185726,-5.366972 z M 49.723082,77.510217 c 0,5.699803 1.352143,10.130671 4.05643,13.292606 2.704286,3.161935 5.803814,4.742902 9.298583,4.742902 2.329847,0 4.784506,-0.686473 7.363979,-2.059418 2.579473,-1.41455 5.034133,-3.49477 7.363979,-6.240661 2.371451,-2.74589 4.306056,-6.219857 5.803815,-10.421902 1.497759,-4.243649 2.246638,-8.487298 2.246638,-12.730947 0,-5.658198 -1.41455,-10.047462 -4.243649,-13.167793 -2.787495,-3.12033 -6.199056,-4.680495 -10.234683,-4.680495 -2.662682,0 -5.179749,0.686473 -7.5512,2.059418 -2.329846,1.331341 -4.597286,3.494769 -6.802319,6.490286 -2.205033,2.995517 -3.97322,6.635903 -5.304561,10.921156 -1.331341,4.285253 -1.997012,8.216869 -1.997012,11.794848 z" style="stroke-width:.887561"/></g></svg>
|
||||
|
Before Width: | Height: | Size: 3.5 KiB |
@@ -1,21 +0,0 @@
|
||||
/* Tweak how the sidebar logo is presented */
|
||||
.sidebar-logo {
|
||||
width: 70%;
|
||||
}
|
||||
.sidebar-brand {
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
/* The landing pages' sidebar-in-content highlights */
|
||||
#features ul {
|
||||
padding-left: 1rem;
|
||||
list-style: none;
|
||||
}
|
||||
#features ul li {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
@media (min-width: 46em) {
|
||||
#features {
|
||||
width: 50%;
|
||||
}
|
||||
}
|
||||
@@ -1,41 +0,0 @@
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# For the full list of built-in configuration values, see the documentation:
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
|
||||
|
||||
project = 'chatmail relay documentation'
|
||||
copyright = '2025, chatmail collective'
|
||||
author = 'chatmail collective'
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
|
||||
|
||||
extensions = [
|
||||
#'sphinx.ext.autodoc',
|
||||
#'sphinx.ext.viewdoc',
|
||||
'sphinxcontrib.mermaid',
|
||||
]
|
||||
|
||||
templates_path = ['_templates']
|
||||
exclude_patterns = []
|
||||
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
|
||||
|
||||
html_theme = 'furo'
|
||||
html_static_path = ['_static']
|
||||
html_css_files = [
|
||||
"custom.css",
|
||||
]
|
||||
|
||||
html_title = "chatmail relay documentation"
|
||||
#html_short_title = f"chatmail-{release}"
|
||||
|
||||
html_logo = "_static/chatmail.svg"
|
||||
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
|
||||
|
||||
Frequently asked questions
|
||||
===========================
|
||||
|
||||
What is the difference between chatmail relays and classic email servers?
|
||||
--------------------------------------------------------------------------
|
||||
|
||||
A chatmail relay is a minimal Mail Transport Agent (MTA) setup that
|
||||
goes beyond what classic email servers offer:
|
||||
|
||||
- **Zero State:** no private data or metadata collected, messages are auto-deleted, low disk usage
|
||||
|
||||
- **Instant/Realtime:** sub-second message delivery, realtime P2P
|
||||
streaming, privacy-preserving Push Notifications for Apple, Google, and `Ubuntu Touch <https://docs.ubports.com/en/latest/appdev/guides/pushnotifications.html>`_;
|
||||
|
||||
- **Security Enforcement**: only strict TLS, DKIM and OpenPGP with minimized metadata accepted
|
||||
|
||||
- **Reliable Federation and Decentralization:** No spam or IP reputation checks, federating
|
||||
depends on established IETF standards and protocols.
|
||||
|
||||
|
||||
How about interoperability with classic email servers?
|
||||
-------------------------------------------------------
|
||||
|
||||
Generally, chatmail relays interoperate well with classic email servers.
|
||||
However, some chatmail relays may be blocked by Big-Tech email
|
||||
providers that use intransparent and proprietary techniques for scanning
|
||||
and looking at cleartext email messages between users, or because they
|
||||
use questionable IP-reputation systems that break interoperability.
|
||||
|
||||
**Chatmail relays instead use and require strong cryptography, allowing
|
||||
anyone to participate, without having to submit to Big-Tech
|
||||
restrictions.**
|
||||
|
||||
.. _selfhosted:
|
||||
|
||||
How are chatmail relays run? Can I run one myself?
|
||||
--------------------------------------------------
|
||||
|
||||
Chatmail relays are designed to be very cheap to run, and are generally
|
||||
self-funded by respective operators. All chatmail relays are
|
||||
automatically deployed and updated using `the chatmail relay
|
||||
repository <https://github.com/chatmail/relay>`__. Chatmail relays are
|
||||
composed of proven standard email server components, Postfix and
|
||||
Dovecot, and are configured to run unattended without much maintenance
|
||||
effort. Chatmail relays happily run on low-end hardware like a Raspberry
|
||||
Pi.
|
||||
|
||||
|
||||
How trustable are chatmail relays?
|
||||
----------------------------------
|
||||
|
||||
Chatmail relays enforce end-to-end encryption,
|
||||
and chatmail clients like `Delta Chat <https://delta.chat>`_
|
||||
enforce end-to-end encryption on their own.
|
||||
|
||||
The end-to-end encryption protection includes attached media, user
|
||||
display names, avatars and group names. What is visible to operators is:
|
||||
message date, sender and receiver addresses.
|
||||
Please see the `Delta Chat FAQ on encryption and security <https://delta.chat/en/help#e2ee>`_ for further info.
|
||||
@@ -1,169 +0,0 @@
|
||||
Setting up a chatmail relay
|
||||
===========================
|
||||
|
||||
This section contains everything needed to setup a ready-to-use chatmail relay.
|
||||
The automated setup is designed and optimized for providing chatmail
|
||||
addresses for immediate permission-free onboarding through chat apps and bots.
|
||||
Chatmail addresses are automatically created at first login,
|
||||
after which the initially specified password is required
|
||||
for sending and receiving messages through them.
|
||||
|
||||
|
||||
Minimal requirements and prerequisites
|
||||
--------------------------------------
|
||||
|
||||
You will need the following:
|
||||
|
||||
- Control over a domain through a DNS provider of your choice.
|
||||
|
||||
- A Debian 12 server with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
|
||||
IPv6 is encouraged if available. Chatmail relay servers only require
|
||||
1GB RAM, one CPU, and perhaps 10GB storage for a few thousand active
|
||||
chatmail addresses.
|
||||
|
||||
- Key-based SSH authentication to the root user. You must add a
|
||||
passphrase-protected private key to your local ssh-agent because you
|
||||
can’t type in your passphrase during deployment. (An ed25519 private
|
||||
key is required due to an `upstream bug in
|
||||
paramiko <https://github.com/paramiko/paramiko/issues/2191>`_)
|
||||
|
||||
|
||||
Setup with ``scripts/cmdeploy``
|
||||
-------------------------------------
|
||||
|
||||
We use ``chat.example.org`` as the chatmail domain in the following
|
||||
steps. Please substitute it with your own domain.
|
||||
|
||||
1. Setup the initial DNS records. The following is an example in the
|
||||
familiar BIND zone file format with a TTL of 1 hour (3600 seconds).
|
||||
Please substitute your domain and IP addresses.
|
||||
|
||||
::
|
||||
|
||||
chat.example.com. 3600 IN A 198.51.100.5
|
||||
chat.example.com. 3600 IN AAAA 2001:db8::5
|
||||
www.chat.example.com. 3600 IN CNAME chat.example.com.
|
||||
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
|
||||
|
||||
2. On your local PC, clone the repository and bootstrap the Python
|
||||
virtualenv.
|
||||
|
||||
::
|
||||
|
||||
git clone https://github.com/chatmail/relay
|
||||
cd relay
|
||||
scripts/initenv.sh
|
||||
|
||||
3. On your local PC, create chatmail configuration file
|
||||
``chatmail.ini``:
|
||||
|
||||
::
|
||||
|
||||
scripts/cmdeploy init chat.example.org # <-- use your domain
|
||||
|
||||
4. Verify that SSH root login to your remote server works:
|
||||
|
||||
::
|
||||
|
||||
ssh root@chat.example.org # <-- use your domain
|
||||
|
||||
5. From your local PC, deploy the remote chatmail relay server:
|
||||
|
||||
::
|
||||
|
||||
scripts/cmdeploy run
|
||||
|
||||
This script will also check that you have all necessary DNS records.
|
||||
If DNS records are missing, it will recommend which you should
|
||||
configure at your DNS provider (it can take some time until they are
|
||||
public).
|
||||
|
||||
Other helpful commands
|
||||
----------------------
|
||||
|
||||
To check the status of your remotely running chatmail service:
|
||||
|
||||
::
|
||||
|
||||
scripts/cmdeploy status
|
||||
|
||||
To display and check all recommended DNS records:
|
||||
|
||||
::
|
||||
|
||||
scripts/cmdeploy dns
|
||||
|
||||
To test whether your chatmail service is working correctly:
|
||||
|
||||
::
|
||||
|
||||
scripts/cmdeploy test
|
||||
|
||||
To measure the performance of your chatmail service:
|
||||
|
||||
::
|
||||
|
||||
scripts/cmdeploy bench
|
||||
|
||||
|
||||
|
||||
Modifying the home page
|
||||
-----------------------
|
||||
|
||||
``cmdeploy run`` also creates default static web pages and deploys them
|
||||
to a Nginx web server with:
|
||||
|
||||
- a default ``index.html`` along with a QR code that users can click to
|
||||
create an address on your chatmail relay
|
||||
|
||||
- a default ``info.html`` that is linked from the home page
|
||||
|
||||
- a default ``policy.html`` that is linked from the home page
|
||||
|
||||
All ``.html`` files are generated by the according markdown ``.md`` file
|
||||
in the ``www/src`` directory.
|
||||
|
||||
Refining the web pages
|
||||
----------------------
|
||||
|
||||
::
|
||||
|
||||
scripts/cmdeploy webdev
|
||||
|
||||
This starts a local live development cycle for chatmail web pages:
|
||||
|
||||
- uses the ``www/src/page-layout.html`` file for producing static HTML
|
||||
pages from ``www/src/*.md`` files
|
||||
|
||||
- continously builds the web presence reading files from ``www/src``
|
||||
directory and generating HTML files and copying assets to the
|
||||
``www/build`` directory.
|
||||
|
||||
- Starts a browser window automatically where you can “refresh” as
|
||||
needed.
|
||||
|
||||
Custom web pages
|
||||
----------------
|
||||
|
||||
You can skip uploading a web page by setting ``www_folder=disabled`` in
|
||||
``chatmail.ini``.
|
||||
|
||||
If you want to manage your web pages outside this git repository, you
|
||||
can set ``www_folder`` in ``chatmail.ini`` to a custom directory on your
|
||||
computer. ``cmdeploy run`` will upload it as the server’s home page, and
|
||||
if it contains a ``src/index.md`` file, will build it with hugo.
|
||||
|
||||
|
||||
Disable automatic address creation
|
||||
--------------------------------------------------------
|
||||
|
||||
If you need to stop address creation, e.g. because some script is wildly
|
||||
creating addresses, login with ssh and run:
|
||||
|
||||
::
|
||||
|
||||
touch /etc/chatmail-nocreate
|
||||
|
||||
Chatmail address creation will be denied while this file is present.
|
||||
|
||||
|
||||
@@ -1,20 +0,0 @@
|
||||
*******************************************
|
||||
chatmail relay documentation
|
||||
*******************************************
|
||||
|
||||
.. image:: ../../www/src/collage-top.png
|
||||
:target: https://testrun.org
|
||||
|
||||
This documentation details how to setup, maintain and understand `chatmail <https://chatmail.at>`_ relays.
|
||||
|
||||
Contributions and feedback welcome through the https://github.com/chatmail/relay repository.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 5
|
||||
|
||||
getting_started
|
||||
proxy
|
||||
migrate
|
||||
overview
|
||||
related
|
||||
faq
|
||||
@@ -1,73 +0,0 @@
|
||||
|
||||
Migrating to a new host
|
||||
-----------------------
|
||||
|
||||
If you want to migrate chatmail relay from an old machine to a new
|
||||
machine, you can use these steps. They were tested with a Linux laptop;
|
||||
you might need to adjust some of the steps to your environment.
|
||||
|
||||
Let’s assume that your ``mail_domain`` is ``mail.example.org``, all
|
||||
involved machines run Debian 12, your old site’s IP address is
|
||||
``13.37.13.37``, and your new site’s IP address is ``13.12.23.42``.
|
||||
|
||||
Note, you should lower the TTLs of your DNS records to a value such as
|
||||
300 (5 minutes) so the migration happens as smoothly as possible.
|
||||
|
||||
During the guide you might get a warning about changed SSH Host keys; in
|
||||
this case, just run ``ssh-keygen -R "mail.example.org"`` as recommended.
|
||||
|
||||
1. First, disable mail services on the old site.
|
||||
|
||||
::
|
||||
|
||||
cmdeploy run --disable-mail --ssh-host 13.37.13.37
|
||||
|
||||
Now your users will notice the migration and will not be able to send
|
||||
or receive messages until the migration is completed.
|
||||
|
||||
2. Now we want to copy ``/home/vmail``, ``/var/lib/acme``,
|
||||
``/etc/dkimkeys``, ``/run/echobot``, and ``/var/spool/postfix`` to
|
||||
the new site. Login to the old site while forwarding your SSH agent
|
||||
so you can copy directly from the old to the new site with your SSH
|
||||
key:
|
||||
|
||||
::
|
||||
|
||||
ssh -A root@13.37.13.37
|
||||
tar c - /home/vmail/mail /var/lib/acme /etc/dkimkeys /run/echobot /var/spool/postfix | ssh root@13.12.23.42 "tar x -C /"
|
||||
|
||||
This transfers all addresses, the TLS certificate, DKIM keys (so DKIM
|
||||
DNS record remains valid), and the echobot’s password so it continues
|
||||
to function. It also preserves the Postfix mail spool so any messages
|
||||
pending delivery will still be delivered.
|
||||
|
||||
3. Install chatmail on the new machine:
|
||||
|
||||
::
|
||||
|
||||
cmdeploy run --disable-mail --ssh-host 13.12.23.42
|
||||
|
||||
Postfix and Dovecot are disabled for now; we will enable them later.
|
||||
We first need to make the new site fully operational.
|
||||
|
||||
4. On the new site, run the following to ensure the ownership is correct
|
||||
in case UIDs/GIDs changed:
|
||||
|
||||
::
|
||||
|
||||
chown root: -R /var/lib/acme
|
||||
chown opendkim: -R /etc/dkimkeys
|
||||
chown vmail: -R /home/vmail/mail
|
||||
chown echobot: -R /run/echobot
|
||||
|
||||
5. Now, update DNS entries.
|
||||
|
||||
If other MTAs try to deliver messages to your chatmail domain they
|
||||
may fail intermittently, as DNS catches up with the new site settings
|
||||
but normally will retry delivering messages for at least a week, so
|
||||
messages will not be lost.
|
||||
|
||||
6. Finally, you can execute ``cmdeploy run --ssh-host 13.12.23.42`` to
|
||||
turn on chatmail on the new relay. Your users will be able to use the
|
||||
chatmail relay as soon as the DNS changes have propagated. Voilà!
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user