mirror of
https://github.com/chatmail/relay.git
synced 2026-05-10 16:04:37 +00:00
Compare commits
3 Commits
hpk/increa
...
missytake/
| Author | SHA1 | Date | |
|---|---|---|---|
|
f26d2da23b | ||
|
3b32387bf8 | ||
|
|
a651fba8b6 |
33
.github/ISSUE_TEMPLATE/bug_report.md
vendored
33
.github/ISSUE_TEMPLATE/bug_report.md
vendored
@@ -1,33 +0,0 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Report something that isn't working.
|
||||
title: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
Please fill out as much of this form as you can (leaving out stuff that is not applicable is ok).
|
||||
-->
|
||||
|
||||
- Server OS (Operating System) - preferably Debian 12:
|
||||
- On which OS you run cmdeploy:
|
||||
- chatmail/relay version: `git rev-parse HEAD`
|
||||
|
||||
## Expected behavior
|
||||
|
||||
*What did you try to achieve?*
|
||||
|
||||
## Actual behavior
|
||||
|
||||
*What happened instead?*
|
||||
|
||||
### Steps to reproduce the problem:
|
||||
|
||||
1.
|
||||
2.
|
||||
|
||||
### Screenshots
|
||||
|
||||
### Logs
|
||||
|
||||
5
.github/ISSUE_TEMPLATE/config.yml
vendored
5
.github/ISSUE_TEMPLATE/config.yml
vendored
@@ -1,5 +0,0 @@
|
||||
blank_issues_enabled: true
|
||||
contact_links:
|
||||
- name: Mutual Help Chat Group
|
||||
url: https://i.delta.chat/#C2846EB4C1CB8DF84B1818F5E3A638FC3FBDC981&a=stalebot1%40nine.testrun.org&g=Chatmail%20Mutual%20Help&x=7sFF7Ik50pWv6J1z7RVC5527&i=d7s1HvOsk5UrSf9AoqRZggg4&s=XmX_9BAW6-g5Ao5E8PyaeKNB
|
||||
about: If you have troubles setting up the relay server, feel free to ask here.
|
||||
14
.github/workflows/test-and-deploy-ipv4only.yaml
vendored
14
.github/workflows/test-and-deploy-ipv4only.yaml
vendored
@@ -38,8 +38,8 @@ jobs:
|
||||
if [ -f dkimkeys-ipv4/dkimkeys/opendkim.private ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" dkimkeys-ipv4 root@ns.testrun.org:/tmp/ || true; fi
|
||||
if [ "$(ls -A acme-ipv4/acme/certs)" ]; then rsync -avz -e "ssh -o StrictHostKeyChecking=accept-new" acme-ipv4 root@ns.testrun.org:/tmp/ || true; fi
|
||||
# make sure CAA record isn't set
|
||||
scp -o StrictHostKeyChecking=accept-new .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging-ipv4.testrun.org.zone
|
||||
scp .github/workflows/staging-ipv4.testrun.org-default.zone root@ns.testrun.org:/etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh -o StrictHostKeyChecking=accept-new root@ns.testrun.org sed -i '/CAA/d' /etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org nsd-checkzone staging-ipv4.testrun.org /etc/nsd/staging-ipv4.testrun.org.zone
|
||||
ssh root@ns.testrun.org systemctl reload nsd
|
||||
|
||||
@@ -49,7 +49,7 @@ jobs:
|
||||
-H "Authorization: Bearer ${{ secrets.HETZNER_API_TOKEN }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"image":"debian-12"}' \
|
||||
"https://api.hetzner.cloud/v1/servers/${{ secrets.STAGING_IPV4_SERVER_ID }}/actions/rebuild"
|
||||
"https://api.hetzner.cloud/v1/servers/${{ secrets.STAGING_SERVER_ID }}/actions/rebuild"
|
||||
|
||||
- run: scripts/initenv.sh
|
||||
|
||||
@@ -63,11 +63,11 @@ jobs:
|
||||
while ! ssh -o ConnectTimeout=180 -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org id -u ; do sleep 1 ; done
|
||||
ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org id -u
|
||||
# download acme & dkim state from ns.testrun.org
|
||||
rsync -e "ssh -o StrictHostKeyChecking=accept-new" -avz root@ns.testrun.org:/tmp/acme-ipv4/acme acme-restore || true
|
||||
rsync -avz root@ns.testrun.org:/tmp/dkimkeys-ipv4/dkimkeys dkimkeys-restore || true
|
||||
rsync -e "ssh -o StrictHostKeyChecking=accept-new" -avz root@ns.testrun.org:/tmp/acme-ipv4 acme-restore || true
|
||||
rsync -avz root@ns.testrun.org:/tmp/dkimkeys-ipv4 dkimkeys-restore || true
|
||||
# restore acme & dkim state to staging2.testrun.org
|
||||
rsync -avz acme-restore/acme root@staging-ipv4.testrun.org:/var/lib/ || true
|
||||
rsync -avz dkimkeys-restore/dkimkeys root@staging-ipv4.testrun.org:/etc/ || true
|
||||
rsync -avz acme-restore/acme-ipv4/acme root@staging-ipv4.testrun.org:/var/lib/ || true
|
||||
rsync -avz dkimkeys-restore/dkimkeys-ipv4/dkimkeys root@staging-ipv4.testrun.org:/etc/ || true
|
||||
ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org chown root:root -R /var/lib/acme || true
|
||||
|
||||
- name: run formatting checks
|
||||
|
||||
263
CHANGELOG.md
263
CHANGELOG.md
@@ -2,149 +2,42 @@
|
||||
|
||||
## untagged
|
||||
|
||||
- Increase dovecot imap-login limits
|
||||
([#577](https://github.com/chatmail/relay/pull/577))
|
||||
|
||||
- Increase nginx connection limits
|
||||
([#576](https://github.com/chatmail/relay/pull/576))
|
||||
|
||||
- If `dns-utils` needs to be installed before cmdeploy run, apt update to make sure it works
|
||||
([#560](https://github.com/chatmail/relay/pull/560))
|
||||
|
||||
- filtermail: respect config message size limit
|
||||
([#572](https://github.com/chatmail/relay/pull/572))
|
||||
|
||||
- Add config value after how many days large files are deleted
|
||||
([#555](https://github.com/chatmail/relay/pull/555))
|
||||
|
||||
- cmdeploy: push relay version to /etc/chatmail-version
|
||||
([#573](https://github.com/chatmail/relay/pull/573))
|
||||
|
||||
- filtermail: allow partial body length in OpenPGP payloads
|
||||
([#570](https://github.com/chatmail/relay/pull/570))
|
||||
|
||||
- chatmaild: allow echobot to receive unencrypted messages by default
|
||||
([#556](https://github.com/chatmail/relay/pull/556))
|
||||
|
||||
|
||||
## 1.6.0 2025-04-11
|
||||
|
||||
- Handle Port-25 connect errors more gracefully (common with VPNs)
|
||||
([#552](https://github.com/chatmail/relay/pull/552))
|
||||
|
||||
- Avoid "acmetool not found" during initial run
|
||||
([#550](https://github.com/chatmail/relay/pull/550))
|
||||
|
||||
- Fix timezone handling such that client/servers do not need to use
|
||||
same timezone.
|
||||
([#553](https://github.com/chatmail/relay/pull/553))
|
||||
|
||||
- Enforce end-to-end encryption for incoming messages.
|
||||
New user address mailboxes now get a `enforceE2EEincoming` file
|
||||
which prohibits incoming cleartext messages from other domains.
|
||||
An outside MTA trying to submit a cleartext message will
|
||||
get a "523 Encryption Needed" response, see RFC5248.
|
||||
If the file does not exist (as it the case for all existing accounts)
|
||||
incoming cleartext messages are accepted.
|
||||
([#538](https://github.com/chatmail/server/pull/538))
|
||||
|
||||
- Enforce end-to-end encryption between local addresses
|
||||
([#535](https://github.com/chatmail/server/pull/535))
|
||||
|
||||
- unbound: check that port 53 is not occupied by a different process
|
||||
([#537](https://github.com/chatmail/server/pull/537))
|
||||
|
||||
- unbound: before unbound is there, use 9.9.9.9 for resolving
|
||||
([#518](https://github.com/chatmail/relay/pull/518))
|
||||
|
||||
- Limit the bind for the HTTPS server on 8443 to 127.0.0.1
|
||||
([#522](https://github.com/chatmail/server/pull/522))
|
||||
([#532](https://github.com/chatmail/server/pull/532))
|
||||
|
||||
- Send SNI when connecting to outside servers
|
||||
([#524](https://github.com/chatmail/server/pull/524))
|
||||
|
||||
- postfix master.cf: use 127.0.0.1 for consistency
|
||||
([#544](https://github.com/chatmail/relay/pull/544))
|
||||
|
||||
- Pass through `original_content` instead of `content` in filtermail
|
||||
([#509](https://github.com/chatmail/server/pull/509))
|
||||
|
||||
- Document TLS requirements in the readme
|
||||
([#514](https://github.com/chatmail/server/pull/514))
|
||||
|
||||
- Remove cleanup service from submission ports
|
||||
([#512](https://github.com/chatmail/server/pull/512))
|
||||
|
||||
- cmdeploy dovecot: delete big messages after 7 days
|
||||
([#504](https://github.com/chatmail/server/pull/504))
|
||||
|
||||
- mtail: fix getting logs from STDIN
|
||||
([#502](https://github.com/chatmail/server/pull/502))
|
||||
|
||||
- filtermail: don't require exactly 2 lines after openPGP payload
|
||||
([#497](https://github.com/chatmail/server/pull/497))
|
||||
|
||||
- cmdeploy dns: offer alternative DKIM record format for some web interfaces
|
||||
([#470](https://github.com/chatmail/server/pull/470))
|
||||
|
||||
- journald: remove old logs from disk
|
||||
([#490](https://github.com/chatmail/server/pull/490))
|
||||
|
||||
- opendkim: restart once every day to mend RAM leaks
|
||||
([#498](https://github.com/chatmail/server/pull/498)
|
||||
([#470](https://github.com/deltachat/chatmail/pull/470))
|
||||
|
||||
- migration guide: let opendkim own the DKIM keys directory
|
||||
([#468](https://github.com/chatmail/server/pull/468))
|
||||
|
||||
- improve secure-join message detection
|
||||
([#473](https://github.com/chatmail/server/pull/473))
|
||||
|
||||
- use old crypt lib in python < 3.11
|
||||
([#483](https://github.com/chatmail/server/pull/483))
|
||||
|
||||
- chatmaild: set umask to 0700 for doveauth + metadata
|
||||
([#490](https://github.com/chatmail/server/pull/492))
|
||||
|
||||
- remove MTA-STS daemon
|
||||
([#488](https://github.com/chatmail/server/pull/488))
|
||||
|
||||
- replace `Subject` with `[...]` for all outgoing mails.
|
||||
([#481](https://github.com/chatmail/server/pull/481))
|
||||
|
||||
- opendkim: use su instead of sudo
|
||||
([#491](https://github.com/chatmail/server/pull/491))
|
||||
([#468](https://github.com/deltachat/chatmail/pull/468))
|
||||
|
||||
## 1.5.0 2024-12-20
|
||||
|
||||
- cmdeploy dns: always show recommended DNS records
|
||||
([#463](https://github.com/chatmail/server/pull/463))
|
||||
([#463](https://github.com/deltachat/chatmail/pull/463))
|
||||
|
||||
- add `--all` to `cmdeploy dns`
|
||||
([#462](https://github.com/chatmail/server/pull/462))
|
||||
([#462](https://github.com/deltachat/chatmail/pull/462))
|
||||
|
||||
- fix `_mta-sts` TXT DNS record
|
||||
([#461](https://github.com/chatmail/server/pull/461)
|
||||
([#461](https://github.com/deltachat/chatmail/pull/461)
|
||||
|
||||
- deploy `iroh-relay` and also update "realtime relay services" in privacy policy.
|
||||
([#434](https://github.com/chatmail/server/pull/434))
|
||||
([#451](https://github.com/chatmail/server/pull/451))
|
||||
([#434](https://github.com/deltachat/chatmail/pull/434))
|
||||
([#451](https://github.com/deltachat/chatmail/pull/451))
|
||||
|
||||
- add guide to migrate chatmail to a new server
|
||||
([#429](https://github.com/chatmail/server/pull/429))
|
||||
([#429](https://github.com/deltachat/chatmail/pull/429))
|
||||
|
||||
- disable anvil authentication penalty
|
||||
([#414](https://github.com/chatmail/server/pull/444)
|
||||
([#414](https://github.com/deltachat/chatmail/pull/444)
|
||||
|
||||
- increase `request_queue_size` for UNIX sockets to 1000.
|
||||
([#437](https://github.com/chatmail/server/pull/437))
|
||||
([#437](https://github.com/deltachat/chatmail/pull/437))
|
||||
|
||||
- add argument to `cmdeploy run` for specifying
|
||||
a different SSH host than `mail_domain`
|
||||
([#439](https://github.com/chatmail/server/pull/439))
|
||||
([#439](https://github.com/deltachat/chatmail/pull/439))
|
||||
|
||||
- query autoritative nameserver to bypass DNS cache
|
||||
([#424](https://github.com/chatmail/server/pull/424))
|
||||
([#424](https://github.com/deltachat/chatmail/pull/424))
|
||||
|
||||
- add mtail support (new optional `mtail_address` ini value)
|
||||
This defines the address on which [`mtail`](https://google.github.io/mtail/)
|
||||
@@ -154,195 +47,195 @@
|
||||
and assign an IP address from this network to the host.
|
||||
If you do not plan to collect metrics,
|
||||
keep this setting unset.
|
||||
([#388](https://github.com/chatmail/server/pull/388))
|
||||
([#388](https://github.com/deltachat/chatmail/pull/388))
|
||||
|
||||
- fix checking for required DNS records
|
||||
([#412](https://github.com/chatmail/server/pull/412))
|
||||
([#412](https://github.com/deltachat/chatmail/pull/412))
|
||||
|
||||
- add support for specifying whole domains for recipient passthrough list
|
||||
([#408](https://github.com/chatmail/server/pull/408))
|
||||
([#408](https://github.com/deltachat/chatmail/pull/408))
|
||||
|
||||
- add a paragraph about "account deletion" to info page
|
||||
([#405](https://github.com/chatmail/server/pull/405))
|
||||
([#405](https://github.com/deltachat/chatmail/pull/405))
|
||||
|
||||
- avoid nginx listening on ipv6 if v6 is dsiabled
|
||||
([#402](https://github.com/chatmail/server/pull/402))
|
||||
([#402](https://github.com/deltachat/chatmail/pull/402))
|
||||
|
||||
- refactor ssh-based execution to allow organizing remote functions in
|
||||
modules.
|
||||
([#396](https://github.com/chatmail/server/pull/396))
|
||||
([#396](https://github.com/deltachat/chatmail/pull/396))
|
||||
|
||||
- trigger "apt upgrade" during "cmdeploy run"
|
||||
([#398](https://github.com/chatmail/server/pull/398))
|
||||
([#398](https://github.com/deltachat/chatmail/pull/398))
|
||||
|
||||
- drop hispanilandia passthrough address
|
||||
([#401](https://github.com/chatmail/server/pull/401))
|
||||
([#401](https://github.com/deltachat/chatmail/pull/401))
|
||||
|
||||
- set CAA record flags to 0
|
||||
|
||||
- add IMAP capabilities instead of overwriting them
|
||||
([#413](https://github.com/chatmail/server/pull/413))
|
||||
([#413](https://github.com/deltachat/chatmail/pull/413))
|
||||
|
||||
- fix OpenPGP payload check
|
||||
([#435](https://github.com/chatmail/server/pull/435))
|
||||
([#435](https://github.com/deltachat/chatmail/pull/435))
|
||||
|
||||
- fix Dovecot quota_max_mail_size to use max_message_size config value
|
||||
([#438](https://github.com/chatmail/server/pull/438))
|
||||
([#438](https://github.com/deltachat/chatmail/pull/438))
|
||||
|
||||
|
||||
## 1.4.1 2024-07-31
|
||||
|
||||
- fix metadata dictproxy which would confuse transactions
|
||||
resulting in missed notifications and other issues.
|
||||
([#393](https://github.com/chatmail/server/pull/393))
|
||||
([#394](https://github.com/chatmail/server/pull/394))
|
||||
([#393](https://github.com/deltachat/chatmail/pull/393))
|
||||
([#394](https://github.com/deltachat/chatmail/pull/394))
|
||||
|
||||
- add optional "imap_rawlog" config option. If true,
|
||||
.in/.out files are created in user home dirs
|
||||
containing the imap protocol messages.
|
||||
([#389](https://github.com/chatmail/server/pull/389))
|
||||
([#389](https://github.com/deltachat/chatmail/pull/389))
|
||||
|
||||
## 1.4.0 2024-07-28
|
||||
|
||||
- Add `disable_ipv6` config option to chatmail.ini.
|
||||
Required if the server doesn't have IPv6 connectivity.
|
||||
([#312](https://github.com/chatmail/server/pull/312))
|
||||
([#312](https://github.com/deltachat/chatmail/pull/312))
|
||||
|
||||
- allow current K9/Thunderbird-mail releases to send encrypted messages
|
||||
outside by accepting their localized "encrypted subject" strings.
|
||||
([#370](https://github.com/chatmail/server/pull/370))
|
||||
([#370](https://github.com/deltachat/chatmail/pull/370))
|
||||
|
||||
- Migrate and remove sqlite database in favor of password/lastlogin tracking
|
||||
in a user's maildir.
|
||||
([#379](https://github.com/chatmail/server/pull/379))
|
||||
([#379](https://github.com/deltachat/chatmail/pull/379))
|
||||
|
||||
- Require pyinfra V3 installed on the client side,
|
||||
run `./scripts/initenv.sh` to upgrade locally.
|
||||
([#378](https://github.com/chatmail/server/pull/378))
|
||||
([#378](https://github.com/deltachat/chatmail/pull/378))
|
||||
|
||||
- don't hardcode "/home/vmail" paths but rather set them
|
||||
once in the config object and use it everywhere else,
|
||||
thereby also improving testability.
|
||||
([#351](https://github.com/chatmail/server/pull/351))
|
||||
([#351](https://github.com/deltachat/chatmail/pull/351))
|
||||
temporarily introduced obligatory "passdb_path" and "mailboxes_dir"
|
||||
settings but they were removed/obsoleted in
|
||||
([#380](https://github.com/chatmail/server/pull/380))
|
||||
([#380](https://github.com/deltachat/chatmail/pull/380))
|
||||
|
||||
- BREAKING: new required chatmail.ini value 'delete_inactive_users_after = 100'
|
||||
which removes users from database and mails after 100 days without any login.
|
||||
([#350](https://github.com/chatmail/server/pull/350))
|
||||
([#350](https://github.com/deltachat/chatmail/pull/350))
|
||||
|
||||
- Refine DNS checking to distinguish between "required" and "recommended" settings
|
||||
([#372](https://github.com/chatmail/server/pull/372))
|
||||
([#372](https://github.com/deltachat/chatmail/pull/372))
|
||||
|
||||
- reload nginx in the acmetool cronjob
|
||||
([#360](https://github.com/chatmail/server/pull/360))
|
||||
([#360](https://github.com/deltachat/chatmail/pull/360))
|
||||
|
||||
- remove checking of reverse-DNS PTR records. Chatmail-servers don't
|
||||
depend on it and even in the wider e-mail system it's not common anymore.
|
||||
If it's an issue, a chatmail operator can still care to properly set reverse DNS.
|
||||
([#348](https://github.com/chatmail/server/pull/348))
|
||||
([#348](https://github.com/deltachat/chatmail/pull/348))
|
||||
|
||||
- Make DNS-checking faster and more interactive, run it fully during "cmdeploy run",
|
||||
also introducing a generic mechanism for rapid remote ssh-based python function execution.
|
||||
([#346](https://github.com/chatmail/server/pull/346))
|
||||
([#346](https://github.com/deltachat/chatmail/pull/346))
|
||||
|
||||
- Don't fix file owner ship of /home/vmail
|
||||
([#345](https://github.com/chatmail/server/pull/345))
|
||||
([#345](https://github.com/deltachat/chatmail/pull/345))
|
||||
|
||||
- Support iterating over all users with doveadm commands
|
||||
([#344](https://github.com/chatmail/server/pull/344))
|
||||
([#344](https://github.com/deltachat/chatmail/pull/344))
|
||||
|
||||
- Test and fix for attempts to create inadmissible accounts
|
||||
([#333](https://github.com/chatmail/server/pull/321))
|
||||
([#333](https://github.com/deltachat/chatmail/pull/321))
|
||||
|
||||
- check that OpenPGP has only PKESK, SKESK and SEIPD packets
|
||||
([#323](https://github.com/chatmail/server/pull/323),
|
||||
[#324](https://github.com/chatmail/server/pull/324))
|
||||
([#323](https://github.com/deltachat/chatmail/pull/323),
|
||||
[#324](https://github.com/deltachat/chatmail/pull/324))
|
||||
|
||||
- improve filtermail checks for encrypted messages and drop support for unencrypted MDNs
|
||||
([#320](https://github.com/chatmail/server/pull/320))
|
||||
([#320](https://github.com/deltachat/chatmail/pull/320))
|
||||
|
||||
- replace `bash` with `/bin/sh`
|
||||
([#334](https://github.com/chatmail/server/pull/334))
|
||||
([#334](https://github.com/deltachat/chatmail/pull/334))
|
||||
|
||||
- Increase number of logged in IMAP sessions to 50000
|
||||
([#335](https://github.com/chatmail/server/pull/335))
|
||||
([#335](https://github.com/deltachat/chatmail/pull/335))
|
||||
|
||||
- filtermail: do not allow ASCII armor without actual payload
|
||||
([#325](https://github.com/chatmail/server/pull/325))
|
||||
([#325](https://github.com/deltachat/chatmail/pull/325))
|
||||
|
||||
- Remove sieve to enable hardlink deduplication in LMTP
|
||||
([#343](https://github.com/chatmail/server/pull/343))
|
||||
([#343](https://github.com/deltachat/chatmail/pull/343))
|
||||
|
||||
- dovecot: enable gzip compression on disk
|
||||
([#341](https://github.com/chatmail/server/pull/341))
|
||||
([#341](https://github.com/deltachat/chatmail/pull/341))
|
||||
|
||||
- DKIM-sign Content-Type and oversign all signed headers
|
||||
([#296](https://github.com/chatmail/server/pull/296))
|
||||
([#296](https://github.com/deltachat/chatmail/pull/296))
|
||||
|
||||
- Add nonci_accounts metric
|
||||
([#347](https://github.com/chatmail/server/pull/347))
|
||||
([#347](https://github.com/deltachat/chatmail/pull/347))
|
||||
|
||||
- doveauth: log when a new account is created
|
||||
([#349](https://github.com/chatmail/server/pull/349))
|
||||
([#349](https://github.com/deltachat/chatmail/pull/349))
|
||||
|
||||
- Multiplex HTTPS, IMAP and SMTP on port 443
|
||||
([#357](https://github.com/chatmail/server/pull/357))
|
||||
([#357](https://github.com/deltachat/chatmail/pull/357))
|
||||
|
||||
## 1.3.0 - 2024-06-06
|
||||
|
||||
- don't check necessary DNS records on cmdeploy init anymore
|
||||
([#316](https://github.com/chatmail/server/pull/316))
|
||||
([#316](https://github.com/deltachat/chatmail/pull/316))
|
||||
|
||||
- ensure cron and acl are installed
|
||||
([#293](https://github.com/chatmail/server/pull/293),
|
||||
[#310](https://github.com/chatmail/server/pull/310))
|
||||
([#293](https://github.com/deltachat/chatmail/pull/293),
|
||||
[#310](https://github.com/deltachat/chatmail/pull/310))
|
||||
|
||||
- change default for delete_mails_after from 40 to 20 days
|
||||
([#300](https://github.com/chatmail/server/pull/300))
|
||||
([#300](https://github.com/deltachat/chatmail/pull/300))
|
||||
|
||||
- save journald logs only to memory and save nginx logs to journald instead of file
|
||||
([#299](https://github.com/chatmail/server/pull/299))
|
||||
([#299](https://github.com/deltachat/chatmail/pull/299))
|
||||
|
||||
- fix writing of multiple obs repositories in `/etc/apt/sources.list`
|
||||
([#290](https://github.com/chatmail/server/pull/290))
|
||||
([#290](https://github.com/deltachat/chatmail/pull/290))
|
||||
|
||||
- metadata: add support for `/shared/vendor/deltachat/irohrelay`
|
||||
([#284](https://github.com/chatmail/server/pull/284))
|
||||
([#284](https://github.com/deltachat/chatmail/pull/284))
|
||||
|
||||
- Emit "XCHATMAIL" capability from IMAP server
|
||||
([#278](https://github.com/chatmail/server/pull/278))
|
||||
([#278](https://github.com/deltachat/chatmail/pull/278))
|
||||
|
||||
- Move echobot `into /var/lib/echobot`
|
||||
([#281](https://github.com/chatmail/server/pull/281))
|
||||
([#281](https://github.com/deltachat/chatmail/pull/281))
|
||||
|
||||
- Accept Let's Encrypt's new Terms of Services
|
||||
([#275](https://github.com/chatmail/server/pull/276))
|
||||
([#275](https://github.com/deltachat/chatmail/pull/276))
|
||||
|
||||
- Reload Dovecot and Postfix when TLS certificate updates
|
||||
([#271](https://github.com/chatmail/server/pull/271))
|
||||
([#271](https://github.com/deltachat/chatmail/pull/271))
|
||||
|
||||
- Use forked version of dovecot without hardcoded delays
|
||||
([#270](https://github.com/chatmail/server/pull/270))
|
||||
([#270](https://github.com/deltachat/chatmail/pull/270))
|
||||
|
||||
## 1.2.0 - 2024-04-04
|
||||
|
||||
- Install dig on the server to resolve DNS records
|
||||
([#267](https://github.com/chatmail/server/pull/267))
|
||||
([#267](https://github.com/deltachat/chatmail/pull/267))
|
||||
|
||||
- preserve notification order and exponentially backoff with
|
||||
retries for tokens where we didn't get a successful return
|
||||
([#265](https://github.com/chatmail/server/pull/263))
|
||||
([#265](https://github.com/deltachat/chatmail/pull/263))
|
||||
|
||||
- Run chatmail-metadata and doveauth as vmail
|
||||
([#261](https://github.com/chatmail/server/pull/261))
|
||||
([#261](https://github.com/deltachat/chatmail/pull/261))
|
||||
|
||||
- Apply systemd restrictions to echobot
|
||||
([#259](https://github.com/chatmail/server/pull/259))
|
||||
([#259](https://github.com/deltachat/chatmail/pull/259))
|
||||
|
||||
- re-enable running the CI in pull requests, but not concurrently
|
||||
([#258](https://github.com/chatmail/server/pull/258))
|
||||
([#258](https://github.com/deltachat/chatmail/pull/258))
|
||||
|
||||
|
||||
## 1.1.0 - 2024-03-28
|
||||
@@ -350,27 +243,27 @@
|
||||
### The changelog starts to record changes from March 15th, 2024
|
||||
|
||||
- Move systemd unit templates to cmdeploy package
|
||||
([#255](https://github.com/chatmail/server/pull/255))
|
||||
([#255](https://github.com/deltachat/chatmail/pull/255))
|
||||
|
||||
- Persist push tokens and support multiple device per address
|
||||
([#254](https://github.com/chatmail/server/pull/254))
|
||||
([#254](https://github.com/deltachat/chatmail/pull/254))
|
||||
|
||||
- Avoid warning for regular doveauth protocol's hello message.
|
||||
([#250](https://github.com/chatmail/server/pull/250))
|
||||
([#250](https://github.com/deltachat/chatmail/pull/250))
|
||||
|
||||
- Fix various tests to pass again with "cmdeploy test".
|
||||
([#245](https://github.com/chatmail/server/pull/245),
|
||||
[#242](https://github.com/chatmail/server/pull/242)
|
||||
([#245](https://github.com/deltachat/chatmail/pull/245),
|
||||
[#242](https://github.com/deltachat/chatmail/pull/242)
|
||||
|
||||
- Ensure lets-encrypt certificates are reloaded after renewal
|
||||
([#244]) https://github.com/chatmail/server/pull/244
|
||||
([#244]) https://github.com/deltachat/chatmail/pull/244
|
||||
|
||||
- Persist tokens to avoid iOS users loosing push-notifications when the
|
||||
chatmail metadata service is restarted (happens regularly during deploys)
|
||||
([#238](https://github.com/chatmail/server/pull/239)
|
||||
([#238](https://github.com/deltachat/chatmail/pull/239)
|
||||
|
||||
- Fix failing sieve-script compile errors on incoming messages
|
||||
([#237](https://github.com/chatmail/server/pull/239)
|
||||
([#237](https://github.com/deltachat/chatmail/pull/239)
|
||||
|
||||
- Fix quota reporting after expunging of old mails
|
||||
([#233](https://github.com/chatmail/server/pull/239)
|
||||
([#233](https://github.com/deltachat/chatmail/pull/239)
|
||||
|
||||
368
README.md
368
README.md
@@ -1,96 +1,48 @@
|
||||
|
||||
<img width="800px" src="www/src/collage-top.png"/>
|
||||
|
||||
# Chatmail relays for end-to-end encrypted e-mail
|
||||
# Chatmail services optimized for Delta Chat apps
|
||||
|
||||
Chatmail relay servers are interoperable Mail Transport Agents (MTAs) designed for:
|
||||
|
||||
- **Convenience:** Low friction instant onboarding
|
||||
|
||||
- **Privacy:** No name, phone numbers, email required or collected
|
||||
|
||||
- **End-to-End Encryption enforced**: only OpenPGP messages with metadata minimization allowed
|
||||
|
||||
- **Instant:** Privacy-preserving Push Notifications for Apple, Google, and Huawei
|
||||
|
||||
- **Speed:** Message delivery in half a second, with optional P2P realtime connections
|
||||
|
||||
- **Transport Security:** Strict TLS and DKIM enforced
|
||||
|
||||
- **Reliability:** No spam or IP reputation checks; rate-limits are suitable for realtime chats
|
||||
|
||||
- **Efficiency:** Messages are only stored for transit and removed automatically
|
||||
|
||||
This repository contains everything needed to setup a ready-to-use chatmail relay
|
||||
This repository helps to setup a ready-to-use chatmail server
|
||||
comprised of a minimal setup of the battle-tested
|
||||
[Postfix SMTP](https://www.postfix.org) and [Dovecot IMAP](https://www.dovecot.org) MTAs/MDAs.
|
||||
[postfix smtp](https://www.postfix.org) and [dovecot imap](https://www.dovecot.org) services.
|
||||
|
||||
The automated setup is designed and optimized for providing chatmail addresses
|
||||
for immediate permission-free onboarding through chat apps and bots.
|
||||
Chatmail addresses are automatically created at first login,
|
||||
after which the initially specified password is required
|
||||
for sending and receiving messages through them.
|
||||
The setup is designed and optimized for providing chatmail accounts
|
||||
for use by [Delta Chat apps](https://delta.chat).
|
||||
|
||||
Please see [this list of known apps and client projects](https://chatmail.at/clients.html)
|
||||
and [this list of known public 3rd party chatmail relay servers](https://chatmail.at/relays).
|
||||
Chatmail accounts are automatically created by a first login,
|
||||
after which the initially specified password is required for using them.
|
||||
|
||||
## Deploying your own chatmail server
|
||||
|
||||
## Minimal requirements, Prerequisites
|
||||
To deploy chatmail on your own server, you must have set-up ssh authentication and need to use an ed25519 key, due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191). You also need to add your private key to the local ssh-agent, because you can't type in your password during deployment.
|
||||
|
||||
You will need the following:
|
||||
|
||||
- Control over a domain through a DNS provider of your choice.
|
||||
|
||||
- A Debian 12 server with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
|
||||
IPv6 is encouraged if available.
|
||||
Chatmail relay servers only require 1GB RAM, one CPU, and perhaps 10GB storage for a
|
||||
few thousand active chatmail addresses.
|
||||
|
||||
- Key-based SSH authentication to the root user.
|
||||
You must add a passphrase-protected private key to your local ssh-agent
|
||||
because you can't type in your passphrase during deployment.
|
||||
(An ed25519 private key is required due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191))
|
||||
|
||||
|
||||
## Getting started
|
||||
|
||||
We use `chat.example.org` as the chatmail domain in the following steps.
|
||||
We use `chat.example.org` as the chatmail domain in the following steps.
|
||||
Please substitute it with your own domain.
|
||||
|
||||
1. Setup the initial DNS records.
|
||||
The following is an example in the familiar BIND zone file format with
|
||||
a TTL of 1 hour (3600 seconds).
|
||||
Please substitute your domain and IP addresses.
|
||||
1. Install the `cmdeploy` command in a virtualenv
|
||||
|
||||
```
|
||||
chat.example.com. 3600 IN A 198.51.100.5
|
||||
chat.example.com. 3600 IN AAAA 2001:db8::5
|
||||
www.chat.example.com. 3600 IN CNAME chat.example.com.
|
||||
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
|
||||
```
|
||||
|
||||
2. Clone the repository and bootstrap the Python virtualenv.
|
||||
|
||||
```
|
||||
git clone https://github.com/chatmail/relay
|
||||
cd relay
|
||||
git clone https://github.com/deltachat/chatmail
|
||||
cd chatmail
|
||||
scripts/initenv.sh
|
||||
```
|
||||
|
||||
3. Create chatmail configuration file `chatmail.ini`:
|
||||
2. Create chatmail configuration file `chatmail.ini`:
|
||||
|
||||
```
|
||||
scripts/cmdeploy init chat.example.org # <-- use your domain
|
||||
```
|
||||
|
||||
4. Verify that SSH root login works:
|
||||
3. Point your domain to the server's IP address,
|
||||
if you haven't done so already.
|
||||
Verify that SSH root login works:
|
||||
|
||||
```
|
||||
ssh root@chat.example.org # <-- use your domain
|
||||
```
|
||||
|
||||
|
||||
5. Deploy the remote chatmail relay server:
|
||||
4. Deploy to the remote chatmail server:
|
||||
|
||||
```
|
||||
scripts/cmdeploy run
|
||||
@@ -130,25 +82,25 @@ scripts/cmdeploy bench
|
||||
|
||||
This repository has four directories:
|
||||
|
||||
- [cmdeploy](https://github.com/chatmail/relay/tree/main/cmdeploy)
|
||||
- [cmdeploy](https://github.com/deltachat/chatmail/tree/main/cmdeploy)
|
||||
is a collection of configuration files
|
||||
and a [pyinfra](https://pyinfra.com)-based deployment script.
|
||||
|
||||
- [chatmaild](https://github.com/chatmail/relay/tree/main/chatmaild)
|
||||
is a Python package containing several small services
|
||||
- [chatmaild](https://github.com/deltachat/chatmail/tree/main/chatmaild)
|
||||
is a python package containing several small services
|
||||
which handle authentication,
|
||||
trigger push notifications on new messages,
|
||||
ensure that outbound mails are encrypted,
|
||||
delete inactive users,
|
||||
and some other minor things.
|
||||
chatmaild can also be installed as a stand-alone Python package.
|
||||
chatmaild can also be installed as a stand-alone python package.
|
||||
|
||||
- [www](https://github.com/chatmail/relay/tree/main/www)
|
||||
- [www](https://github.com/deltachat/chatmail/tree/main/www)
|
||||
contains the html, css, and markdown files
|
||||
which make up a chatmail relay's web page.
|
||||
Edit them before deploying to make your chatmail relay stand out.
|
||||
which make up a chatmail server's web page.
|
||||
Edit them before deploying to make your chatmail server stand out.
|
||||
|
||||
- [scripts](https://github.com/chatmail/relay/tree/main/scripts)
|
||||
- [scripts](https://github.com/deltachat/chatmail/tree/main/scripts)
|
||||
offers two convenience tools for beginners;
|
||||
`initenv.sh` installs the necessary dependencies to a local virtual environment,
|
||||
and the `scripts/cmdeploy` script enables you
|
||||
@@ -159,82 +111,80 @@ This repository has four directories:
|
||||
The `cmdeploy/src/cmdeploy/cmdeploy.py` command line tool
|
||||
helps with setting up and managing the chatmail service.
|
||||
`cmdeploy init` creates the `chatmail.ini` config file.
|
||||
`cmdeploy run` uses a [pyinfra](https://pyinfra.com/)-based [`script`](cmdeploy/src/cmdeploy/__init__.py)
|
||||
to automatically install or upgrade all chatmail components on a relay,
|
||||
`cmdeploy run` uses a [pyinfra](https://pyinfra.com/)-based [script](`cmdeploy/src/cmdeploy/__init__.py`)
|
||||
to automatically install or upgrade all chatmail components on a server,
|
||||
according to the `chatmail.ini` config.
|
||||
|
||||
The components of chatmail are:
|
||||
|
||||
- [Postfix SMTP MTA](https://www.postfix.org) accepts and relays messages
|
||||
(both from your users and from the wider e-mail MTA network)
|
||||
- [postfix smtp server](https://www.postfix.org) accepts sent messages (both from your users and from other servers)
|
||||
|
||||
- [Dovecot IMAP MDA](https://www.dovecot.org) stores messages for your users until they download them
|
||||
- [dovecot imap server](https://www.dovecot.org) stores messages for your users until they download them
|
||||
|
||||
- [Nginx](https://nginx.org/) shows the web page with your privacy policy and additional information
|
||||
- [nginx](https://nginx.org/) shows the web page with your privacy policy and additional information
|
||||
|
||||
- [acmetool](https://hlandau.github.io/acmetool/) manages TLS certificates for Dovecot, Postfix, and Nginx
|
||||
- [acmetool](https://hlandau.github.io/acmetool/) manages TLS certificates for dovecot, postfix, and nginx
|
||||
|
||||
- [OpenDKIM](http://www.opendkim.org/) for signing messages with DKIM and rejecting inbound messages without DKIM
|
||||
- [opendkim](http://www.opendkim.org/) for signing messages with DKIM and rejecting inbound messages without DKIM
|
||||
|
||||
- [mtail](https://google.github.io/mtail/) for collecting anonymized metrics in case you have monitoring
|
||||
|
||||
- [Iroh relay](https://www.iroh.computer/docs/concepts/relay)
|
||||
which helps client devices to establish Peer-to-Peer connections
|
||||
|
||||
- and the chatmaild services, explained in the next section:
|
||||
|
||||
### chatmaild
|
||||
|
||||
`chatmaild` implements various systemd-controlled services
|
||||
that integrate with Dovecot and Postfix to achieve instant-onboarding and
|
||||
only relaying OpenPGP end-to-end messages encrypted messages.
|
||||
A short overview of `chatmaild` services:
|
||||
chatmaild offers several commands
|
||||
which differentiate a *chatmail* server from a classic mail server.
|
||||
If you deploy them with cmdeploy,
|
||||
they are run by systemd services in the background.
|
||||
A short overview:
|
||||
|
||||
- [`doveauth`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/doveauth.py)
|
||||
implements create-on-login address semantics and is used
|
||||
by Dovecot during IMAP login and by Postfix during SMTP/SUBMISSION login
|
||||
- [`doveauth`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/doveauth.py) implements
|
||||
create-on-login account creation semantics and is used
|
||||
by Dovecot during login authentication and by Postfix
|
||||
which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)
|
||||
to authenticate logins.
|
||||
to authenticate users
|
||||
to send mails for them.
|
||||
|
||||
- [`filtermail`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/filtermail.py)
|
||||
prevents unencrypted email from leaving or entering the chatmail service
|
||||
and is integrated into Postfix's outbound and inbound mail pipelines.
|
||||
- [`filtermail`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/filtermail.py) prevents
|
||||
unencrypted e-mail from leaving the chatmail service
|
||||
and is integrated into postfix's outbound mail pipelines.
|
||||
|
||||
- [`chatmail-metadata`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/metadata.py) is contacted by a
|
||||
[Dovecot lua script](https://github.com/chatmail/relay/blob/main/cmdeploy/src/cmdeploy/dovecot/push_notification.lua)
|
||||
to store user-specific relay-side config.
|
||||
- [`chatmail-metadata`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/metadata.py) is contacted by a
|
||||
[dovecot lua script](https://github.com/deltachat/chatmail/blob/main/cmdeploy/src/cmdeploy/dovecot/push_notification.lua)
|
||||
to store user-specific server-side config.
|
||||
On new messages,
|
||||
it [passes the user's push notification token](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/notifier.py)
|
||||
it [passes the user's push notification token](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/notifier.py)
|
||||
to [notifications.delta.chat](https://delta.chat/help#instant-delivery)
|
||||
so the push notifications on the user's phone can be triggered
|
||||
by Apple/Google/Huawei.
|
||||
by Apple/Google.
|
||||
|
||||
- [`delete_inactive_users`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/delete_inactive_users.py)
|
||||
- [`delete_inactive_users`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/delete_inactive_users.py)
|
||||
deletes users if they have not logged in for a very long time.
|
||||
The timeframe can be configured in `chatmail.ini`.
|
||||
|
||||
- [`lastlogin`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/lastlogin.py)
|
||||
is contacted by Dovecot when a user logs in
|
||||
- [`lastlogin`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/lastlogin.py)
|
||||
is contacted by dovecot when a user logs in
|
||||
and stores the date of the login.
|
||||
|
||||
- [`echobot`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/echo.py)
|
||||
- [`echobot`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/echo.py)
|
||||
is a small bot for test purposes.
|
||||
It simply echoes back messages from users.
|
||||
|
||||
- [`chatmail-metrics`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/metrics.py)
|
||||
- [`chatmail-metrics`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/metrics.py)
|
||||
collects some metrics and displays them at `https://example.org/metrics`.
|
||||
|
||||
### Home page and getting started for users
|
||||
|
||||
`cmdeploy run` also creates default static web pages and deploys them
|
||||
to a Nginx web server with:
|
||||
`cmdeploy run` also creates default static Web pages and deploys them
|
||||
to a nginx web server with:
|
||||
|
||||
- a default `index.html` along with a QR code that users can click to
|
||||
create an address on your chatmail relay
|
||||
create accounts on your chatmail provider,
|
||||
|
||||
- a default `info.html` that is linked from the home page
|
||||
- a default `info.html` that is linked from the home page,
|
||||
|
||||
- a default `policy.html` that is linked from the home page
|
||||
- a default `policy.html` that is linked from the home page.
|
||||
|
||||
All `.html` files are generated
|
||||
by the according markdown `.md` file in the `www/src` directory.
|
||||
@@ -242,64 +192,48 @@ by the according markdown `.md` file in the `www/src` directory.
|
||||
|
||||
### Refining the web pages
|
||||
|
||||
|
||||
```
|
||||
scripts/cmdeploy webdev
|
||||
```
|
||||
|
||||
This starts a local live development cycle for chatmail web pages:
|
||||
This starts a local live development cycle for chatmail Web pages:
|
||||
|
||||
- uses the `www/src/page-layout.html` file for producing static
|
||||
HTML pages from `www/src/*.md` files
|
||||
|
||||
- continously builds the web presence reading files from `www/src` directory
|
||||
and generating HTML files and copying assets to the `www/build` directory.
|
||||
and generating html files and copying assets to the `www/build` directory.
|
||||
|
||||
- Starts a browser window automatically where you can "refresh" as needed.
|
||||
|
||||
## Mailbox directory layout
|
||||
|
||||
Fresh chatmail addresses have a mailbox directory that contains:
|
||||
## Emergency Commands to disable automatic account creation
|
||||
|
||||
- a `password` file with the salted password required for authenticating
|
||||
whether a login may use the address to send/receive messages.
|
||||
If you modify the password file manually, you effectively block the user.
|
||||
|
||||
- `enforceE2EEincoming` is a default-created file with each address.
|
||||
If present the file indicates that this chatmail address rejects incoming cleartext messages.
|
||||
If absent the address accepts incoming cleartext messages.
|
||||
|
||||
- `dovecot*`, `cur`, `new` and `tmp` represent IMAP/mailbox state.
|
||||
If the address is only used by one device, the Maildir directories
|
||||
will typically be empty unless the user of that address hasn't been online
|
||||
for a while.
|
||||
|
||||
|
||||
## Emergency Commands to disable automatic address creation
|
||||
|
||||
If you need to stop address creation,
|
||||
e.g. because some script is wildly creating addresses,
|
||||
login with ssh and run:
|
||||
If you need to stop account creation,
|
||||
e.g. because some script is wildly creating accounts,
|
||||
login to the server with ssh and run:
|
||||
|
||||
```
|
||||
touch /etc/chatmail-nocreate
|
||||
```
|
||||
|
||||
Chatmail address creation will be denied while this file is present.
|
||||
While this file is present, account creation will be blocked.
|
||||
|
||||
### Ports
|
||||
|
||||
[Postfix](http://www.postfix.org/) listens on ports 25 (SMTP) and 587 (SUBMISSION) and 465 (SUBMISSIONS).
|
||||
[Dovecot](https://www.dovecot.org/) listens on ports 143 (IMAP) and 993 (IMAPS).
|
||||
[Nginx](https://www.nginx.com/) listens on port 8443 (HTTPS-ALT) and 443 (HTTPS).
|
||||
[Postfix](http://www.postfix.org/) listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
|
||||
[Dovecot](https://www.dovecot.org/) listens on ports 143 (imap) and 993 (imaps).
|
||||
[nginx](https://www.nginx.com/) listens on port 8443 (https-alt) and 443 (https).
|
||||
Port 443 multiplexes HTTPS, IMAP and SMTP using ALPN to redirect connections to ports 8443, 465 or 993.
|
||||
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (HTTP).
|
||||
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (http).
|
||||
|
||||
chatmail-core based apps will, however, discover all ports and configurations
|
||||
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail relay server.
|
||||
Delta Chat apps will, however, discover all ports and configurations
|
||||
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail service.
|
||||
|
||||
## Email authentication
|
||||
|
||||
Chatmail relays enforce [DKIM](https://www.rfc-editor.org/rfc/rfc6376)
|
||||
chatmail servers rely on [DKIM](https://www.rfc-editor.org/rfc/rfc6376)
|
||||
to authenticate incoming emails.
|
||||
Incoming emails must have a valid DKIM signature with
|
||||
Signing Domain Identifier (SDID, `d=` parameter in the DKIM-Signature header)
|
||||
@@ -322,130 +256,101 @@ and rejects incorrectly authenticated emails with [`reject_sender_login_mismatch
|
||||
`From:` header must correspond to envelope MAIL FROM,
|
||||
this is ensured by `filtermail` proxy.
|
||||
|
||||
## TLS requirements
|
||||
## Migrating chatmail server to a new host
|
||||
|
||||
Postfix is configured to require valid TLS
|
||||
by setting [`smtp_tls_security_level`](https://www.postfix.org/postconf.5.html#smtp_tls_security_level) to `verify`.
|
||||
If emails don't arrive at your chatmail relay server,
|
||||
the problem is likely that your relay does not have a valid TLS certificate.
|
||||
|
||||
You can test it by resolving `MX` records of your relay domain
|
||||
and then connecting to MX relays (e.g `mx.example.org`) with
|
||||
`openssl s_client -connect mx.example.org:25 -verify_hostname mx.example.org -verify_return_error -starttls smtp`
|
||||
from the host that has open port 25 to verify that certificate is valid.
|
||||
|
||||
When providing a TLS certificate to your chatmail relay server,
|
||||
make sure to provide the full certificate chain
|
||||
and not just the last certificate.
|
||||
|
||||
If you are running an Exim server and don't see incoming connections
|
||||
from a chatmail relay server in the logs,
|
||||
make sure `smtp_no_mail` log item is enabled in the config
|
||||
with `log_selector = +smtp_no_mail`.
|
||||
By default Exim does not log sessions that are closed
|
||||
before sending the `MAIL` command.
|
||||
This happens if certificate is not recognized as valid by Postfix,
|
||||
so you might think that connection is not established
|
||||
while actually it is a problem with your TLS certificate.
|
||||
|
||||
## Migrating a chatmail relay to a new host
|
||||
|
||||
If you want to migrate chatmail relay from an old machine
|
||||
If you want to migrate chatmail from an old machine
|
||||
to a new machine,
|
||||
you can use these steps.
|
||||
They were tested with a Linux laptop;
|
||||
They were tested with a linux laptop;
|
||||
you might need to adjust some of the steps to your environment.
|
||||
|
||||
Let's assume that your `mail_domain` is `mail.example.org`,
|
||||
all involved machines run Debian 12,
|
||||
your old site's IP address is `13.37.13.37`,
|
||||
and your new site's IP address is `13.12.23.42`.
|
||||
your old server's IP address is `13.37.13.37`,
|
||||
and your new server's IP address is `13.12.23.42`.
|
||||
|
||||
Note, you should lower the TTLs of your DNS records to a value
|
||||
such as 300 (5 minutes) so the migration happens as smoothly as possible.
|
||||
During the guide, you might get a warning about changed SSH Host keys;
|
||||
in this case, just run `ssh-keygen -R "mail.example.org"` as recommended
|
||||
to make sure you can connect with SSH.
|
||||
|
||||
During the guide you might get a warning about changed SSH Host keys;
|
||||
in this case, just run `ssh-keygen -R "mail.example.org"` as recommended.
|
||||
1. First, copy `/var/lib/acme` to the new server with
|
||||
`ssh root@13.37.13.37 tar c /var/lib/acme | ssh root@13.12.23.42 tar x -C /var/lib/`.
|
||||
This transfers your TLS certificate.
|
||||
|
||||
1. First, disable mail services on the old site.
|
||||
2. You should also copy `/etc/dkimkeys` to the new server with
|
||||
`ssh root@13.37.13.37 tar c /etc/dkimkeys | ssh root@13.12.23.42 tar x -C /etc/`
|
||||
so the DKIM DNS record stays correct.
|
||||
|
||||
```
|
||||
cmdeploy run --disable-mail --ssh-host 13.37.13.37
|
||||
```
|
||||
3. On the new server, run `chown root: -R /var/lib/acme` and `chown opendkim: -R /etc/dkimkeys` to make sure the permissions are correct.
|
||||
|
||||
4. Run `cmdeploy run --disable-mail --ssh-host 13.12.23.42` to install chatmail on the new machine.
|
||||
postfix and dovecot are disabled for now,
|
||||
we will enable them later.
|
||||
|
||||
5. Now, point DNS to the new IP addresses.
|
||||
|
||||
You can already remove the old IP addresses from DNS.
|
||||
Existing Delta Chat users will still be able to connect
|
||||
to the old server, send and receive messages,
|
||||
but new users will fail to create new profiles
|
||||
with your chatmail server.
|
||||
|
||||
If other servers try to deliver messages to your new server they will fail,
|
||||
but normally email servers will retry delivering messages
|
||||
for at least a week, so messages will not be lost.
|
||||
|
||||
6. Now you can run `cmdeploy run --disable-mail --ssh-host 13.37.13.37` to disable your old server.
|
||||
|
||||
Now your users will notice the migration
|
||||
and will not be able to send or receive messages
|
||||
until the migration is completed.
|
||||
|
||||
2. Now we want to copy `/home/vmail`, `/var/lib/acme`, `/etc/dkimkeys`, `/run/echobot`, and `/var/spool/postfix` to the new site.
|
||||
Login to the old site while forwarding your SSH agent
|
||||
so you can copy directly from the old to the new site with your SSH key:
|
||||
```
|
||||
ssh -A root@13.37.13.37
|
||||
tar c - /home/vmail/mail /var/lib/acme /etc/dkimkeys /run/echobot /var/spool/postfix | ssh root@13.12.23.42 "tar x -C /"
|
||||
```
|
||||
7. After everything is stopped,
|
||||
you can copy the `/home/vmail/mail` directory to the new server.
|
||||
It includes all user data, messages, password hashes, etc.
|
||||
|
||||
This transfers all addresses, the TLS certificate, DKIM keys (so DKIM DNS record remains valid), and the echobot's password so it continues to function.
|
||||
It also preserves the Postfix mail spool so any messages pending delivery will still be delivered.
|
||||
Just run: `ssh root@13.37.13.37 tar c /home/vmail/mail | ssh root@13.12.23.42 tar x -C /home/vmail/`
|
||||
|
||||
3. Install chatmail on the new machine:
|
||||
After this, your new server has all the necessary files to start operating :)
|
||||
|
||||
```
|
||||
cmdeploy run --disable-mail --ssh-host 13.12.23.42
|
||||
```
|
||||
Postfix and Dovecot are disabled for now; we will enable them later.
|
||||
We first need to make the new site fully operational.
|
||||
8. To be sure the permissions are still fine,
|
||||
run `chown vmail: -R /home/vmail` on the new server.
|
||||
|
||||
3. On the new site, run the following to ensure the ownership is correct in case UIDs/GIDs changed:
|
||||
|
||||
```
|
||||
chown root: -R /var/lib/acme
|
||||
chown opendkim: -R /etc/dkimkeys
|
||||
chown vmail: -R /home/vmail/mail
|
||||
chown echobot: -R /run/echobot
|
||||
```
|
||||
|
||||
4. Now, update DNS entries.
|
||||
|
||||
If other MTAs try to deliver messages to your chatmail domain they may fail intermittently,
|
||||
as DNS catches up with the new site settings
|
||||
but normally will retry delivering messages
|
||||
for at least a week, so messages will not be lost.
|
||||
|
||||
5. Finally, you can execute `cmdeploy run --ssh-host 13.12.23.42` to turn on chatmail on the new relay.
|
||||
Your users will be able to use the chatmail relay as soon as the DNS changes have propagated.
|
||||
Voilà!
|
||||
9. Finally, you can run `cmdeploy run` to turn on chatmail on the new server.
|
||||
Your users can continue using the chatmail server,
|
||||
and messages which were sent after step 6. should arrive now.
|
||||
Voilà!
|
||||
|
||||
## Setting up a reverse proxy
|
||||
|
||||
A chatmail relay MTA does not track or depend on the client IP address
|
||||
A chatmail server does not depend on the client IP address
|
||||
for its operation, so it can be run behind a reverse proxy.
|
||||
This will not even affect incoming mail authentication
|
||||
as DKIM only checks the cryptographic signature
|
||||
of the message and does not use the IP address as the input.
|
||||
|
||||
For example, you may want to self-host your chatmail relay
|
||||
For example, you may want to self-host your chatmail server
|
||||
and only use hosted VPS to provide a public IP address
|
||||
for client connections and incoming mail.
|
||||
You can connect chatmail relay to VPS
|
||||
You can connect chatmail server to VPS
|
||||
using a tunnel protocol
|
||||
such as [WireGuard](https://www.wireguard.com/)
|
||||
and setup a reverse proxy on a VPS
|
||||
to forward connections to the chatmail relay
|
||||
to forward connections to the chatmail server
|
||||
over the tunnel.
|
||||
You can also setup multiple reverse proxies
|
||||
for your chatmail relay in different networks
|
||||
to ensure your relay is reachable even when
|
||||
for your chatmail server in different networks
|
||||
to ensure your server is reachable even when
|
||||
one of the IPs becomes inaccessible due to
|
||||
hosting or routing problems.
|
||||
|
||||
Note that your chatmail relay still needs
|
||||
Note that your server still needs
|
||||
to be able to make outgoing connections on port 25
|
||||
to send messages outside.
|
||||
|
||||
To setup a reverse proxy
|
||||
(or rather Destination NAT, DNAT)
|
||||
for your chatmail relay,
|
||||
for your chatmail server,
|
||||
put the following configuration in `/etc/nftables.conf`:
|
||||
```
|
||||
#!/usr/sbin/nft -f
|
||||
@@ -457,7 +362,7 @@ define wan = eth0
|
||||
# Which ports to proxy.
|
||||
#
|
||||
# Note that SSH is not proxied
|
||||
# so it is possible to log into the proxy server
|
||||
# so it is possible to log into the proxy server
|
||||
# and not the original one.
|
||||
define ports = { smtp, http, https, imap, imaps, submission, submissions }
|
||||
|
||||
@@ -520,7 +425,7 @@ table inet filter {
|
||||
```
|
||||
|
||||
Run `systemctl enable nftables.service`
|
||||
to ensure configuration is reloaded when the proxy relay reboots.
|
||||
to ensure configuration is reloaded when the proxy server reboots.
|
||||
|
||||
Uncomment in `/etc/sysctl.conf` the following two lines:
|
||||
|
||||
@@ -529,16 +434,7 @@ net.ipv4.ip_forward=1
|
||||
net.ipv6.conf.all.forwarding=1
|
||||
```
|
||||
|
||||
Then reboot the relay or do `sysctl -p` and `nft -f /etc/nftables.conf`.
|
||||
Then reboot the server or do `sysctl -p` and `nft -f /etc/nftables.conf`.
|
||||
|
||||
Once proxy relay is set up,
|
||||
Once proxy server is set up,
|
||||
you can add its IP address to the DNS.
|
||||
|
||||
## Neighbors and Acquaintances
|
||||
|
||||
Here are some related projects that you may be interested in:
|
||||
|
||||
- [Mox](https://github.com/mjl-/mox): A Golang email server. [Work is in
|
||||
progress](https://github.com/mjl-/mox/issues/251) to modify it to support all
|
||||
of the features and configuration settings required to operate as a chatmail
|
||||
relay.
|
||||
|
||||
@@ -12,7 +12,7 @@ dependencies = [
|
||||
"deltachat-rpc-client",
|
||||
"filelock",
|
||||
"requests",
|
||||
"crypt-r >= 3.13.1 ; python_version >= '3.11'",
|
||||
"crypt-r",
|
||||
]
|
||||
|
||||
[tool.setuptools]
|
||||
|
||||
59
chatmaild/src/chatmaild/common_encrypted_subjects.py
Normal file
59
chatmaild/src/chatmaild/common_encrypted_subjects.py
Normal file
@@ -0,0 +1,59 @@
|
||||
"""Generated from deltachat, draft-ietf-lamps-header-protection, and
|
||||
encrypted_subject localizations in
|
||||
https://github.com/thunderbird/thunderbird-android/
|
||||
|
||||
"""
|
||||
|
||||
common_encrypted_subjects = {
|
||||
"...",
|
||||
"[...]",
|
||||
"암호화된 메시지",
|
||||
"Ĉifrita mesaĝo",
|
||||
"Courriel chiffré",
|
||||
"Dulrituð skilaboð",
|
||||
"Encrypted Message",
|
||||
"Fersifere berjocht",
|
||||
"Kemennadenn enrineget",
|
||||
"Krüptitud kiri",
|
||||
"Krypterat meddelande",
|
||||
"Krypteret besked",
|
||||
"Kryptert melding",
|
||||
"Mensagem criptografada",
|
||||
"Mensagem encriptada",
|
||||
"Mensaje cifrado",
|
||||
"Mensaxe cifrada",
|
||||
"Mesaj Criptat",
|
||||
"Mesazh i Fshehtëzuar",
|
||||
"Messaggio criptato",
|
||||
"Messaghju cifratu",
|
||||
"Missatge encriptat",
|
||||
"Neges wedi'i Hamgryptio",
|
||||
"Pesan terenkripsi",
|
||||
"Salattu viesti",
|
||||
"Şifreli İleti",
|
||||
"Šifrēta ziņa",
|
||||
"Šifrirana poruka",
|
||||
"Šifrirano sporočilo",
|
||||
"Šifruotas laiškas",
|
||||
"Tin nhắn được mã hóa",
|
||||
"Titkosított üzenet",
|
||||
"Verschlüsselte Nachricht",
|
||||
"Versleuteld bericht",
|
||||
"Zašifrovaná zpráva",
|
||||
"Zaszyfrowana wiadomość",
|
||||
"Zifratu mezua",
|
||||
"Κρυπτογραφημένο μήνυμα",
|
||||
"Зашифроване повідомлення",
|
||||
"Зашифрованное сообщение",
|
||||
"Зашыфраваны ліст",
|
||||
"Криптирано съобщение",
|
||||
"Шифрована порука",
|
||||
"დაშიფრული წერილი",
|
||||
"הודעה מוצפנת",
|
||||
"پیام رمزنگاریشده",
|
||||
"رسالة مشفّرة",
|
||||
"എൻക്രിപ്റ്റുചെയ്ത സന്ദേശം",
|
||||
"加密邮件",
|
||||
"已加密的訊息",
|
||||
"暗号化されたメッセージ",
|
||||
}
|
||||
@@ -11,11 +11,7 @@ def read_config(inipath):
|
||||
assert Path(inipath).exists(), inipath
|
||||
cfg = iniconfig.IniConfig(inipath)
|
||||
params = cfg.sections["params"]
|
||||
default_config_content = get_default_config_content(params["mail_domain"])
|
||||
df_params = iniconfig.IniConfig("ini", data=default_config_content)["params"]
|
||||
new_params = dict(df_params.items())
|
||||
new_params.update(params)
|
||||
return Config(inipath, params=new_params)
|
||||
return Config(inipath, params=params)
|
||||
|
||||
|
||||
class Config:
|
||||
@@ -26,7 +22,6 @@ class Config:
|
||||
self.max_mailbox_size = params["max_mailbox_size"]
|
||||
self.max_message_size = int(params.get("max_message_size", "31457280"))
|
||||
self.delete_mails_after = params["delete_mails_after"]
|
||||
self.delete_large_after = params["delete_large_after"]
|
||||
self.delete_inactive_users_after = int(params["delete_inactive_users_after"])
|
||||
self.username_min_length = int(params["username_min_length"])
|
||||
self.username_max_length = int(params["username_max_length"])
|
||||
@@ -34,13 +29,7 @@ class Config:
|
||||
self.passthrough_senders = params["passthrough_senders"].split()
|
||||
self.passthrough_recipients = params["passthrough_recipients"].split()
|
||||
self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
|
||||
self.filtermail_smtp_port_incoming = int(
|
||||
params["filtermail_smtp_port_incoming"]
|
||||
)
|
||||
self.postfix_reinject_port = int(params["postfix_reinject_port"])
|
||||
self.postfix_reinject_port_incoming = int(
|
||||
params["postfix_reinject_port_incoming"]
|
||||
)
|
||||
self.mtail_address = params.get("mtail_address")
|
||||
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
|
||||
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
|
||||
@@ -65,7 +54,7 @@ class Config:
|
||||
def _getbytefile(self):
|
||||
return open(self._inipath, "rb")
|
||||
|
||||
def get_user(self, addr) -> User:
|
||||
def get_user(self, addr):
|
||||
if not addr or "@" not in addr or "/" in addr:
|
||||
raise ValueError(f"invalid address {addr!r}")
|
||||
|
||||
@@ -80,11 +69,6 @@ class Config:
|
||||
|
||||
def write_initial_config(inipath, mail_domain, overrides):
|
||||
"""Write out default config file, using the specified config value overrides."""
|
||||
content = get_default_config_content(mail_domain, **overrides)
|
||||
inipath.write_text(content)
|
||||
|
||||
|
||||
def get_default_config_content(mail_domain, **overrides):
|
||||
from importlib.resources import files
|
||||
|
||||
inidir = files(__package__).joinpath("ini")
|
||||
@@ -116,7 +100,7 @@ def get_default_config_content(mail_domain, **overrides):
|
||||
lines = []
|
||||
for line in content.split("\n"):
|
||||
for key, value in privacy.items():
|
||||
value_lines = value.format(mail_domain=mail_domain).strip().split("\n")
|
||||
value_lines = value.strip().split("\n")
|
||||
if not line.startswith(f"{key} =") or not value_lines:
|
||||
continue
|
||||
if len(value_lines) == 1:
|
||||
@@ -129,4 +113,5 @@ def get_default_config_content(mail_domain, **overrides):
|
||||
else:
|
||||
lines.append(line)
|
||||
content = "\n".join(lines)
|
||||
return content
|
||||
|
||||
inipath.write_text(content)
|
||||
|
||||
@@ -3,10 +3,7 @@ import logging
|
||||
import os
|
||||
import sys
|
||||
|
||||
try:
|
||||
import crypt_r
|
||||
except ImportError:
|
||||
import crypt as crypt_r
|
||||
import crypt_r
|
||||
|
||||
from .config import Config, read_config
|
||||
from .dictproxy import DictProxy
|
||||
|
||||
@@ -8,7 +8,6 @@ import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from deltachat_rpc_client import Bot, DeltaChat, EventType, Rpc, events
|
||||
|
||||
@@ -98,10 +97,6 @@ def main():
|
||||
if not bot.is_configured():
|
||||
bot.configure(addr, password)
|
||||
|
||||
# write invite link to working directory
|
||||
invitelink = bot.account.get_qr_code()
|
||||
Path("invite-link.txt").write_text(invitelink)
|
||||
|
||||
bot.run_forever()
|
||||
|
||||
|
||||
|
||||
@@ -11,12 +11,10 @@ from email.utils import parseaddr
|
||||
from smtplib import SMTP as SMTPClient
|
||||
|
||||
from aiosmtpd.controller import Controller
|
||||
from aiosmtpd.smtp import SMTP
|
||||
|
||||
from .common_encrypted_subjects import common_encrypted_subjects
|
||||
from .config import read_config
|
||||
|
||||
ENCRYPTION_NEEDED_523 = "523 Encryption Needed: Invalid Unencrypted Mail"
|
||||
|
||||
|
||||
def check_openpgp_payload(payload: bytes):
|
||||
"""Checks the OpenPGP payload.
|
||||
@@ -38,12 +36,6 @@ def check_openpgp_payload(payload: bytes):
|
||||
|
||||
packet_type_id = payload[i] & 0x3F
|
||||
i += 1
|
||||
|
||||
while payload[i] >= 224 and payload[i] < 255:
|
||||
# Partial body length.
|
||||
partial_length = 1 << (payload[i] & 0x1F)
|
||||
i += 1 + partial_length
|
||||
|
||||
if payload[i] < 192:
|
||||
# One-octet length.
|
||||
body_len = payload[i]
|
||||
@@ -62,7 +54,7 @@ def check_openpgp_payload(payload: bytes):
|
||||
)
|
||||
i += 5
|
||||
else:
|
||||
# Impossible, partial body length was processed above.
|
||||
# Partial body length is not allowed.
|
||||
return False
|
||||
|
||||
i += body_len
|
||||
@@ -89,9 +81,7 @@ def check_armored_payload(payload: str):
|
||||
return False
|
||||
payload = payload.removeprefix(prefix)
|
||||
|
||||
while payload.endswith("\r\n"):
|
||||
payload = payload.removesuffix("\r\n")
|
||||
suffix = "-----END PGP MESSAGE-----"
|
||||
suffix = "-----END PGP MESSAGE-----\r\n\r\n"
|
||||
if not payload.endswith(suffix):
|
||||
return False
|
||||
payload = payload.removesuffix(suffix)
|
||||
@@ -110,27 +100,6 @@ def check_armored_payload(payload: str):
|
||||
return False
|
||||
|
||||
|
||||
def is_securejoin(message):
|
||||
if message.get("secure-join") not in ["vc-request", "vg-request"]:
|
||||
return False
|
||||
if not message.is_multipart():
|
||||
return False
|
||||
parts_count = 0
|
||||
for part in message.iter_parts():
|
||||
parts_count += 1
|
||||
if parts_count > 1:
|
||||
return False
|
||||
if part.is_multipart():
|
||||
return False
|
||||
if part.get_content_type() != "text/plain":
|
||||
return False
|
||||
|
||||
payload = part.get_payload().strip().lower()
|
||||
if payload not in ("secure-join: vc-request", "secure-join: vg-request"):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def check_encrypted(message):
|
||||
"""Check that the message is an OpenPGP-encrypted message.
|
||||
|
||||
@@ -138,6 +107,8 @@ def check_encrypted(message):
|
||||
"""
|
||||
if not message.is_multipart():
|
||||
return False
|
||||
if message.get("subject") not in common_encrypted_subjects:
|
||||
return False
|
||||
if message.get_content_type() != "multipart/encrypted":
|
||||
return False
|
||||
parts_count = 0
|
||||
@@ -166,19 +137,9 @@ def check_encrypted(message):
|
||||
return True
|
||||
|
||||
|
||||
async def asyncmain_beforequeue(config, mode):
|
||||
if mode == "outgoing":
|
||||
port = config.filtermail_smtp_port
|
||||
handler = OutgoingBeforeQueueHandler(config)
|
||||
else:
|
||||
port = config.filtermail_smtp_port_incoming
|
||||
handler = IncomingBeforeQueueHandler(config)
|
||||
HackedController(
|
||||
handler,
|
||||
hostname="127.0.0.1",
|
||||
port=port,
|
||||
data_size_limit=config.max_message_size,
|
||||
).start()
|
||||
async def asyncmain_beforequeue(config):
|
||||
port = config.filtermail_smtp_port
|
||||
Controller(BeforeQueueHandler(config), hostname="127.0.0.1", port=port).start()
|
||||
|
||||
|
||||
def recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
@@ -190,21 +151,7 @@ def recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
return False
|
||||
|
||||
|
||||
class HackedController(Controller):
|
||||
def factory(self):
|
||||
return SMTPDiscardRCPTO_options(self.handler, **self.SMTP_kwargs)
|
||||
|
||||
|
||||
class SMTPDiscardRCPTO_options(SMTP):
|
||||
def _getparams(self, params):
|
||||
# aiosmtpd's SMTP daemon fails to handle a request if there are RCPT TO options
|
||||
# We just ignore them for our incoming filtermail purposes
|
||||
if len(params) == 1 and params[0].startswith("ORCPT"):
|
||||
return {}
|
||||
return super()._getparams(params)
|
||||
|
||||
|
||||
class OutgoingBeforeQueueHandler:
|
||||
class BeforeQueueHandler:
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
self.send_rate_limiter = SendRateLimiter()
|
||||
@@ -229,9 +176,7 @@ class OutgoingBeforeQueueHandler:
|
||||
return error
|
||||
logging.info("re-injecting the mail that passed checks")
|
||||
client = SMTPClient("localhost", self.config.postfix_reinject_port)
|
||||
client.sendmail(
|
||||
envelope.mail_from, envelope.rcpt_tos, envelope.original_content
|
||||
)
|
||||
client.sendmail(envelope.mail_from, envelope.rcpt_tos, envelope.content)
|
||||
return "250 OK"
|
||||
|
||||
def check_DATA(self, envelope):
|
||||
@@ -242,86 +187,44 @@ class OutgoingBeforeQueueHandler:
|
||||
mail_encrypted = check_encrypted(message)
|
||||
|
||||
_, from_addr = parseaddr(message.get("from").strip())
|
||||
envelope_from_domain = from_addr.split("@").pop()
|
||||
|
||||
logging.info(f"mime-from: {from_addr} envelope-from: {envelope.mail_from!r}")
|
||||
if envelope.mail_from.lower() != from_addr.lower():
|
||||
return f"500 Invalid FROM <{from_addr!r}> for <{envelope.mail_from!r}>"
|
||||
|
||||
if mail_encrypted or is_securejoin(message):
|
||||
print("Outgoing: Filtering encrypted mail.", file=sys.stderr)
|
||||
return
|
||||
|
||||
print("Outgoing: Filtering unencrypted mail.", file=sys.stderr)
|
||||
if mail_encrypted:
|
||||
print("Filtering encrypted mail.", file=sys.stderr)
|
||||
else:
|
||||
print("Filtering unencrypted mail.", file=sys.stderr)
|
||||
|
||||
if envelope.mail_from in self.config.passthrough_senders:
|
||||
return
|
||||
|
||||
# allow self-sent Autocrypt Setup Message
|
||||
if envelope.rcpt_tos == [from_addr]:
|
||||
if message.get("subject") == "Autocrypt Setup Message":
|
||||
if message.get_content_type() == "multipart/mixed":
|
||||
return
|
||||
|
||||
passthrough_recipients = self.config.passthrough_recipients
|
||||
|
||||
for recipient in envelope.rcpt_tos:
|
||||
if recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
continue
|
||||
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return ENCRYPTION_NEEDED_523
|
||||
|
||||
|
||||
class IncomingBeforeQueueHandler:
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
|
||||
async def handle_DATA(self, server, session, envelope):
|
||||
logging.info("handle_DATA before-queue")
|
||||
error = self.check_DATA(envelope)
|
||||
if error:
|
||||
return error
|
||||
logging.info("re-injecting the mail that passed checks")
|
||||
|
||||
# the smtp daemon on reinject_port_incoming gives it to dkim milter
|
||||
# which looks at source address to determine whether to verify or sign
|
||||
client = SMTPClient(
|
||||
"localhost",
|
||||
self.config.postfix_reinject_port_incoming,
|
||||
source_address=("127.0.0.2", 0),
|
||||
)
|
||||
client.sendmail(
|
||||
envelope.mail_from, envelope.rcpt_tos, envelope.original_content
|
||||
)
|
||||
return "250 OK"
|
||||
|
||||
def check_DATA(self, envelope):
|
||||
"""the central filtering function for e-mails."""
|
||||
logging.info(f"Processing DATA message from {envelope.mail_from}")
|
||||
|
||||
message = BytesParser(policy=policy.default).parsebytes(envelope.content)
|
||||
mail_encrypted = check_encrypted(message)
|
||||
|
||||
if mail_encrypted or is_securejoin(message):
|
||||
print("Incoming: Filtering encrypted mail.", file=sys.stderr)
|
||||
is_securejoin = message.get("secure-join") in [
|
||||
"vc-request",
|
||||
"vg-request",
|
||||
]
|
||||
if is_securejoin:
|
||||
return
|
||||
|
||||
print("Incoming: Filtering unencrypted mail.", file=sys.stderr)
|
||||
|
||||
# we want cleartext mailer-daemon messages to pass through
|
||||
# chatmail core will typically not display them as normal messages
|
||||
if message.get("auto-submitted"):
|
||||
_, from_addr = parseaddr(message.get("from").strip())
|
||||
if from_addr.lower().startswith("mailer-daemon@"):
|
||||
if message.get_content_type() == "multipart/report":
|
||||
return
|
||||
|
||||
for recipient in envelope.rcpt_tos:
|
||||
user = self.config.get_user(recipient)
|
||||
if user is None or user.is_incoming_cleartext_ok():
|
||||
if envelope.mail_from == recipient:
|
||||
# Always allow sending emails to self.
|
||||
continue
|
||||
if recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
continue
|
||||
res = recipient.split("@")
|
||||
if len(res) != 2:
|
||||
return f"500 Invalid address <{recipient}>"
|
||||
_recipient_addr, recipient_domain = res
|
||||
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return ENCRYPTION_NEEDED_523
|
||||
is_outgoing = recipient_domain != envelope_from_domain
|
||||
if is_outgoing and not mail_encrypted:
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return f"500 Invalid unencrypted mail to <{recipient}>"
|
||||
|
||||
|
||||
class SendRateLimiter:
|
||||
@@ -340,14 +243,11 @@ class SendRateLimiter:
|
||||
|
||||
def main():
|
||||
args = sys.argv[1:]
|
||||
assert len(args) == 2
|
||||
assert len(args) == 1
|
||||
config = read_config(args[0])
|
||||
mode = args[1]
|
||||
logging.basicConfig(level=logging.WARN)
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
assert mode in ["incoming", "outgoing"]
|
||||
task = asyncmain_beforequeue(config, mode)
|
||||
task = asyncmain_beforequeue(config)
|
||||
loop.create_task(task)
|
||||
logging.info("entering serving loop")
|
||||
loop.run_forever()
|
||||
|
||||
@@ -23,9 +23,6 @@ max_message_size = 31457280
|
||||
# days after which mails are unconditionally deleted
|
||||
delete_mails_after = 20
|
||||
|
||||
# days after which large messages (>200k) are unconditionally deleted
|
||||
delete_large_after = 7
|
||||
|
||||
# days after which users without a successful login are deleted (database and mails)
|
||||
delete_inactive_users_after = 90
|
||||
|
||||
@@ -43,19 +40,17 @@ passthrough_senders =
|
||||
|
||||
# list of e-mail recipients for which to accept outbound un-encrypted mails
|
||||
# (space-separated, item may start with "@" to whitelist whole recipient domains)
|
||||
passthrough_recipients = xstore@testrun.org echo@{mail_domain}
|
||||
passthrough_recipients = xstore@testrun.org
|
||||
|
||||
#
|
||||
# Deployment Details
|
||||
#
|
||||
|
||||
# SMTP outgoing filtermail and reinjection
|
||||
# where the filtermail SMTP service listens
|
||||
filtermail_smtp_port = 10080
|
||||
postfix_reinject_port = 10025
|
||||
|
||||
# SMTP incoming filtermail and reinjection
|
||||
filtermail_smtp_port_incoming = 10081
|
||||
postfix_reinject_port_incoming = 10026
|
||||
# postfix accepts on the localhost reinject SMTP port
|
||||
postfix_reinject_port = 10025
|
||||
|
||||
# if set to "True" IPv6 is disabled
|
||||
disable_ipv6 = False
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
|
||||
[privacy]
|
||||
|
||||
passthrough_recipients = privacy@testrun.org xstore@testrun.org echo@{mail_domain}
|
||||
passthrough_recipients = privacy@testrun.org xstore@testrun.org
|
||||
|
||||
privacy_postal =
|
||||
Merlinux GmbH, Represented by the managing director H. Krekel,
|
||||
|
||||
@@ -11,8 +11,6 @@ def main(vmail_dir=None):
|
||||
ci_accounts = 0
|
||||
|
||||
for path in Path(vmail_dir).iterdir():
|
||||
if not path.joinpath("cur").is_dir():
|
||||
continue
|
||||
accounts += 1
|
||||
if path.name[:3] in ("ci-", "ac_"):
|
||||
ci_accounts += 1
|
||||
|
||||
@@ -95,12 +95,7 @@ class Notifier:
|
||||
logging.warning(f"removing spurious queue item: {queue_path!r}")
|
||||
queue_path.unlink()
|
||||
continue
|
||||
try:
|
||||
queue_item = PersistentQueueItem.read_from_path(queue_path)
|
||||
except ValueError:
|
||||
logging.warning(f"removing spurious queue item: {queue_path!r}")
|
||||
queue_path.unlink()
|
||||
continue
|
||||
queue_item = PersistentQueueItem.read_from_path(queue_path)
|
||||
self.queue_for_retry(queue_item)
|
||||
|
||||
def queue_for_retry(self, queue_item, retry_num=0):
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
From: {from_addr}
|
||||
To: {to_addr}
|
||||
Autocrypt-Setup-Message: v1
|
||||
Subject: Autocrypt Setup Message
|
||||
Date: Tue, 22 Jan 2019 12:56:29 +0100
|
||||
Content-type: multipart/mixed; boundary="Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ"
|
||||
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
|
||||
Content-Type: text/plain
|
||||
|
||||
This message contains all information to transfer your Autocrypt
|
||||
settings along with your secret key securely from your original
|
||||
device.
|
||||
|
||||
To set up your new device for Autocrypt, please follow the
|
||||
instuctions that should be presented by your new device.
|
||||
|
||||
You can keep this message and use it as a backup for your secret
|
||||
key. If you want to do this, you should write down the Setup Code
|
||||
and store it securely.
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
|
||||
Content-Type: application/autocrypt-setup
|
||||
Content-Disposition: attachment; filename="autocrypt-setup-message.html"
|
||||
|
||||
<html><body>
|
||||
<p>
|
||||
This is the Autocrypt setup file used to transfer settings and
|
||||
keys between clients. You can decrypt it using the Setup Code
|
||||
presented on your old device, and then import the contained key
|
||||
into your keyring.
|
||||
</p>
|
||||
|
||||
<pre>
|
||||
-----BEGIN PGP MESSAGE-----
|
||||
Passphrase-Format: numeric9x4
|
||||
Passphrase-Begin: 17
|
||||
|
||||
jA0EBwMCFAxADoCdzeX/0ukBlqI5+pfpKb751qd/7nLNbkpy3gVcaf1QwRPZYt40
|
||||
Ynp08UqRQ2g48ZlnzHLSwlTGOPTuv2Jt8ka+pgZ45xzvJSG2gau03xP4VsC271kR
|
||||
VmCjdb0Y6Rk96mAwfGzrkbaRQ9Z7fIoL866GOv6h9neiVIkp+JYlTV6ISD0ZQJ4Q
|
||||
I6dOQkB/TWZyVjtiJDOQHdfNWliA6NtqaLq19wlu9L5xXjuNpY95KwR8EJXWe0+o
|
||||
Y3d2U/KxOAkXKghP2Qg1GtlPVeGC5T4p03TGI6pzKT+kHX6Rrm9wK6sM9aTquMmF
|
||||
Vok84Jg1DFnwivWC2RILR81rXi7k/+Y6MUbveFgJ9cQduqpxnmD7TjOblYu7M6zp
|
||||
YGAUxh8DRKlIMn2QsA++DBYQ6ACZvwuY8qTDLkqPDo4WqM313dsMJbyGjDdVE7EM
|
||||
PESS+RlABETpZXz8g/ycr6DIUNdlbPcmYlsBfHWDOuR2GFFTwmlv5slWS39dJv38
|
||||
E0eIe1CwdxI801Se7t7dUUS/ZF8wb6GlmxOcqGbF8eko1Z0S64IAm7/h13MRQCxI
|
||||
geQnHfGYVJ2FOimoCMEKwfa9x++RFTDW0u7spDC2uWvK/1viV8OfRppFhLr/kmKb
|
||||
18lWXuAz80DAjUDUsVqEq2MvJBJGoCJUEyjuRsLkHYRM5jYk4v50LyyR0Om73nWF
|
||||
nZBqmqNzdr7Xb9PHHdFhnEc0VvoYbrcM0RVYcEMW3YbmejM891j1d6Iv+/n/qND/
|
||||
NdebGrfWJMmFLf/iEkzTZ3/v5inW9LpWoRc94ioCjJTaEo8Rib6ARRFaJVIsmNXi
|
||||
YicFGO98D+zX+a2t9Yz6IpPajVslnOp6ScpmXgts/2XWD7oE+JgxSAqo/dLVsHgP
|
||||
Ufo=
|
||||
=pulM
|
||||
-----END PGP MESSAGE-----
|
||||
</pre></body></html>
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ--
|
||||
@@ -1,46 +0,0 @@
|
||||
Date: Fri, 8 Jul 1994 09:21:47 -0400
|
||||
From: Mail Delivery Subsystem <MAILER-DAEMON@example.org>
|
||||
Subject: Returned mail: User unknown
|
||||
To: <owner-ups-mib@CS.UTK.EDU>
|
||||
Auto-Submitted: auto-replied
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/report; report-type=delivery-status;
|
||||
boundary="JAA13167.773673707/CS.UTK.EDU"
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: text/plain; charset=us-ascii
|
||||
|
||||
----- The following addresses had delivery problems -----
|
||||
<arathib@vnet.ibm.com> (unrecoverable error)
|
||||
<wsnell@sdcc13.ucsd.edu> (unrecoverable error)
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: message/delivery-status
|
||||
|
||||
Reporting-MTA: dns; cs.utk.edu
|
||||
|
||||
Original-Recipient: rfc822;arathib@vnet.ibm.com
|
||||
Final-Recipient: rfc822;arathib@vnet.ibm.com
|
||||
Action: failed
|
||||
Status: 5.0.0 (permanent failure)
|
||||
Diagnostic-Code: smtp;
|
||||
550 'arathib@vnet.IBM.COM' is not a registered gateway user
|
||||
Remote-MTA: dns; vnet.ibm.com
|
||||
|
||||
Original-Recipient: rfc822;johnh@hpnjld.njd.hp.com
|
||||
Final-Recipient: rfc822;johnh@hpnjld.njd.hp.com
|
||||
Action: delayed
|
||||
Status: 4.0.0 (hpnjld.njd.jp.com: host name lookup failure)
|
||||
|
||||
Original-Recipient: rfc822;wsnell@sdcc13.ucsd.edu
|
||||
Final-Recipient: rfc822;wsnell@sdcc13.ucsd.edu
|
||||
Action: failed
|
||||
Status: 5.0.0
|
||||
Diagnostic-Code: smtp; 550 user unknown
|
||||
Remote-MTA: dns; sdcc13.ucsd.edu
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: message/rfc822
|
||||
|
||||
[original message goes here]
|
||||
--JAA13167.773673707/CS.UTK.EDU--
|
||||
@@ -1,21 +0,0 @@
|
||||
Subject: Message from {from_addr}
|
||||
From: <{from_addr}>
|
||||
To: <{to_addr}>
|
||||
Date: Sun, 15 Oct 2023 16:43:25 +0000
|
||||
Message-ID: <Mr.78MWtlV7RAi.goCFzBhCYfy@c2.testrun.org>
|
||||
Chat-Version: 1.0
|
||||
Secure-Join: vc-request
|
||||
Secure-Join-Invitenumber: RANDOM-TOKEN
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/mixed; boundary="Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi"
|
||||
|
||||
|
||||
--Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi
|
||||
Content-Type: text/plain; charset=utf-8
|
||||
|
||||
Buy viagra!
|
||||
|
||||
|
||||
--Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi--
|
||||
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
Subject: Message from {from_addr}
|
||||
From: <{from_addr}>
|
||||
To: <{to_addr}>
|
||||
Date: Sun, 15 Oct 2023 16:43:25 +0000
|
||||
Message-ID: <Mr.78MWtlV7RAi.goCFzBhCYfy@c2.testrun.org>
|
||||
Chat-Version: 1.0
|
||||
Secure-Join: vc-request
|
||||
Secure-Join-Invitenumber: RANDOM-TOKEN
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/mixed; boundary="Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi"
|
||||
|
||||
|
||||
--Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi
|
||||
Content-Type: text/plain; charset=utf-8
|
||||
|
||||
Secure-Join: vc-request
|
||||
|
||||
|
||||
--Gl92xgZjOShJ5PGHntqYkoo2OK2Dvi--
|
||||
|
||||
|
||||
@@ -69,11 +69,12 @@ def maildata(request):
|
||||
|
||||
assert datadir.exists(), datadir
|
||||
|
||||
def maildata(name, from_addr, to_addr, subject="[...]"):
|
||||
def maildata(name, from_addr, to_addr, subject="..."):
|
||||
# Using `.read_bytes().decode()` instead of `.read_text()` to preserve newlines.
|
||||
data = datadir.joinpath(name).read_bytes().decode()
|
||||
|
||||
text = data.format(from_addr=from_addr, to_addr=to_addr, subject=subject)
|
||||
return BytesParser(policy=policy.SMTP).parsebytes(text.encode())
|
||||
return BytesParser(policy=policy.default).parsebytes(text.encode())
|
||||
|
||||
return maildata
|
||||
|
||||
|
||||
@@ -15,14 +15,6 @@ def test_read_config_basic(example_config):
|
||||
assert example_config.mail_domain == "chat.example.org"
|
||||
|
||||
|
||||
def test_read_config_basic_using_defaults(tmp_path, maildomain):
|
||||
inipath = tmp_path.joinpath("chatmail.ini")
|
||||
inipath.write_text(f"[params]\nmail_domain = {maildomain}")
|
||||
example_config = read_config(inipath)
|
||||
assert example_config.max_user_send_per_minute == 60
|
||||
assert example_config.filtermail_smtp_port_incoming == 10081
|
||||
|
||||
|
||||
def test_read_config_testrun(make_config):
|
||||
config = make_config("something.testrun.org")
|
||||
assert config.mail_domain == "something.testrun.org"
|
||||
@@ -35,7 +27,6 @@ def test_read_config_testrun(make_config):
|
||||
assert config.max_user_send_per_minute == 60
|
||||
assert config.max_mailbox_size == "100M"
|
||||
assert config.delete_mails_after == "20"
|
||||
assert config.delete_large_after == "7"
|
||||
assert config.username_min_length == 9
|
||||
assert config.username_max_length == 9
|
||||
assert config.password_min_length == 9
|
||||
|
||||
@@ -1,12 +1,11 @@
|
||||
import pytest
|
||||
|
||||
from chatmaild.filtermail import (
|
||||
IncomingBeforeQueueHandler,
|
||||
OutgoingBeforeQueueHandler,
|
||||
BeforeQueueHandler,
|
||||
SendRateLimiter,
|
||||
check_armored_payload,
|
||||
check_encrypted,
|
||||
is_securejoin,
|
||||
common_encrypted_subjects,
|
||||
)
|
||||
|
||||
|
||||
@@ -19,13 +18,7 @@ def maildomain():
|
||||
@pytest.fixture
|
||||
def handler(make_config, maildomain):
|
||||
config = make_config(maildomain)
|
||||
return OutgoingBeforeQueueHandler(config)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def inhandler(make_config, maildomain):
|
||||
config = make_config(maildomain)
|
||||
return IncomingBeforeQueueHandler(config)
|
||||
return BeforeQueueHandler(config)
|
||||
|
||||
|
||||
def test_reject_forged_from(maildata, gencreds, handler):
|
||||
@@ -36,14 +29,14 @@ def test_reject_forged_from(maildata, gencreds, handler):
|
||||
# test that the filter lets good mail through
|
||||
to_addr = gencreds()[0]
|
||||
env.content = maildata(
|
||||
"encrypted.eml", from_addr=env.mail_from, to_addr=to_addr
|
||||
"plain.eml", from_addr=env.mail_from, to_addr=to_addr
|
||||
).as_bytes()
|
||||
|
||||
assert not handler.check_DATA(envelope=env)
|
||||
|
||||
# test that the filter rejects forged mail
|
||||
env.content = maildata(
|
||||
"encrypted.eml", from_addr="forged@c3.testrun.org", to_addr=to_addr
|
||||
"plain.eml", from_addr="forged@c3.testrun.org", to_addr=to_addr
|
||||
).as_bytes()
|
||||
error = handler.check_DATA(envelope=env)
|
||||
assert "500" in error
|
||||
@@ -62,28 +55,19 @@ def test_filtermail_no_encryption_detection(maildata):
|
||||
assert not check_encrypted(msg)
|
||||
|
||||
|
||||
def test_filtermail_securejoin_detection(maildata):
|
||||
msg = maildata(
|
||||
"securejoin-vc.eml", from_addr="some@example.org", to_addr="other@example.org"
|
||||
)
|
||||
assert is_securejoin(msg)
|
||||
|
||||
msg = maildata(
|
||||
"securejoin-vc-fake.eml",
|
||||
from_addr="some@example.org",
|
||||
to_addr="other@example.org",
|
||||
)
|
||||
assert not is_securejoin(msg)
|
||||
|
||||
|
||||
def test_filtermail_encryption_detection(maildata):
|
||||
msg = maildata(
|
||||
"encrypted.eml",
|
||||
from_addr="1@example.org",
|
||||
to_addr="2@example.org",
|
||||
subject="Subject does not matter, will be replaced anyway",
|
||||
)
|
||||
assert check_encrypted(msg)
|
||||
for subject in common_encrypted_subjects:
|
||||
msg = maildata(
|
||||
"encrypted.eml",
|
||||
from_addr="1@example.org",
|
||||
to_addr="2@example.org",
|
||||
subject=subject,
|
||||
)
|
||||
assert check_encrypted(msg)
|
||||
|
||||
# if the subject is not a known encrypted subject value, it is not considered ac-encrypted
|
||||
msg.replace_header("Subject", "Click this link")
|
||||
assert not check_encrypted(msg)
|
||||
|
||||
|
||||
def test_filtermail_no_literal_packets(maildata):
|
||||
@@ -113,7 +97,7 @@ def test_send_rate_limiter():
|
||||
break
|
||||
|
||||
|
||||
def test_cleartext_excempt_privacy(maildata, gencreds, handler):
|
||||
def test_excempt_privacy(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = "privacy@testrun.org"
|
||||
handler.config.passthrough_recipients = [to_addr]
|
||||
@@ -134,73 +118,10 @@ def test_cleartext_excempt_privacy(maildata, gencreds, handler):
|
||||
rcpt_tos = [to_addr, false_to]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert "523" in handler.check_DATA(envelope=env2)
|
||||
assert "500" in handler.check_DATA(envelope=env2)
|
||||
|
||||
|
||||
def test_cleartext_self_send_autocrypt_setup_message(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = from_addr
|
||||
|
||||
msg = maildata("asm.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert not handler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_send_fails(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = gencreds()[0]
|
||||
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
res = handler.check_DATA(envelope=env)
|
||||
assert "523 Encryption Needed" in res
|
||||
|
||||
|
||||
def test_cleartext_incoming_fails(maildata, gencreds, inhandler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr, password = gencreds()
|
||||
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
user = inhandler.config.get_user(to_addr)
|
||||
user.set_password(password)
|
||||
res = inhandler.check_DATA(envelope=env)
|
||||
assert "523 Encryption Needed" in res
|
||||
|
||||
user.allow_incoming_cleartext()
|
||||
assert not inhandler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_incoming_mailer_daemon(maildata, gencreds, inhandler):
|
||||
from_addr = "mailer-daemon@example.org"
|
||||
to_addr = gencreds()[0]
|
||||
|
||||
msg = maildata("mailer-daemon.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert not inhandler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_passthrough_domains(maildata, gencreds, handler):
|
||||
def test_passthrough_domains(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = "privacy@x.y.z"
|
||||
handler.config.passthrough_recipients = ["@x.y.z"]
|
||||
@@ -221,10 +142,10 @@ def test_cleartext_passthrough_domains(maildata, gencreds, handler):
|
||||
rcpt_tos = [to_addr, false_to]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert "523" in handler.check_DATA(envelope=env2)
|
||||
assert "500" in handler.check_DATA(envelope=env2)
|
||||
|
||||
|
||||
def test_cleartext_passthrough_senders(gencreds, handler, maildata):
|
||||
def test_passthrough_senders(gencreds, handler, maildata):
|
||||
acc1 = gencreds()[0]
|
||||
to_addr = "recipient@something.org"
|
||||
handler.config.passthrough_senders = [acc1]
|
||||
@@ -275,20 +196,10 @@ UN4fiB0KR9JyG2ayUdNJVkXZSZLnHyRgiaadlpUo16LVvw==\r
|
||||
=b5Kp\r
|
||||
-----END PGP MESSAGE-----\r
|
||||
\r
|
||||
\r
|
||||
"""
|
||||
|
||||
assert check_armored_payload(payload) == True
|
||||
|
||||
payload = payload.removesuffix("\r\n")
|
||||
assert check_armored_payload(payload) == True
|
||||
|
||||
payload = payload.removesuffix("\r\n")
|
||||
assert check_armored_payload(payload) == True
|
||||
|
||||
payload = payload.removesuffix("\r\n")
|
||||
assert check_armored_payload(payload) == True
|
||||
|
||||
payload = """-----BEGIN PGP MESSAGE-----\r
|
||||
\r
|
||||
HELLOWORLD
|
||||
@@ -304,45 +215,3 @@ HELLOWORLD
|
||||
\r
|
||||
"""
|
||||
assert check_armored_payload(payload) == False
|
||||
|
||||
# Test payload using partial body length
|
||||
# as generated by GopenPGP.
|
||||
payload = """-----BEGIN PGP MESSAGE-----\r
|
||||
\r
|
||||
wV4DdCVjRfOT3TQSAQdAY5+pjT6mlCxPGdR3be4w7oJJRUGIPI/Vnh+mJxGSm34w\r
|
||||
LNlVc89S1g22uQYFif2sUJsQWbpoHpNkuWpkSgOaHmNvrZiY/YU5iv+cZ3LbmtUG\r
|
||||
0uoBisSHh9O1c+5sYZSbrvYZ1NOwlD7Fv/U5/Mw4E5+CjxfdgNGp5o3DDddzPK78\r
|
||||
jseDhdSXxnaiIJC93hxNX6R1RPt3G2gukyzx69wciPQShcF8zf3W3o75Ed7B8etV\r
|
||||
QEeB16xzdFhKa9JxdjTu3osgCs21IO7wpcFkjc7nZzlW6jPnELJJaNmv4yOOCjMp\r
|
||||
6YAkaN/BkL+jHTznHDuDsT5ilnTXpwHDU1Cm9PIx/KFcNCQnIB+2DcdIHPHUH1ci\r
|
||||
jvqoeXAVWjKXEjS7PqPFuP/xGbrWG2ugs+toXJOKbgRkExvKs1dwPFKrgghvCVbW\r
|
||||
AcKejQKAPArLwpkA7aD875TZQShvGt74fNs45XBlGOYOnNOAJ1KAmzrXLIDViyyB\r
|
||||
kDsmTBk785xofuCkjBpXSe6vsMprPzCteDfaUibh8FHeJjucxPerwuOPEmnogNaf\r
|
||||
YyL4+iy8H8I9/p7pmUqILprxTG0jTOtlk0bTVzeiF56W1xbtSEMuOo4oFbQTyOM2\r
|
||||
bKXaYo774Jm+rRtKAnnI2dtf9RpK19cog6YNzfYjesLKbXDsPZbN5rmwyFiCvvxC\r
|
||||
kQ6JLob+B2fPdY2gzy7LypxktS8Zi1HJcWDHJGVmQodaDLqKUObb4M26bXDe6oxI\r
|
||||
NS8PJz5exVbM3KhZnUOEn6PJRBBf5a/ZqxlhZPcQo/oBuhKpBRpO5kSDwPIUByu3\r
|
||||
UlXLSkpMqe9pUarAOEuQjfl2RVY7U+RrQYp4YP5keMO+i8NCefAFbowTTufO1JIq\r
|
||||
2nVgCi/QVnxZyEc9OYt/8AE3g4cdojE+vsSDifZLSWYIetpfrohHv3dT3StD1QRG\r
|
||||
0QE6qq6oKpg/IL0cjvuX4c7a7bslv2fXp8t75y37RU6253qdIebhxc/cRhPbc/yu\r
|
||||
p0YLyD4SrvKTLP2ZV95jT4IPEpqm4AN3QmiOzdtqR2gLyb62L8QfqI/FdwsIiRiM\r
|
||||
hqydwoqt/lfSqG1WKPh+6EkMkH+TDiCC1BQdbN1MNcyUtcjb35PR2c8Ld2TF3guA\r
|
||||
jLIqMt/Vb7hBoMb2FcsOYY25ka9oV62OwgKWLXnFzk+modMR5fzb4kxVVAYEqP+D\r
|
||||
T5KO1Vs76v1fyPGOq6BbBCvLwTqe/e6IZInJles4v5jrhnLcGKmNGivCUDe6X6NY\r
|
||||
UKNt5RsZllwDQpaAb5dMNhyrk8SgIE7TBI7rvqIdUCE52Vy+0JDxFg5olRpFUfO6\r
|
||||
/MyTW3Yo/ekk/npHr7iYYqJTCc21bDGLWQcIo/XO7WPxrKNWGBNPFnkRdw0MaKr4\r
|
||||
+cEM3V8NFnSEpC12xA+RX/CezuJtwXZK5MpG76eYqMO6qyC+c25YcFecEufDZDxx\r
|
||||
ZLqRszVRyxyWPtk/oIeQK2v9wOqY6N9/ff01gHz69vqYqN5bUw/QKZsmx1zW+gPw\r
|
||||
6x2tDK2BHeYl182gCbhlKISRFwCtbjqZSkiKWao/VtygHkw0fK34avJuyQ/X9YaN\r
|
||||
BRy+7Lf3VA53pnB5WJ1xwRXN8VDvmZeXzv2krHveCMemj0OjnRoCLu117xN0A5m9\r
|
||||
Fm/RoDix5PolDHtWTtr2m1n2hp2LHnj8at9lFEd0SKhAYHVL9KjzycwWODZRXt+x\r
|
||||
zGDDuooEeTvdY5NLyKcl4gETz1ZP4Ez5jGGjhPSwSpq1mU7UaJ9ZXXdr4KHyifW6\r
|
||||
ggNzNsGhXTap7IWZpTtqXABydfiBshmH2NjqtNDwBweJVSgP10+r0WhMWlaZs6xl\r
|
||||
V3o5yskJt6GlkwpJxZrTvN6Tiww/eW7HFV6NGf7IRSWY5tJc/iA7/92tOmkdvJ1q\r
|
||||
myLbG7cJB787QjplEyVe2P/JBO6xYvbkJLf9Q+HaviTO25rugRSrYsoKMDfO8VlQ\r
|
||||
1CcnTPVtApPZJEQzAWJEgVAM8uIlkqWJJMgyWT34sTkdBeCUFGloXQFs9Yxd0AGf\r
|
||||
/zHEkYZSTKpVSvAIGu4=\r
|
||||
=6iHb\r
|
||||
-----END PGP MESSAGE-----\r
|
||||
"""
|
||||
assert check_armored_payload(payload) == True
|
||||
|
||||
@@ -242,22 +242,6 @@ def test_requeue_removes_tmp_files(notifier, metadata, testaddr, caplog):
|
||||
assert queue_item.addr == testaddr
|
||||
|
||||
|
||||
def test_requeue_removes_invalid_files(notifier, metadata, testaddr, caplog):
|
||||
metadata.add_token_to_addr(testaddr, "01234")
|
||||
notifier.new_message_for_addr(testaddr, metadata)
|
||||
# empty/invalid files should be ignored
|
||||
p = notifier.queue_dir.joinpath("1203981203")
|
||||
p.touch()
|
||||
notifier2 = notifier.__class__(notifier.queue_dir)
|
||||
notifier2.requeue_persistent_queue_items()
|
||||
assert "spurious" in caplog.records[0].msg
|
||||
assert not p.exists()
|
||||
assert notifier2.retry_queues[0].qsize() == 1
|
||||
when, queue_item = notifier2.retry_queues[0].get()
|
||||
assert when <= int(time.time())
|
||||
assert queue_item.addr == testaddr
|
||||
|
||||
|
||||
def test_start_and_stop_notification_threads(notifier, testaddr):
|
||||
threads = notifier.start_notification_threads(None)
|
||||
for retry_num, threadlist in threads.items():
|
||||
|
||||
@@ -2,15 +2,8 @@ from chatmaild.metrics import main
|
||||
|
||||
|
||||
def test_main(tmp_path, capsys):
|
||||
paths = []
|
||||
for x in ("ci-asllkj", "ac_12l3kj", "qweqwe", "ci-l1k2j31l2k3"):
|
||||
p = tmp_path.joinpath(x)
|
||||
p.mkdir()
|
||||
p.joinpath("cur").mkdir()
|
||||
paths.append(p)
|
||||
|
||||
tmp_path.joinpath("nomailbox").mkdir()
|
||||
|
||||
tmp_path.joinpath(x).mkdir()
|
||||
main(tmp_path)
|
||||
out, _ = capsys.readouterr()
|
||||
d = {}
|
||||
|
||||
@@ -40,17 +40,3 @@ def test_no_mailboxes_dir(testaddr, example_config, tmp_path):
|
||||
user.set_password("someeqkjwelkqwjleqwe")
|
||||
user.set_last_login_timestamp(100000)
|
||||
assert user.get_last_login_timestamp() == 86400
|
||||
|
||||
|
||||
def test_set_get_cleartext_flag(testaddr, example_config, tmp_path):
|
||||
p = tmp_path.joinpath("a", "mailboxes")
|
||||
example_config.mailboxes_dir = p
|
||||
|
||||
user = example_config.get_user(testaddr)
|
||||
user.set_password("someeqkjwelkqwjleqwe")
|
||||
user.set_last_login_timestamp(100000)
|
||||
assert user.get_last_login_timestamp() == 86400
|
||||
|
||||
assert not user.is_incoming_cleartext_ok()
|
||||
user.allow_incoming_cleartext()
|
||||
assert user.is_incoming_cleartext_ok()
|
||||
|
||||
@@ -13,7 +13,6 @@ class User:
|
||||
self.maildir = maildir
|
||||
self.addr = addr
|
||||
self.password_path = password_path
|
||||
self.enforce_E2EE_path = maildir.joinpath("enforceE2EEincoming")
|
||||
self.uid = uid
|
||||
self.gid = gid
|
||||
|
||||
@@ -36,13 +35,6 @@ class User:
|
||||
home = str(self.maildir)
|
||||
return dict(addr=self.addr, home=home, uid=self.uid, gid=self.gid, password=pw)
|
||||
|
||||
def is_incoming_cleartext_ok(self):
|
||||
return not self.enforce_E2EE_path.exists()
|
||||
|
||||
def allow_incoming_cleartext(self):
|
||||
if self.enforce_E2EE_path.exists():
|
||||
self.enforce_E2EE_path.unlink()
|
||||
|
||||
def set_password(self, enc_password):
|
||||
"""Set the specified password for this user.
|
||||
|
||||
@@ -58,8 +50,6 @@ class User:
|
||||
if not self.addr.startswith("echo@"):
|
||||
logging.error(f"could not write password for: {self.addr}")
|
||||
raise
|
||||
if not self.addr.startswith("echo@"):
|
||||
self.enforce_E2EE_path.touch()
|
||||
|
||||
def set_last_login_timestamp(self, timestamp):
|
||||
"""Track login time with daily granularity
|
||||
|
||||
@@ -7,35 +7,17 @@ import io
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from io import StringIO
|
||||
from pathlib import Path
|
||||
|
||||
from chatmaild.config import Config, read_config
|
||||
from pyinfra import facts, host
|
||||
from pyinfra.api import FactBase
|
||||
from pyinfra.facts.files import File
|
||||
from pyinfra.facts.server import Sysctl
|
||||
from pyinfra.facts.systemd import SystemdEnabled
|
||||
from pyinfra.operations import apt, files, pip, server, systemd
|
||||
|
||||
from .acmetool import deploy_acmetool
|
||||
|
||||
|
||||
class Port(FactBase):
|
||||
"""
|
||||
Returns the process occuping a port.
|
||||
"""
|
||||
|
||||
def command(self, port: int) -> str:
|
||||
return (
|
||||
"ss -lptn 'src :%d' | awk 'NR>1 {print $6,$7}' | sed 's/users:((\"//;s/\".*//'"
|
||||
% (port,)
|
||||
)
|
||||
|
||||
def process(self, output: [str]) -> str:
|
||||
return output[0]
|
||||
|
||||
|
||||
def _build_chatmaild(dist_dir) -> None:
|
||||
dist_dir = Path(dist_dir).resolve()
|
||||
if dist_dir.exists():
|
||||
@@ -97,8 +79,8 @@ def _install_remote_venv_with_chatmaild(config) -> None:
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
name="install gcc and headers to build crypt_r source package",
|
||||
packages=["gcc", "python3-dev"],
|
||||
name="install python3-dev to build crypt_r source package",
|
||||
packages=["python3-dev"],
|
||||
)
|
||||
|
||||
server.shell(
|
||||
@@ -124,14 +106,12 @@ def _install_remote_venv_with_chatmaild(config) -> None:
|
||||
for fn in (
|
||||
"doveauth",
|
||||
"filtermail",
|
||||
"filtermail-incoming",
|
||||
"echobot",
|
||||
"chatmail-metadata",
|
||||
"lastlogin",
|
||||
):
|
||||
execpath = fn if fn != "filtermail-incoming" else "filtermail"
|
||||
params = dict(
|
||||
execpath=f"{remote_venv_dir}/bin/{execpath}",
|
||||
execpath=f"{remote_venv_dir}/bin/{fn}",
|
||||
config_path=remote_chatmail_inipath,
|
||||
remote_venv_dir=remote_venv_dir,
|
||||
mail_domain=config.mail_domain,
|
||||
@@ -235,37 +215,51 @@ def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
|
||||
server.shell(
|
||||
name="Generate OpenDKIM domain keys",
|
||||
commands=[
|
||||
f"/usr/sbin/opendkim-genkey -D /etc/dkimkeys -d {domain} -s {dkim_selector}"
|
||||
f"opendkim-genkey -D /etc/dkimkeys -d {domain} -s {dkim_selector}"
|
||||
],
|
||||
_use_su_login=True,
|
||||
_su_user="opendkim",
|
||||
_sudo=True,
|
||||
_sudo_user="opendkim",
|
||||
)
|
||||
|
||||
service_file = files.put(
|
||||
name="Configure opendkim to restart once a day",
|
||||
src=importlib.resources.files(__package__).joinpath("opendkim/systemd.conf"),
|
||||
dest="/etc/systemd/system/opendkim.service.d/10-prevent-memory-leak.conf",
|
||||
)
|
||||
need_restart |= service_file.changed
|
||||
|
||||
return need_restart
|
||||
|
||||
|
||||
def _uninstall_mta_sts_daemon() -> None:
|
||||
# Remove configuration.
|
||||
files.file("/etc/mta-sts-daemon.yml", present=False)
|
||||
def _install_mta_sts_daemon() -> bool:
|
||||
need_restart = False
|
||||
|
||||
files.directory("/usr/local/lib/postfix-mta-sts-resolver", present=False)
|
||||
|
||||
files.file("/etc/systemd/system/mta-sts-daemon.service", present=False)
|
||||
|
||||
systemd.service(
|
||||
name="Stop MTA-STS daemon",
|
||||
service="mta-sts-daemon.service",
|
||||
daemon_reload=True,
|
||||
running=False,
|
||||
enabled=False,
|
||||
config = files.put(
|
||||
name="upload postfix-mta-sts-resolver config",
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"postfix/mta-sts-daemon.yml"
|
||||
),
|
||||
dest="/etc/mta-sts-daemon.yml",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= config.changed
|
||||
|
||||
server.shell(
|
||||
name="install postfix-mta-sts-resolver with pip",
|
||||
commands=[
|
||||
"python3 -m virtualenv /usr/local/lib/postfix-mta-sts-resolver",
|
||||
"/usr/local/lib/postfix-mta-sts-resolver/bin/pip install postfix-mta-sts-resolver",
|
||||
],
|
||||
)
|
||||
|
||||
systemd_unit = files.put(
|
||||
name="upload mta-sts-daemon systemd unit",
|
||||
src=importlib.resources.files(__package__).joinpath(
|
||||
"postfix/mta-sts-daemon.service"
|
||||
),
|
||||
dest="/etc/systemd/system/mta-sts-daemon.service",
|
||||
user="root",
|
||||
group="root",
|
||||
mode="644",
|
||||
)
|
||||
need_restart |= systemd_unit.changed
|
||||
|
||||
return need_restart
|
||||
|
||||
|
||||
def _configure_postfix(config: Config, debug: bool = False) -> bool:
|
||||
@@ -365,10 +359,6 @@ def _configure_dovecot(config: Config, debug: bool = False) -> bool:
|
||||
# it is recommended to set the following inotify limits
|
||||
for name in ("max_user_instances", "max_user_watches"):
|
||||
key = f"fs.inotify.{name}"
|
||||
if host.get_fact(Sysctl)[key] > 65535:
|
||||
# Skip updating limits if already sufficient
|
||||
# (enables running in incus containers where sysctl readonly)
|
||||
continue
|
||||
server.sysctl(
|
||||
name=f"Change {key}",
|
||||
key=key,
|
||||
@@ -564,6 +554,7 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
|
||||
server.group(name="Create vmail group", group="vmail", system=True)
|
||||
server.user(name="Create vmail user", user="vmail", group="vmail", system=True)
|
||||
server.user(name="Create filtermail user", user="filtermail", system=True)
|
||||
server.group(name="Create opendkim group", group="opendkim", system=True)
|
||||
server.user(
|
||||
name="Create opendkim user",
|
||||
@@ -598,12 +589,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
ensure_newline=True,
|
||||
)
|
||||
|
||||
if host.get_fact(Port, port=53) != "unbound":
|
||||
files.line(
|
||||
name="Add 9.9.9.9 to resolv.conf",
|
||||
path="/etc/resolv.conf",
|
||||
line="nameserver 9.9.9.9",
|
||||
)
|
||||
apt.update(name="apt update", cache_time=24 * 3600)
|
||||
apt.upgrade(name="upgrade apt packages", auto_remove=True)
|
||||
|
||||
@@ -615,12 +600,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
# Run local DNS resolver `unbound`.
|
||||
# `resolvconf` takes care of setting up /etc/resolv.conf
|
||||
# to use 127.0.0.1 as the resolver.
|
||||
from cmdeploy.cmdeploy import Out
|
||||
|
||||
process_on_53 = host.get_fact(Port, port=53)
|
||||
if process_on_53 not in (None, "unbound"):
|
||||
Out().red(f"Can't install unbound: port 53 is occupied by: {process_on_53}")
|
||||
exit(1)
|
||||
apt.packages(
|
||||
name="Install unbound",
|
||||
packages=["unbound", "unbound-anchor", "dnsutils"],
|
||||
@@ -684,8 +663,8 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
debug = False
|
||||
dovecot_need_restart = _configure_dovecot(config, debug=debug)
|
||||
postfix_need_restart = _configure_postfix(config, debug=debug)
|
||||
mta_sts_need_restart = _install_mta_sts_daemon()
|
||||
nginx_need_restart = _configure_nginx(config)
|
||||
_uninstall_mta_sts_daemon()
|
||||
|
||||
_remove_rspamd()
|
||||
opendkim_need_restart = _configure_opendkim(mail_domain, "opendkim")
|
||||
@@ -695,10 +674,18 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
service="opendkim.service",
|
||||
running=True,
|
||||
enabled=True,
|
||||
daemon_reload=opendkim_need_restart,
|
||||
restarted=opendkim_need_restart,
|
||||
)
|
||||
|
||||
systemd.service(
|
||||
name="Start and enable MTA-STS daemon",
|
||||
service="mta-sts-daemon.service",
|
||||
daemon_reload=True,
|
||||
running=True,
|
||||
enabled=True,
|
||||
restarted=mta_sts_need_restart,
|
||||
)
|
||||
|
||||
# Dovecot should be started before Postfix
|
||||
# because it creates authentication socket
|
||||
# required by Postfix.
|
||||
@@ -748,23 +735,10 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
enabled=True,
|
||||
restarted=journald_conf.changed,
|
||||
)
|
||||
files.directory(
|
||||
name="Ensure old logs on disk are deleted",
|
||||
path="/var/log/journal/",
|
||||
present=False,
|
||||
)
|
||||
|
||||
apt.packages(
|
||||
name="Ensure cron is installed",
|
||||
packages=["cron"],
|
||||
)
|
||||
git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode()
|
||||
git_diff = subprocess.check_output(["git", "diff"]).decode()
|
||||
files.put(
|
||||
name="Upload chatmail relay git commiit hash",
|
||||
src=StringIO(git_hash + git_diff),
|
||||
dest="/etc/chatmail-version",
|
||||
mode="700",
|
||||
)
|
||||
|
||||
deploy_mtail(config)
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
"acme-enter-email": "{{ email }}"
|
||||
"acme-agreement:https://letsencrypt.org/documents/LE-SA-v1.5-February-24-2025.pdf": true
|
||||
"acme-agreement:https://letsencrypt.org/documents/LE-SA-v1.4-April-3-2024.pdf": true
|
||||
|
||||
@@ -16,7 +16,7 @@ www.{{ mail_domain }}. CNAME {{ mail_domain }}.
|
||||
;
|
||||
; Recommended DNS entries for interoperability and security-hardening
|
||||
;
|
||||
{{ mail_domain }}. TXT "v=spf1 a ~all"
|
||||
{{ mail_domain }}. TXT "v=spf1 a:{{ mail_domain }} ~all"
|
||||
_dmarc.{{ mail_domain }}. TXT "v=DMARC1;p=reject;adkim=s;aspf=s"
|
||||
|
||||
{% if acme_account_url %}
|
||||
|
||||
@@ -86,19 +86,15 @@ def run_cmd(args, out):
|
||||
out.red("Please re-run scripts/initenv.sh to update pyinfra to version 3.")
|
||||
return 1
|
||||
|
||||
try:
|
||||
retcode = out.check_call(cmd, env=env)
|
||||
if retcode == 0:
|
||||
out.green("Deploy completed, call `cmdeploy dns` next.")
|
||||
elif not remote_data["acme_account_url"]:
|
||||
out.red("Deploy completed but letsencrypt not configured")
|
||||
out.red("Run 'cmdeploy run' again")
|
||||
retcode = 0
|
||||
else:
|
||||
out.red("Deploy failed")
|
||||
except subprocess.CalledProcessError:
|
||||
retcode = out.check_call(cmd, env=env)
|
||||
if retcode == 0:
|
||||
out.green("Deploy completed, call `cmdeploy dns` next.")
|
||||
elif not remote_data["acme_account_url"]:
|
||||
out.red("Deploy completed but letsencrypt not configured")
|
||||
out.red("Run 'cmdeploy run' again")
|
||||
retcode = 0
|
||||
else:
|
||||
out.red("Deploy failed")
|
||||
retcode = 1
|
||||
return retcode
|
||||
|
||||
|
||||
|
||||
@@ -177,20 +177,17 @@ service auth-worker {
|
||||
}
|
||||
|
||||
service imap-login {
|
||||
# We are not using "high-security-mode" because even if dovecot
|
||||
# login would be compromised, there are only E2EE messages to be
|
||||
# found or sent, and an attacker doesn't have the key used with this address.
|
||||
# High-security mode.
|
||||
# Each process serves a single connection and exits afterwards.
|
||||
# This is the default, but we set it explicitly to be sure.
|
||||
# See <https://doc.dovecot.org/admin_manual/login_processes/#high-security-mode> for details.
|
||||
|
||||
# Performance Mode. Each process serves up to 100 connections and exits afterwards.
|
||||
service_count = 100
|
||||
service_count = 1
|
||||
|
||||
# Inrease the number of simultaneous connections.
|
||||
#
|
||||
# As of Dovecot 2.3.19.1 the default is 100 processes.
|
||||
# Combined with `service_count = 1` it means only 100 connections
|
||||
# can be handled simultaneously.
|
||||
# We allow up to 5000 * 100 = 500K connections
|
||||
process_limit = 10000
|
||||
|
||||
# Avoid startup latency for new connections.
|
||||
@@ -212,7 +209,7 @@ ssl = required
|
||||
ssl_cert = </var/lib/acme/live/{{ config.mail_domain }}/fullchain
|
||||
ssl_key = </var/lib/acme/live/{{ config.mail_domain }}/privkey
|
||||
ssl_dh = </usr/share/dovecot/dh.pem
|
||||
ssl_min_protocol = TLSv1.3
|
||||
ssl_min_protocol = TLSv1.2
|
||||
ssl_prefer_server_ciphers = yes
|
||||
|
||||
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
# delete already seen big mails after 7 days, in the INBOX
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/cur/*' -mtime +{{ config.delete_large_after }} -size +200k -type f -delete
|
||||
# delete all mails after {{ config.delete_mails_after }} days, in the Inbox
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/cur/*' -mtime +{{ config.delete_mails_after }} -type f -delete
|
||||
# or in any IMAP subfolder
|
||||
|
||||
@@ -2,6 +2,15 @@ function dovecot_lua_notify_begin_txn(user)
|
||||
return user
|
||||
end
|
||||
|
||||
function contains(v, needle)
|
||||
for _, keyword in ipairs(v) do
|
||||
if keyword == needle then
|
||||
return true
|
||||
end
|
||||
end
|
||||
return false
|
||||
end
|
||||
|
||||
function dovecot_lua_notify_event_message_new(user, event)
|
||||
local mbox = user:mailbox(event.mailbox)
|
||||
mbox:sync()
|
||||
|
||||
@@ -3,7 +3,7 @@ Description=mtail
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/bin/sh -c "journalctl -f -o short-iso -n 0 | /usr/bin/mtail --address={{ address }} --port={{ port }} --progs /etc/mtail --logtostderr --logs /dev/stdin"
|
||||
ExecStart=/bin/sh -c "journalctl -f -o short-iso -n 0 | /usr/bin/mtail --address={{ address }} --port={{ port }} --progs /etc/mtail --logtostderr --logs -"
|
||||
Restart=on-failure
|
||||
|
||||
[Install]
|
||||
|
||||
@@ -2,25 +2,11 @@ load_module modules/ngx_stream_module.so;
|
||||
|
||||
user www-data;
|
||||
worker_processes auto;
|
||||
|
||||
# Increase the number of connections
|
||||
# that a worker process can open
|
||||
# to avoid errors such as
|
||||
# accept4() failed (24: Too many open files)
|
||||
# and
|
||||
# socket() failed (24: Too many open files) while connecting to upstream
|
||||
# in the logs.
|
||||
# <https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile>
|
||||
worker_rlimit_nofile 2048;
|
||||
pid /run/nginx.pid;
|
||||
error_log syslog:server=unix:/dev/log,facility=local3;
|
||||
|
||||
events {
|
||||
# Increase to avoid errors such as
|
||||
# 768 worker_connections are not enough while connecting to upstream
|
||||
# in the logs.
|
||||
# <https://nginx.org/en/docs/ngx_core_module.html#worker_connections>
|
||||
worker_connections 2048;
|
||||
worker_connections 768;
|
||||
# multi_accept on;
|
||||
}
|
||||
|
||||
@@ -60,7 +46,10 @@ http {
|
||||
|
||||
server {
|
||||
|
||||
listen 127.0.0.1:8443 ssl default_server;
|
||||
listen 8443 ssl default_server;
|
||||
{% if not disable_ipv6 %}
|
||||
listen [::]:8443 ssl default_server;
|
||||
{% endif %}
|
||||
|
||||
root /var/www/html;
|
||||
|
||||
@@ -131,7 +120,10 @@ http {
|
||||
|
||||
# Redirect www. to non-www
|
||||
server {
|
||||
listen 127.0.0.1:8443 ssl;
|
||||
listen 8443 ssl;
|
||||
{% if not disable_ipv6 %}
|
||||
listen [::]:8443 ssl;
|
||||
{% endif %}
|
||||
server_name www.{{ config.domain_name }};
|
||||
return 301 $scheme://{{ config.domain_name }}$request_uri;
|
||||
access_log syslog:server=unix:/dev/log,facility=local7;
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
[Service]
|
||||
Restart=always
|
||||
RuntimeMaxSec=1d
|
||||
@@ -20,12 +20,9 @@ smtpd_tls_key_file=/var/lib/acme/live/{{ config.mail_domain }}/privkey
|
||||
smtpd_tls_security_level=may
|
||||
|
||||
smtp_tls_CApath=/etc/ssl/certs
|
||||
smtp_tls_security_level=verify
|
||||
# Send SNI extension when connecting to other servers.
|
||||
# <https://www.postfix.org/postconf.5.html#smtp_tls_servername>
|
||||
smtp_tls_servername = hostname
|
||||
smtp_tls_security_level=may
|
||||
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
|
||||
smtp_tls_policy_maps = inline:{nauta.cu=may}
|
||||
smtp_tls_policy_maps = socketmap:inet:127.0.0.1:8461:postfix
|
||||
smtpd_tls_protocols = >=TLSv1.2
|
||||
|
||||
# Disable anonymous cipher suites
|
||||
|
||||
@@ -14,11 +14,10 @@ smtp inet n - y - - smtpd -v
|
||||
{%- else %}
|
||||
smtp inet n - y - - smtpd
|
||||
{%- endif %}
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port_incoming }}
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
submission inet n - y - 5000 smtpd
|
||||
-o syslog_name=postfix/submission
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
-o smtpd_tls_mandatory_protocols=>=TLSv1.3
|
||||
-o smtpd_sasl_auth_enable=yes
|
||||
-o smtpd_sasl_type=dovecot
|
||||
-o smtpd_sasl_path=private/auth
|
||||
@@ -32,11 +31,11 @@ submission inet n - y - 5000 smtpd
|
||||
-o milter_macro_daemon_name=ORIGINATING
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
-o cleanup_service_name=authclean
|
||||
smtps inet n - y - 5000 smtpd
|
||||
-o syslog_name=postfix/smtps
|
||||
-o smtpd_tls_wrappermode=yes
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
-o smtpd_tls_mandatory_protocols=>=TLSv1.3
|
||||
-o smtpd_sasl_auth_enable=yes
|
||||
-o smtpd_sasl_type=dovecot
|
||||
-o smtpd_sasl_path=private/auth
|
||||
@@ -49,6 +48,7 @@ smtps inet n - y - 5000 smtpd
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o milter_macro_daemon_name=ORIGINATING
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
-o cleanup_service_name=authclean
|
||||
#628 inet n - y - - qmqpd
|
||||
pickup unix n - y 60 1 pickup
|
||||
cleanup unix n - y - 0 cleanup
|
||||
@@ -76,27 +76,17 @@ anvil unix - - y - 1 anvil
|
||||
scache unix - - y - 1 scache
|
||||
postlog unix-dgram n - n - 1 postlogd
|
||||
filter unix - n n - - lmtp
|
||||
# Local SMTP server for reinjecting outgoing filtered mail.
|
||||
127.0.0.1:{{ config.postfix_reinject_port }} inet n - n - 10 smtpd
|
||||
# Local SMTP server for reinjecting filered mail.
|
||||
localhost:{{ config.postfix_reinject_port }} inet n - n - 10 smtpd
|
||||
-o syslog_name=postfix/reinject
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
-o cleanup_service_name=authclean
|
||||
|
||||
# Local SMTP server for reinjecting incoming filtered mail
|
||||
127.0.0.1:{{ config.postfix_reinject_port_incoming }} inet n - n - 10 smtpd
|
||||
-o syslog_name=postfix/reinject_incoming
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
|
||||
# Cleanup `Received` headers for authenticated mail
|
||||
# to avoid leaking client IP.
|
||||
#
|
||||
# We do not do this for received mails
|
||||
# as this will break DKIM signatures
|
||||
# if `Received` header is signed.
|
||||
#
|
||||
# This service also rewrites
|
||||
# Subject with `[...]`
|
||||
# to make sure the users
|
||||
# cannot send unprotected Subject.
|
||||
authclean unix n - - - 0 cleanup
|
||||
-o header_checks=regexp:/etc/postfix/submission_header_cleanup
|
||||
|
||||
@@ -2,4 +2,3 @@
|
||||
/^X-Originating-IP:/ IGNORE
|
||||
/^X-Mailer:/ IGNORE
|
||||
/^User-Agent:/ IGNORE
|
||||
/^Subject:/ REPLACE Subject: [...]
|
||||
|
||||
@@ -19,7 +19,7 @@ def perform_initial_checks(mail_domain):
|
||||
"""Collecting initial DNS settings."""
|
||||
assert mail_domain
|
||||
if not shell("dig", fail_ok=True):
|
||||
shell("apt-get update && apt-get install -y dnsutils")
|
||||
shell("apt-get install -y dnsutils")
|
||||
A = query_dns("A", mail_domain)
|
||||
AAAA = query_dns("AAAA", mail_domain)
|
||||
MTA_STS = query_dns("CNAME", f"mta-sts.{mail_domain}")
|
||||
|
||||
@@ -1,13 +1,10 @@
|
||||
from subprocess import DEVNULL, CalledProcessError, check_output
|
||||
from subprocess import CalledProcessError, check_output
|
||||
|
||||
|
||||
def shell(command, fail_ok=False):
|
||||
print(f"$ {command}")
|
||||
args = dict(shell=True)
|
||||
if fail_ok:
|
||||
args["stderr"] = DEVNULL
|
||||
try:
|
||||
return check_output(command, **args).decode().rstrip()
|
||||
return check_output(command, shell=True).decode().rstrip()
|
||||
except CalledProcessError:
|
||||
if not fail_ok:
|
||||
raise
|
||||
@@ -17,22 +14,3 @@ def shell(command, fail_ok=False):
|
||||
def get_systemd_running():
|
||||
lines = shell("systemctl --type=service --state=running").split("\n")
|
||||
return [line for line in lines if line.startswith(" ")]
|
||||
|
||||
|
||||
def write_numbytes(path, num):
|
||||
with open(path, "w") as f:
|
||||
f.write("x" * num)
|
||||
|
||||
|
||||
def dovecot_recalc_quota(user):
|
||||
shell(f"doveadm quota recalc -u {user}")
|
||||
output = shell(f"doveadm quota get -u {user}")
|
||||
#
|
||||
# Quota name Type Value Limit %
|
||||
# User quota STORAGE 5 102400 0
|
||||
# User quota MESSAGE 2 - 0
|
||||
#
|
||||
for line in output.split("\n"):
|
||||
parts = line.split()
|
||||
if parts[2] == "STORAGE":
|
||||
return dict(value=int(parts[3]), limit=int(parts[4]), percent=int(parts[5]))
|
||||
|
||||
@@ -7,7 +7,6 @@ Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
RuntimeDirectory=chatmail-metadata
|
||||
UMask=0077
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -7,7 +7,6 @@ Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
RuntimeDirectory=doveauth
|
||||
UMask=0077
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
[Unit]
|
||||
Description=Incoming Chatmail Postfix before queue filter
|
||||
|
||||
[Service]
|
||||
ExecStart={execpath} {config_path} incoming
|
||||
Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
[Unit]
|
||||
Description=Outgoing Chatmail Postfix before queue filter
|
||||
Description=Chatmail Postfix before queue filter
|
||||
|
||||
[Service]
|
||||
ExecStart={execpath} {config_path} outgoing
|
||||
ExecStart={execpath} {config_path}
|
||||
Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
User=filtermail
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -37,7 +37,7 @@ class TestDC:
|
||||
|
||||
def test_ping_pong(self, benchmark, cmfactory):
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
def dc_ping_pong():
|
||||
chat.send_text("ping")
|
||||
@@ -49,7 +49,7 @@ class TestDC:
|
||||
|
||||
def test_send_10_receive_10(self, benchmark, cmfactory, lp):
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
def dc_send_10_receive_10():
|
||||
for i in range(10):
|
||||
|
||||
@@ -90,13 +90,8 @@ def test_concurrent_logins_same_account(
|
||||
|
||||
|
||||
def test_no_vrfy(chatmail_config):
|
||||
domain = chatmail_config.mail_domain
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.settimeout(10)
|
||||
try:
|
||||
sock.connect((domain, 25))
|
||||
except socket.timeout:
|
||||
pytest.skip(f"port 25 not reachable for {domain}")
|
||||
sock.connect((chatmail_config.mail_domain, 25))
|
||||
banner = sock.recv(1024)
|
||||
print(banner)
|
||||
sock.send(b"VRFY wrongaddress@%s\r\n" % (chatmail_config.mail_domain.encode(),))
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import datetime
|
||||
import smtplib
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -54,15 +52,6 @@ class TestSSHExecutor:
|
||||
else:
|
||||
pytest.fail("didn't raise exception")
|
||||
|
||||
def test_opendkim_restarted(self, sshexec):
|
||||
"""check that opendkim is not running for longer than a day."""
|
||||
cmd = "systemctl show opendkim --timestamp=utc --property=ActiveEnterTimestamp"
|
||||
out = sshexec(call=remote.rshell.shell, kwargs=dict(command=cmd))
|
||||
datestring = out.split("=")[1]
|
||||
since_date = datetime.datetime.strptime(datestring, "%a %Y-%m-%d %H:%M:%S %Z")
|
||||
now = datetime.datetime.now(since_date.tzinfo)
|
||||
assert (now - since_date).total_seconds() < 60 * 60 * 51
|
||||
|
||||
|
||||
def test_remote(remote, imap_or_smtp):
|
||||
lineproducer = remote.iter_output(imap_or_smtp.logcmd)
|
||||
@@ -118,41 +107,14 @@ def test_authenticated_from(cmsetup, maildata):
|
||||
|
||||
@pytest.mark.parametrize("from_addr", ["fake@example.org", "fake@testrun.org"])
|
||||
def test_reject_missing_dkim(cmsetup, maildata, from_addr):
|
||||
"""Test that emails with missing or wrong DMARC, DKIM, and SPF entries are rejected."""
|
||||
recipient = cmsetup.gen_users(1)[0]
|
||||
msg = maildata(
|
||||
"encrypted.eml", from_addr=from_addr, to_addr=recipient.addr
|
||||
).as_string()
|
||||
try:
|
||||
conn = smtplib.SMTP(cmsetup.maildomain, 25, timeout=10)
|
||||
except TimeoutError:
|
||||
pytest.skip(f"port 25 not reachable for {cmsetup.maildomain}")
|
||||
|
||||
with conn as s:
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=recipient.addr).as_string()
|
||||
with smtplib.SMTP(cmsetup.maildomain, 25) as s:
|
||||
with pytest.raises(smtplib.SMTPDataError, match="No valid DKIM signature"):
|
||||
s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg)
|
||||
|
||||
|
||||
def test_rewrite_subject(cmsetup, maildata):
|
||||
"""Test that subject gets replaced with [...]."""
|
||||
user1, user2 = cmsetup.gen_users(2)
|
||||
|
||||
sent_msg = maildata(
|
||||
"encrypted.eml",
|
||||
from_addr=user1.addr,
|
||||
to_addr=user2.addr,
|
||||
subject="Unencrypted subject",
|
||||
).as_string()
|
||||
user1.smtp.sendmail(from_addr=user1.addr, to_addrs=[user2.addr], msg=sent_msg)
|
||||
|
||||
messages = user2.imap.fetch_all_messages()
|
||||
assert len(messages) == 1
|
||||
rcvd_msg = messages[0]
|
||||
assert "Subject: [...]" not in sent_msg
|
||||
assert "Subject: [...]" in rcvd_msg
|
||||
assert "Subject: Unencrypted subject" in sent_msg
|
||||
assert "Subject: Unencrypted subject" not in rcvd_msg
|
||||
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_exceed_rate_limit(cmsetup, gencreds, maildata, chatmail_config):
|
||||
"""Test that the per-account send-mail limit is exceeded."""
|
||||
@@ -185,25 +147,6 @@ def test_expunged(remote, chatmail_config):
|
||||
f"find {chatmail_config.mailboxes_dir} -path '*/tmp/*' -mtime +{outdated_days} -type f",
|
||||
f"find {chatmail_config.mailboxes_dir} -path '*/.*/tmp/*' -mtime +{outdated_days} -type f",
|
||||
]
|
||||
outdated_days = int(chatmail_config.delete_large_after) + 1
|
||||
find_cmds.append(
|
||||
"find {chatmail_config.mailboxes_dir} -path '*/cur/*' -mtime +{outdated_days} -size +200k -type f"
|
||||
)
|
||||
for cmd in find_cmds:
|
||||
for line in remote.iter_output(cmd):
|
||||
assert not line
|
||||
|
||||
|
||||
def test_deployed_state(remote):
|
||||
git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode()
|
||||
git_diff = subprocess.check_output(["git", "diff"]).decode()
|
||||
git_status = [git_hash.strip()]
|
||||
for line in git_diff.splitlines():
|
||||
git_status.append(line.strip().lower())
|
||||
remote_version = []
|
||||
for line in remote.iter_output("cat /etc/chatmail-version"):
|
||||
print(line)
|
||||
remote_version.append(line)
|
||||
# assert len(git_status) == len(remote_version) # for some reason, we only get 11 lines from remote.iter_output()
|
||||
for i in range(len(remote_version)):
|
||||
assert git_status[i] == remote_version[i], "You have undeployed changes."
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import ipaddress
|
||||
import random
|
||||
import re
|
||||
import time
|
||||
|
||||
@@ -6,9 +7,6 @@ import imap_tools
|
||||
import pytest
|
||||
import requests
|
||||
|
||||
from cmdeploy.remote import rshell
|
||||
from cmdeploy.sshexec import SSHExec
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def imap_mailbox(cmfactory):
|
||||
@@ -56,23 +54,22 @@ class TestEndToEndDeltaChat:
|
||||
"""Test that a DC account can send a message to a second DC account
|
||||
on the same chat-mail instance."""
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
lp.sec("ac1: prepare and send text message to ac2")
|
||||
chat.send_text("message0")
|
||||
|
||||
lp.sec("wait for ac2 to receive message")
|
||||
msg2 = ac2._evtracker.wait_next_incoming_message()
|
||||
assert msg2.text == "message0"
|
||||
|
||||
def test_exceed_quota(
|
||||
self, cmfactory, lp, tmpdir, remote, chatmail_config, sshdomain
|
||||
):
|
||||
@pytest.mark.slow
|
||||
def test_exceed_quota(self, cmfactory, lp, tmpdir, remote, chatmail_config):
|
||||
"""This is a very slow test as it needs to upload >100MB of mail data
|
||||
before quota is exceeded, and thus depends on the speed of the upload.
|
||||
"""
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
|
||||
user = ac2.get_config("configured_addr")
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
def parse_size_limit(limit: str) -> int:
|
||||
"""Parse a size limit and return the number of bytes as integer.
|
||||
@@ -85,27 +82,49 @@ class TestEndToEndDeltaChat:
|
||||
return int(float(number) * units[unit])
|
||||
|
||||
quota = parse_size_limit(chatmail_config.max_mailbox_size)
|
||||
attachsize = 1 * 1024 * 1024
|
||||
num_to_send = quota // attachsize + 2
|
||||
lp.sec(f"ac1: send {num_to_send} large files to ac2")
|
||||
lp.indent(f"per-user quota is assumed to be: {quota / (1024 * 1024)}MB")
|
||||
alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
|
||||
msgs = []
|
||||
for i in range(num_to_send):
|
||||
attachment = tmpdir / f"attachment{i}"
|
||||
data = "".join(random.choice(alphanumeric) for i in range(1024))
|
||||
with open(attachment, "w+") as f:
|
||||
for j in range(attachsize // len(data)):
|
||||
f.write(data)
|
||||
|
||||
lp.sec(f"filling remote inbox for {user}")
|
||||
fn = f"7743102289.M843172P2484002.c20,S={quota},W=2398:2,"
|
||||
path = chatmail_config.mailboxes_dir.joinpath(user, "cur", fn)
|
||||
sshexec = SSHExec(sshdomain)
|
||||
sshexec(call=rshell.write_numbytes, kwargs=dict(path=str(path), num=120))
|
||||
res = sshexec(call=rshell.dovecot_recalc_quota, kwargs=dict(user=user))
|
||||
assert res["percent"] >= 100
|
||||
msg = chat.send_file(str(attachment))
|
||||
msgs.append(msg)
|
||||
lp.indent(f"Sent out msg {i}, size {attachsize / (1024 * 1024)}MB")
|
||||
|
||||
lp.sec("ac2: check quota is triggered")
|
||||
lp.sec("ac2: check messages are arriving until quota is reached")
|
||||
|
||||
starting = True
|
||||
addr = ac2.get_config("addr").lower()
|
||||
saved_ok = 0
|
||||
for line in remote.iter_output("journalctl -n0 -f -u dovecot"):
|
||||
if starting:
|
||||
chat.send_text("hello")
|
||||
starting = False
|
||||
if user not in line:
|
||||
if addr not in line:
|
||||
# print(line)
|
||||
continue
|
||||
if "quota exceeded" in line:
|
||||
return
|
||||
if "quota" in line:
|
||||
if "quota exceeded" in line:
|
||||
if saved_ok < num_to_send // 2:
|
||||
pytest.fail(
|
||||
f"quota exceeded too early: after {saved_ok} messages already"
|
||||
)
|
||||
lp.indent("good, message sending failed because quota was exceeded")
|
||||
return
|
||||
if (
|
||||
"stored mail into mailbox 'inbox'" in line
|
||||
or "saved mail to inbox" in line
|
||||
):
|
||||
saved_ok += 1
|
||||
print(f"{saved_ok}: {line}")
|
||||
if saved_ok >= num_to_send:
|
||||
break
|
||||
|
||||
pytest.fail("sending succeeded although messages should exceed quota")
|
||||
|
||||
def test_securejoin(self, cmfactory, lp, maildomain2):
|
||||
ac1 = cmfactory.new_online_configuring_account(cache=False)
|
||||
@@ -153,7 +172,7 @@ def test_hide_senders_ip_address(cmfactory):
|
||||
assert ipaddress.ip_address(public_ip)
|
||||
|
||||
user1, user2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(user1, user2)
|
||||
chat = cmfactory.get_accepted_chat(user1, user2)
|
||||
|
||||
chat.send_text("testing submission header cleanup")
|
||||
user2._evtracker.wait_next_incoming_message()
|
||||
@@ -162,18 +181,11 @@ def test_hide_senders_ip_address(cmfactory):
|
||||
assert public_ip not in msg.obj.as_string()
|
||||
|
||||
|
||||
def test_echobot(cmfactory, chatmail_config, lp, sshdomain):
|
||||
def test_echobot(cmfactory, chatmail_config, lp):
|
||||
ac = cmfactory.get_online_accounts(1)[0]
|
||||
|
||||
# establish contact with echobot
|
||||
sshexec = SSHExec(sshdomain)
|
||||
command = "cat /var/lib/echobot/invite-link.txt"
|
||||
echo_invite_link = sshexec(call=rshell.shell, kwargs=dict(command=command))
|
||||
chat = ac.qr_setup_contact(echo_invite_link)
|
||||
ac._evtracker.wait_securejoin_joiner_progress(1000)
|
||||
|
||||
# send message and check it gets replied back
|
||||
lp.sec("Send message to echobot")
|
||||
lp.sec(f"Send message to echo@{chatmail_config.mail_domain}")
|
||||
chat = ac.create_chat(f"echo@{chatmail_config.mail_domain}")
|
||||
text = "hi, I hope you text me back"
|
||||
chat.send_text(text)
|
||||
lp.sec("Wait for reply from echobot")
|
||||
|
||||
@@ -62,7 +62,7 @@ def sshdomain(maildomain):
|
||||
def maildomain2():
|
||||
domain = os.environ.get("CHATMAIL_DOMAIN2")
|
||||
if not domain:
|
||||
pytest.skip("set CHATMAIL_DOMAIN2 to a second chatmail server")
|
||||
pytest.skip("set CHATMAIL_DOMAIN2 to a ssh-reachable chatmail instance")
|
||||
return domain
|
||||
|
||||
|
||||
@@ -302,13 +302,10 @@ def cmfactory(request, gencreds, tmpdir, maildomain):
|
||||
pytest.importorskip("deltachat")
|
||||
from deltachat.testplugin import ACFactory
|
||||
|
||||
data = request.getfixturevalue("data")
|
||||
|
||||
testproc = ChatmailTestProcess(request.config, maildomain, gencreds)
|
||||
|
||||
class Data:
|
||||
def read_path(self, path):
|
||||
return
|
||||
|
||||
am = ACFactory(request=request, tmpdir=tmpdir, testprocess=testproc, data=Data())
|
||||
am = ACFactory(request=request, tmpdir=tmpdir, testprocess=testproc, data=data)
|
||||
|
||||
# nb. a bit hacky
|
||||
# would probably be better if deltachat's test machinery grows native support
|
||||
|
||||
@@ -1,18 +1,5 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
if command -v lsb_release 2>&1 >/dev/null; then
|
||||
case "$(lsb_release -is)" in
|
||||
Ubuntu | Debian )
|
||||
if ! dpkg -l | grep python3-dev 2>&1 >/dev/null
|
||||
then
|
||||
echo "You need to install python3-dev for installing the other dependencies."
|
||||
exit 1
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
python3 -m venv --upgrade-deps venv
|
||||
|
||||
venv/bin/pip install -e chatmaild
|
||||
|
||||
@@ -6,6 +6,29 @@ interoperable e-mail service for everyone. What's behind a `chatmail` is
|
||||
effectively a normal e-mail address just like any other but optimized
|
||||
for the usage in chats, especially DeltaChat.
|
||||
|
||||
### Choosing a chatmail address instead of using a random one
|
||||
|
||||
In the Delta Chat account setup you may tap `Create a profile` then `Use other server` and choose `Classic e-mail login`. Here fill the two fields like this:
|
||||
|
||||
- `E-Mail Address`: invent a word with
|
||||
{% if username_min_length == username_max_length %}
|
||||
*exactly* {{ username_min_length }}
|
||||
{% else %}
|
||||
{{ username_min_length}}
|
||||
{% if username_max_length == "more" %}
|
||||
or more
|
||||
{% else %}
|
||||
to {{ username_max_length }}
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
characters
|
||||
and append `@{{config.mail_domain}}` to it.
|
||||
|
||||
- `Existing Password`: invent at least {{ password_min_length }} characters.
|
||||
|
||||
If the e-mail address is not yet taken, you'll get that account.
|
||||
The first login sets your password.
|
||||
|
||||
|
||||
### Rate and storage limits
|
||||
|
||||
@@ -15,10 +38,9 @@ for the usage in chats, especially DeltaChat.
|
||||
|
||||
- You may send up to {{ config.max_user_send_per_minute }} messages per minute.
|
||||
|
||||
- You can store up to [{{ config.max_mailbox_size }} messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server).
|
||||
- Messages are unconditionally removed {{ config.delete_mails_after }} days after arriving on the server.
|
||||
|
||||
- Messages are unconditionally removed latest {{ config.delete_mails_after }} days after arriving on the server.
|
||||
Earlier, if storage may exceed otherwise.
|
||||
- You can store up to [{{ config.max_mailbox_size }} messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server).
|
||||
|
||||
|
||||
### <a name="account-deletion"></a> Account deletion
|
||||
|
||||
Reference in New Issue
Block a user