mirror of
https://github.com/chatmail/relay.git
synced 2026-05-10 16:04:37 +00:00
Compare commits
58 Commits
link2xt/un
...
1.6.0
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a0a1dd65a6 | ||
|
|
046552061e | ||
|
|
1fba4a3cdf | ||
|
|
44ff6da5d2 | ||
|
|
71160b8f65 | ||
|
|
9f74d0a608 | ||
|
|
c9078d7c92 | ||
|
|
aa4259477f | ||
|
|
21f9885ffe | ||
|
|
f9e885c442 | ||
|
|
b45be700a8 | ||
|
|
9c381e1fbf | ||
|
|
3cc9bc3ceb | ||
|
|
2a89be8209 | ||
|
|
c848b61346 | ||
|
|
49787044ff | ||
|
|
04ae0b86fb | ||
|
|
b0434dc927 | ||
|
|
7578c5f1d3 | ||
|
|
5ba99dc782 | ||
|
|
6d898d5431 | ||
|
|
fc3fb93432 | ||
|
|
c4f0146e16 | ||
|
|
194030a456 | ||
|
|
ce240083c4 | ||
|
|
0722876603 | ||
|
|
724020ec2a | ||
|
|
b01348d313 | ||
|
|
46e31bbce3 | ||
|
|
a4f4627a75 | ||
|
|
8d34e036ec | ||
|
|
e004a5e2f6 | ||
|
|
acf6e862d0 | ||
|
|
31faf2c78e | ||
|
|
f8c28d8b9f | ||
|
|
f69a2355f6 | ||
|
|
388c01105c | ||
|
|
f8996e1d7d | ||
|
|
6b3d5025d9 | ||
|
|
ed271189d2 | ||
|
|
65f8a9a652 | ||
|
|
6c5b9fde1f | ||
|
|
258436442f | ||
|
|
05a32efa50 | ||
|
|
1142d06fdb | ||
|
|
35fe189be7 | ||
|
|
a78e8e10d2 | ||
|
|
9af37ccfbf | ||
|
|
803f3e6181 | ||
|
|
f188aef11e | ||
|
|
76d7e60018 | ||
|
|
fe749159e4 | ||
|
|
3c3532a292 | ||
|
|
710ca0070f | ||
|
|
4038fefefd | ||
|
|
cdcdc0b724 | ||
|
|
2313093b55 | ||
|
|
3f2ec54725 |
32
.github/ISSUE_TEMPLATE/bug_report.md
vendored
Normal file
32
.github/ISSUE_TEMPLATE/bug_report.md
vendored
Normal file
@@ -0,0 +1,32 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Report something that isn't working.
|
||||
title: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
Please fill out as much of this form as you can (leaving out stuff that is not applicable is ok).
|
||||
-->
|
||||
|
||||
- Server OS (Operating System) - preferably Debian 12:
|
||||
- On which OS you run cmdeploy:
|
||||
|
||||
## Expected behavior
|
||||
|
||||
*What did you try to achieve?*
|
||||
|
||||
## Actual behavior
|
||||
|
||||
*What happened instead?*
|
||||
|
||||
### Steps to reproduce the problem:
|
||||
|
||||
1.
|
||||
2.
|
||||
|
||||
### Screenshots
|
||||
|
||||
### Logs
|
||||
|
||||
5
.github/ISSUE_TEMPLATE/config.yml
vendored
Normal file
5
.github/ISSUE_TEMPLATE/config.yml
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
blank_issues_enabled: true
|
||||
contact_links:
|
||||
- name: Mutual Help Chat Group
|
||||
url: https://i.delta.chat/#C2846EB4C1CB8DF84B1818F5E3A638FC3FBDC981&a=stalebot1%40nine.testrun.org&g=Chatmail%20Mutual%20Help&x=7sFF7Ik50pWv6J1z7RVC5527&i=d7s1HvOsk5UrSf9AoqRZggg4&s=XmX_9BAW6-g5Ao5E8PyaeKNB
|
||||
about: If you have troubles setting up the relay server, feel free to ask here.
|
||||
230
CHANGELOG.md
230
CHANGELOG.md
@@ -2,69 +2,125 @@
|
||||
|
||||
## untagged
|
||||
|
||||
|
||||
## 1.6.0 2025-04-11
|
||||
|
||||
- Handle Port-25 connect errors more gracefully (common with VPNs)
|
||||
([#552](https://github.com/chatmail/relay/pull/552))
|
||||
|
||||
- Avoid "acmetool not found" during initial run
|
||||
([#550](https://github.com/chatmail/relay/pull/550))
|
||||
|
||||
- Fix timezone handling such that client/servers do not need to use
|
||||
same timezone.
|
||||
([#553](https://github.com/chatmail/relay/pull/553))
|
||||
|
||||
- Enforce end-to-end encryption for incoming messages.
|
||||
New user address mailboxes now get a `enforceE2EEincoming` file
|
||||
which prohibits incoming cleartext messages from other domains.
|
||||
An outside MTA trying to submit a cleartext message will
|
||||
get a "523 Encryption Needed" response, see RFC5248.
|
||||
If the file does not exist (as it the case for all existing accounts)
|
||||
incoming cleartext messages are accepted.
|
||||
([#538](https://github.com/chatmail/server/pull/538))
|
||||
|
||||
- Enforce end-to-end encryption between local addresses
|
||||
([#535](https://github.com/chatmail/server/pull/535))
|
||||
|
||||
- unbound: check that port 53 is not occupied by a different process
|
||||
([#537](https://github.com/chatmail/server/pull/537))
|
||||
|
||||
- unbound: before unbound is there, use 9.9.9.9 for resolving
|
||||
([#518](https://github.com/chatmail/relay/pull/518))
|
||||
|
||||
- Limit the bind for the HTTPS server on 8443 to 127.0.0.1
|
||||
([#522](https://github.com/chatmail/server/pull/522))
|
||||
([#532](https://github.com/chatmail/server/pull/532))
|
||||
|
||||
- Send SNI when connecting to outside servers
|
||||
([#524](https://github.com/chatmail/server/pull/524))
|
||||
|
||||
- postfix master.cf: use 127.0.0.1 for consistency
|
||||
([#544](https://github.com/chatmail/relay/pull/544))
|
||||
|
||||
- Pass through `original_content` instead of `content` in filtermail
|
||||
([#509](https://github.com/chatmail/server/pull/509))
|
||||
|
||||
- Document TLS requirements in the readme
|
||||
([#514](https://github.com/chatmail/server/pull/514))
|
||||
|
||||
- Remove cleanup service from submission ports
|
||||
([#512](https://github.com/chatmail/server/pull/512))
|
||||
|
||||
- cmdeploy dovecot: delete big messages after 7 days
|
||||
([#504](https://github.com/chatmail/server/pull/504))
|
||||
|
||||
- mtail: fix getting logs from STDIN
|
||||
([#502](https://github.com/chatmail/server/pull/502))
|
||||
|
||||
- filtermail: don't require exactly 2 lines after openPGP payload
|
||||
([#497](https://github.com/chatmail/chatmail/pull/497))
|
||||
([#497](https://github.com/chatmail/server/pull/497))
|
||||
|
||||
- cmdeploy dns: offer alternative DKIM record format for some web interfaces
|
||||
([#470](https://github.com/deltachat/chatmail/pull/470))
|
||||
([#470](https://github.com/chatmail/server/pull/470))
|
||||
|
||||
- journald: remove old logs from disk
|
||||
([#490](https://github.com/deltachat/chatmail/pull/490))
|
||||
([#490](https://github.com/chatmail/server/pull/490))
|
||||
|
||||
- opendkim: restart once every day to mend RAM leaks
|
||||
([#498](https://github.com/chatmail/chatmail/pull/498)
|
||||
([#498](https://github.com/chatmail/server/pull/498)
|
||||
|
||||
- migration guide: let opendkim own the DKIM keys directory
|
||||
([#468](https://github.com/deltachat/chatmail/pull/468))
|
||||
([#468](https://github.com/chatmail/server/pull/468))
|
||||
|
||||
- improve secure-join message detection
|
||||
([#473](https://github.com/deltachat/chatmail/pull/473))
|
||||
([#473](https://github.com/chatmail/server/pull/473))
|
||||
|
||||
- use old crypt lib in python < 3.11
|
||||
([#483](https://github.com/deltachat/chatmail/pull/483))
|
||||
([#483](https://github.com/chatmail/server/pull/483))
|
||||
|
||||
- chatmaild: set umask to 0700 for doveauth + metadata
|
||||
([#490](https://github.com/deltachat/chatmail/pull/492))
|
||||
([#490](https://github.com/chatmail/server/pull/492))
|
||||
|
||||
- remove MTA-STS daemon
|
||||
([#488](https://github.com/deltachat/chatmail/pull/488))
|
||||
([#488](https://github.com/chatmail/server/pull/488))
|
||||
|
||||
- replace `Subject` with `[...]` for all outgoing mails.
|
||||
([#481](https://github.com/deltachat/chatmail/pull/481))
|
||||
([#481](https://github.com/chatmail/server/pull/481))
|
||||
|
||||
- opendkim: use su instead of sudo
|
||||
([#491](https://github.com/deltachat/chatmail/pull/491))
|
||||
([#491](https://github.com/chatmail/server/pull/491))
|
||||
|
||||
## 1.5.0 2024-12-20
|
||||
|
||||
- cmdeploy dns: always show recommended DNS records
|
||||
([#463](https://github.com/deltachat/chatmail/pull/463))
|
||||
([#463](https://github.com/chatmail/server/pull/463))
|
||||
|
||||
- add `--all` to `cmdeploy dns`
|
||||
([#462](https://github.com/deltachat/chatmail/pull/462))
|
||||
([#462](https://github.com/chatmail/server/pull/462))
|
||||
|
||||
- fix `_mta-sts` TXT DNS record
|
||||
([#461](https://github.com/deltachat/chatmail/pull/461)
|
||||
([#461](https://github.com/chatmail/server/pull/461)
|
||||
|
||||
- deploy `iroh-relay` and also update "realtime relay services" in privacy policy.
|
||||
([#434](https://github.com/deltachat/chatmail/pull/434))
|
||||
([#451](https://github.com/deltachat/chatmail/pull/451))
|
||||
([#434](https://github.com/chatmail/server/pull/434))
|
||||
([#451](https://github.com/chatmail/server/pull/451))
|
||||
|
||||
- add guide to migrate chatmail to a new server
|
||||
([#429](https://github.com/deltachat/chatmail/pull/429))
|
||||
([#429](https://github.com/chatmail/server/pull/429))
|
||||
|
||||
- disable anvil authentication penalty
|
||||
([#414](https://github.com/deltachat/chatmail/pull/444)
|
||||
([#414](https://github.com/chatmail/server/pull/444)
|
||||
|
||||
- increase `request_queue_size` for UNIX sockets to 1000.
|
||||
([#437](https://github.com/deltachat/chatmail/pull/437))
|
||||
([#437](https://github.com/chatmail/server/pull/437))
|
||||
|
||||
- add argument to `cmdeploy run` for specifying
|
||||
a different SSH host than `mail_domain`
|
||||
([#439](https://github.com/deltachat/chatmail/pull/439))
|
||||
([#439](https://github.com/chatmail/server/pull/439))
|
||||
|
||||
- query autoritative nameserver to bypass DNS cache
|
||||
([#424](https://github.com/deltachat/chatmail/pull/424))
|
||||
([#424](https://github.com/chatmail/server/pull/424))
|
||||
|
||||
- add mtail support (new optional `mtail_address` ini value)
|
||||
This defines the address on which [`mtail`](https://google.github.io/mtail/)
|
||||
@@ -74,195 +130,195 @@
|
||||
and assign an IP address from this network to the host.
|
||||
If you do not plan to collect metrics,
|
||||
keep this setting unset.
|
||||
([#388](https://github.com/deltachat/chatmail/pull/388))
|
||||
([#388](https://github.com/chatmail/server/pull/388))
|
||||
|
||||
- fix checking for required DNS records
|
||||
([#412](https://github.com/deltachat/chatmail/pull/412))
|
||||
([#412](https://github.com/chatmail/server/pull/412))
|
||||
|
||||
- add support for specifying whole domains for recipient passthrough list
|
||||
([#408](https://github.com/deltachat/chatmail/pull/408))
|
||||
([#408](https://github.com/chatmail/server/pull/408))
|
||||
|
||||
- add a paragraph about "account deletion" to info page
|
||||
([#405](https://github.com/deltachat/chatmail/pull/405))
|
||||
([#405](https://github.com/chatmail/server/pull/405))
|
||||
|
||||
- avoid nginx listening on ipv6 if v6 is dsiabled
|
||||
([#402](https://github.com/deltachat/chatmail/pull/402))
|
||||
([#402](https://github.com/chatmail/server/pull/402))
|
||||
|
||||
- refactor ssh-based execution to allow organizing remote functions in
|
||||
modules.
|
||||
([#396](https://github.com/deltachat/chatmail/pull/396))
|
||||
([#396](https://github.com/chatmail/server/pull/396))
|
||||
|
||||
- trigger "apt upgrade" during "cmdeploy run"
|
||||
([#398](https://github.com/deltachat/chatmail/pull/398))
|
||||
([#398](https://github.com/chatmail/server/pull/398))
|
||||
|
||||
- drop hispanilandia passthrough address
|
||||
([#401](https://github.com/deltachat/chatmail/pull/401))
|
||||
([#401](https://github.com/chatmail/server/pull/401))
|
||||
|
||||
- set CAA record flags to 0
|
||||
|
||||
- add IMAP capabilities instead of overwriting them
|
||||
([#413](https://github.com/deltachat/chatmail/pull/413))
|
||||
([#413](https://github.com/chatmail/server/pull/413))
|
||||
|
||||
- fix OpenPGP payload check
|
||||
([#435](https://github.com/deltachat/chatmail/pull/435))
|
||||
([#435](https://github.com/chatmail/server/pull/435))
|
||||
|
||||
- fix Dovecot quota_max_mail_size to use max_message_size config value
|
||||
([#438](https://github.com/deltachat/chatmail/pull/438))
|
||||
([#438](https://github.com/chatmail/server/pull/438))
|
||||
|
||||
|
||||
## 1.4.1 2024-07-31
|
||||
|
||||
- fix metadata dictproxy which would confuse transactions
|
||||
resulting in missed notifications and other issues.
|
||||
([#393](https://github.com/deltachat/chatmail/pull/393))
|
||||
([#394](https://github.com/deltachat/chatmail/pull/394))
|
||||
([#393](https://github.com/chatmail/server/pull/393))
|
||||
([#394](https://github.com/chatmail/server/pull/394))
|
||||
|
||||
- add optional "imap_rawlog" config option. If true,
|
||||
.in/.out files are created in user home dirs
|
||||
containing the imap protocol messages.
|
||||
([#389](https://github.com/deltachat/chatmail/pull/389))
|
||||
([#389](https://github.com/chatmail/server/pull/389))
|
||||
|
||||
## 1.4.0 2024-07-28
|
||||
|
||||
- Add `disable_ipv6` config option to chatmail.ini.
|
||||
Required if the server doesn't have IPv6 connectivity.
|
||||
([#312](https://github.com/deltachat/chatmail/pull/312))
|
||||
([#312](https://github.com/chatmail/server/pull/312))
|
||||
|
||||
- allow current K9/Thunderbird-mail releases to send encrypted messages
|
||||
outside by accepting their localized "encrypted subject" strings.
|
||||
([#370](https://github.com/deltachat/chatmail/pull/370))
|
||||
([#370](https://github.com/chatmail/server/pull/370))
|
||||
|
||||
- Migrate and remove sqlite database in favor of password/lastlogin tracking
|
||||
in a user's maildir.
|
||||
([#379](https://github.com/deltachat/chatmail/pull/379))
|
||||
([#379](https://github.com/chatmail/server/pull/379))
|
||||
|
||||
- Require pyinfra V3 installed on the client side,
|
||||
run `./scripts/initenv.sh` to upgrade locally.
|
||||
([#378](https://github.com/deltachat/chatmail/pull/378))
|
||||
([#378](https://github.com/chatmail/server/pull/378))
|
||||
|
||||
- don't hardcode "/home/vmail" paths but rather set them
|
||||
once in the config object and use it everywhere else,
|
||||
thereby also improving testability.
|
||||
([#351](https://github.com/deltachat/chatmail/pull/351))
|
||||
([#351](https://github.com/chatmail/server/pull/351))
|
||||
temporarily introduced obligatory "passdb_path" and "mailboxes_dir"
|
||||
settings but they were removed/obsoleted in
|
||||
([#380](https://github.com/deltachat/chatmail/pull/380))
|
||||
([#380](https://github.com/chatmail/server/pull/380))
|
||||
|
||||
- BREAKING: new required chatmail.ini value 'delete_inactive_users_after = 100'
|
||||
which removes users from database and mails after 100 days without any login.
|
||||
([#350](https://github.com/deltachat/chatmail/pull/350))
|
||||
([#350](https://github.com/chatmail/server/pull/350))
|
||||
|
||||
- Refine DNS checking to distinguish between "required" and "recommended" settings
|
||||
([#372](https://github.com/deltachat/chatmail/pull/372))
|
||||
([#372](https://github.com/chatmail/server/pull/372))
|
||||
|
||||
- reload nginx in the acmetool cronjob
|
||||
([#360](https://github.com/deltachat/chatmail/pull/360))
|
||||
([#360](https://github.com/chatmail/server/pull/360))
|
||||
|
||||
- remove checking of reverse-DNS PTR records. Chatmail-servers don't
|
||||
depend on it and even in the wider e-mail system it's not common anymore.
|
||||
If it's an issue, a chatmail operator can still care to properly set reverse DNS.
|
||||
([#348](https://github.com/deltachat/chatmail/pull/348))
|
||||
([#348](https://github.com/chatmail/server/pull/348))
|
||||
|
||||
- Make DNS-checking faster and more interactive, run it fully during "cmdeploy run",
|
||||
also introducing a generic mechanism for rapid remote ssh-based python function execution.
|
||||
([#346](https://github.com/deltachat/chatmail/pull/346))
|
||||
([#346](https://github.com/chatmail/server/pull/346))
|
||||
|
||||
- Don't fix file owner ship of /home/vmail
|
||||
([#345](https://github.com/deltachat/chatmail/pull/345))
|
||||
([#345](https://github.com/chatmail/server/pull/345))
|
||||
|
||||
- Support iterating over all users with doveadm commands
|
||||
([#344](https://github.com/deltachat/chatmail/pull/344))
|
||||
([#344](https://github.com/chatmail/server/pull/344))
|
||||
|
||||
- Test and fix for attempts to create inadmissible accounts
|
||||
([#333](https://github.com/deltachat/chatmail/pull/321))
|
||||
([#333](https://github.com/chatmail/server/pull/321))
|
||||
|
||||
- check that OpenPGP has only PKESK, SKESK and SEIPD packets
|
||||
([#323](https://github.com/deltachat/chatmail/pull/323),
|
||||
[#324](https://github.com/deltachat/chatmail/pull/324))
|
||||
([#323](https://github.com/chatmail/server/pull/323),
|
||||
[#324](https://github.com/chatmail/server/pull/324))
|
||||
|
||||
- improve filtermail checks for encrypted messages and drop support for unencrypted MDNs
|
||||
([#320](https://github.com/deltachat/chatmail/pull/320))
|
||||
([#320](https://github.com/chatmail/server/pull/320))
|
||||
|
||||
- replace `bash` with `/bin/sh`
|
||||
([#334](https://github.com/deltachat/chatmail/pull/334))
|
||||
([#334](https://github.com/chatmail/server/pull/334))
|
||||
|
||||
- Increase number of logged in IMAP sessions to 50000
|
||||
([#335](https://github.com/deltachat/chatmail/pull/335))
|
||||
([#335](https://github.com/chatmail/server/pull/335))
|
||||
|
||||
- filtermail: do not allow ASCII armor without actual payload
|
||||
([#325](https://github.com/deltachat/chatmail/pull/325))
|
||||
([#325](https://github.com/chatmail/server/pull/325))
|
||||
|
||||
- Remove sieve to enable hardlink deduplication in LMTP
|
||||
([#343](https://github.com/deltachat/chatmail/pull/343))
|
||||
([#343](https://github.com/chatmail/server/pull/343))
|
||||
|
||||
- dovecot: enable gzip compression on disk
|
||||
([#341](https://github.com/deltachat/chatmail/pull/341))
|
||||
([#341](https://github.com/chatmail/server/pull/341))
|
||||
|
||||
- DKIM-sign Content-Type and oversign all signed headers
|
||||
([#296](https://github.com/deltachat/chatmail/pull/296))
|
||||
([#296](https://github.com/chatmail/server/pull/296))
|
||||
|
||||
- Add nonci_accounts metric
|
||||
([#347](https://github.com/deltachat/chatmail/pull/347))
|
||||
([#347](https://github.com/chatmail/server/pull/347))
|
||||
|
||||
- doveauth: log when a new account is created
|
||||
([#349](https://github.com/deltachat/chatmail/pull/349))
|
||||
([#349](https://github.com/chatmail/server/pull/349))
|
||||
|
||||
- Multiplex HTTPS, IMAP and SMTP on port 443
|
||||
([#357](https://github.com/deltachat/chatmail/pull/357))
|
||||
([#357](https://github.com/chatmail/server/pull/357))
|
||||
|
||||
## 1.3.0 - 2024-06-06
|
||||
|
||||
- don't check necessary DNS records on cmdeploy init anymore
|
||||
([#316](https://github.com/deltachat/chatmail/pull/316))
|
||||
([#316](https://github.com/chatmail/server/pull/316))
|
||||
|
||||
- ensure cron and acl are installed
|
||||
([#293](https://github.com/deltachat/chatmail/pull/293),
|
||||
[#310](https://github.com/deltachat/chatmail/pull/310))
|
||||
([#293](https://github.com/chatmail/server/pull/293),
|
||||
[#310](https://github.com/chatmail/server/pull/310))
|
||||
|
||||
- change default for delete_mails_after from 40 to 20 days
|
||||
([#300](https://github.com/deltachat/chatmail/pull/300))
|
||||
([#300](https://github.com/chatmail/server/pull/300))
|
||||
|
||||
- save journald logs only to memory and save nginx logs to journald instead of file
|
||||
([#299](https://github.com/deltachat/chatmail/pull/299))
|
||||
([#299](https://github.com/chatmail/server/pull/299))
|
||||
|
||||
- fix writing of multiple obs repositories in `/etc/apt/sources.list`
|
||||
([#290](https://github.com/deltachat/chatmail/pull/290))
|
||||
([#290](https://github.com/chatmail/server/pull/290))
|
||||
|
||||
- metadata: add support for `/shared/vendor/deltachat/irohrelay`
|
||||
([#284](https://github.com/deltachat/chatmail/pull/284))
|
||||
([#284](https://github.com/chatmail/server/pull/284))
|
||||
|
||||
- Emit "XCHATMAIL" capability from IMAP server
|
||||
([#278](https://github.com/deltachat/chatmail/pull/278))
|
||||
([#278](https://github.com/chatmail/server/pull/278))
|
||||
|
||||
- Move echobot `into /var/lib/echobot`
|
||||
([#281](https://github.com/deltachat/chatmail/pull/281))
|
||||
([#281](https://github.com/chatmail/server/pull/281))
|
||||
|
||||
- Accept Let's Encrypt's new Terms of Services
|
||||
([#275](https://github.com/deltachat/chatmail/pull/276))
|
||||
([#275](https://github.com/chatmail/server/pull/276))
|
||||
|
||||
- Reload Dovecot and Postfix when TLS certificate updates
|
||||
([#271](https://github.com/deltachat/chatmail/pull/271))
|
||||
([#271](https://github.com/chatmail/server/pull/271))
|
||||
|
||||
- Use forked version of dovecot without hardcoded delays
|
||||
([#270](https://github.com/deltachat/chatmail/pull/270))
|
||||
([#270](https://github.com/chatmail/server/pull/270))
|
||||
|
||||
## 1.2.0 - 2024-04-04
|
||||
|
||||
- Install dig on the server to resolve DNS records
|
||||
([#267](https://github.com/deltachat/chatmail/pull/267))
|
||||
([#267](https://github.com/chatmail/server/pull/267))
|
||||
|
||||
- preserve notification order and exponentially backoff with
|
||||
retries for tokens where we didn't get a successful return
|
||||
([#265](https://github.com/deltachat/chatmail/pull/263))
|
||||
([#265](https://github.com/chatmail/server/pull/263))
|
||||
|
||||
- Run chatmail-metadata and doveauth as vmail
|
||||
([#261](https://github.com/deltachat/chatmail/pull/261))
|
||||
([#261](https://github.com/chatmail/server/pull/261))
|
||||
|
||||
- Apply systemd restrictions to echobot
|
||||
([#259](https://github.com/deltachat/chatmail/pull/259))
|
||||
([#259](https://github.com/chatmail/server/pull/259))
|
||||
|
||||
- re-enable running the CI in pull requests, but not concurrently
|
||||
([#258](https://github.com/deltachat/chatmail/pull/258))
|
||||
([#258](https://github.com/chatmail/server/pull/258))
|
||||
|
||||
|
||||
## 1.1.0 - 2024-03-28
|
||||
@@ -270,27 +326,27 @@
|
||||
### The changelog starts to record changes from March 15th, 2024
|
||||
|
||||
- Move systemd unit templates to cmdeploy package
|
||||
([#255](https://github.com/deltachat/chatmail/pull/255))
|
||||
([#255](https://github.com/chatmail/server/pull/255))
|
||||
|
||||
- Persist push tokens and support multiple device per address
|
||||
([#254](https://github.com/deltachat/chatmail/pull/254))
|
||||
([#254](https://github.com/chatmail/server/pull/254))
|
||||
|
||||
- Avoid warning for regular doveauth protocol's hello message.
|
||||
([#250](https://github.com/deltachat/chatmail/pull/250))
|
||||
([#250](https://github.com/chatmail/server/pull/250))
|
||||
|
||||
- Fix various tests to pass again with "cmdeploy test".
|
||||
([#245](https://github.com/deltachat/chatmail/pull/245),
|
||||
[#242](https://github.com/deltachat/chatmail/pull/242)
|
||||
([#245](https://github.com/chatmail/server/pull/245),
|
||||
[#242](https://github.com/chatmail/server/pull/242)
|
||||
|
||||
- Ensure lets-encrypt certificates are reloaded after renewal
|
||||
([#244]) https://github.com/deltachat/chatmail/pull/244
|
||||
([#244]) https://github.com/chatmail/server/pull/244
|
||||
|
||||
- Persist tokens to avoid iOS users loosing push-notifications when the
|
||||
chatmail metadata service is restarted (happens regularly during deploys)
|
||||
([#238](https://github.com/deltachat/chatmail/pull/239)
|
||||
([#238](https://github.com/chatmail/server/pull/239)
|
||||
|
||||
- Fix failing sieve-script compile errors on incoming messages
|
||||
([#237](https://github.com/deltachat/chatmail/pull/239)
|
||||
([#237](https://github.com/chatmail/server/pull/239)
|
||||
|
||||
- Fix quota reporting after expunging of old mails
|
||||
([#233](https://github.com/deltachat/chatmail/pull/239)
|
||||
([#233](https://github.com/chatmail/server/pull/239)
|
||||
|
||||
357
README.md
357
README.md
@@ -1,48 +1,96 @@
|
||||
|
||||
<img width="800px" src="www/src/collage-top.png"/>
|
||||
|
||||
# Chatmail services optimized for Delta Chat apps
|
||||
# Chatmail relays for end-to-end encrypted e-mail
|
||||
|
||||
This repository helps to setup a ready-to-use chatmail server
|
||||
Chatmail relay servers are interoperable Mail Transport Agents (MTAs) designed for:
|
||||
|
||||
- **Convenience:** Low friction instant onboarding
|
||||
|
||||
- **Privacy:** No name, phone numbers, email required or collected
|
||||
|
||||
- **End-to-End Encryption enforced**: only OpenPGP messages with metadata minimization allowed
|
||||
|
||||
- **Instant:** Privacy-preserving Push Notifications for Apple, Google, and Huawei
|
||||
|
||||
- **Speed:** Message delivery in half a second, with optional P2P realtime connections
|
||||
|
||||
- **Transport Security:** Strict TLS and DKIM enforced
|
||||
|
||||
- **Reliability:** No spam or IP reputation checks; rate-limits are suitable for realtime chats
|
||||
|
||||
- **Efficiency:** Messages are only stored for transit and removed automatically
|
||||
|
||||
This repository contains everything needed to setup a ready-to-use chatmail relay
|
||||
comprised of a minimal setup of the battle-tested
|
||||
[postfix smtp](https://www.postfix.org) and [dovecot imap](https://www.dovecot.org) services.
|
||||
[Postfix SMTP](https://www.postfix.org) and [Dovecot IMAP](https://www.dovecot.org) MTAs/MDAs.
|
||||
|
||||
The setup is designed and optimized for providing chatmail accounts
|
||||
for use by [Delta Chat apps](https://delta.chat).
|
||||
The automated setup is designed and optimized for providing chatmail addresses
|
||||
for immediate permission-free onboarding through chat apps and bots.
|
||||
Chatmail addresses are automatically created at first login,
|
||||
after which the initially specified password is required
|
||||
for sending and receiving messages through them.
|
||||
|
||||
Chatmail accounts are automatically created by a first login,
|
||||
after which the initially specified password is required for using them.
|
||||
Please see [this list of known apps and client projects](https://chatmail.at/clients.html)
|
||||
and [this list of known public 3rd party chatmail relay servers](https://chatmail.at/relays).
|
||||
|
||||
## Deploying your own chatmail server
|
||||
|
||||
To deploy chatmail on your own server, you must have set-up ssh authentication and need to use an ed25519 key, due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191). You also need to add your private key to the local ssh-agent, because you can't type in your password during deployment.
|
||||
## Minimal requirements, Prerequisites
|
||||
|
||||
We use `chat.example.org` as the chatmail domain in the following steps.
|
||||
You will need the following:
|
||||
|
||||
- Control over a domain through a DNS provider of your choice.
|
||||
|
||||
- A Debian 12 server with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
|
||||
IPv6 is encouraged if available.
|
||||
Chatmail relay servers only require 1GB RAM, one CPU, and perhaps 10GB storage for a
|
||||
few thousand active chatmail addresses.
|
||||
|
||||
- Key-based SSH authentication to the root user.
|
||||
You must add a passphrase-protected private key to your local ssh-agent
|
||||
because you can't type in your passphrase during deployment.
|
||||
(An ed25519 private key is required due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191))
|
||||
|
||||
|
||||
## Getting started
|
||||
|
||||
We use `chat.example.org` as the chatmail domain in the following steps.
|
||||
Please substitute it with your own domain.
|
||||
|
||||
1. Install the `cmdeploy` command in a virtualenv
|
||||
1. Setup the initial DNS records.
|
||||
The following is an example in the familiar BIND zone file format with
|
||||
a TTL of 1 hour (3600 seconds).
|
||||
Please substitute your domain and IP addresses.
|
||||
|
||||
```
|
||||
git clone https://github.com/deltachat/chatmail
|
||||
cd chatmail
|
||||
chat.example.com. 3600 IN A 198.51.100.5
|
||||
chat.example.com. 3600 IN AAAA 2001:db8::5
|
||||
www.chat.example.com. 3600 IN CNAME chat.example.com.
|
||||
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
|
||||
```
|
||||
|
||||
2. Clone the repository and bootstrap the Python virtualenv.
|
||||
|
||||
```
|
||||
git clone https://github.com/chatmail/relay
|
||||
cd relay
|
||||
scripts/initenv.sh
|
||||
```
|
||||
|
||||
2. Create chatmail configuration file `chatmail.ini`:
|
||||
3. Create chatmail configuration file `chatmail.ini`:
|
||||
|
||||
```
|
||||
scripts/cmdeploy init chat.example.org # <-- use your domain
|
||||
```
|
||||
|
||||
3. Point your domain to the server's IP address,
|
||||
if you haven't done so already.
|
||||
Verify that SSH root login works:
|
||||
4. Verify that SSH root login works:
|
||||
|
||||
```
|
||||
ssh root@chat.example.org # <-- use your domain
|
||||
```
|
||||
|
||||
4. Deploy to the remote chatmail server:
|
||||
|
||||
5. Deploy the remote chatmail relay server:
|
||||
|
||||
```
|
||||
scripts/cmdeploy run
|
||||
@@ -82,25 +130,25 @@ scripts/cmdeploy bench
|
||||
|
||||
This repository has four directories:
|
||||
|
||||
- [cmdeploy](https://github.com/deltachat/chatmail/tree/main/cmdeploy)
|
||||
- [cmdeploy](https://github.com/chatmail/relay/tree/main/cmdeploy)
|
||||
is a collection of configuration files
|
||||
and a [pyinfra](https://pyinfra.com)-based deployment script.
|
||||
|
||||
- [chatmaild](https://github.com/deltachat/chatmail/tree/main/chatmaild)
|
||||
is a python package containing several small services
|
||||
- [chatmaild](https://github.com/chatmail/relay/tree/main/chatmaild)
|
||||
is a Python package containing several small services
|
||||
which handle authentication,
|
||||
trigger push notifications on new messages,
|
||||
ensure that outbound mails are encrypted,
|
||||
delete inactive users,
|
||||
and some other minor things.
|
||||
chatmaild can also be installed as a stand-alone python package.
|
||||
chatmaild can also be installed as a stand-alone Python package.
|
||||
|
||||
- [www](https://github.com/deltachat/chatmail/tree/main/www)
|
||||
- [www](https://github.com/chatmail/relay/tree/main/www)
|
||||
contains the html, css, and markdown files
|
||||
which make up a chatmail server's web page.
|
||||
Edit them before deploying to make your chatmail server stand out.
|
||||
which make up a chatmail relay's web page.
|
||||
Edit them before deploying to make your chatmail relay stand out.
|
||||
|
||||
- [scripts](https://github.com/deltachat/chatmail/tree/main/scripts)
|
||||
- [scripts](https://github.com/chatmail/relay/tree/main/scripts)
|
||||
offers two convenience tools for beginners;
|
||||
`initenv.sh` installs the necessary dependencies to a local virtual environment,
|
||||
and the `scripts/cmdeploy` script enables you
|
||||
@@ -112,79 +160,81 @@ The `cmdeploy/src/cmdeploy/cmdeploy.py` command line tool
|
||||
helps with setting up and managing the chatmail service.
|
||||
`cmdeploy init` creates the `chatmail.ini` config file.
|
||||
`cmdeploy run` uses a [pyinfra](https://pyinfra.com/)-based [script](`cmdeploy/src/cmdeploy/__init__.py`)
|
||||
to automatically install or upgrade all chatmail components on a server,
|
||||
to automatically install or upgrade all chatmail components on a relay,
|
||||
according to the `chatmail.ini` config.
|
||||
|
||||
The components of chatmail are:
|
||||
|
||||
- [postfix smtp server](https://www.postfix.org) accepts sent messages (both from your users and from other servers)
|
||||
- [Postfix SMTP MTA](https://www.postfix.org) accepts and relays messages
|
||||
(both from your users and from the wider e-mail MTA network)
|
||||
|
||||
- [dovecot imap server](https://www.dovecot.org) stores messages for your users until they download them
|
||||
- [Dovecot IMAP MDA](https://www.dovecot.org) stores messages for your users until they download them
|
||||
|
||||
- [nginx](https://nginx.org/) shows the web page with your privacy policy and additional information
|
||||
- [Nginx](https://nginx.org/) shows the web page with your privacy policy and additional information
|
||||
|
||||
- [acmetool](https://hlandau.github.io/acmetool/) manages TLS certificates for dovecot, postfix, and nginx
|
||||
- [acmetool](https://hlandau.github.io/acmetool/) manages TLS certificates for Dovecot, Postfix, and Nginx
|
||||
|
||||
- [opendkim](http://www.opendkim.org/) for signing messages with DKIM and rejecting inbound messages without DKIM
|
||||
- [OpenDKIM](http://www.opendkim.org/) for signing messages with DKIM and rejecting inbound messages without DKIM
|
||||
|
||||
- [mtail](https://google.github.io/mtail/) for collecting anonymized metrics in case you have monitoring
|
||||
|
||||
- [Iroh relay](https://www.iroh.computer/docs/concepts/relay)
|
||||
which helps client devices to establish Peer-to-Peer connections
|
||||
|
||||
- and the chatmaild services, explained in the next section:
|
||||
|
||||
### chatmaild
|
||||
|
||||
chatmaild offers several commands
|
||||
which differentiate a *chatmail* server from a classic mail server.
|
||||
If you deploy them with cmdeploy,
|
||||
they are run by systemd services in the background.
|
||||
A short overview:
|
||||
`chatmaild` implements various systemd-controlled services
|
||||
that integrate with Dovecot and Postfix to achieve instant-onboarding and
|
||||
only relaying OpenPGP end-to-end messages encrypted messages.
|
||||
A short overview of `chatmaild` services:
|
||||
|
||||
- [`doveauth`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/doveauth.py) implements
|
||||
create-on-login account creation semantics and is used
|
||||
by Dovecot during login authentication and by Postfix
|
||||
- [`doveauth`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/doveauth.py)
|
||||
implements create-on-login address semantics and is used
|
||||
by Dovecot during IMAP login and by Postfix during SMTP/SUBMISSION login
|
||||
which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)
|
||||
to authenticate users
|
||||
to send mails for them.
|
||||
to authenticate logins.
|
||||
|
||||
- [`filtermail`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/filtermail.py) prevents
|
||||
unencrypted e-mail from leaving the chatmail service
|
||||
and is integrated into postfix's outbound mail pipelines.
|
||||
- [`filtermail`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/filtermail.py)
|
||||
prevents unencrypted email from leaving or entering the chatmail service
|
||||
and is integrated into Postfix's outbound and inbound mail pipelines.
|
||||
|
||||
- [`chatmail-metadata`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/metadata.py) is contacted by a
|
||||
[dovecot lua script](https://github.com/deltachat/chatmail/blob/main/cmdeploy/src/cmdeploy/dovecot/push_notification.lua)
|
||||
to store user-specific server-side config.
|
||||
- [`chatmail-metadata`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/metadata.py) is contacted by a
|
||||
[Dovecot lua script](https://github.com/chatmail/relay/blob/main/cmdeploy/src/cmdeploy/dovecot/push_notification.lua)
|
||||
to store user-specific relay-side config.
|
||||
On new messages,
|
||||
it [passes the user's push notification token](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/notifier.py)
|
||||
it [passes the user's push notification token](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/notifier.py)
|
||||
to [notifications.delta.chat](https://delta.chat/help#instant-delivery)
|
||||
so the push notifications on the user's phone can be triggered
|
||||
by Apple/Google.
|
||||
by Apple/Google/Huawei.
|
||||
|
||||
- [`delete_inactive_users`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/delete_inactive_users.py)
|
||||
- [`delete_inactive_users`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/delete_inactive_users.py)
|
||||
deletes users if they have not logged in for a very long time.
|
||||
The timeframe can be configured in `chatmail.ini`.
|
||||
|
||||
- [`lastlogin`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/lastlogin.py)
|
||||
is contacted by dovecot when a user logs in
|
||||
- [`lastlogin`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/lastlogin.py)
|
||||
is contacted by Dovecot when a user logs in
|
||||
and stores the date of the login.
|
||||
|
||||
- [`echobot`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/echo.py)
|
||||
- [`echobot`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/echo.py)
|
||||
is a small bot for test purposes.
|
||||
It simply echoes back messages from users.
|
||||
|
||||
- [`chatmail-metrics`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/metrics.py)
|
||||
- [`chatmail-metrics`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/metrics.py)
|
||||
collects some metrics and displays them at `https://example.org/metrics`.
|
||||
|
||||
### Home page and getting started for users
|
||||
|
||||
`cmdeploy run` also creates default static Web pages and deploys them
|
||||
to a nginx web server with:
|
||||
`cmdeploy run` also creates default static web pages and deploys them
|
||||
to a Nginx web server with:
|
||||
|
||||
- a default `index.html` along with a QR code that users can click to
|
||||
create accounts on your chatmail provider,
|
||||
create an address on your chatmail relay
|
||||
|
||||
- a default `info.html` that is linked from the home page,
|
||||
- a default `info.html` that is linked from the home page
|
||||
|
||||
- a default `policy.html` that is linked from the home page.
|
||||
- a default `policy.html` that is linked from the home page
|
||||
|
||||
All `.html` files are generated
|
||||
by the according markdown `.md` file in the `www/src` directory.
|
||||
@@ -192,48 +242,64 @@ by the according markdown `.md` file in the `www/src` directory.
|
||||
|
||||
### Refining the web pages
|
||||
|
||||
|
||||
```
|
||||
scripts/cmdeploy webdev
|
||||
```
|
||||
|
||||
This starts a local live development cycle for chatmail Web pages:
|
||||
This starts a local live development cycle for chatmail web pages:
|
||||
|
||||
- uses the `www/src/page-layout.html` file for producing static
|
||||
HTML pages from `www/src/*.md` files
|
||||
|
||||
- continously builds the web presence reading files from `www/src` directory
|
||||
and generating html files and copying assets to the `www/build` directory.
|
||||
and generating HTML files and copying assets to the `www/build` directory.
|
||||
|
||||
- Starts a browser window automatically where you can "refresh" as needed.
|
||||
|
||||
## Mailbox directory layout
|
||||
|
||||
## Emergency Commands to disable automatic account creation
|
||||
Fresh chatmail addresses have a mailbox directory that contains:
|
||||
|
||||
If you need to stop account creation,
|
||||
e.g. because some script is wildly creating accounts,
|
||||
login to the server with ssh and run:
|
||||
- a `password` file with the salted password required for authenticating
|
||||
whether a login may use the address to send/receive messages.
|
||||
If you modify the password file manually, you effectively block the user.
|
||||
|
||||
- `enforceE2EEincoming` is a default-created file with each address.
|
||||
If present the file indicates that this chatmail address rejects incoming cleartext messages.
|
||||
If absent the address accepts incoming cleartext messages.
|
||||
|
||||
- `dovecot*`, `cur`, `new` and `tmp` represent IMAP/mailbox state.
|
||||
If the address is only used by one device, the Maildir directories
|
||||
will typically be empty unless the user of that address hasn't been online
|
||||
for a while.
|
||||
|
||||
|
||||
## Emergency Commands to disable automatic address creation
|
||||
|
||||
If you need to stop address creation,
|
||||
e.g. because some script is wildly creating addresses,
|
||||
login with ssh and run:
|
||||
|
||||
```
|
||||
touch /etc/chatmail-nocreate
|
||||
```
|
||||
|
||||
While this file is present, account creation will be blocked.
|
||||
Chatmail address creation will be denied while this file is present.
|
||||
|
||||
### Ports
|
||||
|
||||
[Postfix](http://www.postfix.org/) listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
|
||||
[Dovecot](https://www.dovecot.org/) listens on ports 143 (imap) and 993 (imaps).
|
||||
[nginx](https://www.nginx.com/) listens on port 8443 (https-alt) and 443 (https).
|
||||
[Postfix](http://www.postfix.org/) listens on ports 25 (SMTP) and 587 (SUBMISSION) and 465 (SUBMISSIONS).
|
||||
[Dovecot](https://www.dovecot.org/) listens on ports 143 (IMAP) and 993 (IMAPS).
|
||||
[Nginx](https://www.nginx.com/) listens on port 8443 (HTTPS-ALT) and 443 (HTTPS).
|
||||
Port 443 multiplexes HTTPS, IMAP and SMTP using ALPN to redirect connections to ports 8443, 465 or 993.
|
||||
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (http).
|
||||
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (HTTP).
|
||||
|
||||
Delta Chat apps will, however, discover all ports and configurations
|
||||
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail service.
|
||||
chatmail-core based apps will, however, discover all ports and configurations
|
||||
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail relay server.
|
||||
|
||||
## Email authentication
|
||||
|
||||
chatmail servers rely on [DKIM](https://www.rfc-editor.org/rfc/rfc6376)
|
||||
Chatmail relays enforce [DKIM](https://www.rfc-editor.org/rfc/rfc6376)
|
||||
to authenticate incoming emails.
|
||||
Incoming emails must have a valid DKIM signature with
|
||||
Signing Domain Identifier (SDID, `d=` parameter in the DKIM-Signature header)
|
||||
@@ -256,101 +322,130 @@ and rejects incorrectly authenticated emails with [`reject_sender_login_mismatch
|
||||
`From:` header must correspond to envelope MAIL FROM,
|
||||
this is ensured by `filtermail` proxy.
|
||||
|
||||
## Migrating chatmail server to a new host
|
||||
## TLS requirements
|
||||
|
||||
If you want to migrate chatmail from an old machine
|
||||
Postfix is configured to require valid TLS
|
||||
by setting [`smtp_tls_security_level`](https://www.postfix.org/postconf.5.html#smtp_tls_security_level) to `verify`.
|
||||
If emails don't arrive at your chatmail relay server,
|
||||
the problem is likely that your relay does not have a valid TLS certificate.
|
||||
|
||||
You can test it by resolving `MX` records of your relay domain
|
||||
and then connecting to MX relays (e.g `mx.example.org`) with
|
||||
`openssl s_client -connect mx.example.org:25 -verify_hostname mx.example.org -verify_return_error -starttls smtp`
|
||||
from the host that has open port 25 to verify that certificate is valid.
|
||||
|
||||
When providing a TLS certificate to your chatmail relay server,
|
||||
make sure to provide the full certificate chain
|
||||
and not just the last certificate.
|
||||
|
||||
If you are running an Exim server and don't see incoming connections
|
||||
from a chatmail relay server in the logs,
|
||||
make sure `smtp_no_mail` log item is enabled in the config
|
||||
with `log_selector = +smtp_no_mail`.
|
||||
By default Exim does not log sessions that are closed
|
||||
before sending the `MAIL` command.
|
||||
This happens if certificate is not recognized as valid by Postfix,
|
||||
so you might think that connection is not established
|
||||
while actually it is a problem with your TLS certificate.
|
||||
|
||||
## Migrating a chatmail relay to a new host
|
||||
|
||||
If you want to migrate chatmail relay from an old machine
|
||||
to a new machine,
|
||||
you can use these steps.
|
||||
They were tested with a linux laptop;
|
||||
They were tested with a Linux laptop;
|
||||
you might need to adjust some of the steps to your environment.
|
||||
|
||||
Let's assume that your `mail_domain` is `mail.example.org`,
|
||||
all involved machines run Debian 12,
|
||||
your old server's IP address is `13.37.13.37`,
|
||||
and your new server's IP address is `13.12.23.42`.
|
||||
your old site's IP address is `13.37.13.37`,
|
||||
and your new site's IP address is `13.12.23.42`.
|
||||
|
||||
During the guide, you might get a warning about changed SSH Host keys;
|
||||
in this case, just run `ssh-keygen -R "mail.example.org"` as recommended
|
||||
to make sure you can connect with SSH.
|
||||
Note, you should lower the TTLs of your DNS records to a value
|
||||
such as 300 (5 minutes) so the migration happens as smoothly as possible.
|
||||
|
||||
1. First, copy `/var/lib/acme` to the new server with
|
||||
`ssh root@13.37.13.37 tar c /var/lib/acme | ssh root@13.12.23.42 tar x -C /var/lib/`.
|
||||
This transfers your TLS certificate.
|
||||
During the guide you might get a warning about changed SSH Host keys;
|
||||
in this case, just run `ssh-keygen -R "mail.example.org"` as recommended.
|
||||
|
||||
2. You should also copy `/etc/dkimkeys` to the new server with
|
||||
`ssh root@13.37.13.37 tar c /etc/dkimkeys | ssh root@13.12.23.42 tar x -C /etc/`
|
||||
so the DKIM DNS record stays correct.
|
||||
1. First, disable mail services on the old site.
|
||||
|
||||
3. On the new server, run `chown root: -R /var/lib/acme` and `chown opendkim: -R /etc/dkimkeys` to make sure the permissions are correct.
|
||||
|
||||
4. Run `cmdeploy run --disable-mail --ssh-host 13.12.23.42` to install chatmail on the new machine.
|
||||
postfix and dovecot are disabled for now,
|
||||
we will enable them later.
|
||||
|
||||
5. Now, point DNS to the new IP addresses.
|
||||
|
||||
You can already remove the old IP addresses from DNS.
|
||||
Existing Delta Chat users will still be able to connect
|
||||
to the old server, send and receive messages,
|
||||
but new users will fail to create new profiles
|
||||
with your chatmail server.
|
||||
|
||||
If other servers try to deliver messages to your new server they will fail,
|
||||
but normally email servers will retry delivering messages
|
||||
for at least a week, so messages will not be lost.
|
||||
|
||||
6. Now you can run `cmdeploy run --disable-mail --ssh-host 13.37.13.37` to disable your old server.
|
||||
```
|
||||
cmdeploy run --disable-mail --ssh-host 13.37.13.37
|
||||
```
|
||||
|
||||
Now your users will notice the migration
|
||||
and will not be able to send or receive messages
|
||||
until the migration is completed.
|
||||
|
||||
7. After everything is stopped,
|
||||
you can copy the `/home/vmail/mail` directory to the new server.
|
||||
It includes all user data, messages, password hashes, etc.
|
||||
2. Now we want to copy `/home/vmail`, `/var/lib/acme`, `/etc/dkimkeys`, `/run/echobot`, and `/var/spool/postfix` to the new site.
|
||||
Login to the old site while forwarding your SSH agent
|
||||
so you can copy directly from the old to the new site with your SSH key:
|
||||
```
|
||||
ssh -A root@13.37.13.37
|
||||
tar c - /home/vmail/mail /var/lib/acme /etc/dkimkeys /run/echobot /var/spool/postfix | ssh root@13.12.23.42 "tar x -C /"
|
||||
```
|
||||
|
||||
Just run: `ssh root@13.37.13.37 tar c /home/vmail/mail | ssh root@13.12.23.42 tar x -C /home/vmail/`
|
||||
This transfers all addresses, the TLS certificate, DKIM keys (so DKIM DNS record remains valid), and the echobot's password so it continues to function.
|
||||
It also preserves the Postfix mail spool so any messages pending delivery will still be delivered.
|
||||
|
||||
After this, your new server has all the necessary files to start operating :)
|
||||
3. Install chatmail on the new machine:
|
||||
|
||||
8. To be sure the permissions are still fine,
|
||||
run `chown vmail: -R /home/vmail` on the new server.
|
||||
```
|
||||
cmdeploy run --disable-mail --ssh-host 13.12.23.42
|
||||
```
|
||||
Postfix and Dovecot are disabled for now; we will enable them later.
|
||||
We first need to make the new site fully operational.
|
||||
|
||||
9. Finally, you can run `cmdeploy run` to turn on chatmail on the new server.
|
||||
Your users can continue using the chatmail server,
|
||||
and messages which were sent after step 6. should arrive now.
|
||||
Voilà!
|
||||
3. On the new site, run the following to ensure the ownership is correct in case UIDs/GIDs changed:
|
||||
|
||||
```
|
||||
chown root: -R /var/lib/acme
|
||||
chown opendkim: -R /etc/dkimkeys
|
||||
chown vmail: -R /home/vmail/mail
|
||||
chown echobot: -R /run/echobot
|
||||
```
|
||||
|
||||
4. Now, update DNS entries.
|
||||
|
||||
If other MTAs try to deliver messages to your chatmail domain they may fail intermittently,
|
||||
as DNS catches up with the new site settings
|
||||
but normally will retry delivering messages
|
||||
for at least a week, so messages will not be lost.
|
||||
|
||||
5. Finally, you can execute `cmdeploy run --ssh-host 13.12.23.42` to turn on chatmail on the new relay.
|
||||
Your users will be able to use the chatmail relay as soon as the DNS changes have propagated.
|
||||
Voilà!
|
||||
|
||||
## Setting up a reverse proxy
|
||||
|
||||
A chatmail server does not depend on the client IP address
|
||||
A chatmail relay MTA does not track or depend on the client IP address
|
||||
for its operation, so it can be run behind a reverse proxy.
|
||||
This will not even affect incoming mail authentication
|
||||
as DKIM only checks the cryptographic signature
|
||||
of the message and does not use the IP address as the input.
|
||||
|
||||
For example, you may want to self-host your chatmail server
|
||||
For example, you may want to self-host your chatmail relay
|
||||
and only use hosted VPS to provide a public IP address
|
||||
for client connections and incoming mail.
|
||||
You can connect chatmail server to VPS
|
||||
You can connect chatmail relay to VPS
|
||||
using a tunnel protocol
|
||||
such as [WireGuard](https://www.wireguard.com/)
|
||||
and setup a reverse proxy on a VPS
|
||||
to forward connections to the chatmail server
|
||||
to forward connections to the chatmail relay
|
||||
over the tunnel.
|
||||
You can also setup multiple reverse proxies
|
||||
for your chatmail server in different networks
|
||||
to ensure your server is reachable even when
|
||||
for your chatmail relay in different networks
|
||||
to ensure your relay is reachable even when
|
||||
one of the IPs becomes inaccessible due to
|
||||
hosting or routing problems.
|
||||
|
||||
Note that your server still needs
|
||||
Note that your chatmail relay still needs
|
||||
to be able to make outgoing connections on port 25
|
||||
to send messages outside.
|
||||
|
||||
To setup a reverse proxy
|
||||
(or rather Destination NAT, DNAT)
|
||||
for your chatmail server,
|
||||
for your chatmail relay,
|
||||
put the following configuration in `/etc/nftables.conf`:
|
||||
```
|
||||
#!/usr/sbin/nft -f
|
||||
@@ -362,7 +457,7 @@ define wan = eth0
|
||||
# Which ports to proxy.
|
||||
#
|
||||
# Note that SSH is not proxied
|
||||
# so it is possible to log into the proxy server
|
||||
# so it is possible to log into the proxy server
|
||||
# and not the original one.
|
||||
define ports = { smtp, http, https, imap, imaps, submission, submissions }
|
||||
|
||||
@@ -425,7 +520,7 @@ table inet filter {
|
||||
```
|
||||
|
||||
Run `systemctl enable nftables.service`
|
||||
to ensure configuration is reloaded when the proxy server reboots.
|
||||
to ensure configuration is reloaded when the proxy relay reboots.
|
||||
|
||||
Uncomment in `/etc/sysctl.conf` the following two lines:
|
||||
|
||||
@@ -434,7 +529,7 @@ net.ipv4.ip_forward=1
|
||||
net.ipv6.conf.all.forwarding=1
|
||||
```
|
||||
|
||||
Then reboot the server or do `sysctl -p` and `nft -f /etc/nftables.conf`.
|
||||
Then reboot the relay or do `sysctl -p` and `nft -f /etc/nftables.conf`.
|
||||
|
||||
Once proxy server is set up,
|
||||
Once proxy relay is set up,
|
||||
you can add its IP address to the DNS.
|
||||
|
||||
@@ -11,7 +11,11 @@ def read_config(inipath):
|
||||
assert Path(inipath).exists(), inipath
|
||||
cfg = iniconfig.IniConfig(inipath)
|
||||
params = cfg.sections["params"]
|
||||
return Config(inipath, params=params)
|
||||
default_config_content = get_default_config_content(params["mail_domain"])
|
||||
df_params = iniconfig.IniConfig("ini", data=default_config_content)["params"]
|
||||
new_params = dict(df_params.items())
|
||||
new_params.update(params)
|
||||
return Config(inipath, params=new_params)
|
||||
|
||||
|
||||
class Config:
|
||||
@@ -29,7 +33,13 @@ class Config:
|
||||
self.passthrough_senders = params["passthrough_senders"].split()
|
||||
self.passthrough_recipients = params["passthrough_recipients"].split()
|
||||
self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
|
||||
self.filtermail_smtp_port_incoming = int(
|
||||
params["filtermail_smtp_port_incoming"]
|
||||
)
|
||||
self.postfix_reinject_port = int(params["postfix_reinject_port"])
|
||||
self.postfix_reinject_port_incoming = int(
|
||||
params["postfix_reinject_port_incoming"]
|
||||
)
|
||||
self.mtail_address = params.get("mtail_address")
|
||||
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
|
||||
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
|
||||
@@ -69,6 +79,11 @@ class Config:
|
||||
|
||||
def write_initial_config(inipath, mail_domain, overrides):
|
||||
"""Write out default config file, using the specified config value overrides."""
|
||||
content = get_default_config_content(mail_domain, **overrides)
|
||||
inipath.write_text(content)
|
||||
|
||||
|
||||
def get_default_config_content(mail_domain, **overrides):
|
||||
from importlib.resources import files
|
||||
|
||||
inidir = files(__package__).joinpath("ini")
|
||||
@@ -113,5 +128,4 @@ def write_initial_config(inipath, mail_domain, overrides):
|
||||
else:
|
||||
lines.append(line)
|
||||
content = "\n".join(lines)
|
||||
|
||||
inipath.write_text(content)
|
||||
return content
|
||||
|
||||
@@ -8,6 +8,7 @@ import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from deltachat_rpc_client import Bot, DeltaChat, EventType, Rpc, events
|
||||
|
||||
@@ -97,6 +98,10 @@ def main():
|
||||
if not bot.is_configured():
|
||||
bot.configure(addr, password)
|
||||
|
||||
# write invite link to working directory
|
||||
invitelink = bot.account.get_qr_code()
|
||||
Path("invite-link.txt").write_text(invitelink)
|
||||
|
||||
bot.run_forever()
|
||||
|
||||
|
||||
|
||||
@@ -11,9 +11,12 @@ from email.utils import parseaddr
|
||||
from smtplib import SMTP as SMTPClient
|
||||
|
||||
from aiosmtpd.controller import Controller
|
||||
from aiosmtpd.smtp import SMTP
|
||||
|
||||
from .config import read_config
|
||||
|
||||
ENCRYPTION_NEEDED_523 = "523 Encryption Needed: Invalid Unencrypted Mail"
|
||||
|
||||
|
||||
def check_openpgp_payload(payload: bytes):
|
||||
"""Checks the OpenPGP payload.
|
||||
@@ -157,9 +160,14 @@ def check_encrypted(message):
|
||||
return True
|
||||
|
||||
|
||||
async def asyncmain_beforequeue(config):
|
||||
port = config.filtermail_smtp_port
|
||||
Controller(BeforeQueueHandler(config), hostname="127.0.0.1", port=port).start()
|
||||
async def asyncmain_beforequeue(config, mode):
|
||||
if mode == "outgoing":
|
||||
port = config.filtermail_smtp_port
|
||||
handler = OutgoingBeforeQueueHandler(config)
|
||||
else:
|
||||
port = config.filtermail_smtp_port_incoming
|
||||
handler = IncomingBeforeQueueHandler(config)
|
||||
HackedController(handler, hostname="127.0.0.1", port=port).start()
|
||||
|
||||
|
||||
def recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
@@ -171,7 +179,21 @@ def recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
return False
|
||||
|
||||
|
||||
class BeforeQueueHandler:
|
||||
class HackedController(Controller):
|
||||
def factory(self):
|
||||
return SMTPDiscardRCPTO_options(self.handler, **self.SMTP_kwargs)
|
||||
|
||||
|
||||
class SMTPDiscardRCPTO_options(SMTP):
|
||||
def _getparams(self, params):
|
||||
# aiosmtpd's SMTP daemon fails to handle a request if there are RCPT TO options
|
||||
# We just ignore them for our incoming filtermail purposes
|
||||
if len(params) == 1 and params[0].startswith("ORCPT"):
|
||||
return {}
|
||||
return super()._getparams(params)
|
||||
|
||||
|
||||
class OutgoingBeforeQueueHandler:
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
self.send_rate_limiter = SendRateLimiter()
|
||||
@@ -196,7 +218,9 @@ class BeforeQueueHandler:
|
||||
return error
|
||||
logging.info("re-injecting the mail that passed checks")
|
||||
client = SMTPClient("localhost", self.config.postfix_reinject_port)
|
||||
client.sendmail(envelope.mail_from, envelope.rcpt_tos, envelope.content)
|
||||
client.sendmail(
|
||||
envelope.mail_from, envelope.rcpt_tos, envelope.original_content
|
||||
)
|
||||
return "250 OK"
|
||||
|
||||
def check_DATA(self, envelope):
|
||||
@@ -207,40 +231,86 @@ class BeforeQueueHandler:
|
||||
mail_encrypted = check_encrypted(message)
|
||||
|
||||
_, from_addr = parseaddr(message.get("from").strip())
|
||||
envelope_from_domain = from_addr.split("@").pop()
|
||||
|
||||
logging.info(f"mime-from: {from_addr} envelope-from: {envelope.mail_from!r}")
|
||||
if envelope.mail_from.lower() != from_addr.lower():
|
||||
return f"500 Invalid FROM <{from_addr!r}> for <{envelope.mail_from!r}>"
|
||||
|
||||
if mail_encrypted:
|
||||
print("Filtering encrypted mail.", file=sys.stderr)
|
||||
else:
|
||||
print("Filtering unencrypted mail.", file=sys.stderr)
|
||||
if mail_encrypted or is_securejoin(message):
|
||||
print("Outgoing: Filtering encrypted mail.", file=sys.stderr)
|
||||
return
|
||||
|
||||
print("Outgoing: Filtering unencrypted mail.", file=sys.stderr)
|
||||
|
||||
if envelope.mail_from in self.config.passthrough_senders:
|
||||
return
|
||||
|
||||
# allow self-sent Autocrypt Setup Message
|
||||
if envelope.rcpt_tos == [from_addr]:
|
||||
if message.get("subject") == "Autocrypt Setup Message":
|
||||
if message.get_content_type() == "multipart/mixed":
|
||||
return
|
||||
|
||||
passthrough_recipients = self.config.passthrough_recipients
|
||||
|
||||
if mail_encrypted or is_securejoin(message):
|
||||
return
|
||||
|
||||
for recipient in envelope.rcpt_tos:
|
||||
if envelope.mail_from == recipient:
|
||||
# Always allow sending emails to self.
|
||||
continue
|
||||
if recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
continue
|
||||
res = recipient.split("@")
|
||||
if len(res) != 2:
|
||||
return f"500 Invalid address <{recipient}>"
|
||||
_recipient_addr, recipient_domain = res
|
||||
|
||||
is_outgoing = recipient_domain != envelope_from_domain
|
||||
if is_outgoing:
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return f"500 Invalid unencrypted mail to <{recipient}>"
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return ENCRYPTION_NEEDED_523
|
||||
|
||||
|
||||
class IncomingBeforeQueueHandler:
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
|
||||
async def handle_DATA(self, server, session, envelope):
|
||||
logging.info("handle_DATA before-queue")
|
||||
error = self.check_DATA(envelope)
|
||||
if error:
|
||||
return error
|
||||
logging.info("re-injecting the mail that passed checks")
|
||||
|
||||
# the smtp daemon on reinject_port_incoming gives it to dkim milter
|
||||
# which looks at source address to determine whether to verify or sign
|
||||
client = SMTPClient(
|
||||
"localhost",
|
||||
self.config.postfix_reinject_port_incoming,
|
||||
source_address=("127.0.0.2", 0),
|
||||
)
|
||||
client.sendmail(
|
||||
envelope.mail_from, envelope.rcpt_tos, envelope.original_content
|
||||
)
|
||||
return "250 OK"
|
||||
|
||||
def check_DATA(self, envelope):
|
||||
"""the central filtering function for e-mails."""
|
||||
logging.info(f"Processing DATA message from {envelope.mail_from}")
|
||||
|
||||
message = BytesParser(policy=policy.default).parsebytes(envelope.content)
|
||||
mail_encrypted = check_encrypted(message)
|
||||
|
||||
if mail_encrypted or is_securejoin(message):
|
||||
print("Incoming: Filtering encrypted mail.", file=sys.stderr)
|
||||
return
|
||||
|
||||
print("Incoming: Filtering unencrypted mail.", file=sys.stderr)
|
||||
|
||||
# we want cleartext mailer-daemon messages to pass through
|
||||
# chatmail core will typically not display them as normal messages
|
||||
if message.get("auto-submitted"):
|
||||
_, from_addr = parseaddr(message.get("from").strip())
|
||||
if from_addr.lower().startswith("mailer-daemon@"):
|
||||
if message.get_content_type() == "multipart/report":
|
||||
return
|
||||
|
||||
for recipient in envelope.rcpt_tos:
|
||||
user = self.config.get_user(recipient)
|
||||
if user is None or user.is_incoming_cleartext_ok():
|
||||
continue
|
||||
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return ENCRYPTION_NEEDED_523
|
||||
|
||||
|
||||
class SendRateLimiter:
|
||||
@@ -259,11 +329,14 @@ class SendRateLimiter:
|
||||
|
||||
def main():
|
||||
args = sys.argv[1:]
|
||||
assert len(args) == 1
|
||||
assert len(args) == 2
|
||||
config = read_config(args[0])
|
||||
mode = args[1]
|
||||
logging.basicConfig(level=logging.WARN)
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
task = asyncmain_beforequeue(config)
|
||||
assert mode in ["incoming", "outgoing"]
|
||||
task = asyncmain_beforequeue(config, mode)
|
||||
loop.create_task(task)
|
||||
logging.info("entering serving loop")
|
||||
loop.run_forever()
|
||||
|
||||
@@ -46,12 +46,14 @@ passthrough_recipients = xstore@testrun.org
|
||||
# Deployment Details
|
||||
#
|
||||
|
||||
# where the filtermail SMTP service listens
|
||||
# SMTP outgoing filtermail and reinjection
|
||||
filtermail_smtp_port = 10080
|
||||
|
||||
# postfix accepts on the localhost reinject SMTP port
|
||||
postfix_reinject_port = 10025
|
||||
|
||||
# SMTP incoming filtermail and reinjection
|
||||
filtermail_smtp_port_incoming = 10081
|
||||
postfix_reinject_port_incoming = 10026
|
||||
|
||||
# if set to "True" IPv6 is disabled
|
||||
disable_ipv6 = False
|
||||
|
||||
|
||||
@@ -11,6 +11,8 @@ def main(vmail_dir=None):
|
||||
ci_accounts = 0
|
||||
|
||||
for path in Path(vmail_dir).iterdir():
|
||||
if not path.joinpath("cur").is_dir():
|
||||
continue
|
||||
accounts += 1
|
||||
if path.name[:3] in ("ci-", "ac_"):
|
||||
ci_accounts += 1
|
||||
|
||||
56
chatmaild/src/chatmaild/tests/mail-data/asm.eml
Normal file
56
chatmaild/src/chatmaild/tests/mail-data/asm.eml
Normal file
@@ -0,0 +1,56 @@
|
||||
From: {from_addr}
|
||||
To: {to_addr}
|
||||
Autocrypt-Setup-Message: v1
|
||||
Subject: Autocrypt Setup Message
|
||||
Date: Tue, 22 Jan 2019 12:56:29 +0100
|
||||
Content-type: multipart/mixed; boundary="Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ"
|
||||
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
|
||||
Content-Type: text/plain
|
||||
|
||||
This message contains all information to transfer your Autocrypt
|
||||
settings along with your secret key securely from your original
|
||||
device.
|
||||
|
||||
To set up your new device for Autocrypt, please follow the
|
||||
instuctions that should be presented by your new device.
|
||||
|
||||
You can keep this message and use it as a backup for your secret
|
||||
key. If you want to do this, you should write down the Setup Code
|
||||
and store it securely.
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
|
||||
Content-Type: application/autocrypt-setup
|
||||
Content-Disposition: attachment; filename="autocrypt-setup-message.html"
|
||||
|
||||
<html><body>
|
||||
<p>
|
||||
This is the Autocrypt setup file used to transfer settings and
|
||||
keys between clients. You can decrypt it using the Setup Code
|
||||
presented on your old device, and then import the contained key
|
||||
into your keyring.
|
||||
</p>
|
||||
|
||||
<pre>
|
||||
-----BEGIN PGP MESSAGE-----
|
||||
Passphrase-Format: numeric9x4
|
||||
Passphrase-Begin: 17
|
||||
|
||||
jA0EBwMCFAxADoCdzeX/0ukBlqI5+pfpKb751qd/7nLNbkpy3gVcaf1QwRPZYt40
|
||||
Ynp08UqRQ2g48ZlnzHLSwlTGOPTuv2Jt8ka+pgZ45xzvJSG2gau03xP4VsC271kR
|
||||
VmCjdb0Y6Rk96mAwfGzrkbaRQ9Z7fIoL866GOv6h9neiVIkp+JYlTV6ISD0ZQJ4Q
|
||||
I6dOQkB/TWZyVjtiJDOQHdfNWliA6NtqaLq19wlu9L5xXjuNpY95KwR8EJXWe0+o
|
||||
Y3d2U/KxOAkXKghP2Qg1GtlPVeGC5T4p03TGI6pzKT+kHX6Rrm9wK6sM9aTquMmF
|
||||
Vok84Jg1DFnwivWC2RILR81rXi7k/+Y6MUbveFgJ9cQduqpxnmD7TjOblYu7M6zp
|
||||
YGAUxh8DRKlIMn2QsA++DBYQ6ACZvwuY8qTDLkqPDo4WqM313dsMJbyGjDdVE7EM
|
||||
PESS+RlABETpZXz8g/ycr6DIUNdlbPcmYlsBfHWDOuR2GFFTwmlv5slWS39dJv38
|
||||
E0eIe1CwdxI801Se7t7dUUS/ZF8wb6GlmxOcqGbF8eko1Z0S64IAm7/h13MRQCxI
|
||||
geQnHfGYVJ2FOimoCMEKwfa9x++RFTDW0u7spDC2uWvK/1viV8OfRppFhLr/kmKb
|
||||
18lWXuAz80DAjUDUsVqEq2MvJBJGoCJUEyjuRsLkHYRM5jYk4v50LyyR0Om73nWF
|
||||
nZBqmqNzdr7Xb9PHHdFhnEc0VvoYbrcM0RVYcEMW3YbmejM891j1d6Iv+/n/qND/
|
||||
NdebGrfWJMmFLf/iEkzTZ3/v5inW9LpWoRc94ioCjJTaEo8Rib6ARRFaJVIsmNXi
|
||||
YicFGO98D+zX+a2t9Yz6IpPajVslnOp6ScpmXgts/2XWD7oE+JgxSAqo/dLVsHgP
|
||||
Ufo=
|
||||
=pulM
|
||||
-----END PGP MESSAGE-----
|
||||
</pre></body></html>
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ--
|
||||
46
chatmaild/src/chatmaild/tests/mail-data/mailer-daemon.eml
Normal file
46
chatmaild/src/chatmaild/tests/mail-data/mailer-daemon.eml
Normal file
@@ -0,0 +1,46 @@
|
||||
Date: Fri, 8 Jul 1994 09:21:47 -0400
|
||||
From: Mail Delivery Subsystem <MAILER-DAEMON@example.org>
|
||||
Subject: Returned mail: User unknown
|
||||
To: <owner-ups-mib@CS.UTK.EDU>
|
||||
Auto-Submitted: auto-replied
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/report; report-type=delivery-status;
|
||||
boundary="JAA13167.773673707/CS.UTK.EDU"
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: text/plain; charset=us-ascii
|
||||
|
||||
----- The following addresses had delivery problems -----
|
||||
<arathib@vnet.ibm.com> (unrecoverable error)
|
||||
<wsnell@sdcc13.ucsd.edu> (unrecoverable error)
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: message/delivery-status
|
||||
|
||||
Reporting-MTA: dns; cs.utk.edu
|
||||
|
||||
Original-Recipient: rfc822;arathib@vnet.ibm.com
|
||||
Final-Recipient: rfc822;arathib@vnet.ibm.com
|
||||
Action: failed
|
||||
Status: 5.0.0 (permanent failure)
|
||||
Diagnostic-Code: smtp;
|
||||
550 'arathib@vnet.IBM.COM' is not a registered gateway user
|
||||
Remote-MTA: dns; vnet.ibm.com
|
||||
|
||||
Original-Recipient: rfc822;johnh@hpnjld.njd.hp.com
|
||||
Final-Recipient: rfc822;johnh@hpnjld.njd.hp.com
|
||||
Action: delayed
|
||||
Status: 4.0.0 (hpnjld.njd.jp.com: host name lookup failure)
|
||||
|
||||
Original-Recipient: rfc822;wsnell@sdcc13.ucsd.edu
|
||||
Final-Recipient: rfc822;wsnell@sdcc13.ucsd.edu
|
||||
Action: failed
|
||||
Status: 5.0.0
|
||||
Diagnostic-Code: smtp; 550 user unknown
|
||||
Remote-MTA: dns; sdcc13.ucsd.edu
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: message/rfc822
|
||||
|
||||
[original message goes here]
|
||||
--JAA13167.773673707/CS.UTK.EDU--
|
||||
@@ -72,9 +72,8 @@ def maildata(request):
|
||||
def maildata(name, from_addr, to_addr, subject="[...]"):
|
||||
# Using `.read_bytes().decode()` instead of `.read_text()` to preserve newlines.
|
||||
data = datadir.joinpath(name).read_bytes().decode()
|
||||
|
||||
text = data.format(from_addr=from_addr, to_addr=to_addr, subject=subject)
|
||||
return BytesParser(policy=policy.default).parsebytes(text.encode())
|
||||
return BytesParser(policy=policy.SMTP).parsebytes(text.encode())
|
||||
|
||||
return maildata
|
||||
|
||||
|
||||
@@ -15,6 +15,14 @@ def test_read_config_basic(example_config):
|
||||
assert example_config.mail_domain == "chat.example.org"
|
||||
|
||||
|
||||
def test_read_config_basic_using_defaults(tmp_path, maildomain):
|
||||
inipath = tmp_path.joinpath("chatmail.ini")
|
||||
inipath.write_text(f"[params]\nmail_domain = {maildomain}")
|
||||
example_config = read_config(inipath)
|
||||
assert example_config.max_user_send_per_minute == 60
|
||||
assert example_config.filtermail_smtp_port_incoming == 10081
|
||||
|
||||
|
||||
def test_read_config_testrun(make_config):
|
||||
config = make_config("something.testrun.org")
|
||||
assert config.mail_domain == "something.testrun.org"
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import pytest
|
||||
|
||||
from chatmaild.filtermail import (
|
||||
BeforeQueueHandler,
|
||||
IncomingBeforeQueueHandler,
|
||||
OutgoingBeforeQueueHandler,
|
||||
SendRateLimiter,
|
||||
check_armored_payload,
|
||||
check_encrypted,
|
||||
@@ -18,7 +19,13 @@ def maildomain():
|
||||
@pytest.fixture
|
||||
def handler(make_config, maildomain):
|
||||
config = make_config(maildomain)
|
||||
return BeforeQueueHandler(config)
|
||||
return OutgoingBeforeQueueHandler(config)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def inhandler(make_config, maildomain):
|
||||
config = make_config(maildomain)
|
||||
return IncomingBeforeQueueHandler(config)
|
||||
|
||||
|
||||
def test_reject_forged_from(maildata, gencreds, handler):
|
||||
@@ -29,14 +36,14 @@ def test_reject_forged_from(maildata, gencreds, handler):
|
||||
# test that the filter lets good mail through
|
||||
to_addr = gencreds()[0]
|
||||
env.content = maildata(
|
||||
"plain.eml", from_addr=env.mail_from, to_addr=to_addr
|
||||
"encrypted.eml", from_addr=env.mail_from, to_addr=to_addr
|
||||
).as_bytes()
|
||||
|
||||
assert not handler.check_DATA(envelope=env)
|
||||
|
||||
# test that the filter rejects forged mail
|
||||
env.content = maildata(
|
||||
"plain.eml", from_addr="forged@c3.testrun.org", to_addr=to_addr
|
||||
"encrypted.eml", from_addr="forged@c3.testrun.org", to_addr=to_addr
|
||||
).as_bytes()
|
||||
error = handler.check_DATA(envelope=env)
|
||||
assert "500" in error
|
||||
@@ -106,7 +113,7 @@ def test_send_rate_limiter():
|
||||
break
|
||||
|
||||
|
||||
def test_excempt_privacy(maildata, gencreds, handler):
|
||||
def test_cleartext_excempt_privacy(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = "privacy@testrun.org"
|
||||
handler.config.passthrough_recipients = [to_addr]
|
||||
@@ -127,10 +134,73 @@ def test_excempt_privacy(maildata, gencreds, handler):
|
||||
rcpt_tos = [to_addr, false_to]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert "500" in handler.check_DATA(envelope=env2)
|
||||
assert "523" in handler.check_DATA(envelope=env2)
|
||||
|
||||
|
||||
def test_passthrough_domains(maildata, gencreds, handler):
|
||||
def test_cleartext_self_send_autocrypt_setup_message(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = from_addr
|
||||
|
||||
msg = maildata("asm.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert not handler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_send_fails(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = gencreds()[0]
|
||||
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
res = handler.check_DATA(envelope=env)
|
||||
assert "523 Encryption Needed" in res
|
||||
|
||||
|
||||
def test_cleartext_incoming_fails(maildata, gencreds, inhandler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr, password = gencreds()
|
||||
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
user = inhandler.config.get_user(to_addr)
|
||||
user.set_password(password)
|
||||
res = inhandler.check_DATA(envelope=env)
|
||||
assert "523 Encryption Needed" in res
|
||||
|
||||
user.allow_incoming_cleartext()
|
||||
assert not inhandler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_incoming_mailer_daemon(maildata, gencreds, inhandler):
|
||||
from_addr = "mailer-daemon@example.org"
|
||||
to_addr = gencreds()[0]
|
||||
|
||||
msg = maildata("mailer-daemon.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert not inhandler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_passthrough_domains(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = "privacy@x.y.z"
|
||||
handler.config.passthrough_recipients = ["@x.y.z"]
|
||||
@@ -151,10 +221,10 @@ def test_passthrough_domains(maildata, gencreds, handler):
|
||||
rcpt_tos = [to_addr, false_to]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert "500" in handler.check_DATA(envelope=env2)
|
||||
assert "523" in handler.check_DATA(envelope=env2)
|
||||
|
||||
|
||||
def test_passthrough_senders(gencreds, handler, maildata):
|
||||
def test_cleartext_passthrough_senders(gencreds, handler, maildata):
|
||||
acc1 = gencreds()[0]
|
||||
to_addr = "recipient@something.org"
|
||||
handler.config.passthrough_senders = [acc1]
|
||||
|
||||
@@ -2,8 +2,15 @@ from chatmaild.metrics import main
|
||||
|
||||
|
||||
def test_main(tmp_path, capsys):
|
||||
paths = []
|
||||
for x in ("ci-asllkj", "ac_12l3kj", "qweqwe", "ci-l1k2j31l2k3"):
|
||||
tmp_path.joinpath(x).mkdir()
|
||||
p = tmp_path.joinpath(x)
|
||||
p.mkdir()
|
||||
p.joinpath("cur").mkdir()
|
||||
paths.append(p)
|
||||
|
||||
tmp_path.joinpath("nomailbox").mkdir()
|
||||
|
||||
main(tmp_path)
|
||||
out, _ = capsys.readouterr()
|
||||
d = {}
|
||||
|
||||
@@ -40,3 +40,17 @@ def test_no_mailboxes_dir(testaddr, example_config, tmp_path):
|
||||
user.set_password("someeqkjwelkqwjleqwe")
|
||||
user.set_last_login_timestamp(100000)
|
||||
assert user.get_last_login_timestamp() == 86400
|
||||
|
||||
|
||||
def test_set_get_cleartext_flag(testaddr, example_config, tmp_path):
|
||||
p = tmp_path.joinpath("a", "mailboxes")
|
||||
example_config.mailboxes_dir = p
|
||||
|
||||
user = example_config.get_user(testaddr)
|
||||
user.set_password("someeqkjwelkqwjleqwe")
|
||||
user.set_last_login_timestamp(100000)
|
||||
assert user.get_last_login_timestamp() == 86400
|
||||
|
||||
assert not user.is_incoming_cleartext_ok()
|
||||
user.allow_incoming_cleartext()
|
||||
assert user.is_incoming_cleartext_ok()
|
||||
|
||||
@@ -13,6 +13,7 @@ class User:
|
||||
self.maildir = maildir
|
||||
self.addr = addr
|
||||
self.password_path = password_path
|
||||
self.enforce_E2EE_path = maildir.joinpath("enforceE2EEincoming")
|
||||
self.uid = uid
|
||||
self.gid = gid
|
||||
|
||||
@@ -35,6 +36,13 @@ class User:
|
||||
home = str(self.maildir)
|
||||
return dict(addr=self.addr, home=home, uid=self.uid, gid=self.gid, password=pw)
|
||||
|
||||
def is_incoming_cleartext_ok(self):
|
||||
return not self.enforce_E2EE_path.exists()
|
||||
|
||||
def allow_incoming_cleartext(self):
|
||||
if self.enforce_E2EE_path.exists():
|
||||
self.enforce_E2EE_path.unlink()
|
||||
|
||||
def set_password(self, enc_password):
|
||||
"""Set the specified password for this user.
|
||||
|
||||
@@ -50,6 +58,7 @@ class User:
|
||||
if not self.addr.startswith("echo@"):
|
||||
logging.error(f"could not write password for: {self.addr}")
|
||||
raise
|
||||
self.enforce_E2EE_path.touch()
|
||||
|
||||
def set_last_login_timestamp(self, timestamp):
|
||||
"""Track login time with daily granularity
|
||||
|
||||
@@ -11,6 +11,7 @@ from pathlib import Path
|
||||
|
||||
from chatmaild.config import Config, read_config
|
||||
from pyinfra import facts, host
|
||||
from pyinfra.api import FactBase
|
||||
from pyinfra.facts.files import File
|
||||
from pyinfra.facts.systemd import SystemdEnabled
|
||||
from pyinfra.operations import apt, files, pip, server, systemd
|
||||
@@ -18,6 +19,21 @@ from pyinfra.operations import apt, files, pip, server, systemd
|
||||
from .acmetool import deploy_acmetool
|
||||
|
||||
|
||||
class Port(FactBase):
|
||||
"""
|
||||
Returns the process occuping a port.
|
||||
"""
|
||||
|
||||
def command(self, port: int) -> str:
|
||||
return (
|
||||
"ss -lptn 'src :%d' | awk 'NR>1 {print $6,$7}' | sed 's/users:((\"//;s/\".*//'"
|
||||
% (port,)
|
||||
)
|
||||
|
||||
def process(self, output: [str]) -> str:
|
||||
return output[0]
|
||||
|
||||
|
||||
def _build_chatmaild(dist_dir) -> None:
|
||||
dist_dir = Path(dist_dir).resolve()
|
||||
if dist_dir.exists():
|
||||
@@ -106,12 +122,14 @@ def _install_remote_venv_with_chatmaild(config) -> None:
|
||||
for fn in (
|
||||
"doveauth",
|
||||
"filtermail",
|
||||
"filtermail-incoming",
|
||||
"echobot",
|
||||
"chatmail-metadata",
|
||||
"lastlogin",
|
||||
):
|
||||
execpath = fn if fn != "filtermail-incoming" else "filtermail"
|
||||
params = dict(
|
||||
execpath=f"{remote_venv_dir}/bin/{fn}",
|
||||
execpath=f"{remote_venv_dir}/bin/{execpath}",
|
||||
config_path=remote_chatmail_inipath,
|
||||
remote_venv_dir=remote_venv_dir,
|
||||
mail_domain=config.mail_domain,
|
||||
@@ -215,7 +233,7 @@ def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
|
||||
server.shell(
|
||||
name="Generate OpenDKIM domain keys",
|
||||
commands=[
|
||||
f"opendkim-genkey -D /etc/dkimkeys -d {domain} -s {dkim_selector}"
|
||||
f"/usr/sbin/opendkim-genkey -D /etc/dkimkeys -d {domain} -s {dkim_selector}"
|
||||
],
|
||||
_use_su_login=True,
|
||||
_su_user="opendkim",
|
||||
@@ -228,7 +246,6 @@ def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
|
||||
)
|
||||
need_restart |= service_file.changed
|
||||
|
||||
|
||||
return need_restart
|
||||
|
||||
|
||||
@@ -541,7 +558,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
|
||||
server.group(name="Create vmail group", group="vmail", system=True)
|
||||
server.user(name="Create vmail user", user="vmail", group="vmail", system=True)
|
||||
server.user(name="Create filtermail user", user="filtermail", system=True)
|
||||
server.group(name="Create opendkim group", group="opendkim", system=True)
|
||||
server.user(
|
||||
name="Create opendkim user",
|
||||
@@ -576,6 +592,12 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
ensure_newline=True,
|
||||
)
|
||||
|
||||
if host.get_fact(Port, port=53) != "unbound":
|
||||
files.line(
|
||||
name="Add 9.9.9.9 to resolv.conf",
|
||||
path="/etc/resolv.conf",
|
||||
line="nameserver 9.9.9.9",
|
||||
)
|
||||
apt.update(name="apt update", cache_time=24 * 3600)
|
||||
apt.upgrade(name="upgrade apt packages", auto_remove=True)
|
||||
|
||||
@@ -587,6 +609,12 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
# Run local DNS resolver `unbound`.
|
||||
# `resolvconf` takes care of setting up /etc/resolv.conf
|
||||
# to use 127.0.0.1 as the resolver.
|
||||
from cmdeploy.cmdeploy import Out
|
||||
|
||||
process_on_53 = host.get_fact(Port, port=53)
|
||||
if process_on_53 not in (None, "unbound"):
|
||||
Out().red(f"Can't install unbound: port 53 is occupied by: {process_on_53}")
|
||||
exit(1)
|
||||
apt.packages(
|
||||
name="Install unbound",
|
||||
packages=["unbound", "unbound-anchor", "dnsutils"],
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
"acme-enter-email": "{{ email }}"
|
||||
"acme-agreement:https://letsencrypt.org/documents/LE-SA-v1.4-April-3-2024.pdf": true
|
||||
"acme-agreement:https://letsencrypt.org/documents/LE-SA-v1.5-February-24-2025.pdf": true
|
||||
|
||||
@@ -86,15 +86,19 @@ def run_cmd(args, out):
|
||||
out.red("Please re-run scripts/initenv.sh to update pyinfra to version 3.")
|
||||
return 1
|
||||
|
||||
retcode = out.check_call(cmd, env=env)
|
||||
if retcode == 0:
|
||||
out.green("Deploy completed, call `cmdeploy dns` next.")
|
||||
elif not remote_data["acme_account_url"]:
|
||||
out.red("Deploy completed but letsencrypt not configured")
|
||||
out.red("Run 'cmdeploy run' again")
|
||||
retcode = 0
|
||||
else:
|
||||
try:
|
||||
retcode = out.check_call(cmd, env=env)
|
||||
if retcode == 0:
|
||||
out.green("Deploy completed, call `cmdeploy dns` next.")
|
||||
elif not remote_data["acme_account_url"]:
|
||||
out.red("Deploy completed but letsencrypt not configured")
|
||||
out.red("Run 'cmdeploy run' again")
|
||||
retcode = 0
|
||||
else:
|
||||
out.red("Deploy failed")
|
||||
except subprocess.CalledProcessError:
|
||||
out.red("Deploy failed")
|
||||
retcode = 1
|
||||
return retcode
|
||||
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
# delete already seen big mails after 7 days, in the INBOX
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/cur/*' -mtime +7 -size +200k -type f -delete
|
||||
# delete all mails after {{ config.delete_mails_after }} days, in the Inbox
|
||||
2 0 * * * vmail find {{ config.mailboxes_dir }} -path '*/cur/*' -mtime +{{ config.delete_mails_after }} -type f -delete
|
||||
# or in any IMAP subfolder
|
||||
|
||||
@@ -3,7 +3,7 @@ Description=mtail
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/bin/sh -c "journalctl -f -o short-iso -n 0 | /usr/bin/mtail --address={{ address }} --port={{ port }} --progs /etc/mtail --logtostderr --logs -"
|
||||
ExecStart=/bin/sh -c "journalctl -f -o short-iso -n 0 | /usr/bin/mtail --address={{ address }} --port={{ port }} --progs /etc/mtail --logtostderr --logs /dev/stdin"
|
||||
Restart=on-failure
|
||||
|
||||
[Install]
|
||||
|
||||
@@ -46,10 +46,7 @@ http {
|
||||
|
||||
server {
|
||||
|
||||
listen 8443 ssl default_server;
|
||||
{% if not disable_ipv6 %}
|
||||
listen [::]:8443 ssl default_server;
|
||||
{% endif %}
|
||||
listen 127.0.0.1:8443 ssl default_server;
|
||||
|
||||
root /var/www/html;
|
||||
|
||||
@@ -120,10 +117,7 @@ http {
|
||||
|
||||
# Redirect www. to non-www
|
||||
server {
|
||||
listen 8443 ssl;
|
||||
{% if not disable_ipv6 %}
|
||||
listen [::]:8443 ssl;
|
||||
{% endif %}
|
||||
listen 127.0.0.1:8443 ssl;
|
||||
server_name www.{{ config.domain_name }};
|
||||
return 301 $scheme://{{ config.domain_name }}$request_uri;
|
||||
access_log syslog:server=unix:/dev/log,facility=local7;
|
||||
|
||||
@@ -21,6 +21,9 @@ smtpd_tls_security_level=may
|
||||
|
||||
smtp_tls_CApath=/etc/ssl/certs
|
||||
smtp_tls_security_level=verify
|
||||
# Send SNI extension when connecting to other servers.
|
||||
# <https://www.postfix.org/postconf.5.html#smtp_tls_servername>
|
||||
smtp_tls_servername = hostname
|
||||
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
|
||||
smtp_tls_policy_maps = inline:{nauta.cu=may}
|
||||
smtpd_tls_protocols = >=TLSv1.2
|
||||
|
||||
@@ -14,7 +14,7 @@ smtp inet n - y - - smtpd -v
|
||||
{%- else %}
|
||||
smtp inet n - y - - smtpd
|
||||
{%- endif %}
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port_incoming }}
|
||||
submission inet n - y - 5000 smtpd
|
||||
-o syslog_name=postfix/submission
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
@@ -32,7 +32,6 @@ submission inet n - y - 5000 smtpd
|
||||
-o milter_macro_daemon_name=ORIGINATING
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
-o cleanup_service_name=authclean
|
||||
smtps inet n - y - 5000 smtpd
|
||||
-o syslog_name=postfix/smtps
|
||||
-o smtpd_tls_wrappermode=yes
|
||||
@@ -50,7 +49,6 @@ smtps inet n - y - 5000 smtpd
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o milter_macro_daemon_name=ORIGINATING
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
-o cleanup_service_name=authclean
|
||||
#628 inet n - y - - qmqpd
|
||||
pickup unix n - y 60 1 pickup
|
||||
cleanup unix n - y - 0 cleanup
|
||||
@@ -78,17 +76,27 @@ anvil unix - - y - 1 anvil
|
||||
scache unix - - y - 1 scache
|
||||
postlog unix-dgram n - n - 1 postlogd
|
||||
filter unix - n n - - lmtp
|
||||
# Local SMTP server for reinjecting filered mail.
|
||||
localhost:{{ config.postfix_reinject_port }} inet n - n - 10 smtpd
|
||||
# Local SMTP server for reinjecting outgoing filtered mail.
|
||||
127.0.0.1:{{ config.postfix_reinject_port }} inet n - n - 10 smtpd
|
||||
-o syslog_name=postfix/reinject
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
-o cleanup_service_name=authclean
|
||||
|
||||
# Local SMTP server for reinjecting incoming filtered mail
|
||||
127.0.0.1:{{ config.postfix_reinject_port_incoming }} inet n - n - 10 smtpd
|
||||
-o syslog_name=postfix/reinject_incoming
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
|
||||
# Cleanup `Received` headers for authenticated mail
|
||||
# to avoid leaking client IP.
|
||||
#
|
||||
# We do not do this for received mails
|
||||
# as this will break DKIM signatures
|
||||
# if `Received` header is signed.
|
||||
#
|
||||
# This service also rewrites
|
||||
# Subject with `[...]`
|
||||
# to make sure the users
|
||||
# cannot send unprotected Subject.
|
||||
authclean unix n - - - 0 cleanup
|
||||
-o header_checks=regexp:/etc/postfix/submission_header_cleanup
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
from subprocess import CalledProcessError, check_output
|
||||
from subprocess import DEVNULL, CalledProcessError, check_output
|
||||
|
||||
|
||||
def shell(command, fail_ok=False):
|
||||
print(f"$ {command}")
|
||||
args = dict(shell=True)
|
||||
if fail_ok:
|
||||
args["stderr"] = DEVNULL
|
||||
try:
|
||||
return check_output(command, shell=True).decode().rstrip()
|
||||
return check_output(command, **args).decode().rstrip()
|
||||
except CalledProcessError:
|
||||
if not fail_ok:
|
||||
raise
|
||||
@@ -14,3 +17,22 @@ def shell(command, fail_ok=False):
|
||||
def get_systemd_running():
|
||||
lines = shell("systemctl --type=service --state=running").split("\n")
|
||||
return [line for line in lines if line.startswith(" ")]
|
||||
|
||||
|
||||
def write_numbytes(path, num):
|
||||
with open(path, "w") as f:
|
||||
f.write("x" * num)
|
||||
|
||||
|
||||
def dovecot_recalc_quota(user):
|
||||
shell(f"doveadm quota recalc -u {user}")
|
||||
output = shell(f"doveadm quota get -u {user}")
|
||||
#
|
||||
# Quota name Type Value Limit %
|
||||
# User quota STORAGE 5 102400 0
|
||||
# User quota MESSAGE 2 - 0
|
||||
#
|
||||
for line in output.split("\n"):
|
||||
parts = line.split()
|
||||
if parts[2] == "STORAGE":
|
||||
return dict(value=int(parts[3]), limit=int(parts[4]), percent=int(parts[5]))
|
||||
|
||||
12
cmdeploy/src/cmdeploy/service/filtermail-incoming.service.f
Normal file
12
cmdeploy/src/cmdeploy/service/filtermail-incoming.service.f
Normal file
@@ -0,0 +1,12 @@
|
||||
[Unit]
|
||||
Description=Incoming Chatmail Postfix before queue filter
|
||||
|
||||
[Service]
|
||||
ExecStart={execpath} {config_path} incoming
|
||||
Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
[Unit]
|
||||
Description=Chatmail Postfix before queue filter
|
||||
Description=Outgoing Chatmail Postfix before queue filter
|
||||
|
||||
[Service]
|
||||
ExecStart={execpath} {config_path}
|
||||
ExecStart={execpath} {config_path} outgoing
|
||||
Restart=always
|
||||
RestartSec=30
|
||||
User=filtermail
|
||||
User=vmail
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -37,7 +37,7 @@ class TestDC:
|
||||
|
||||
def test_ping_pong(self, benchmark, cmfactory):
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
|
||||
def dc_ping_pong():
|
||||
chat.send_text("ping")
|
||||
@@ -49,7 +49,7 @@ class TestDC:
|
||||
|
||||
def test_send_10_receive_10(self, benchmark, cmfactory, lp):
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
|
||||
def dc_send_10_receive_10():
|
||||
for i in range(10):
|
||||
|
||||
@@ -90,8 +90,13 @@ def test_concurrent_logins_same_account(
|
||||
|
||||
|
||||
def test_no_vrfy(chatmail_config):
|
||||
domain = chatmail_config.mail_domain
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.connect((chatmail_config.mail_domain, 25))
|
||||
sock.settimeout(10)
|
||||
try:
|
||||
sock.connect((domain, 25))
|
||||
except socket.timeout:
|
||||
pytest.skip(f"port 25 not reachable for {domain}")
|
||||
banner = sock.recv(1024)
|
||||
print(banner)
|
||||
sock.send(b"VRFY wrongaddress@%s\r\n" % (chatmail_config.mail_domain.encode(),))
|
||||
|
||||
@@ -55,11 +55,12 @@ class TestSSHExecutor:
|
||||
|
||||
def test_opendkim_restarted(self, sshexec):
|
||||
"""check that opendkim is not running for longer than a day."""
|
||||
out = sshexec(call=remote.rshell.shell, kwargs=dict(command="systemctl status opendkim"))
|
||||
assert type(out) == str
|
||||
since_date_str = out.split("since ")[1].split(";")[0]
|
||||
since_date = datetime.datetime.strptime(since_date_str, "%a %Y-%m-%d %H:%M:%S %Z")
|
||||
assert (datetime.datetime.now() - since_date).total_seconds() < 60 * 60 * 24
|
||||
cmd = "systemctl show opendkim --timestamp=utc --property=ActiveEnterTimestamp"
|
||||
out = sshexec(call=remote.rshell.shell, kwargs=dict(command=cmd))
|
||||
datestring = out.split("=")[1]
|
||||
since_date = datetime.datetime.strptime(datestring, "%a %Y-%m-%d %H:%M:%S %Z")
|
||||
now = datetime.datetime.now(since_date.tzinfo)
|
||||
assert (now - since_date).total_seconds() < 60 * 60 * 51
|
||||
|
||||
|
||||
def test_remote(remote, imap_or_smtp):
|
||||
@@ -116,10 +117,14 @@ def test_authenticated_from(cmsetup, maildata):
|
||||
|
||||
@pytest.mark.parametrize("from_addr", ["fake@example.org", "fake@testrun.org"])
|
||||
def test_reject_missing_dkim(cmsetup, maildata, from_addr):
|
||||
"""Test that emails with missing or wrong DMARC, DKIM, and SPF entries are rejected."""
|
||||
recipient = cmsetup.gen_users(1)[0]
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=recipient.addr).as_string()
|
||||
with smtplib.SMTP(cmsetup.maildomain, 25) as s:
|
||||
msg = maildata("encrypted.eml", from_addr=from_addr, to_addr=recipient.addr).as_string()
|
||||
try:
|
||||
conn = smtplib.SMTP(cmsetup.maildomain, 25, timeout=10)
|
||||
except TimeoutError:
|
||||
pytest.skip(f"port 25 not reachable for {cmsetup.maildomain}")
|
||||
|
||||
with conn as s:
|
||||
with pytest.raises(smtplib.SMTPDataError, match="No valid DKIM signature"):
|
||||
s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg)
|
||||
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import ipaddress
|
||||
import random
|
||||
import re
|
||||
import time
|
||||
|
||||
@@ -7,6 +6,9 @@ import imap_tools
|
||||
import pytest
|
||||
import requests
|
||||
|
||||
from cmdeploy.remote import rshell
|
||||
from cmdeploy.sshexec import SSHExec
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def imap_mailbox(cmfactory):
|
||||
@@ -54,22 +56,23 @@ class TestEndToEndDeltaChat:
|
||||
"""Test that a DC account can send a message to a second DC account
|
||||
on the same chat-mail instance."""
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
lp.sec("ac1: prepare and send text message to ac2")
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
chat.send_text("message0")
|
||||
|
||||
lp.sec("wait for ac2 to receive message")
|
||||
msg2 = ac2._evtracker.wait_next_incoming_message()
|
||||
assert msg2.text == "message0"
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_exceed_quota(self, cmfactory, lp, tmpdir, remote, chatmail_config):
|
||||
def test_exceed_quota(
|
||||
self, cmfactory, lp, tmpdir, remote, chatmail_config, sshdomain
|
||||
):
|
||||
"""This is a very slow test as it needs to upload >100MB of mail data
|
||||
before quota is exceeded, and thus depends on the speed of the upload.
|
||||
"""
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
|
||||
user = ac2.get_config("configured_addr")
|
||||
|
||||
def parse_size_limit(limit: str) -> int:
|
||||
"""Parse a size limit and return the number of bytes as integer.
|
||||
@@ -82,49 +85,27 @@ class TestEndToEndDeltaChat:
|
||||
return int(float(number) * units[unit])
|
||||
|
||||
quota = parse_size_limit(chatmail_config.max_mailbox_size)
|
||||
attachsize = 1 * 1024 * 1024
|
||||
num_to_send = quota // attachsize + 2
|
||||
lp.sec(f"ac1: send {num_to_send} large files to ac2")
|
||||
lp.indent(f"per-user quota is assumed to be: {quota / (1024 * 1024)}MB")
|
||||
alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
|
||||
msgs = []
|
||||
for i in range(num_to_send):
|
||||
attachment = tmpdir / f"attachment{i}"
|
||||
data = "".join(random.choice(alphanumeric) for i in range(1024))
|
||||
with open(attachment, "w+") as f:
|
||||
for j in range(attachsize // len(data)):
|
||||
f.write(data)
|
||||
|
||||
msg = chat.send_file(str(attachment))
|
||||
msgs.append(msg)
|
||||
lp.indent(f"Sent out msg {i}, size {attachsize / (1024 * 1024)}MB")
|
||||
lp.sec(f"filling remote inbox for {user}")
|
||||
fn = f"7743102289.M843172P2484002.c20,S={quota},W=2398:2,"
|
||||
path = chatmail_config.mailboxes_dir.joinpath(user, "cur", fn)
|
||||
sshexec = SSHExec(sshdomain)
|
||||
sshexec(call=rshell.write_numbytes, kwargs=dict(path=str(path), num=120))
|
||||
res = sshexec(call=rshell.dovecot_recalc_quota, kwargs=dict(user=user))
|
||||
assert res["percent"] >= 100
|
||||
|
||||
lp.sec("ac2: check messages are arriving until quota is reached")
|
||||
lp.sec("ac2: check quota is triggered")
|
||||
|
||||
addr = ac2.get_config("addr").lower()
|
||||
saved_ok = 0
|
||||
starting = True
|
||||
for line in remote.iter_output("journalctl -n0 -f -u dovecot"):
|
||||
if addr not in line:
|
||||
if starting:
|
||||
chat.send_text("hello")
|
||||
starting = False
|
||||
if user not in line:
|
||||
# print(line)
|
||||
continue
|
||||
if "quota" in line:
|
||||
if "quota exceeded" in line:
|
||||
if saved_ok < num_to_send // 2:
|
||||
pytest.fail(
|
||||
f"quota exceeded too early: after {saved_ok} messages already"
|
||||
)
|
||||
lp.indent("good, message sending failed because quota was exceeded")
|
||||
return
|
||||
if (
|
||||
"stored mail into mailbox 'inbox'" in line
|
||||
or "saved mail to inbox" in line
|
||||
):
|
||||
saved_ok += 1
|
||||
print(f"{saved_ok}: {line}")
|
||||
if saved_ok >= num_to_send:
|
||||
break
|
||||
|
||||
pytest.fail("sending succeeded although messages should exceed quota")
|
||||
if "quota exceeded" in line:
|
||||
return
|
||||
|
||||
def test_securejoin(self, cmfactory, lp, maildomain2):
|
||||
ac1 = cmfactory.new_online_configuring_account(cache=False)
|
||||
@@ -172,7 +153,7 @@ def test_hide_senders_ip_address(cmfactory):
|
||||
assert ipaddress.ip_address(public_ip)
|
||||
|
||||
user1, user2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_accepted_chat(user1, user2)
|
||||
chat = cmfactory.get_protected_chat(user1, user2)
|
||||
|
||||
chat.send_text("testing submission header cleanup")
|
||||
user2._evtracker.wait_next_incoming_message()
|
||||
@@ -181,11 +162,18 @@ def test_hide_senders_ip_address(cmfactory):
|
||||
assert public_ip not in msg.obj.as_string()
|
||||
|
||||
|
||||
def test_echobot(cmfactory, chatmail_config, lp):
|
||||
def test_echobot(cmfactory, chatmail_config, lp, sshdomain):
|
||||
ac = cmfactory.get_online_accounts(1)[0]
|
||||
|
||||
lp.sec(f"Send message to echo@{chatmail_config.mail_domain}")
|
||||
chat = ac.create_chat(f"echo@{chatmail_config.mail_domain}")
|
||||
# establish contact with echobot
|
||||
sshexec = SSHExec(sshdomain)
|
||||
command = "cat /var/lib/echobot/invite-link.txt"
|
||||
echo_invite_link = sshexec(call=rshell.shell, kwargs=dict(command=command))
|
||||
chat = ac.qr_setup_contact(echo_invite_link)
|
||||
ac._evtracker.wait_securejoin_joiner_progress(1000)
|
||||
|
||||
# send message and check it gets replied back
|
||||
lp.sec("Send message to echobot")
|
||||
text = "hi, I hope you text me back"
|
||||
chat.send_text(text)
|
||||
lp.sec("Wait for reply from echobot")
|
||||
|
||||
@@ -62,7 +62,7 @@ def sshdomain(maildomain):
|
||||
def maildomain2():
|
||||
domain = os.environ.get("CHATMAIL_DOMAIN2")
|
||||
if not domain:
|
||||
pytest.skip("set CHATMAIL_DOMAIN2 to a ssh-reachable chatmail instance")
|
||||
pytest.skip("set CHATMAIL_DOMAIN2 to a second chatmail server")
|
||||
return domain
|
||||
|
||||
|
||||
@@ -302,10 +302,12 @@ def cmfactory(request, gencreds, tmpdir, maildomain):
|
||||
pytest.importorskip("deltachat")
|
||||
from deltachat.testplugin import ACFactory
|
||||
|
||||
data = request.getfixturevalue("data")
|
||||
|
||||
testproc = ChatmailTestProcess(request.config, maildomain, gencreds)
|
||||
am = ACFactory(request=request, tmpdir=tmpdir, testprocess=testproc, data=data)
|
||||
|
||||
class Data:
|
||||
def read_path(self, path):
|
||||
return
|
||||
am = ACFactory(request=request, tmpdir=tmpdir, testprocess=testproc, data=Data())
|
||||
|
||||
# nb. a bit hacky
|
||||
# would probably be better if deltachat's test machinery grows native support
|
||||
|
||||
@@ -6,29 +6,6 @@ interoperable e-mail service for everyone. What's behind a `chatmail` is
|
||||
effectively a normal e-mail address just like any other but optimized
|
||||
for the usage in chats, especially DeltaChat.
|
||||
|
||||
### Choosing a chatmail address instead of using a random one
|
||||
|
||||
In the Delta Chat account setup you may tap `Create a profile` then `Use other server` and choose `Classic e-mail login`. Here fill the two fields like this:
|
||||
|
||||
- `E-Mail Address`: invent a word with
|
||||
{% if username_min_length == username_max_length %}
|
||||
*exactly* {{ username_min_length }}
|
||||
{% else %}
|
||||
{{ username_min_length}}
|
||||
{% if username_max_length == "more" %}
|
||||
or more
|
||||
{% else %}
|
||||
to {{ username_max_length }}
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
characters
|
||||
and append `@{{config.mail_domain}}` to it.
|
||||
|
||||
- `Existing Password`: invent at least {{ password_min_length }} characters.
|
||||
|
||||
If the e-mail address is not yet taken, you'll get that account.
|
||||
The first login sets your password.
|
||||
|
||||
|
||||
### Rate and storage limits
|
||||
|
||||
@@ -38,10 +15,11 @@ The first login sets your password.
|
||||
|
||||
- You may send up to {{ config.max_user_send_per_minute }} messages per minute.
|
||||
|
||||
- Messages are unconditionally removed {{ config.delete_mails_after }} days after arriving on the server.
|
||||
|
||||
- You can store up to [{{ config.max_mailbox_size }} messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server).
|
||||
|
||||
- Messages are unconditionally removed latest {{ config.delete_mails_after }} days after arriving on the server.
|
||||
Earlier, if storage may exceed otherwise.
|
||||
|
||||
|
||||
### <a name="account-deletion"></a> Account deletion
|
||||
|
||||
|
||||
Reference in New Issue
Block a user