mirror of
https://github.com/chatmail/relay.git
synced 2026-05-10 16:04:37 +00:00
Compare commits
1 Commits
postfix-lo
...
link2xt/re
| Author | SHA1 | Date | |
|---|---|---|---|
|
9c4da90377 |
32
.github/ISSUE_TEMPLATE/bug_report.md
vendored
32
.github/ISSUE_TEMPLATE/bug_report.md
vendored
@@ -1,32 +0,0 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Report something that isn't working.
|
||||
title: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
Please fill out as much of this form as you can (leaving out stuff that is not applicable is ok).
|
||||
-->
|
||||
|
||||
- Server OS (Operating System) - preferably Debian 12:
|
||||
- On which OS you run cmdeploy:
|
||||
|
||||
## Expected behavior
|
||||
|
||||
*What did you try to achieve?*
|
||||
|
||||
## Actual behavior
|
||||
|
||||
*What happened instead?*
|
||||
|
||||
### Steps to reproduce the problem:
|
||||
|
||||
1.
|
||||
2.
|
||||
|
||||
### Screenshots
|
||||
|
||||
### Logs
|
||||
|
||||
5
.github/ISSUE_TEMPLATE/config.yml
vendored
5
.github/ISSUE_TEMPLATE/config.yml
vendored
@@ -1,5 +0,0 @@
|
||||
blank_issues_enabled: true
|
||||
contact_links:
|
||||
- name: Mutual Help Chat Group
|
||||
url: https://i.delta.chat/#C2846EB4C1CB8DF84B1818F5E3A638FC3FBDC981&a=stalebot1%40nine.testrun.org&g=Chatmail%20Mutual%20Help&x=7sFF7Ik50pWv6J1z7RVC5527&i=d7s1HvOsk5UrSf9AoqRZggg4&s=XmX_9BAW6-g5Ao5E8PyaeKNB
|
||||
about: If you have troubles setting up the relay server, feel free to ask here.
|
||||
34
CHANGELOG.md
34
CHANGELOG.md
@@ -2,40 +2,6 @@
|
||||
|
||||
## untagged
|
||||
|
||||
- Avoid "acmetool not found" during initial run
|
||||
([#550](https://github.com/chatmail/relay/pull/550))
|
||||
|
||||
- Enforce end-to-end encryption for incoming messages.
|
||||
New user address mailboxes now get a `enforceE2EEincoming` file
|
||||
which prohibits incoming cleartext messages from other domains.
|
||||
An outside MTA trying to submit a cleartext message will
|
||||
get a "523 Encryption Needed" response, see RFC5248.
|
||||
If the file does not exist (as it the case for all existing accounts)
|
||||
incoming cleartext messages are accepted.
|
||||
([#538](https://github.com/chatmail/server/pull/538))
|
||||
|
||||
- Enforce end-to-end encryption between local addresses
|
||||
([#535](https://github.com/chatmail/server/pull/535))
|
||||
|
||||
- Limit the bind for the HTTPS server on 8443 to 127.0.0.1
|
||||
([#522](https://github.com/chatmail/server/pull/522))
|
||||
([#532](https://github.com/chatmail/server/pull/532))
|
||||
|
||||
- Send SNI when connecting to outside servers
|
||||
([#524](https://github.com/chatmail/server/pull/524))
|
||||
|
||||
- postfix master.cf: use 127.0.0.1 for consistency
|
||||
([#544](https://github.com/chatmail/relay/pull/544))
|
||||
|
||||
- Pass through `original_content` instead of `content` in filtermail
|
||||
([#509](https://github.com/chatmail/server/pull/509))
|
||||
|
||||
- Document TLS requirements in the readme
|
||||
([#514](https://github.com/chatmail/server/pull/514))
|
||||
|
||||
- Remove cleanup service from submission ports
|
||||
([#512](https://github.com/chatmail/server/pull/512))
|
||||
|
||||
- cmdeploy dovecot: delete big messages after 7 days
|
||||
([#504](https://github.com/chatmail/server/pull/504))
|
||||
|
||||
|
||||
357
README.md
357
README.md
@@ -1,96 +1,48 @@
|
||||
|
||||
<img width="800px" src="www/src/collage-top.png"/>
|
||||
|
||||
# Chatmail relays for end-to-end encrypted e-mail
|
||||
# Chatmail services optimized for Delta Chat apps
|
||||
|
||||
Chatmail relay servers are interoperable Mail Transport Agents (MTAs) designed for:
|
||||
|
||||
- **Convenience:** Low friction instant onboarding
|
||||
|
||||
- **Privacy:** No name, phone numbers, email required or collected
|
||||
|
||||
- **End-to-End Encryption enforced**: only OpenPGP messages with metadata minimization allowed
|
||||
|
||||
- **Instant:** Privacy-preserving Push Notifications for Apple, Google, and Huawei
|
||||
|
||||
- **Speed:** Message delivery in half a second, with optional P2P realtime connections
|
||||
|
||||
- **Transport Security:** Strict TLS and DKIM enforced
|
||||
|
||||
- **Reliability:** No spam or IP reputation checks; rate-limits are suitable for realtime chats
|
||||
|
||||
- **Efficiency:** Messages are only stored for transit and removed automatically
|
||||
|
||||
This repository contains everything needed to setup a ready-to-use chatmail relay
|
||||
This repository helps to setup a ready-to-use chatmail server
|
||||
comprised of a minimal setup of the battle-tested
|
||||
[Postfix SMTP](https://www.postfix.org) and [Dovecot IMAP](https://www.dovecot.org) MTAs/MDAs.
|
||||
[postfix smtp](https://www.postfix.org) and [dovecot imap](https://www.dovecot.org) services.
|
||||
|
||||
The automated setup is designed and optimized for providing chatmail addresses
|
||||
for immediate permission-free onboarding through chat apps and bots.
|
||||
Chatmail addresses are automatically created at first login,
|
||||
after which the initially specified password is required
|
||||
for sending and receiving messages through them.
|
||||
The setup is designed and optimized for providing chatmail accounts
|
||||
for use by [Delta Chat apps](https://delta.chat).
|
||||
|
||||
Please see [this list of known apps and client projects](https://chatmail.at/clients.html)
|
||||
and [this list of known public 3rd party chatmail relay servers](https://chatmail.at/relays).
|
||||
Chatmail accounts are automatically created by a first login,
|
||||
after which the initially specified password is required for using them.
|
||||
|
||||
## Deploying your own chatmail server
|
||||
|
||||
## Minimal requirements, Prerequisites
|
||||
To deploy chatmail on your own server, you must have set-up ssh authentication and need to use an ed25519 key, due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191). You also need to add your private key to the local ssh-agent, because you can't type in your password during deployment.
|
||||
|
||||
You will need the following:
|
||||
|
||||
- Control over a domain through a DNS provider of your choice.
|
||||
|
||||
- A Debian 12 server with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
|
||||
IPv6 is encouraged if available.
|
||||
Chatmail relay servers only require 1GB RAM, one CPU, and perhaps 10GB storage for a
|
||||
few thousand active chatmail addresses.
|
||||
|
||||
- Key-based SSH authentication to the root user.
|
||||
You must add a passphrase-protected private key to your local ssh-agent
|
||||
because you can't type in your passphrase during deployment.
|
||||
(An ed25519 private key is required due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191))
|
||||
|
||||
|
||||
## Getting started
|
||||
|
||||
We use `chat.example.org` as the chatmail domain in the following steps.
|
||||
We use `chat.example.org` as the chatmail domain in the following steps.
|
||||
Please substitute it with your own domain.
|
||||
|
||||
1. Setup the initial DNS records.
|
||||
The following is an example in the familiar BIND zone file format with
|
||||
a TTL of 1 hour (3600 seconds).
|
||||
Please substitute your domain and IP addresses.
|
||||
1. Install the `cmdeploy` command in a virtualenv
|
||||
|
||||
```
|
||||
chat.example.com. 3600 IN A 198.51.100.5
|
||||
chat.example.com. 3600 IN AAAA 2001:db8::5
|
||||
www.chat.example.com. 3600 IN CNAME chat.example.com.
|
||||
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
|
||||
```
|
||||
|
||||
2. Clone the repository and bootstrap the Python virtualenv.
|
||||
|
||||
```
|
||||
git clone https://github.com/chatmail/relay
|
||||
cd relay
|
||||
git clone https://github.com/deltachat/chatmail
|
||||
cd chatmail
|
||||
scripts/initenv.sh
|
||||
```
|
||||
|
||||
3. Create chatmail configuration file `chatmail.ini`:
|
||||
2. Create chatmail configuration file `chatmail.ini`:
|
||||
|
||||
```
|
||||
scripts/cmdeploy init chat.example.org # <-- use your domain
|
||||
```
|
||||
|
||||
4. Verify that SSH root login works:
|
||||
3. Point your domain to the server's IP address,
|
||||
if you haven't done so already.
|
||||
Verify that SSH root login works:
|
||||
|
||||
```
|
||||
ssh root@chat.example.org # <-- use your domain
|
||||
```
|
||||
|
||||
|
||||
5. Deploy the remote chatmail relay server:
|
||||
4. Deploy to the remote chatmail server:
|
||||
|
||||
```
|
||||
scripts/cmdeploy run
|
||||
@@ -130,25 +82,25 @@ scripts/cmdeploy bench
|
||||
|
||||
This repository has four directories:
|
||||
|
||||
- [cmdeploy](https://github.com/chatmail/relay/tree/main/cmdeploy)
|
||||
- [cmdeploy](https://github.com/deltachat/chatmail/tree/main/cmdeploy)
|
||||
is a collection of configuration files
|
||||
and a [pyinfra](https://pyinfra.com)-based deployment script.
|
||||
|
||||
- [chatmaild](https://github.com/chatmail/relay/tree/main/chatmaild)
|
||||
is a Python package containing several small services
|
||||
- [chatmaild](https://github.com/deltachat/chatmail/tree/main/chatmaild)
|
||||
is a python package containing several small services
|
||||
which handle authentication,
|
||||
trigger push notifications on new messages,
|
||||
ensure that outbound mails are encrypted,
|
||||
delete inactive users,
|
||||
and some other minor things.
|
||||
chatmaild can also be installed as a stand-alone Python package.
|
||||
chatmaild can also be installed as a stand-alone python package.
|
||||
|
||||
- [www](https://github.com/chatmail/relay/tree/main/www)
|
||||
- [www](https://github.com/deltachat/chatmail/tree/main/www)
|
||||
contains the html, css, and markdown files
|
||||
which make up a chatmail relay's web page.
|
||||
Edit them before deploying to make your chatmail relay stand out.
|
||||
which make up a chatmail server's web page.
|
||||
Edit them before deploying to make your chatmail server stand out.
|
||||
|
||||
- [scripts](https://github.com/chatmail/relay/tree/main/scripts)
|
||||
- [scripts](https://github.com/deltachat/chatmail/tree/main/scripts)
|
||||
offers two convenience tools for beginners;
|
||||
`initenv.sh` installs the necessary dependencies to a local virtual environment,
|
||||
and the `scripts/cmdeploy` script enables you
|
||||
@@ -160,81 +112,79 @@ The `cmdeploy/src/cmdeploy/cmdeploy.py` command line tool
|
||||
helps with setting up and managing the chatmail service.
|
||||
`cmdeploy init` creates the `chatmail.ini` config file.
|
||||
`cmdeploy run` uses a [pyinfra](https://pyinfra.com/)-based [script](`cmdeploy/src/cmdeploy/__init__.py`)
|
||||
to automatically install or upgrade all chatmail components on a relay,
|
||||
to automatically install or upgrade all chatmail components on a server,
|
||||
according to the `chatmail.ini` config.
|
||||
|
||||
The components of chatmail are:
|
||||
|
||||
- [Postfix SMTP MTA](https://www.postfix.org) accepts and relays messages
|
||||
(both from your users and from the wider e-mail MTA network)
|
||||
- [postfix smtp server](https://www.postfix.org) accepts sent messages (both from your users and from other servers)
|
||||
|
||||
- [Dovecot IMAP MDA](https://www.dovecot.org) stores messages for your users until they download them
|
||||
- [dovecot imap server](https://www.dovecot.org) stores messages for your users until they download them
|
||||
|
||||
- [Nginx](https://nginx.org/) shows the web page with your privacy policy and additional information
|
||||
- [nginx](https://nginx.org/) shows the web page with your privacy policy and additional information
|
||||
|
||||
- [acmetool](https://hlandau.github.io/acmetool/) manages TLS certificates for Dovecot, Postfix, and Nginx
|
||||
- [acmetool](https://hlandau.github.io/acmetool/) manages TLS certificates for dovecot, postfix, and nginx
|
||||
|
||||
- [OpenDKIM](http://www.opendkim.org/) for signing messages with DKIM and rejecting inbound messages without DKIM
|
||||
- [opendkim](http://www.opendkim.org/) for signing messages with DKIM and rejecting inbound messages without DKIM
|
||||
|
||||
- [mtail](https://google.github.io/mtail/) for collecting anonymized metrics in case you have monitoring
|
||||
|
||||
- [Iroh relay](https://www.iroh.computer/docs/concepts/relay)
|
||||
which helps client devices to establish Peer-to-Peer connections
|
||||
|
||||
- and the chatmaild services, explained in the next section:
|
||||
|
||||
### chatmaild
|
||||
|
||||
`chatmaild` implements various systemd-controlled services
|
||||
that integrate with Dovecot and Postfix to achieve instant-onboarding and
|
||||
only relaying OpenPGP end-to-end messages encrypted messages.
|
||||
A short overview of `chatmaild` services:
|
||||
chatmaild offers several commands
|
||||
which differentiate a *chatmail* server from a classic mail server.
|
||||
If you deploy them with cmdeploy,
|
||||
they are run by systemd services in the background.
|
||||
A short overview:
|
||||
|
||||
- [`doveauth`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/doveauth.py)
|
||||
implements create-on-login address semantics and is used
|
||||
by Dovecot during IMAP login and by Postfix during SMTP/SUBMISSION login
|
||||
- [`doveauth`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/doveauth.py) implements
|
||||
create-on-login account creation semantics and is used
|
||||
by Dovecot during login authentication and by Postfix
|
||||
which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)
|
||||
to authenticate logins.
|
||||
to authenticate users
|
||||
to send mails for them.
|
||||
|
||||
- [`filtermail`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/filtermail.py)
|
||||
prevents unencrypted email from leaving or entering the chatmail service
|
||||
and is integrated into Postfix's outbound and inbound mail pipelines.
|
||||
- [`filtermail`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/filtermail.py) prevents
|
||||
unencrypted e-mail from leaving the chatmail service
|
||||
and is integrated into postfix's outbound mail pipelines.
|
||||
|
||||
- [`chatmail-metadata`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/metadata.py) is contacted by a
|
||||
[Dovecot lua script](https://github.com/chatmail/relay/blob/main/cmdeploy/src/cmdeploy/dovecot/push_notification.lua)
|
||||
to store user-specific relay-side config.
|
||||
- [`chatmail-metadata`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/metadata.py) is contacted by a
|
||||
[dovecot lua script](https://github.com/deltachat/chatmail/blob/main/cmdeploy/src/cmdeploy/dovecot/push_notification.lua)
|
||||
to store user-specific server-side config.
|
||||
On new messages,
|
||||
it [passes the user's push notification token](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/notifier.py)
|
||||
it [passes the user's push notification token](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/notifier.py)
|
||||
to [notifications.delta.chat](https://delta.chat/help#instant-delivery)
|
||||
so the push notifications on the user's phone can be triggered
|
||||
by Apple/Google/Huawei.
|
||||
by Apple/Google.
|
||||
|
||||
- [`delete_inactive_users`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/delete_inactive_users.py)
|
||||
- [`delete_inactive_users`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/delete_inactive_users.py)
|
||||
deletes users if they have not logged in for a very long time.
|
||||
The timeframe can be configured in `chatmail.ini`.
|
||||
|
||||
- [`lastlogin`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/lastlogin.py)
|
||||
is contacted by Dovecot when a user logs in
|
||||
- [`lastlogin`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/lastlogin.py)
|
||||
is contacted by dovecot when a user logs in
|
||||
and stores the date of the login.
|
||||
|
||||
- [`echobot`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/echo.py)
|
||||
- [`echobot`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/echo.py)
|
||||
is a small bot for test purposes.
|
||||
It simply echoes back messages from users.
|
||||
|
||||
- [`chatmail-metrics`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/metrics.py)
|
||||
- [`chatmail-metrics`](https://github.com/deltachat/chatmail/blob/main/chatmaild/src/chatmaild/metrics.py)
|
||||
collects some metrics and displays them at `https://example.org/metrics`.
|
||||
|
||||
### Home page and getting started for users
|
||||
|
||||
`cmdeploy run` also creates default static web pages and deploys them
|
||||
to a Nginx web server with:
|
||||
`cmdeploy run` also creates default static Web pages and deploys them
|
||||
to a nginx web server with:
|
||||
|
||||
- a default `index.html` along with a QR code that users can click to
|
||||
create an address on your chatmail relay
|
||||
create accounts on your chatmail provider,
|
||||
|
||||
- a default `info.html` that is linked from the home page
|
||||
- a default `info.html` that is linked from the home page,
|
||||
|
||||
- a default `policy.html` that is linked from the home page
|
||||
- a default `policy.html` that is linked from the home page.
|
||||
|
||||
All `.html` files are generated
|
||||
by the according markdown `.md` file in the `www/src` directory.
|
||||
@@ -242,64 +192,48 @@ by the according markdown `.md` file in the `www/src` directory.
|
||||
|
||||
### Refining the web pages
|
||||
|
||||
|
||||
```
|
||||
scripts/cmdeploy webdev
|
||||
```
|
||||
|
||||
This starts a local live development cycle for chatmail web pages:
|
||||
This starts a local live development cycle for chatmail Web pages:
|
||||
|
||||
- uses the `www/src/page-layout.html` file for producing static
|
||||
HTML pages from `www/src/*.md` files
|
||||
|
||||
- continously builds the web presence reading files from `www/src` directory
|
||||
and generating HTML files and copying assets to the `www/build` directory.
|
||||
and generating html files and copying assets to the `www/build` directory.
|
||||
|
||||
- Starts a browser window automatically where you can "refresh" as needed.
|
||||
|
||||
## Mailbox directory layout
|
||||
|
||||
Fresh chatmail addresses have a mailbox directory that contains:
|
||||
## Emergency Commands to disable automatic account creation
|
||||
|
||||
- a `password` file with the salted password required for authenticating
|
||||
whether a login may use the address to send/receive messages.
|
||||
If you modify the password file manually, you effectively block the user.
|
||||
|
||||
- `enforceE2EEincoming` is a default-created file with each address.
|
||||
If present the file indicates that this chatmail address rejects incoming cleartext messages.
|
||||
If absent the address accepts incoming cleartext messages.
|
||||
|
||||
- `dovecot*`, `cur`, `new` and `tmp` represent IMAP/mailbox state.
|
||||
If the address is only used by one device, the Maildir directories
|
||||
will typically be empty unless the user of that address hasn't been online
|
||||
for a while.
|
||||
|
||||
|
||||
## Emergency Commands to disable automatic address creation
|
||||
|
||||
If you need to stop address creation,
|
||||
e.g. because some script is wildly creating addresses,
|
||||
login with ssh and run:
|
||||
If you need to stop account creation,
|
||||
e.g. because some script is wildly creating accounts,
|
||||
login to the server with ssh and run:
|
||||
|
||||
```
|
||||
touch /etc/chatmail-nocreate
|
||||
```
|
||||
|
||||
Chatmail address creation will be denied while this file is present.
|
||||
While this file is present, account creation will be blocked.
|
||||
|
||||
### Ports
|
||||
|
||||
[Postfix](http://www.postfix.org/) listens on ports 25 (SMTP) and 587 (SUBMISSION) and 465 (SUBMISSIONS).
|
||||
[Dovecot](https://www.dovecot.org/) listens on ports 143 (IMAP) and 993 (IMAPS).
|
||||
[Nginx](https://www.nginx.com/) listens on port 8443 (HTTPS-ALT) and 443 (HTTPS).
|
||||
[Postfix](http://www.postfix.org/) listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
|
||||
[Dovecot](https://www.dovecot.org/) listens on ports 143 (imap) and 993 (imaps).
|
||||
[nginx](https://www.nginx.com/) listens on port 8443 (https-alt) and 443 (https).
|
||||
Port 443 multiplexes HTTPS, IMAP and SMTP using ALPN to redirect connections to ports 8443, 465 or 993.
|
||||
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (HTTP).
|
||||
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (http).
|
||||
|
||||
chatmail-core based apps will, however, discover all ports and configurations
|
||||
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail relay server.
|
||||
Delta Chat apps will, however, discover all ports and configurations
|
||||
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail service.
|
||||
|
||||
## Email authentication
|
||||
|
||||
Chatmail relays enforce [DKIM](https://www.rfc-editor.org/rfc/rfc6376)
|
||||
chatmail servers rely on [DKIM](https://www.rfc-editor.org/rfc/rfc6376)
|
||||
to authenticate incoming emails.
|
||||
Incoming emails must have a valid DKIM signature with
|
||||
Signing Domain Identifier (SDID, `d=` parameter in the DKIM-Signature header)
|
||||
@@ -322,130 +256,101 @@ and rejects incorrectly authenticated emails with [`reject_sender_login_mismatch
|
||||
`From:` header must correspond to envelope MAIL FROM,
|
||||
this is ensured by `filtermail` proxy.
|
||||
|
||||
## TLS requirements
|
||||
## Migrating chatmail server to a new host
|
||||
|
||||
Postfix is configured to require valid TLS
|
||||
by setting [`smtp_tls_security_level`](https://www.postfix.org/postconf.5.html#smtp_tls_security_level) to `verify`.
|
||||
If emails don't arrive at your chatmail relay server,
|
||||
the problem is likely that your relay does not have a valid TLS certificate.
|
||||
|
||||
You can test it by resolving `MX` records of your relay domain
|
||||
and then connecting to MX relays (e.g `mx.example.org`) with
|
||||
`openssl s_client -connect mx.example.org:25 -verify_hostname mx.example.org -verify_return_error -starttls smtp`
|
||||
from the host that has open port 25 to verify that certificate is valid.
|
||||
|
||||
When providing a TLS certificate to your chatmail relay server,
|
||||
make sure to provide the full certificate chain
|
||||
and not just the last certificate.
|
||||
|
||||
If you are running an Exim server and don't see incoming connections
|
||||
from a chatmail relay server in the logs,
|
||||
make sure `smtp_no_mail` log item is enabled in the config
|
||||
with `log_selector = +smtp_no_mail`.
|
||||
By default Exim does not log sessions that are closed
|
||||
before sending the `MAIL` command.
|
||||
This happens if certificate is not recognized as valid by Postfix,
|
||||
so you might think that connection is not established
|
||||
while actually it is a problem with your TLS certificate.
|
||||
|
||||
## Migrating a chatmail relay to a new host
|
||||
|
||||
If you want to migrate chatmail relay from an old machine
|
||||
If you want to migrate chatmail from an old machine
|
||||
to a new machine,
|
||||
you can use these steps.
|
||||
They were tested with a Linux laptop;
|
||||
They were tested with a linux laptop;
|
||||
you might need to adjust some of the steps to your environment.
|
||||
|
||||
Let's assume that your `mail_domain` is `mail.example.org`,
|
||||
all involved machines run Debian 12,
|
||||
your old site's IP address is `13.37.13.37`,
|
||||
and your new site's IP address is `13.12.23.42`.
|
||||
your old server's IP address is `13.37.13.37`,
|
||||
and your new server's IP address is `13.12.23.42`.
|
||||
|
||||
Note, you should lower the TTLs of your DNS records to a value
|
||||
such as 300 (5 minutes) so the migration happens as smoothly as possible.
|
||||
During the guide, you might get a warning about changed SSH Host keys;
|
||||
in this case, just run `ssh-keygen -R "mail.example.org"` as recommended
|
||||
to make sure you can connect with SSH.
|
||||
|
||||
During the guide you might get a warning about changed SSH Host keys;
|
||||
in this case, just run `ssh-keygen -R "mail.example.org"` as recommended.
|
||||
1. First, copy `/var/lib/acme` to the new server with
|
||||
`ssh root@13.37.13.37 tar c /var/lib/acme | ssh root@13.12.23.42 tar x -C /var/lib/`.
|
||||
This transfers your TLS certificate.
|
||||
|
||||
1. First, disable mail services on the old site.
|
||||
2. You should also copy `/etc/dkimkeys` to the new server with
|
||||
`ssh root@13.37.13.37 tar c /etc/dkimkeys | ssh root@13.12.23.42 tar x -C /etc/`
|
||||
so the DKIM DNS record stays correct.
|
||||
|
||||
```
|
||||
cmdeploy run --disable-mail --ssh-host 13.37.13.37
|
||||
```
|
||||
3. On the new server, run `chown root: -R /var/lib/acme` and `chown opendkim: -R /etc/dkimkeys` to make sure the permissions are correct.
|
||||
|
||||
4. Run `cmdeploy run --disable-mail --ssh-host 13.12.23.42` to install chatmail on the new machine.
|
||||
postfix and dovecot are disabled for now,
|
||||
we will enable them later.
|
||||
|
||||
5. Now, point DNS to the new IP addresses.
|
||||
|
||||
You can already remove the old IP addresses from DNS.
|
||||
Existing Delta Chat users will still be able to connect
|
||||
to the old server, send and receive messages,
|
||||
but new users will fail to create new profiles
|
||||
with your chatmail server.
|
||||
|
||||
If other servers try to deliver messages to your new server they will fail,
|
||||
but normally email servers will retry delivering messages
|
||||
for at least a week, so messages will not be lost.
|
||||
|
||||
6. Now you can run `cmdeploy run --disable-mail --ssh-host 13.37.13.37` to disable your old server.
|
||||
|
||||
Now your users will notice the migration
|
||||
and will not be able to send or receive messages
|
||||
until the migration is completed.
|
||||
|
||||
2. Now we want to copy `/home/vmail`, `/var/lib/acme`, `/etc/dkimkeys`, `/run/echobot`, and `/var/spool/postfix` to the new site.
|
||||
Login to the old site while forwarding your SSH agent
|
||||
so you can copy directly from the old to the new site with your SSH key:
|
||||
```
|
||||
ssh -A root@13.37.13.37
|
||||
tar c - /home/vmail/mail /var/lib/acme /etc/dkimkeys /run/echobot /var/spool/postfix | ssh root@13.12.23.42 "tar x -C /"
|
||||
```
|
||||
7. After everything is stopped,
|
||||
you can copy the `/home/vmail/mail` directory to the new server.
|
||||
It includes all user data, messages, password hashes, etc.
|
||||
|
||||
This transfers all addresses, the TLS certificate, DKIM keys (so DKIM DNS record remains valid), and the echobot's password so it continues to function.
|
||||
It also preserves the Postfix mail spool so any messages pending delivery will still be delivered.
|
||||
Just run: `ssh root@13.37.13.37 tar c /home/vmail/mail | ssh root@13.12.23.42 tar x -C /home/vmail/`
|
||||
|
||||
3. Install chatmail on the new machine:
|
||||
After this, your new server has all the necessary files to start operating :)
|
||||
|
||||
```
|
||||
cmdeploy run --disable-mail --ssh-host 13.12.23.42
|
||||
```
|
||||
Postfix and Dovecot are disabled for now; we will enable them later.
|
||||
We first need to make the new site fully operational.
|
||||
8. To be sure the permissions are still fine,
|
||||
run `chown vmail: -R /home/vmail` on the new server.
|
||||
|
||||
3. On the new site, run the following to ensure the ownership is correct in case UIDs/GIDs changed:
|
||||
|
||||
```
|
||||
chown root: -R /var/lib/acme
|
||||
chown opendkim: -R /etc/dkimkeys
|
||||
chown vmail: -R /home/vmail/mail
|
||||
chown echobot: -R /run/echobot
|
||||
```
|
||||
|
||||
4. Now, update DNS entries.
|
||||
|
||||
If other MTAs try to deliver messages to your chatmail domain they may fail intermittently,
|
||||
as DNS catches up with the new site settings
|
||||
but normally will retry delivering messages
|
||||
for at least a week, so messages will not be lost.
|
||||
|
||||
5. Finally, you can execute `cmdeploy run --ssh-host 13.12.23.42` to turn on chatmail on the new relay.
|
||||
Your users will be able to use the chatmail relay as soon as the DNS changes have propagated.
|
||||
Voilà!
|
||||
9. Finally, you can run `cmdeploy run` to turn on chatmail on the new server.
|
||||
Your users can continue using the chatmail server,
|
||||
and messages which were sent after step 6. should arrive now.
|
||||
Voilà!
|
||||
|
||||
## Setting up a reverse proxy
|
||||
|
||||
A chatmail relay MTA does not track or depend on the client IP address
|
||||
A chatmail server does not depend on the client IP address
|
||||
for its operation, so it can be run behind a reverse proxy.
|
||||
This will not even affect incoming mail authentication
|
||||
as DKIM only checks the cryptographic signature
|
||||
of the message and does not use the IP address as the input.
|
||||
|
||||
For example, you may want to self-host your chatmail relay
|
||||
For example, you may want to self-host your chatmail server
|
||||
and only use hosted VPS to provide a public IP address
|
||||
for client connections and incoming mail.
|
||||
You can connect chatmail relay to VPS
|
||||
You can connect chatmail server to VPS
|
||||
using a tunnel protocol
|
||||
such as [WireGuard](https://www.wireguard.com/)
|
||||
and setup a reverse proxy on a VPS
|
||||
to forward connections to the chatmail relay
|
||||
to forward connections to the chatmail server
|
||||
over the tunnel.
|
||||
You can also setup multiple reverse proxies
|
||||
for your chatmail relay in different networks
|
||||
to ensure your relay is reachable even when
|
||||
for your chatmail server in different networks
|
||||
to ensure your server is reachable even when
|
||||
one of the IPs becomes inaccessible due to
|
||||
hosting or routing problems.
|
||||
|
||||
Note that your chatmail relay still needs
|
||||
Note that your server still needs
|
||||
to be able to make outgoing connections on port 25
|
||||
to send messages outside.
|
||||
|
||||
To setup a reverse proxy
|
||||
(or rather Destination NAT, DNAT)
|
||||
for your chatmail relay,
|
||||
for your chatmail server,
|
||||
put the following configuration in `/etc/nftables.conf`:
|
||||
```
|
||||
#!/usr/sbin/nft -f
|
||||
@@ -457,7 +362,7 @@ define wan = eth0
|
||||
# Which ports to proxy.
|
||||
#
|
||||
# Note that SSH is not proxied
|
||||
# so it is possible to log into the proxy server
|
||||
# so it is possible to log into the proxy server
|
||||
# and not the original one.
|
||||
define ports = { smtp, http, https, imap, imaps, submission, submissions }
|
||||
|
||||
@@ -520,7 +425,7 @@ table inet filter {
|
||||
```
|
||||
|
||||
Run `systemctl enable nftables.service`
|
||||
to ensure configuration is reloaded when the proxy relay reboots.
|
||||
to ensure configuration is reloaded when the proxy server reboots.
|
||||
|
||||
Uncomment in `/etc/sysctl.conf` the following two lines:
|
||||
|
||||
@@ -529,7 +434,7 @@ net.ipv4.ip_forward=1
|
||||
net.ipv6.conf.all.forwarding=1
|
||||
```
|
||||
|
||||
Then reboot the relay or do `sysctl -p` and `nft -f /etc/nftables.conf`.
|
||||
Then reboot the server or do `sysctl -p` and `nft -f /etc/nftables.conf`.
|
||||
|
||||
Once proxy relay is set up,
|
||||
Once proxy server is set up,
|
||||
you can add its IP address to the DNS.
|
||||
|
||||
@@ -11,11 +11,7 @@ def read_config(inipath):
|
||||
assert Path(inipath).exists(), inipath
|
||||
cfg = iniconfig.IniConfig(inipath)
|
||||
params = cfg.sections["params"]
|
||||
default_config_content = get_default_config_content(params["mail_domain"])
|
||||
df_params = iniconfig.IniConfig("ini", data=default_config_content)["params"]
|
||||
new_params = dict(df_params.items())
|
||||
new_params.update(params)
|
||||
return Config(inipath, params=new_params)
|
||||
return Config(inipath, params=params)
|
||||
|
||||
|
||||
class Config:
|
||||
@@ -33,13 +29,7 @@ class Config:
|
||||
self.passthrough_senders = params["passthrough_senders"].split()
|
||||
self.passthrough_recipients = params["passthrough_recipients"].split()
|
||||
self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
|
||||
self.filtermail_smtp_port_incoming = int(
|
||||
params["filtermail_smtp_port_incoming"]
|
||||
)
|
||||
self.postfix_reinject_port = int(params["postfix_reinject_port"])
|
||||
self.postfix_reinject_port_incoming = int(
|
||||
params["postfix_reinject_port_incoming"]
|
||||
)
|
||||
self.mtail_address = params.get("mtail_address")
|
||||
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
|
||||
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
|
||||
@@ -79,11 +69,6 @@ class Config:
|
||||
|
||||
def write_initial_config(inipath, mail_domain, overrides):
|
||||
"""Write out default config file, using the specified config value overrides."""
|
||||
content = get_default_config_content(mail_domain, **overrides)
|
||||
inipath.write_text(content)
|
||||
|
||||
|
||||
def get_default_config_content(mail_domain, **overrides):
|
||||
from importlib.resources import files
|
||||
|
||||
inidir = files(__package__).joinpath("ini")
|
||||
@@ -128,4 +113,5 @@ def get_default_config_content(mail_domain, **overrides):
|
||||
else:
|
||||
lines.append(line)
|
||||
content = "\n".join(lines)
|
||||
return content
|
||||
|
||||
inipath.write_text(content)
|
||||
|
||||
@@ -8,7 +8,6 @@ import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from deltachat_rpc_client import Bot, DeltaChat, EventType, Rpc, events
|
||||
|
||||
@@ -98,10 +97,6 @@ def main():
|
||||
if not bot.is_configured():
|
||||
bot.configure(addr, password)
|
||||
|
||||
# write invite link to working directory
|
||||
invitelink = bot.account.get_qr_code()
|
||||
Path("invite-link.txt").write_text(invitelink)
|
||||
|
||||
bot.run_forever()
|
||||
|
||||
|
||||
|
||||
@@ -11,12 +11,9 @@ from email.utils import parseaddr
|
||||
from smtplib import SMTP as SMTPClient
|
||||
|
||||
from aiosmtpd.controller import Controller
|
||||
from aiosmtpd.smtp import SMTP
|
||||
|
||||
from .config import read_config
|
||||
|
||||
ENCRYPTION_NEEDED_523 = "523 Encryption Needed: Invalid Unencrypted Mail"
|
||||
|
||||
|
||||
def check_openpgp_payload(payload: bytes):
|
||||
"""Checks the OpenPGP payload.
|
||||
@@ -160,14 +157,9 @@ def check_encrypted(message):
|
||||
return True
|
||||
|
||||
|
||||
async def asyncmain_beforequeue(config, mode):
|
||||
if mode == "outgoing":
|
||||
port = config.filtermail_smtp_port
|
||||
handler = OutgoingBeforeQueueHandler(config)
|
||||
else:
|
||||
port = config.filtermail_smtp_port_incoming
|
||||
handler = IncomingBeforeQueueHandler(config)
|
||||
HackedController(handler, hostname="127.0.0.1", port=port).start()
|
||||
async def asyncmain_beforequeue(config):
|
||||
port = config.filtermail_smtp_port
|
||||
Controller(BeforeQueueHandler(config), hostname="127.0.0.1", port=port).start()
|
||||
|
||||
|
||||
def recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
@@ -179,21 +171,7 @@ def recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
return False
|
||||
|
||||
|
||||
class HackedController(Controller):
|
||||
def factory(self):
|
||||
return SMTPDiscardRCPTO_options(self.handler, **self.SMTP_kwargs)
|
||||
|
||||
|
||||
class SMTPDiscardRCPTO_options(SMTP):
|
||||
def _getparams(self, params):
|
||||
# aiosmtpd's SMTP daemon fails to handle a request if there are RCPT TO options
|
||||
# We just ignore them for our incoming filtermail purposes
|
||||
if len(params) == 1 and params[0].startswith("ORCPT"):
|
||||
return {}
|
||||
return super()._getparams(params)
|
||||
|
||||
|
||||
class OutgoingBeforeQueueHandler:
|
||||
class BeforeQueueHandler:
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
self.send_rate_limiter = SendRateLimiter()
|
||||
@@ -218,9 +196,7 @@ class OutgoingBeforeQueueHandler:
|
||||
return error
|
||||
logging.info("re-injecting the mail that passed checks")
|
||||
client = SMTPClient("localhost", self.config.postfix_reinject_port)
|
||||
client.sendmail(
|
||||
envelope.mail_from, envelope.rcpt_tos, envelope.original_content
|
||||
)
|
||||
client.sendmail(envelope.mail_from, envelope.rcpt_tos, envelope.content)
|
||||
return "250 OK"
|
||||
|
||||
def check_DATA(self, envelope):
|
||||
@@ -231,86 +207,40 @@ class OutgoingBeforeQueueHandler:
|
||||
mail_encrypted = check_encrypted(message)
|
||||
|
||||
_, from_addr = parseaddr(message.get("from").strip())
|
||||
envelope_from_domain = from_addr.split("@").pop()
|
||||
|
||||
logging.info(f"mime-from: {from_addr} envelope-from: {envelope.mail_from!r}")
|
||||
if envelope.mail_from.lower() != from_addr.lower():
|
||||
return f"500 Invalid FROM <{from_addr!r}> for <{envelope.mail_from!r}>"
|
||||
|
||||
if mail_encrypted or is_securejoin(message):
|
||||
print("Outgoing: Filtering encrypted mail.", file=sys.stderr)
|
||||
return
|
||||
|
||||
print("Outgoing: Filtering unencrypted mail.", file=sys.stderr)
|
||||
if mail_encrypted:
|
||||
print("Filtering encrypted mail.", file=sys.stderr)
|
||||
else:
|
||||
print("Filtering unencrypted mail.", file=sys.stderr)
|
||||
|
||||
if envelope.mail_from in self.config.passthrough_senders:
|
||||
return
|
||||
|
||||
# allow self-sent Autocrypt Setup Message
|
||||
if envelope.rcpt_tos == [from_addr]:
|
||||
if message.get("subject") == "Autocrypt Setup Message":
|
||||
if message.get_content_type() == "multipart/mixed":
|
||||
return
|
||||
|
||||
passthrough_recipients = self.config.passthrough_recipients
|
||||
|
||||
for recipient in envelope.rcpt_tos:
|
||||
if recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
continue
|
||||
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return ENCRYPTION_NEEDED_523
|
||||
|
||||
|
||||
class IncomingBeforeQueueHandler:
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
|
||||
async def handle_DATA(self, server, session, envelope):
|
||||
logging.info("handle_DATA before-queue")
|
||||
error = self.check_DATA(envelope)
|
||||
if error:
|
||||
return error
|
||||
logging.info("re-injecting the mail that passed checks")
|
||||
|
||||
# the smtp daemon on reinject_port_incoming gives it to dkim milter
|
||||
# which looks at source address to determine whether to verify or sign
|
||||
client = SMTPClient(
|
||||
"localhost",
|
||||
self.config.postfix_reinject_port_incoming,
|
||||
source_address=("127.0.0.2", 0),
|
||||
)
|
||||
client.sendmail(
|
||||
envelope.mail_from, envelope.rcpt_tos, envelope.original_content
|
||||
)
|
||||
return "250 OK"
|
||||
|
||||
def check_DATA(self, envelope):
|
||||
"""the central filtering function for e-mails."""
|
||||
logging.info(f"Processing DATA message from {envelope.mail_from}")
|
||||
|
||||
message = BytesParser(policy=policy.default).parsebytes(envelope.content)
|
||||
mail_encrypted = check_encrypted(message)
|
||||
|
||||
if mail_encrypted or is_securejoin(message):
|
||||
print("Incoming: Filtering encrypted mail.", file=sys.stderr)
|
||||
return
|
||||
|
||||
print("Incoming: Filtering unencrypted mail.", file=sys.stderr)
|
||||
|
||||
# we want cleartext mailer-daemon messages to pass through
|
||||
# chatmail core will typically not display them as normal messages
|
||||
if message.get("auto-submitted"):
|
||||
_, from_addr = parseaddr(message.get("from").strip())
|
||||
if from_addr.lower().startswith("mailer-daemon@"):
|
||||
if message.get_content_type() == "multipart/report":
|
||||
return
|
||||
|
||||
for recipient in envelope.rcpt_tos:
|
||||
user = self.config.get_user(recipient)
|
||||
if user is None or user.is_incoming_cleartext_ok():
|
||||
if envelope.mail_from == recipient:
|
||||
# Always allow sending emails to self.
|
||||
continue
|
||||
if recipient_matches_passthrough(recipient, passthrough_recipients):
|
||||
continue
|
||||
res = recipient.split("@")
|
||||
if len(res) != 2:
|
||||
return f"500 Invalid address <{recipient}>"
|
||||
_recipient_addr, recipient_domain = res
|
||||
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return ENCRYPTION_NEEDED_523
|
||||
is_outgoing = recipient_domain != envelope_from_domain
|
||||
if is_outgoing:
|
||||
print("Rejected unencrypted mail.", file=sys.stderr)
|
||||
return f"500 Invalid unencrypted mail to <{recipient}>"
|
||||
|
||||
|
||||
class SendRateLimiter:
|
||||
@@ -329,14 +259,11 @@ class SendRateLimiter:
|
||||
|
||||
def main():
|
||||
args = sys.argv[1:]
|
||||
assert len(args) == 2
|
||||
assert len(args) == 1
|
||||
config = read_config(args[0])
|
||||
mode = args[1]
|
||||
logging.basicConfig(level=logging.WARN)
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
assert mode in ["incoming", "outgoing"]
|
||||
task = asyncmain_beforequeue(config, mode)
|
||||
task = asyncmain_beforequeue(config)
|
||||
loop.create_task(task)
|
||||
logging.info("entering serving loop")
|
||||
loop.run_forever()
|
||||
|
||||
@@ -46,13 +46,11 @@ passthrough_recipients = xstore@testrun.org
|
||||
# Deployment Details
|
||||
#
|
||||
|
||||
# SMTP outgoing filtermail and reinjection
|
||||
# where the filtermail SMTP service listens
|
||||
filtermail_smtp_port = 10080
|
||||
postfix_reinject_port = 10025
|
||||
|
||||
# SMTP incoming filtermail and reinjection
|
||||
filtermail_smtp_port_incoming = 10081
|
||||
postfix_reinject_port_incoming = 10026
|
||||
# postfix accepts on the localhost reinject SMTP port
|
||||
postfix_reinject_port = 10025
|
||||
|
||||
# if set to "True" IPv6 is disabled
|
||||
disable_ipv6 = False
|
||||
|
||||
@@ -11,8 +11,6 @@ def main(vmail_dir=None):
|
||||
ci_accounts = 0
|
||||
|
||||
for path in Path(vmail_dir).iterdir():
|
||||
if not path.joinpath("cur").is_dir():
|
||||
continue
|
||||
accounts += 1
|
||||
if path.name[:3] in ("ci-", "ac_"):
|
||||
ci_accounts += 1
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
From: {from_addr}
|
||||
To: {to_addr}
|
||||
Autocrypt-Setup-Message: v1
|
||||
Subject: Autocrypt Setup Message
|
||||
Date: Tue, 22 Jan 2019 12:56:29 +0100
|
||||
Content-type: multipart/mixed; boundary="Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ"
|
||||
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
|
||||
Content-Type: text/plain
|
||||
|
||||
This message contains all information to transfer your Autocrypt
|
||||
settings along with your secret key securely from your original
|
||||
device.
|
||||
|
||||
To set up your new device for Autocrypt, please follow the
|
||||
instuctions that should be presented by your new device.
|
||||
|
||||
You can keep this message and use it as a backup for your secret
|
||||
key. If you want to do this, you should write down the Setup Code
|
||||
and store it securely.
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
|
||||
Content-Type: application/autocrypt-setup
|
||||
Content-Disposition: attachment; filename="autocrypt-setup-message.html"
|
||||
|
||||
<html><body>
|
||||
<p>
|
||||
This is the Autocrypt setup file used to transfer settings and
|
||||
keys between clients. You can decrypt it using the Setup Code
|
||||
presented on your old device, and then import the contained key
|
||||
into your keyring.
|
||||
</p>
|
||||
|
||||
<pre>
|
||||
-----BEGIN PGP MESSAGE-----
|
||||
Passphrase-Format: numeric9x4
|
||||
Passphrase-Begin: 17
|
||||
|
||||
jA0EBwMCFAxADoCdzeX/0ukBlqI5+pfpKb751qd/7nLNbkpy3gVcaf1QwRPZYt40
|
||||
Ynp08UqRQ2g48ZlnzHLSwlTGOPTuv2Jt8ka+pgZ45xzvJSG2gau03xP4VsC271kR
|
||||
VmCjdb0Y6Rk96mAwfGzrkbaRQ9Z7fIoL866GOv6h9neiVIkp+JYlTV6ISD0ZQJ4Q
|
||||
I6dOQkB/TWZyVjtiJDOQHdfNWliA6NtqaLq19wlu9L5xXjuNpY95KwR8EJXWe0+o
|
||||
Y3d2U/KxOAkXKghP2Qg1GtlPVeGC5T4p03TGI6pzKT+kHX6Rrm9wK6sM9aTquMmF
|
||||
Vok84Jg1DFnwivWC2RILR81rXi7k/+Y6MUbveFgJ9cQduqpxnmD7TjOblYu7M6zp
|
||||
YGAUxh8DRKlIMn2QsA++DBYQ6ACZvwuY8qTDLkqPDo4WqM313dsMJbyGjDdVE7EM
|
||||
PESS+RlABETpZXz8g/ycr6DIUNdlbPcmYlsBfHWDOuR2GFFTwmlv5slWS39dJv38
|
||||
E0eIe1CwdxI801Se7t7dUUS/ZF8wb6GlmxOcqGbF8eko1Z0S64IAm7/h13MRQCxI
|
||||
geQnHfGYVJ2FOimoCMEKwfa9x++RFTDW0u7spDC2uWvK/1viV8OfRppFhLr/kmKb
|
||||
18lWXuAz80DAjUDUsVqEq2MvJBJGoCJUEyjuRsLkHYRM5jYk4v50LyyR0Om73nWF
|
||||
nZBqmqNzdr7Xb9PHHdFhnEc0VvoYbrcM0RVYcEMW3YbmejM891j1d6Iv+/n/qND/
|
||||
NdebGrfWJMmFLf/iEkzTZ3/v5inW9LpWoRc94ioCjJTaEo8Rib6ARRFaJVIsmNXi
|
||||
YicFGO98D+zX+a2t9Yz6IpPajVslnOp6ScpmXgts/2XWD7oE+JgxSAqo/dLVsHgP
|
||||
Ufo=
|
||||
=pulM
|
||||
-----END PGP MESSAGE-----
|
||||
</pre></body></html>
|
||||
--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ--
|
||||
@@ -1,46 +0,0 @@
|
||||
Date: Fri, 8 Jul 1994 09:21:47 -0400
|
||||
From: Mail Delivery Subsystem <MAILER-DAEMON@example.org>
|
||||
Subject: Returned mail: User unknown
|
||||
To: <owner-ups-mib@CS.UTK.EDU>
|
||||
Auto-Submitted: auto-replied
|
||||
MIME-Version: 1.0
|
||||
Content-Type: multipart/report; report-type=delivery-status;
|
||||
boundary="JAA13167.773673707/CS.UTK.EDU"
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: text/plain; charset=us-ascii
|
||||
|
||||
----- The following addresses had delivery problems -----
|
||||
<arathib@vnet.ibm.com> (unrecoverable error)
|
||||
<wsnell@sdcc13.ucsd.edu> (unrecoverable error)
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: message/delivery-status
|
||||
|
||||
Reporting-MTA: dns; cs.utk.edu
|
||||
|
||||
Original-Recipient: rfc822;arathib@vnet.ibm.com
|
||||
Final-Recipient: rfc822;arathib@vnet.ibm.com
|
||||
Action: failed
|
||||
Status: 5.0.0 (permanent failure)
|
||||
Diagnostic-Code: smtp;
|
||||
550 'arathib@vnet.IBM.COM' is not a registered gateway user
|
||||
Remote-MTA: dns; vnet.ibm.com
|
||||
|
||||
Original-Recipient: rfc822;johnh@hpnjld.njd.hp.com
|
||||
Final-Recipient: rfc822;johnh@hpnjld.njd.hp.com
|
||||
Action: delayed
|
||||
Status: 4.0.0 (hpnjld.njd.jp.com: host name lookup failure)
|
||||
|
||||
Original-Recipient: rfc822;wsnell@sdcc13.ucsd.edu
|
||||
Final-Recipient: rfc822;wsnell@sdcc13.ucsd.edu
|
||||
Action: failed
|
||||
Status: 5.0.0
|
||||
Diagnostic-Code: smtp; 550 user unknown
|
||||
Remote-MTA: dns; sdcc13.ucsd.edu
|
||||
|
||||
--JAA13167.773673707/CS.UTK.EDU
|
||||
content-type: message/rfc822
|
||||
|
||||
[original message goes here]
|
||||
--JAA13167.773673707/CS.UTK.EDU--
|
||||
@@ -72,8 +72,9 @@ def maildata(request):
|
||||
def maildata(name, from_addr, to_addr, subject="[...]"):
|
||||
# Using `.read_bytes().decode()` instead of `.read_text()` to preserve newlines.
|
||||
data = datadir.joinpath(name).read_bytes().decode()
|
||||
|
||||
text = data.format(from_addr=from_addr, to_addr=to_addr, subject=subject)
|
||||
return BytesParser(policy=policy.SMTP).parsebytes(text.encode())
|
||||
return BytesParser(policy=policy.default).parsebytes(text.encode())
|
||||
|
||||
return maildata
|
||||
|
||||
|
||||
@@ -15,14 +15,6 @@ def test_read_config_basic(example_config):
|
||||
assert example_config.mail_domain == "chat.example.org"
|
||||
|
||||
|
||||
def test_read_config_basic_using_defaults(tmp_path, maildomain):
|
||||
inipath = tmp_path.joinpath("chatmail.ini")
|
||||
inipath.write_text(f"[params]\nmail_domain = {maildomain}")
|
||||
example_config = read_config(inipath)
|
||||
assert example_config.max_user_send_per_minute == 60
|
||||
assert example_config.filtermail_smtp_port_incoming == 10081
|
||||
|
||||
|
||||
def test_read_config_testrun(make_config):
|
||||
config = make_config("something.testrun.org")
|
||||
assert config.mail_domain == "something.testrun.org"
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
import pytest
|
||||
|
||||
from chatmaild.filtermail import (
|
||||
IncomingBeforeQueueHandler,
|
||||
OutgoingBeforeQueueHandler,
|
||||
BeforeQueueHandler,
|
||||
SendRateLimiter,
|
||||
check_armored_payload,
|
||||
check_encrypted,
|
||||
@@ -19,13 +18,7 @@ def maildomain():
|
||||
@pytest.fixture
|
||||
def handler(make_config, maildomain):
|
||||
config = make_config(maildomain)
|
||||
return OutgoingBeforeQueueHandler(config)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def inhandler(make_config, maildomain):
|
||||
config = make_config(maildomain)
|
||||
return IncomingBeforeQueueHandler(config)
|
||||
return BeforeQueueHandler(config)
|
||||
|
||||
|
||||
def test_reject_forged_from(maildata, gencreds, handler):
|
||||
@@ -36,14 +29,14 @@ def test_reject_forged_from(maildata, gencreds, handler):
|
||||
# test that the filter lets good mail through
|
||||
to_addr = gencreds()[0]
|
||||
env.content = maildata(
|
||||
"encrypted.eml", from_addr=env.mail_from, to_addr=to_addr
|
||||
"plain.eml", from_addr=env.mail_from, to_addr=to_addr
|
||||
).as_bytes()
|
||||
|
||||
assert not handler.check_DATA(envelope=env)
|
||||
|
||||
# test that the filter rejects forged mail
|
||||
env.content = maildata(
|
||||
"encrypted.eml", from_addr="forged@c3.testrun.org", to_addr=to_addr
|
||||
"plain.eml", from_addr="forged@c3.testrun.org", to_addr=to_addr
|
||||
).as_bytes()
|
||||
error = handler.check_DATA(envelope=env)
|
||||
assert "500" in error
|
||||
@@ -113,7 +106,7 @@ def test_send_rate_limiter():
|
||||
break
|
||||
|
||||
|
||||
def test_cleartext_excempt_privacy(maildata, gencreds, handler):
|
||||
def test_excempt_privacy(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = "privacy@testrun.org"
|
||||
handler.config.passthrough_recipients = [to_addr]
|
||||
@@ -134,73 +127,10 @@ def test_cleartext_excempt_privacy(maildata, gencreds, handler):
|
||||
rcpt_tos = [to_addr, false_to]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert "523" in handler.check_DATA(envelope=env2)
|
||||
assert "500" in handler.check_DATA(envelope=env2)
|
||||
|
||||
|
||||
def test_cleartext_self_send_autocrypt_setup_message(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = from_addr
|
||||
|
||||
msg = maildata("asm.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert not handler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_send_fails(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = gencreds()[0]
|
||||
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
res = handler.check_DATA(envelope=env)
|
||||
assert "523 Encryption Needed" in res
|
||||
|
||||
|
||||
def test_cleartext_incoming_fails(maildata, gencreds, inhandler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr, password = gencreds()
|
||||
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
user = inhandler.config.get_user(to_addr)
|
||||
user.set_password(password)
|
||||
res = inhandler.check_DATA(envelope=env)
|
||||
assert "523 Encryption Needed" in res
|
||||
|
||||
user.allow_incoming_cleartext()
|
||||
assert not inhandler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_incoming_mailer_daemon(maildata, gencreds, inhandler):
|
||||
from_addr = "mailer-daemon@example.org"
|
||||
to_addr = gencreds()[0]
|
||||
|
||||
msg = maildata("mailer-daemon.eml", from_addr=from_addr, to_addr=to_addr)
|
||||
|
||||
class env:
|
||||
mail_from = from_addr
|
||||
rcpt_tos = [to_addr]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert not inhandler.check_DATA(envelope=env)
|
||||
|
||||
|
||||
def test_cleartext_passthrough_domains(maildata, gencreds, handler):
|
||||
def test_passthrough_domains(maildata, gencreds, handler):
|
||||
from_addr = gencreds()[0]
|
||||
to_addr = "privacy@x.y.z"
|
||||
handler.config.passthrough_recipients = ["@x.y.z"]
|
||||
@@ -221,10 +151,10 @@ def test_cleartext_passthrough_domains(maildata, gencreds, handler):
|
||||
rcpt_tos = [to_addr, false_to]
|
||||
content = msg.as_bytes()
|
||||
|
||||
assert "523" in handler.check_DATA(envelope=env2)
|
||||
assert "500" in handler.check_DATA(envelope=env2)
|
||||
|
||||
|
||||
def test_cleartext_passthrough_senders(gencreds, handler, maildata):
|
||||
def test_passthrough_senders(gencreds, handler, maildata):
|
||||
acc1 = gencreds()[0]
|
||||
to_addr = "recipient@something.org"
|
||||
handler.config.passthrough_senders = [acc1]
|
||||
|
||||
@@ -2,15 +2,8 @@ from chatmaild.metrics import main
|
||||
|
||||
|
||||
def test_main(tmp_path, capsys):
|
||||
paths = []
|
||||
for x in ("ci-asllkj", "ac_12l3kj", "qweqwe", "ci-l1k2j31l2k3"):
|
||||
p = tmp_path.joinpath(x)
|
||||
p.mkdir()
|
||||
p.joinpath("cur").mkdir()
|
||||
paths.append(p)
|
||||
|
||||
tmp_path.joinpath("nomailbox").mkdir()
|
||||
|
||||
tmp_path.joinpath(x).mkdir()
|
||||
main(tmp_path)
|
||||
out, _ = capsys.readouterr()
|
||||
d = {}
|
||||
|
||||
@@ -40,17 +40,3 @@ def test_no_mailboxes_dir(testaddr, example_config, tmp_path):
|
||||
user.set_password("someeqkjwelkqwjleqwe")
|
||||
user.set_last_login_timestamp(100000)
|
||||
assert user.get_last_login_timestamp() == 86400
|
||||
|
||||
|
||||
def test_set_get_cleartext_flag(testaddr, example_config, tmp_path):
|
||||
p = tmp_path.joinpath("a", "mailboxes")
|
||||
example_config.mailboxes_dir = p
|
||||
|
||||
user = example_config.get_user(testaddr)
|
||||
user.set_password("someeqkjwelkqwjleqwe")
|
||||
user.set_last_login_timestamp(100000)
|
||||
assert user.get_last_login_timestamp() == 86400
|
||||
|
||||
assert not user.is_incoming_cleartext_ok()
|
||||
user.allow_incoming_cleartext()
|
||||
assert user.is_incoming_cleartext_ok()
|
||||
|
||||
@@ -13,7 +13,6 @@ class User:
|
||||
self.maildir = maildir
|
||||
self.addr = addr
|
||||
self.password_path = password_path
|
||||
self.enforce_E2EE_path = maildir.joinpath("enforceE2EEincoming")
|
||||
self.uid = uid
|
||||
self.gid = gid
|
||||
|
||||
@@ -36,13 +35,6 @@ class User:
|
||||
home = str(self.maildir)
|
||||
return dict(addr=self.addr, home=home, uid=self.uid, gid=self.gid, password=pw)
|
||||
|
||||
def is_incoming_cleartext_ok(self):
|
||||
return not self.enforce_E2EE_path.exists()
|
||||
|
||||
def allow_incoming_cleartext(self):
|
||||
if self.enforce_E2EE_path.exists():
|
||||
self.enforce_E2EE_path.unlink()
|
||||
|
||||
def set_password(self, enc_password):
|
||||
"""Set the specified password for this user.
|
||||
|
||||
@@ -58,7 +50,6 @@ class User:
|
||||
if not self.addr.startswith("echo@"):
|
||||
logging.error(f"could not write password for: {self.addr}")
|
||||
raise
|
||||
self.enforce_E2EE_path.touch()
|
||||
|
||||
def set_last_login_timestamp(self, timestamp):
|
||||
"""Track login time with daily granularity
|
||||
|
||||
@@ -106,14 +106,12 @@ def _install_remote_venv_with_chatmaild(config) -> None:
|
||||
for fn in (
|
||||
"doveauth",
|
||||
"filtermail",
|
||||
"filtermail-incoming",
|
||||
"echobot",
|
||||
"chatmail-metadata",
|
||||
"lastlogin",
|
||||
):
|
||||
execpath = fn if fn != "filtermail-incoming" else "filtermail"
|
||||
params = dict(
|
||||
execpath=f"{remote_venv_dir}/bin/{execpath}",
|
||||
execpath=f"{remote_venv_dir}/bin/{fn}",
|
||||
config_path=remote_chatmail_inipath,
|
||||
remote_venv_dir=remote_venv_dir,
|
||||
mail_domain=config.mail_domain,
|
||||
@@ -217,7 +215,7 @@ def _configure_opendkim(domain: str, dkim_selector: str = "dkim") -> bool:
|
||||
server.shell(
|
||||
name="Generate OpenDKIM domain keys",
|
||||
commands=[
|
||||
f"/usr/sbin/opendkim-genkey -D /etc/dkimkeys -d {domain} -s {dkim_selector}"
|
||||
f"opendkim-genkey -D /etc/dkimkeys -d {domain} -s {dkim_selector}"
|
||||
],
|
||||
_use_su_login=True,
|
||||
_su_user="opendkim",
|
||||
@@ -543,6 +541,7 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
|
||||
|
||||
server.group(name="Create vmail group", group="vmail", system=True)
|
||||
server.user(name="Create vmail user", user="vmail", group="vmail", system=True)
|
||||
server.user(name="Create filtermail user", user="filtermail", system=True)
|
||||
server.group(name="Create opendkim group", group="opendkim", system=True)
|
||||
server.user(
|
||||
name="Create opendkim user",
|
||||
|
||||
@@ -46,7 +46,10 @@ http {
|
||||
|
||||
server {
|
||||
|
||||
listen 127.0.0.1:8443 ssl default_server;
|
||||
listen 8443 ssl default_server;
|
||||
{% if not disable_ipv6 %}
|
||||
listen [::]:8443 ssl default_server;
|
||||
{% endif %}
|
||||
|
||||
root /var/www/html;
|
||||
|
||||
@@ -117,7 +120,10 @@ http {
|
||||
|
||||
# Redirect www. to non-www
|
||||
server {
|
||||
listen 127.0.0.1:8443 ssl;
|
||||
listen 8443 ssl;
|
||||
{% if not disable_ipv6 %}
|
||||
listen [::]:8443 ssl;
|
||||
{% endif %}
|
||||
server_name www.{{ config.domain_name }};
|
||||
return 301 $scheme://{{ config.domain_name }}$request_uri;
|
||||
access_log syslog:server=unix:/dev/log,facility=local7;
|
||||
|
||||
@@ -21,9 +21,6 @@ smtpd_tls_security_level=may
|
||||
|
||||
smtp_tls_CApath=/etc/ssl/certs
|
||||
smtp_tls_security_level=verify
|
||||
# Send SNI extension when connecting to other servers.
|
||||
# <https://www.postfix.org/postconf.5.html#smtp_tls_servername>
|
||||
smtp_tls_servername = hostname
|
||||
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
|
||||
smtp_tls_policy_maps = inline:{nauta.cu=may}
|
||||
smtpd_tls_protocols = >=TLSv1.2
|
||||
|
||||
@@ -14,7 +14,7 @@ smtp inet n - y - - smtpd -v
|
||||
{%- else %}
|
||||
smtp inet n - y - - smtpd
|
||||
{%- endif %}
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port_incoming }}
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
submission inet n - y - 5000 smtpd
|
||||
-o syslog_name=postfix/submission
|
||||
-o smtpd_tls_security_level=encrypt
|
||||
@@ -32,6 +32,7 @@ submission inet n - y - 5000 smtpd
|
||||
-o milter_macro_daemon_name=ORIGINATING
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
-o cleanup_service_name=authclean
|
||||
smtps inet n - y - 5000 smtpd
|
||||
-o syslog_name=postfix/smtps
|
||||
-o smtpd_tls_wrappermode=yes
|
||||
@@ -49,6 +50,7 @@ smtps inet n - y - 5000 smtpd
|
||||
-o smtpd_client_connection_count_limit=1000
|
||||
-o milter_macro_daemon_name=ORIGINATING
|
||||
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port }}
|
||||
-o cleanup_service_name=authclean
|
||||
#628 inet n - y - - qmqpd
|
||||
pickup unix n - y 60 1 pickup
|
||||
cleanup unix n - y - 0 cleanup
|
||||
@@ -76,16 +78,10 @@ anvil unix - - y - 1 anvil
|
||||
scache unix - - y - 1 scache
|
||||
postlog unix-dgram n - n - 1 postlogd
|
||||
filter unix - n n - - lmtp
|
||||
# Local SMTP server for reinjecting outgoing filtered mail.
|
||||
127.0.0.1:{{ config.postfix_reinject_port }} inet n - n - 10 smtpd
|
||||
# Local SMTP server for reinjecting filered mail.
|
||||
localhost:{{ config.postfix_reinject_port }} inet n - n - 10 smtpd
|
||||
-o syslog_name=postfix/reinject
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
-o cleanup_service_name=authclean
|
||||
|
||||
# Local SMTP server for reinjecting incoming filtered mail
|
||||
127.0.0.1:{{ config.postfix_reinject_port_incoming }} inet n - n - 10 smtpd
|
||||
-o syslog_name=postfix/reinject_incoming
|
||||
-o smtpd_milters=unix:opendkim/opendkim.sock
|
||||
|
||||
# Cleanup `Received` headers for authenticated mail
|
||||
# to avoid leaking client IP.
|
||||
@@ -93,10 +89,5 @@ filter unix - n n - - lmtp
|
||||
# We do not do this for received mails
|
||||
# as this will break DKIM signatures
|
||||
# if `Received` header is signed.
|
||||
#
|
||||
# This service also rewrites
|
||||
# Subject with `[...]`
|
||||
# to make sure the users
|
||||
# cannot send unprotected Subject.
|
||||
authclean unix n - - - 0 cleanup
|
||||
-o header_checks=regexp:/etc/postfix/submission_header_cleanup
|
||||
|
||||
@@ -1,13 +1,10 @@
|
||||
from subprocess import DEVNULL, CalledProcessError, check_output
|
||||
from subprocess import CalledProcessError, check_output
|
||||
|
||||
|
||||
def shell(command, fail_ok=False):
|
||||
print(f"$ {command}")
|
||||
args = dict(shell=True)
|
||||
if fail_ok:
|
||||
args["stderr"] = DEVNULL
|
||||
try:
|
||||
return check_output(command, **args).decode().rstrip()
|
||||
return check_output(command, shell=True).decode().rstrip()
|
||||
except CalledProcessError:
|
||||
if not fail_ok:
|
||||
raise
|
||||
@@ -17,22 +14,3 @@ def shell(command, fail_ok=False):
|
||||
def get_systemd_running():
|
||||
lines = shell("systemctl --type=service --state=running").split("\n")
|
||||
return [line for line in lines if line.startswith(" ")]
|
||||
|
||||
|
||||
def write_numbytes(path, num):
|
||||
with open(path, "w") as f:
|
||||
f.write("x" * num)
|
||||
|
||||
|
||||
def dovecot_recalc_quota(user):
|
||||
shell(f"doveadm quota recalc -u {user}")
|
||||
output = shell(f"doveadm quota get -u {user}")
|
||||
#
|
||||
# Quota name Type Value Limit %
|
||||
# User quota STORAGE 5 102400 0
|
||||
# User quota MESSAGE 2 - 0
|
||||
#
|
||||
for line in output.split("\n"):
|
||||
parts = line.split()
|
||||
if parts[2] == "STORAGE":
|
||||
return dict(value=int(parts[3]), limit=int(parts[4]), percent=int(parts[5]))
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
[Unit]
|
||||
Description=Incoming Chatmail Postfix before queue filter
|
||||
|
||||
[Service]
|
||||
ExecStart={execpath} {config_path} incoming
|
||||
Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
[Unit]
|
||||
Description=Outgoing Chatmail Postfix before queue filter
|
||||
Description=Chatmail Postfix before queue filter
|
||||
|
||||
[Service]
|
||||
ExecStart={execpath} {config_path} outgoing
|
||||
ExecStart={execpath} {config_path}
|
||||
Restart=always
|
||||
RestartSec=30
|
||||
User=vmail
|
||||
User=filtermail
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -37,7 +37,7 @@ class TestDC:
|
||||
|
||||
def test_ping_pong(self, benchmark, cmfactory):
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
def dc_ping_pong():
|
||||
chat.send_text("ping")
|
||||
@@ -49,7 +49,7 @@ class TestDC:
|
||||
|
||||
def test_send_10_receive_10(self, benchmark, cmfactory, lp):
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
def dc_send_10_receive_10():
|
||||
for i in range(10):
|
||||
|
||||
@@ -116,8 +116,9 @@ def test_authenticated_from(cmsetup, maildata):
|
||||
|
||||
@pytest.mark.parametrize("from_addr", ["fake@example.org", "fake@testrun.org"])
|
||||
def test_reject_missing_dkim(cmsetup, maildata, from_addr):
|
||||
"""Test that emails with missing or wrong DMARC, DKIM, and SPF entries are rejected."""
|
||||
recipient = cmsetup.gen_users(1)[0]
|
||||
msg = maildata("encrypted.eml", from_addr=from_addr, to_addr=recipient.addr).as_string()
|
||||
msg = maildata("plain.eml", from_addr=from_addr, to_addr=recipient.addr).as_string()
|
||||
with smtplib.SMTP(cmsetup.maildomain, 25) as s:
|
||||
with pytest.raises(smtplib.SMTPDataError, match="No valid DKIM signature"):
|
||||
s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg)
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import ipaddress
|
||||
import random
|
||||
import re
|
||||
import time
|
||||
|
||||
@@ -6,9 +7,6 @@ import imap_tools
|
||||
import pytest
|
||||
import requests
|
||||
|
||||
from cmdeploy.remote import rshell
|
||||
from cmdeploy.sshexec import SSHExec
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def imap_mailbox(cmfactory):
|
||||
@@ -56,23 +54,22 @@ class TestEndToEndDeltaChat:
|
||||
"""Test that a DC account can send a message to a second DC account
|
||||
on the same chat-mail instance."""
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
lp.sec("ac1: prepare and send text message to ac2")
|
||||
chat.send_text("message0")
|
||||
|
||||
lp.sec("wait for ac2 to receive message")
|
||||
msg2 = ac2._evtracker.wait_next_incoming_message()
|
||||
assert msg2.text == "message0"
|
||||
|
||||
def test_exceed_quota(
|
||||
self, cmfactory, lp, tmpdir, remote, chatmail_config, sshdomain
|
||||
):
|
||||
@pytest.mark.slow
|
||||
def test_exceed_quota(self, cmfactory, lp, tmpdir, remote, chatmail_config):
|
||||
"""This is a very slow test as it needs to upload >100MB of mail data
|
||||
before quota is exceeded, and thus depends on the speed of the upload.
|
||||
"""
|
||||
ac1, ac2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(ac1, ac2)
|
||||
|
||||
user = ac2.get_config("configured_addr")
|
||||
chat = cmfactory.get_accepted_chat(ac1, ac2)
|
||||
|
||||
def parse_size_limit(limit: str) -> int:
|
||||
"""Parse a size limit and return the number of bytes as integer.
|
||||
@@ -85,27 +82,49 @@ class TestEndToEndDeltaChat:
|
||||
return int(float(number) * units[unit])
|
||||
|
||||
quota = parse_size_limit(chatmail_config.max_mailbox_size)
|
||||
attachsize = 1 * 1024 * 1024
|
||||
num_to_send = quota // attachsize + 2
|
||||
lp.sec(f"ac1: send {num_to_send} large files to ac2")
|
||||
lp.indent(f"per-user quota is assumed to be: {quota / (1024 * 1024)}MB")
|
||||
alphanumeric = "abcdefghijklmnopqrstuvwxyz1234567890"
|
||||
msgs = []
|
||||
for i in range(num_to_send):
|
||||
attachment = tmpdir / f"attachment{i}"
|
||||
data = "".join(random.choice(alphanumeric) for i in range(1024))
|
||||
with open(attachment, "w+") as f:
|
||||
for j in range(attachsize // len(data)):
|
||||
f.write(data)
|
||||
|
||||
lp.sec(f"filling remote inbox for {user}")
|
||||
fn = f"7743102289.M843172P2484002.c20,S={quota},W=2398:2,"
|
||||
path = chatmail_config.mailboxes_dir.joinpath(user, "cur", fn)
|
||||
sshexec = SSHExec(sshdomain)
|
||||
sshexec(call=rshell.write_numbytes, kwargs=dict(path=str(path), num=120))
|
||||
res = sshexec(call=rshell.dovecot_recalc_quota, kwargs=dict(user=user))
|
||||
assert res["percent"] >= 100
|
||||
msg = chat.send_file(str(attachment))
|
||||
msgs.append(msg)
|
||||
lp.indent(f"Sent out msg {i}, size {attachsize / (1024 * 1024)}MB")
|
||||
|
||||
lp.sec("ac2: check quota is triggered")
|
||||
lp.sec("ac2: check messages are arriving until quota is reached")
|
||||
|
||||
starting = True
|
||||
addr = ac2.get_config("addr").lower()
|
||||
saved_ok = 0
|
||||
for line in remote.iter_output("journalctl -n0 -f -u dovecot"):
|
||||
if starting:
|
||||
chat.send_text("hello")
|
||||
starting = False
|
||||
if user not in line:
|
||||
if addr not in line:
|
||||
# print(line)
|
||||
continue
|
||||
if "quota exceeded" in line:
|
||||
return
|
||||
if "quota" in line:
|
||||
if "quota exceeded" in line:
|
||||
if saved_ok < num_to_send // 2:
|
||||
pytest.fail(
|
||||
f"quota exceeded too early: after {saved_ok} messages already"
|
||||
)
|
||||
lp.indent("good, message sending failed because quota was exceeded")
|
||||
return
|
||||
if (
|
||||
"stored mail into mailbox 'inbox'" in line
|
||||
or "saved mail to inbox" in line
|
||||
):
|
||||
saved_ok += 1
|
||||
print(f"{saved_ok}: {line}")
|
||||
if saved_ok >= num_to_send:
|
||||
break
|
||||
|
||||
pytest.fail("sending succeeded although messages should exceed quota")
|
||||
|
||||
def test_securejoin(self, cmfactory, lp, maildomain2):
|
||||
ac1 = cmfactory.new_online_configuring_account(cache=False)
|
||||
@@ -153,7 +172,7 @@ def test_hide_senders_ip_address(cmfactory):
|
||||
assert ipaddress.ip_address(public_ip)
|
||||
|
||||
user1, user2 = cmfactory.get_online_accounts(2)
|
||||
chat = cmfactory.get_protected_chat(user1, user2)
|
||||
chat = cmfactory.get_accepted_chat(user1, user2)
|
||||
|
||||
chat.send_text("testing submission header cleanup")
|
||||
user2._evtracker.wait_next_incoming_message()
|
||||
@@ -162,18 +181,11 @@ def test_hide_senders_ip_address(cmfactory):
|
||||
assert public_ip not in msg.obj.as_string()
|
||||
|
||||
|
||||
def test_echobot(cmfactory, chatmail_config, lp, sshdomain):
|
||||
def test_echobot(cmfactory, chatmail_config, lp):
|
||||
ac = cmfactory.get_online_accounts(1)[0]
|
||||
|
||||
# establish contact with echobot
|
||||
sshexec = SSHExec(sshdomain)
|
||||
command = "cat /var/lib/echobot/invite-link.txt"
|
||||
echo_invite_link = sshexec(call=rshell.shell, kwargs=dict(command=command))
|
||||
chat = ac.qr_setup_contact(echo_invite_link)
|
||||
ac._evtracker.wait_securejoin_joiner_progress(1000)
|
||||
|
||||
# send message and check it gets replied back
|
||||
lp.sec("Send message to echobot")
|
||||
lp.sec(f"Send message to echo@{chatmail_config.mail_domain}")
|
||||
chat = ac.create_chat(f"echo@{chatmail_config.mail_domain}")
|
||||
text = "hi, I hope you text me back"
|
||||
chat.send_text(text)
|
||||
lp.sec("Wait for reply from echobot")
|
||||
|
||||
@@ -62,7 +62,7 @@ def sshdomain(maildomain):
|
||||
def maildomain2():
|
||||
domain = os.environ.get("CHATMAIL_DOMAIN2")
|
||||
if not domain:
|
||||
pytest.skip("set CHATMAIL_DOMAIN2 to a second chatmail server")
|
||||
pytest.skip("set CHATMAIL_DOMAIN2 to a ssh-reachable chatmail instance")
|
||||
return domain
|
||||
|
||||
|
||||
@@ -302,12 +302,10 @@ def cmfactory(request, gencreds, tmpdir, maildomain):
|
||||
pytest.importorskip("deltachat")
|
||||
from deltachat.testplugin import ACFactory
|
||||
|
||||
testproc = ChatmailTestProcess(request.config, maildomain, gencreds)
|
||||
data = request.getfixturevalue("data")
|
||||
|
||||
class Data:
|
||||
def read_path(self, path):
|
||||
return
|
||||
am = ACFactory(request=request, tmpdir=tmpdir, testprocess=testproc, data=Data())
|
||||
testproc = ChatmailTestProcess(request.config, maildomain, gencreds)
|
||||
am = ACFactory(request=request, tmpdir=tmpdir, testprocess=testproc, data=data)
|
||||
|
||||
# nb. a bit hacky
|
||||
# would probably be better if deltachat's test machinery grows native support
|
||||
|
||||
@@ -6,6 +6,29 @@ interoperable e-mail service for everyone. What's behind a `chatmail` is
|
||||
effectively a normal e-mail address just like any other but optimized
|
||||
for the usage in chats, especially DeltaChat.
|
||||
|
||||
### Choosing a chatmail address instead of using a random one
|
||||
|
||||
In the Delta Chat account setup you may tap `Create a profile` then `Use other server` and choose `Classic e-mail login`. Here fill the two fields like this:
|
||||
|
||||
- `E-Mail Address`: invent a word with
|
||||
{% if username_min_length == username_max_length %}
|
||||
*exactly* {{ username_min_length }}
|
||||
{% else %}
|
||||
{{ username_min_length}}
|
||||
{% if username_max_length == "more" %}
|
||||
or more
|
||||
{% else %}
|
||||
to {{ username_max_length }}
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
characters
|
||||
and append `@{{config.mail_domain}}` to it.
|
||||
|
||||
- `Existing Password`: invent at least {{ password_min_length }} characters.
|
||||
|
||||
If the e-mail address is not yet taken, you'll get that account.
|
||||
The first login sets your password.
|
||||
|
||||
|
||||
### Rate and storage limits
|
||||
|
||||
@@ -15,10 +38,9 @@ for the usage in chats, especially DeltaChat.
|
||||
|
||||
- You may send up to {{ config.max_user_send_per_minute }} messages per minute.
|
||||
|
||||
- You can store up to [{{ config.max_mailbox_size }} messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server).
|
||||
- Messages are unconditionally removed {{ config.delete_mails_after }} days after arriving on the server.
|
||||
|
||||
- Messages are unconditionally removed latest {{ config.delete_mails_after }} days after arriving on the server.
|
||||
Earlier, if storage may exceed otherwise.
|
||||
- You can store up to [{{ config.max_mailbox_size }} messages on the server](https://delta.chat/en/help#what-happens-if-i-turn-on-delete-old-messages-from-server).
|
||||
|
||||
|
||||
### <a name="account-deletion"></a> Account deletion
|
||||
|
||||
Reference in New Issue
Block a user