cqrenet/relay
3
0
Fork 0
mirror of https://github.com/chatmail/relay.git synced 2026-05-10 07:54:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

Various README improvements

Browse Source 
- e-mail -> email
- capitalization of software and protocols (Postfix, Dovecot, SMTP, etc)

Rephrased the install instructions to start by recommending the DNS records that cmdeploy is going to check for immediately anyway.

Refactored the migration guide to fix incorrect tar commands (wrong usage of -C flag) and suggest how to copy directly from server to server by logging in with SSH agent forwarding. The mail services should be disabled first before proceeding with any changes and also ensure the echobot and mail spool are copied over to the new server so no messages are lost.
This commit is contained in:
Mark Felder
2025-03-24 10:38:26 -07:00
committed by holger krekel
parent 8d34e036ec
commit a4f4627a75
1 changed files with 107 additions and 95 deletions
Download Patch File Download Diff File Expand all files Collapse all files

202
README.md
View File

@@ -3,80 +3,90 @@
# Chatmail servers for secure instant messaging
Chatmail servers are minimal interoperable e-mail routing machines designed for:
Chatmail servers are interoperable email routing machines optimized for:
- **Convenience:** Instant onboarding, with optional Google/Apple/Huawei push notifications
- **Convenience:** Low friction instant onboarding
- **Privacy:** Just login, no questions asked, no name, numbers or e-mail needed
- **Privacy:** No name, telephone numbers, or email required to sign-up permitting you to create unique and disposable identities. Push notification architecture is privacy-preserving. (Apple, Google, and Huawei)
- **Speed:** End-to-End Message delivery in well under a second
- **Speed:** Message delivery in well under a second.
- **Security:** Strict TLS, DKIM and OpenPGP with metadata-minimization enforced.
- **Security:** Strict TLS, DKIM and OpenPGP with metadata-minimization rules enforced.
- **Relaxation:** No annoying spam-checking, IP reputation or rate limits
- **Reliability:** No spam or IP reputation checks to disrupt message delivery. Default per-user message rate limit is high enough for realtime chats.
- **Efficiency:** messages are only stored for transit and removed automatically.
- **Efficiency:** Messages are only stored for transit and removed automatically.
This repository contains everything needed to setup a ready-to-use chatmail server
comprised of a minimal setup of the battle-tested
[postfix smtp](https://www.postfix.org) and [dovecot imap](https://www.dovecot.org) services.
[Postfix SMTP](https://www.postfix.org) and [Dovecot IMAP](https://www.dovecot.org) services.
The automated setup is designed and optimized for providing chatmail addresses
for immediate permission-free onboarding through chat apps and bots.
Chatmail addresses are automatically created by a first login,
after which the initially specified password is required
for sending and receiving messages through them.
The automated setup is designed and optimized for providing chatmail addresses
for immediate permission-free onboarding through chat apps and bots.
Chatmail addresses are automatically created at first login,
after which the initially specified password is required
for sending and receiving messages through them.
Please see [this list of known apps and client projects](https://chatmail.at/apps.html) which offer instant onboarding on chatmail servers,
and [this list of known public 3rd party chatmail servers](https://delta.chat/en/chatmail).
Please see [this list of known apps and client projects](https://chatmail.at/apps.html) which offer instant onboarding on chatmail servers
and [this list of known public 3rd party chatmail servers](https://delta.chat/en/chatmail).
## Minimal requirements, Prerequisites
You will need the following:
- control over a domain through a DNS provider of your choice,
- Control over a domain through a DNS provider of your choice.
- a remote Debian 12 machine with IPV4 and preferably also IPV6 addresses and
reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
Machine needs 1GB RAM, one slow CPU and maybe 10GB storage for a
few thousand active chatmail addresses,
- A Debian 12 server with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.
IPv6 is encouraged if available.
Chatmail servers only require 1GB RAM, one CPU, and perhaps 10GB storage for a
few thousand active chatmail addresses.
- a terminal window with password-less ssh root login to the remote machine;
you must have set up ssh authentication and need to use an ed25519 key,
due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191);
you also need to add your private key to the local ssh-agent,
because you can't type in your password during deployment.
- Key-based SSH authentication to the server's root user.
You must add a passphrase-protected private key to the local ssh-agent
because you can't type in your passphrase during deployment.
(An ed25519 private key is required due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191))
## Getting started
We use `chat.example.org` as the chatmail domain in the following steps.
We use `chat.example.org` as the chatmail domain in the following steps.
Please substitute it with your own domain.
1. Install the `cmdeploy` command in a virtualenv
1. Setup the initial DNS records.
The following is an example in the familiar BIND zone file format with
a TTL of 1 hour (3600 seconds).
Please substitute your domain and IP addresses.
```
chat.example.com. 3600 IN A 198.51.100.5
chat.example.com. 3600 IN AAAA 2001:db8::5
www.chat.example.com. 3600 IN CNAME chat.example.com.
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
```
2. Clone the repository and bootstrap the Python virtualenv.
```
git clone https://github.com/chatmail/server
cd chatmail
cd server
scripts/initenv.sh
```
2. Create chatmail configuration file `chatmail.ini`:
3. Create chatmail configuration file `chatmail.ini`:
```
scripts/cmdeploy init chat.example.org # <-- use your domain
```
3. Point your domain to the server's IP address,
if you haven't done so already.
Verify that SSH root login works:
4. Verify that SSH root login works:
```
ssh root@chat.example.org # <-- use your domain
```
4. Deploy to the remote chatmail server:
5. Deploy to the remote chatmail server:
```
scripts/cmdeploy run
@@ -121,13 +131,13 @@ This repository has four directories:
and a [pyinfra](https://pyinfra.com)-based deployment script.
- [chatmaild](https://github.com/chatmail/server/tree/main/chatmaild)
is a python package containing several small services
is a Python package containing several small services
which handle authentication,
trigger push notifications on new messages,
ensure that outbound mails are encrypted,
delete inactive users,
and some other minor things.
chatmaild can also be installed as a stand-alone python package.
chatmaild can also be installed as a stand-alone Python package.
- [www](https://github.com/chatmail/server/tree/main/www)
contains the html, css, and markdown files
@@ -151,15 +161,15 @@ according to the `chatmail.ini` config.
The components of chatmail are:
- [postfix smtp server](https://www.postfix.org) accepts sent messages (both from your users and from other servers)
- [Postfix SMTP server](https://www.postfix.org) accepts sent messages (both from your users and from other servers)
- [dovecot imap server](https://www.dovecot.org) stores messages for your users until they download them
- [Dovecot IMAP server](https://www.dovecot.org) stores messages for your users until they download them
- [nginx](https://nginx.org/) shows the web page with your privacy policy and additional information
- [Nginx](https://nginx.org/) shows the web page with your privacy policy and additional information
- [acmetool](https://hlandau.github.io/acmetool/) manages TLS certificates for dovecot, postfix, and nginx
- [acmetool](https://hlandau.github.io/acmetool/) manages TLS certificates for Dovecot, Postfix, and Nginx
- [opendkim](http://www.opendkim.org/) for signing messages with DKIM and rejecting inbound messages without DKIM
- [OpenDKIM](http://www.opendkim.org/) for signing messages with DKIM and rejecting inbound messages without DKIM
- [mtail](https://google.github.io/mtail/) for collecting anonymized metrics in case you have monitoring
@@ -181,8 +191,8 @@ A short overview:
to send mails for them.
- [`filtermail`](https://github.com/chatmail/server/blob/main/chatmaild/src/chatmaild/filtermail.py) prevents
unencrypted e-mail from leaving the chatmail service
and is integrated into postfix's outbound mail pipelines.
unencrypted email from leaving the chatmail service
and is integrated into Postfix's outbound mail pipelines.
- [`chatmail-metadata`](https://github.com/chatmail/server/blob/main/chatmaild/src/chatmaild/metadata.py) is contacted by a
[dovecot lua script](https://github.com/chatmail/server/blob/main/cmdeploy/src/cmdeploy/dovecot/push_notification.lua)
@@ -191,7 +201,7 @@ A short overview:
it [passes the user's push notification token](https://github.com/chatmail/server/blob/main/chatmaild/src/chatmaild/notifier.py)
to [notifications.delta.chat](https://delta.chat/help#instant-delivery)
so the push notifications on the user's phone can be triggered
by Apple/Google.
by Apple/Google/Huawei.
- [`delete_inactive_users`](https://github.com/chatmail/server/blob/main/chatmaild/src/chatmaild/delete_inactive_users.py)
deletes users if they have not logged in for a very long time.
@@ -210,15 +220,15 @@ A short overview:
### Home page and getting started for users
`cmdeploy run` also creates default static Web pages and deploys them
to a nginx web server with:
`cmdeploy run` also creates default static web pages and deploys them
to a Nginx web server with:
- a default `index.html` along with a QR code that users can click to
create accounts on your chatmail provider,
create accounts on your chatmail provider
- a default `info.html` that is linked from the home page,
- a default `info.html` that is linked from the home page
- a default `policy.html` that is linked from the home page.
- a default `policy.html` that is linked from the home page
All `.html` files are generated
by the according markdown `.md` file in the `www/src` directory.
@@ -231,13 +241,13 @@ by the according markdown `.md` file in the `www/src` directory.
scripts/cmdeploy webdev
```
This starts a local live development cycle for chatmail Web pages:
This starts a local live development cycle for chatmail web pages:
- uses the `www/src/page-layout.html` file for producing static
HTML pages from `www/src/*.md` files
- continously builds the web presence reading files from `www/src` directory
and generating html files and copying assets to the `www/build` directory.
and generating HTML files and copying assets to the `www/build` directory.
- Starts a browser window automatically where you can "refresh" as needed.
@@ -252,17 +262,17 @@ login to the server with ssh and run:
touch /etc/chatmail-nocreate
```
While this file is present, account creation will be blocked.
Account creation will be denied while this file is present.
### Ports
[Postfix](http://www.postfix.org/) listens on ports 25 (smtp) and 587 (submission) and 465 (submissions).
[Dovecot](https://www.dovecot.org/) listens on ports 143 (imap) and 993 (imaps).
[nginx](https://www.nginx.com/) listens on port 8443 (https-alt) and 443 (https).
[Postfix](http://www.postfix.org/) listens on ports 25 (SMTP) and 587 (SUBMISSION) and 465 (SUBMISSIONS).
[Dovecot](https://www.dovecot.org/) listens on ports 143 (IMAP) and 993 (IMAPS).
[Nginx](https://www.nginx.com/) listens on port 8443 (HTTPS-ALT) and 443 (HTTPS).
Port 443 multiplexes HTTPS, IMAP and SMTP using ALPN to redirect connections to ports 8443, 465 or 993.
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (http).
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (HTTP).
Chatmail-core based apps will, however, discover all ports and configurations
chatmail-core based apps will, however, discover all ports and configurations
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail server.
## Email authentication
@@ -321,7 +331,7 @@ while actually it is a problem with your TLS certificate.
If you want to migrate chatmail from an old machine
to a new machine,
you can use these steps.
They were tested with a linux laptop;
They were tested with a Linux laptop;
you might need to adjust some of the steps to your environment.
Let's assume that your `mail_domain` is `mail.example.org`,
@@ -329,56 +339,58 @@ all involved machines run Debian 12,
your old server's IP address is `13.37.13.37`,
and your new server's IP address is `13.12.23.42`.
During the guide, you might get a warning about changed SSH Host keys;
in this case, just run `ssh-keygen -R "mail.example.org"` as recommended
to make sure you can connect with SSH.
Note, you should lower the TTLs of your DNS records to a value
such as 300 (5 minutes) so the migration happens as smoothly as possible.
1. First, copy `/var/lib/acme` to the new server with
`ssh root@13.37.13.37 tar c /var/lib/acme | ssh root@13.12.23.42 tar x -C /var/lib/`.
This transfers your TLS certificate.
During the guide you might get a warning about changed SSH Host keys;
in this case, just run `ssh-keygen -R "mail.example.org"` as recommended.
2. You should also copy `/etc/dkimkeys` to the new server with
`ssh root@13.37.13.37 tar c /etc/dkimkeys | ssh root@13.12.23.42 tar x -C /etc/`
so the DKIM DNS record stays correct.
1. First, disable mail services on the old server.
3. On the new server, run `chown root: -R /var/lib/acme` and `chown opendkim: -R /etc/dkimkeys` to make sure the permissions are correct.
4. Run `cmdeploy run --disable-mail --ssh-host 13.12.23.42` to install chatmail on the new machine.
postfix and dovecot are disabled for now,
we will enable them later.
5. Now, point DNS to the new IP addresses.
You can already remove the old IP addresses from DNS.
Existing Chatmail app users or bots will still be able to connect
to the old server, send and receive messages,
but new ones will fail to create new profiles
with your chatmail server.
If other servers try to deliver messages to your new server they will fail,
but normally email servers will retry delivering messages
for at least a week, so messages will not be lost.
6. Now you can run `cmdeploy run --disable-mail --ssh-host 13.37.13.37` to disable your old server.
```
cmdeploy run --disable-mail --ssh-host 13.37.13.37
```
Now your users will notice the migration
and will not be able to send or receive messages
until the migration is completed.
7. After everything is stopped,
you can copy the `/home/vmail/mail` directory to the new server.
It includes all user data, messages, password hashes, etc.
2. Now we want to copy `/home/vmail`, `/var/lib/acme`, `/etc/dkimkeys`, `/run/echobot`, and `/var/spool/postfix` to the new server.
Login to the old server while forwarding your SSH agent so you can copy directly from the old server to the new server with
your SSH key:
Just run: `ssh root@13.37.13.37 tar c /home/vmail/mail | ssh root@13.12.23.42 tar x -C /home/vmail/`
```
ssh -A root@13.37.13.37
tar c - /home/vmail/mail /var/lib/acme /etc/dkimkeys /run/echobot /var/spool/postfix | ssh root@13.12.23.42 "tar x -C /"
```
After this, your new server has all the necessary files to start operating :)
This transfers all accounts, the TLS certificate, DKIM keys (so DKIM DNS record remains valid), and the echobot's password so it continues to function.
It also preserves the Postfix mail spool so any messages pending delivery will still be delivered.
8. To be sure the permissions are still fine,
run `chown vmail: -R /home/vmail` on the new server.
3. Install chatmail on the new machine:
9. Finally, you can run `cmdeploy run` to turn on chatmail on the new server.
Your users can continue using the chatmail server,
and messages which were sent after step 6. should arrive now.
```
cmdeploy run --disable-mail --ssh-host 13.12.23.42
```
Postfix and Dovecot are disabled for now; we will enable them later. This ensures all user accounts are provisioned.
3. On the new server, run the following to ensure the ownership is correct in case UIDs/GIDs changed:
```
chown root: -R /var/lib/acme
chown opendkim: -R /etc/dkimkeys
chown vmail: -R /home/vmail/mail
chown echobot: -R /run/echobot
```
4. Now, update DNS to the new IP addresses.
If other servers try to deliver messages to your server they may fail,
but normally email servers will retry delivering messages
for at least a week, so messages will not be lost.
5. Finally, you can execute `cmdeploy run --ssh-host 13.12.23.42` to turn on chatmail on the new server.
Your users will be able to use the chatmail server as soon as the DNS changes have propagated.
Voilà!
## Setting up a reverse proxy