Commit Graph
100 Commits
Author SHA1 Message Date
link2xt d4f8a29243 docs: fix link to Maddy and update madmail URL 2026-02-13 09:49:29 +00:00
link2xt dd2beb226a test(test_exceed_rate_limit): print timestamps when sending messages 2026-01-26 14:25:06 +00:00
link2xt 9c7508cc33 test: fix flaky test_exceed_rate_limit
filtermail rate limiter is using leaky bucket
algorithm (GCRA).
Exceeting the limit requires sending
at least max_user_send_per_minute
messages to exhaust allowed burst,
and then sending messages faster
than the leak rate.
As we don't know how fast is the network
between the server and test runner,
try to send 3 times max_user_send_per_minute
messages to ensure the test does not
fail randomly.
2026-01-26 12:07:10 +00:00
link2xt c45fe03652 fix(mtail): separate metrics for incoming and outgoing messages 2026-01-22 14:45:33 +00:00
link2xt c2acbad802 docs: pin Dovecot documentation URLs to version 2.3
At least some old URLs are 404 already.
2026-01-07 20:08:45 +00:00
link2xt 77a6f49c9b ci: remove jsok/serialize-workflow-action dependency
Deployments to test servers will not be cancelled anymore,
but it is not clear if we even want it.
This setup is much simpler because it only depends
on GitHub Actions features and does not allocate
a runner just to sleep there and wait in the queue.
2025-12-27 14:36:39 +00:00
link2xt 0d890274fd feat: use daemon_name for OpenDKIM sign-verify decision instead of IP
On FreeBSD 127.0.0.2 is not assigned to any interface by default,
so 127.0.0.2 source address hack cannot be used to make OpenDKIM
verify the signature instead of signing.

This change sets InternalHosts to `-` so no IP addresses
make OpenDKIM sign the message. Instead of IP address,
OpenDKIM in the outgoing pipeline is explicitly told
to sign messages by setting `{daemon_name}` macro to `ORIGINATING`.
2025-12-19 17:09:33 +00:00
link2xt 7191329a9f chore(release): prepare for 1.9.0 2025-12-18 23:49:48 +00:00
link2xt 1ae4c8451a ci: run tests against ci-chatmail.testrun.org instead of nine.testrun.org 2025-12-18 23:06:05 +00:00
link2xtandholger krekel 610843a44a docs: add RELEASE.md and CONTRIBUTING.md 2025-12-18 09:21:19 +01:00
link2xtandholger krekel 966754a346 chore: setup git-cliff
I am running git-cliff 2.11.0.
Ran `git-cliff --init` to generate `cliff.toml`.
Removed emojis, replaced `doc` with `docs` to match chatmail core
convention.
2025-12-18 09:21:19 +01:00
link2xt 87153667ed chore: update the heading in the CHANGELOG.md
I have checked that nobody added any entries since 1.8.0 was released.
2025-12-17 21:18:02 +00:00
link2xtandholger krekel 51d16b6bb8 Add hpk42 SSH key to staging server for debugging 2025-12-07 20:13:38 +01:00
link2xtandholger krekel 2beba8c455 ci: add deployment environments for all deployment workflows
Code posting the link to comments is removed
as deployment URLs are directly visible in the UI.
2025-12-07 15:21:44 +01:00
link2xtandholger krekel 33c67d22fa Add execnet dependency 2025-12-07 15:21:44 +01:00
link2xt 2e3e3101b6 Add robots.txt to exclude all web crawlers 2025-11-16 10:31:14 +00:00
link2xt 68cc6676ef Update changelog 2025-11-15 10:51:04 +00:00
link2xt 14ca95d25a fix(postfix): set smtpd_tls_mandatory_protocols for port 25
smtp_tls_mandatory_protocols does not affect port 25
because we require STARTTLS on port 25 since commit
8d7e1dad0e

We don't have any smtpd ports with opportunistic TLS.
Submission ports require TLSv1.3 and starting with this commit
MX port will require TLSv1.2 instead of TLSv1.

I have not managed to connect using TLSv1.1
even without this fix to reproduce the problem,
but I have checked that setting
`-o smtpd_tls_mandatory_protocols=>=TLSv1.3`
does not allow to connect using TLSv1.2 anymore using
`openssl s_client -connect example.org:25 -starttls smtp -tls1_2`.

`smtpd_tls_protocols` setting is removed
because it does not affect anything except the internal ports
and its `git blame` points to the wrong commit.
2025-11-15 10:51:04 +00:00
link2xt 3524b055db fix(postfix): set smtp_tls_mandatory_protocols to require TLSv1.2 for outgoing connections
According to
<https://www.postfix.org/postconf.5.html#smtp_tls_security_level>
for outgoing connections with smtp_tls_security_level
`encrypt` and higher (such as `verify` that we currently use)
the setting `smtp_tls_mandatory_protocols`
is used instead of `smtp_tls_protocols`.
According to `postconf -d`
(and `postconf` because the default is not changed)
current setting value is `smtp_tls_mandatory_protocols = >=TLSv1`.
But we only want to connect outside with TLS 1.2 and TLS 1.3.

`smtp_tls_protocols` which was already set to `>= TLSv1.2`
in commit 0155f32df6
only affected outgoing connections with the `may` level
exception set for nauta.cu domain via `smtp_tls_policy_maps`
which does not support STARTTLS at all.
2025-11-15 10:51:04 +00:00
link2xt a0ebb2bdbc ci: pin jsok/serialize-workflow-action 2025-11-08 21:03:48 +00:00
link2xt 132bdcb5e5 Update the changelog 2025-11-08 19:20:39 +00:00
link2xt 7d593841bb fix: change hook permissions from 744 to 755
There is no reason for it to be not executable by non-owner.
2025-11-08 19:20:39 +00:00
link2xt 83e7caeaf8 Replace acmetool cronjob with a timer 2025-11-08 19:20:39 +00:00
link2xt 1cff4a94f1 Setup acmetool hook into correct place 2025-11-08 19:20:39 +00:00
link2xt 8d7e1dad0e Require STARTTLS for incoming port 25 connections
We already require that outgoing connections
use STARTTLS so other servers need a valid TLS
certificate to accept messages from us.
It is then very unlikely that they cannot use TLS
to send messages to us.

Conversely, if they only can send messages to use without TLS,
it likely does not have STARTLS on its port 25
and then we don't want to accept messages from them
because we will likely not be able to reply.
2025-10-28 01:44:14 +00:00
link2xt c0da7bb3bf docs: chatmail-turn listens on 3478 UDP, not TCP port 2025-10-28 01:08:06 +00:00
link2xt 9148b16d81 acmetool: use ECDSA keys instead of RSA 2025-10-25 08:00:31 +00:00
link2xt 0155f32df6 Require TLS 1.2 for outgoing SMTP connections 2025-10-22 02:46:29 +00:00
link2xt 2b90f7db37 filtermail: run CPU-intensive handle_DATA in a thread pool executor
See
<https://docs.python.org/3/library/asyncio-eventloop.html#executing-code-in-thread-or-process-pools>
for the documentation.

This should avoid processing of large messages from hogging asyncio
thread and delaying async operations like accepting new connections.
2025-10-19 10:43:11 +00:00
link2xt d7f50183ea feat: setup TURN server 2025-10-10 18:32:32 +00:00
link2xt 0ed7c360a9 Update changelog 2025-10-05 02:37:50 +00:00
link2xt af272545dd Restart iroh-relay if the binary is updated 2025-10-05 02:37:23 +00:00
link2xt 7725a73cf5 Ensure that downloaded iroh-relay matches expected SHA-256 sum
Previously we only used SHA-256 sum
to check if we need to update the binary.
2025-10-05 02:37:23 +00:00
link2xt e65311c0df Update iroh-relay to 0.35.0 2025-10-05 02:37:23 +00:00
link2xt d091b865c7 fix: ignore all RCPT TO: parameters
Stalwart sends `NOTIFY=DELAY,FAILURE`
to request Delivery Status Notifications.
aiosmtpd does not support any parameters,
not just ORCPT, so we have to ignore all of them.
2025-10-05 02:36:40 +00:00
link2xt c56805211f Increase maxproc for reinjecting ports from 10 to 100
Otherwise under high load filtermail
starts printing "Connection refused" errors to the log.
2025-09-24 16:10:26 +00:00
link2xt 290e80e795 Revert "dovecot: keep mailbox index only in memory (#632)"
This reverts commit 7bf2dfd62e.
2025-09-22 22:55:57 +00:00
link2xt 00ab53800e Update changelog 2025-09-18 15:28:15 +00:00
link2xt fc65072edb Allow ports 143 and 993 to be used by dovecot process 2025-09-18 15:26:58 +00:00
link2xt 96a1dbac08 Expire push notification tokens after 90 days 2025-06-10 22:27:21 +00:00
link2xt 5215e1dc2b Update changelog 2025-06-04 20:57:31 +00:00
link2xt 624a33a61e Use static binary from official mtail release instead of Debian package
Debian has outdated version that does not actually work
with logs from stdin. It gets stuck after some time.
2025-06-04 20:56:27 +00:00
link2xt 6bc751213f Checkout non-merge commit in CI 2025-06-04 20:12:22 +00:00
link2xt 4b721bfcd4 Reconfigure imap-login to high-performance mode
High-security mode could be configured
to handle more connections by increasing process_limit,
but has problems logging in many users at once after
each Dovecot restart or config reload.
2025-06-03 16:30:06 +00:00
link2xt 4a6aa446cd Increase nginx connection limits 2025-06-02 18:28:57 +00:00
link2xt 0638bea363 filtermail: allow partial body length in OpenPGP payloads 2025-05-05 07:03:09 +00:00
link2xt 5f29c53232 Fix mox URL in the README 2025-04-23 16:59:26 +00:00
link2xt 49787044ff Do not encourage non-random addresses and weak passwords 2025-04-08 11:49:39 +00:00
link2xt 05a32efa50 fix: send SNI when connecting to outside servers
Otherwise email providers which allow to bring your own domain
and use the same IP addresses for all customers
send wildcard certificate instead of the correct one
and Postfix refuses to connect with an error

    server certificate verification failed for example.org[A.B.C.D]:25: num=62:hostname mismatch
2025-03-16 11:21:16 +00:00
link2xt 35fe189be7 Pass through original_content instead of content in filtermail
This avoids unnecessary UTF-8 recoding and passes bytestring through.
2025-03-11 13:27:16 +00:00
landGitHub 803f3e6181 Merge pull request #514 from chatmail/link2xt/readme-tls
Document TLS requirements in the readme
2025-03-10 22:47:43 +00:00
link2xt f188aef11e Document TLS requirements in the readme 2025-03-09 15:52:44 +00:00
link2xt 76d7e60018 Remove cleanup service from submission ports
It does not work because of `smtpd_proxy_filter`
forwarding the message to filtermail
and we cleanup the message once
filtermail reinjects it on port 10025.
2025-03-09 10:26:53 +00:00
link2xt fe749159e4 Document that authclean cleans up the Subject 2025-03-08 02:42:35 +00:00
link2xtandmissytake 8ec6e6e985 opendkim: use su instead of sudo 2025-02-17 19:09:50 +01:00
link2xt 1a35cdc7a9 Require TLS 1.3 on client-facing ports
I tested with -tls1_2 option
of openssl s_client
that TLS 1.2 connections
are no longer possible
on any ports except port 25.

Port 25 requires at least TLS 1.2
for encrypted connections.
2025-02-16 23:01:56 +00:00
link2xt 2daac76574 Replace subject with [...] for outgoing mail
`authclean` cleanup server is used by
reinjecting smtpd running on localhost:10025 by default.
It runs after filtermail
and currently removes `Received` header
to avoid leaking IP address.
Can as well be used to replace `Subject` lines
with `Subject: [...]`.
If there are multiple `Subject` lines,
all of them should be replaced.

This allows us to avoid dealing with
localized subjects, including SecureJoin
messages `vc-request` and `vg-request`
which can have Subject lines like
Subject: =?utf-8?q?Nachricht_von_nrn178fi4=40nine=2Etestrun=2Eorg?=
2025-02-16 22:35:51 +00:00
link2xt 5633582d31 Add changelog entry for MTA-STS daemon removal 2025-02-16 21:27:15 +00:00
link2xt 667a987dfc Remove MTA-STS daemon 2025-02-16 20:31:07 +00:00
link2xt 49907c78a3 Add changelog entry for crypt compatibility 2025-02-16 15:15:31 +00:00
link2xt 7e6f8ddfba Simplify SPF record
There is no need to explicitly specify domain for `a` rule.
2025-02-15 03:51:49 +00:00
landGitHub 9e6ba1a164 fix: install gcc and python3-dev (#477)
These are needed to build crypt-r
2025-01-27 14:38:18 +00:00
link2xt b2995551a2 ci: remove iroh relay from zonefiles
iroh subdomain is not needed
since 95f8c4b269
2025-01-26 19:22:45 +00:00
link2xtandholger krekel c8f46147e0 chore: ruff 0.9.2 fixes and formatting 2025-01-24 20:57:13 +01:00
link2xt 95f8c4b269 Update iroh and remove iroh. subdomain 2024-11-09 01:02:20 +00:00
link2xtandmissytake 5048bde6d0 Deploy iroh relay 2024-10-30 13:30:44 +01:00
link2xtandmissytake c35c44ad8d Replace rsync with tar 2024-10-29 17:17:17 +01:00
link2xt 7f3f69fa72 fix: increase request_queue_size for UNIX sockets to 1000
Default value is 5.
This setting was lost during refactoring in commit bf0f6e2303
2024-10-27 14:20:42 +00:00
link2xt 5055434e48 Fix OpenPGP payload check
Replace \r\r\n in literal.eml test with \r\n
to make `test_filtermail_no_literal_packets`
actually reach `check_openpgp_payload()`
and make `check_openpgp_payload()` more strict.
2024-10-22 18:41:27 +00:00
link2xtandholger krekel 737ab54bf2 ci: test cmdeploy dns only once
It should be reliable.
2024-10-16 12:06:55 +02:00
link2xt 20fa5d9656 Query autoritative nameserver directly to bypass DNS cache
unbound-control is not installed out of the box
and even once installed `flush_zone` does not seem
to work reliably.

Instead of trying to flush the cache from unbound,
we now query authoritative nameserver directly using `dig`.
2024-10-15 22:19:47 +00:00
link2xt a2f2e04ff9 fix: set acme_account_url even if some DNS records are not set
perform_initial_checks may exit early
and not add `acme_account_url` if required DNS
records are not found.
In this case other `cmdeploy run` fails
with KeyError.

To avoid this, `acme_account_url` should always be set.

Unlike DNS checks, running acmetool
may not fail due to network errors,
so it is more reliable and should be checked first.
2024-10-15 16:10:36 +00:00
link2xtandholger krekel 7573ef928f mention wireguard 2024-10-14 12:22:02 +02:00
link2xtandholger krekel 46297d4839 Document setting up DNAT 2024-10-14 12:22:02 +02:00
5515607b63 Setup mtail (#388)
Co-authored-by: holger krekel <holger@merlinux.eu>
2024-10-14 09:18:35 +00:00
link2xtandholger krekel d0ed8830f7 Add IMAP capabilities instead of overwriting them
I wanted to add `COMPRESS=DEFLATE`,
but it should be added only for sessions
that are logged in because `COMPRESS`
command does not work before logging in.

Dovecot already does it correctly
if we don't overwrite the capability string.
2024-10-13 20:18:34 +02:00
link2xt a6bdbb748b Set CAA record flags to 0 2024-09-15 02:57:38 +00:00
link2xt a1e80fdca1 Fix ruff warnings 2024-08-23 11:57:47 +00:00
link2xt 1f1d1fdf59 fix: use separate transaction storage for each DictProxy handler
DictProxy can have transactions with the same name
(most frequently `1`) processed in parallel.
Dovecot expects that transaction names on each connection
are independent.
2024-07-31 18:49:18 +00:00
link2xtandholger krekel e3f0bb195d Fix doveauth logging for created accounts
Currently logs look like this:
`Created address: <chatmaild.user.User object at 0x7fafac36bcd0>`
2024-07-30 12:09:12 +02:00
link2xt 052fb64a3d nginx: use numbers for upstream ports
Otherwise nginx fails when user actually tries to connect,
logs have errors such as
`invalid port in upstream "127.0.0.1:imaps"`
and
`invalid port in upstream "127.0.0.1:submissions"`.
2024-07-17 17:13:05 +00:00
link2xt e8bf051cd0 refactor: use f-string in logging where it is easy
% is only interpreted if there are two or more arguments:
<https://docs.python.org/3/library/logging.html#logging.Logger.debug>
So it is safe to pass a single argument with already formatted
string.
2024-07-16 09:13:56 +00:00
link2xtandholger krekel 02df395dab filtermail: do not inject addresses into fromat string 2024-07-13 11:46:49 +02:00
link2xtandholger krekel 39584c7b7d postfix: fix timeout to 300s on submission ports
Otherwise smtpd reduces it to 10s on "overload".
2024-07-13 11:46:20 +02:00
link2xtandholger krekel 4ebc4f3069 postfix: do not lookup client hostnames 2024-07-13 11:45:54 +02:00
link2xt d73e896e66 Add changelog entry for HTTPS/IMAP/SMTP multiplexing 2024-07-11 10:31:45 +00:00
link2xt 283045dc4a Multiplex HTTPS, IMAP and SMTP on port 443
Services are distinguished based on ALPN.
For example,
    openssl s_client -connect example.org:443 -alpn smtp
gives SMTP connection and
    openssl s_client -connect example.org:443 -alpn imap
gives IMAP connection.
2024-07-11 10:30:46 +00:00
link2xtandholger krekel 62c60d3070 doveauth: log when a new account is created 2024-07-09 00:24:06 +02:00
link2xt 4292355310 Add nonci_accounts metric
Calculating this with PromQL is not easy
due to interpolation.

Also add HELP and TYPE metadata for each metric.
2024-07-08 18:33:18 +00:00
link2xt 0d61c13c58 DKIM-sign Content-Type and oversign all signed headers
Oversigning (including header name in DKIM-Signature
more times that it appears in the mail) prevents
adding more headers with the same name
without invalidating DKIM signature.

We don't want middleboxes to insert a second From header,
adding Cc field to mails that don't have one etc.
2024-07-08 14:27:11 +00:00
link2xt 733b9604ba dovecot: enable gzip compression on disk 2024-07-05 20:13:03 +00:00
link2xt 969fdd7995 Remove sieve to enable hardlink deduplication in LMTP
LMTP does not deduplicate messages
if sieve plugin is used.

We don't check for Auto-Submitted header anymore
as iOS application has a notification service
and should not display "You have a new message".
2024-07-05 19:22:26 +00:00
link2xt b1d11d7747 Revert 57c29c14a4
Apparently this causes outlook.com messages to be rejected
even though they don't use `l=` tag.
2024-07-03 20:36:31 +00:00
link2xt e948bdaea8 filtermail: do not allow ASCII armor without actual payload
Last line is removed as "optional checksum",
so it can contain anything.
Make sure that there is at least some actual payload
besides this line.
2024-07-03 19:36:07 +00:00
link2xt 17389b8667 Increase number of logged in IMAP sessions to 50000 2024-07-01 17:20:23 +00:00
link2xtandholger krekel 635b5de304 Replace bash with /bin/sh 2024-07-01 11:47:38 +02:00
link2xt 57c29c14a4 Reject DKIM signatures that do not cover the whole message body 2024-06-18 02:48:54 +00:00
link2xtandholger krekel 2b5d903cc5 Allow SKESK packets in encrypted mails
They are not used by Delta Chat now,
but this will allow to start using them in the future.
2024-06-13 19:48:59 +02:00
link2xtandGitHub c8d270a853 Check that OpenPGP has only PKESK and SEIPD packets (#323) 2024-06-12 17:21:37 +00:00
link2xt 72f4e9edbf filtermail: remove support for unencrypted MDNs
Delta Chat does not send them since 1.43.
1.44 has been released for a while already
and 1.46 is in the process of being released.
2024-06-11 16:18:39 +00:00