Commit Graph

1477 Commits

Author SHA1 Message Date
Jagoda Estera Ślązak a9dd9fe3e0 docs: Update overview diagrams (#995)
Adds a detailed diagram describing
all paths a message can take,
that takes into account postfix services.

Additionally, adds OpenDKIM to dependency
diagram.

Fixes: #771
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
Co-authored-by: missytake <missytake@systemli.org>
2026-06-03 12:24:32 +02:00
missytake aa846c3478 fix: expire empty directories (#994)
* fix: respect --dry when expiring empty directories

Co-authored-by: j4n <j4n@systemli.org>
2026-06-03 10:42:28 +02:00
feld 921080125f Aggressive LMTP header cleanup (#816)
This will remove all headers possible during LMTP delivery, except:

- From: required or core does not process the message correctly.
  Also required for cleartext compatibility.
- Message-Id: required for clients to know which messages have been
  downloaded
- Chat-Is-Post-Message: is required for our attachment previews
- Content-Type: required
- For Cleartext compability: To, CC, In-Reply-To, References, Subject,
  and Date
- For Chatmail future expansion, allow Chat-*
- Permit the entire Secure-Join* namespace

Co-authored-by: holger krekel  <holger@merlinux.eu>
2026-06-03 08:43:36 +02:00
Andrey 0xdc09 8a60245183 fix(resolver): disable negative caching (#170) 2026-06-02 12:04:32 +02:00
Jagoda Estera Ślązak d898f41064 fix: Always deploy unbound.conf.d/chatmail.conf (#993)
This fixes issue with negative cache
only disabled in ipv4-only mode.

Follow up to #992

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-02 12:02:07 +02:00
Jagoda Estera Ślązak e9e012234b feat: Disable negative cache in unbound (#992)
Related:
- https://github.com/chatmail/relay/issues/543
- https://github.com/chatmail/filtermail/pull/170

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-02 10:48:28 +02:00
j4n 6d6517f8f3 chore: add filtermail.mtail so filtermail failures can be monitored (#169)
Example Prometheus queries:

Outbound errors by reason:
  sum by (reason) (rate(filtermail_transport_error_total[5m]))

filtermail-incoming drops by reason:
  sum by (reason) (rate(filtermail_inbound_drop_total[5m]))
2026-06-01 21:51:34 +02:00
Jagoda Estera Ślązak e9a63222e6 fix(smtp-server): Correct error when EOF while reading DATA (#168)
Log
`Unexpected EoF while receiving DATA!`
instead of
`Malformed DATA line without CRLF ending!`

Fixes: #165

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-01 16:27:30 +02:00
Jagoda Estera Ślązak bb40c5bb21 fix: Check if all required ports are available for filtermail (#983)
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-01 11:10:25 +02:00
Jagoda Estera Ślązak a229f1bc45 chore(deps): Upgrade filtermail to v0.7 (#982)
## 0.7.0 - 2026-05-26

### Bug Fixes

- Do not crash if accepting new connection fails

### Documentation

- *(readme)* Remove docs for options removed in da9a116

### Features

- [**breaking**] Remove passthrough options that allowed unencrypted mail to pass

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-01 09:55:50 +02:00
dependabot[bot] 893ceab107 chore(deps): Bump taiki-e/install-action from 2.75.28 to 2.79.6 (#164)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.75.28 to 2.79.6.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/51cd0b8c0499559d9a4d75c0f5c67bec3a894ec8...f48d2f8ba2b452934c948b7be1a768079c3632ff)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.79.6
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-01 09:51:57 +02:00
l a6361a2194 fix: switch ratelimiter to MonotonicClock (#167)
There is a known bug that `quanta` clock
used by default sometimes jumps back:
<https://github.com/metrics-rs/quanta/issues/111>.
This results in `governor` rate limiter
incorrectly rejecting the messages
even when less than the configured burst size
has been sent.

Switched to standard monotonic clock
which does not have this problem.
I tested that this fixed the problem for me,
chatmail relay test `test_exceed_rate_limit`
stopped failing randomly.

Fixes <https://github.com/chatmail/filtermail/issues/166>
2026-06-01 09:51:33 +02:00
link2xt 4ba19b0031 test: set socket security for IMAP and SMTP to "TLS" in "dclogin"
With "default" (like it was for SMTP) or not set (like it was for IMAP),
both TLS and STARTTLS are tried.
Trying STARTTLS against TLS port is going to timeout
because in STARTTLS server talks first,
but when connected to TLS port the server
waits for TLS client hello and does not send anything.

Should not actually matter in tests which connect successfully
on the first try because implicit TLS is tried first.
2026-05-28 22:29:08 +00:00
holger krekel 5eab3a5a25 try using cmlxc main branch fix for delete-server issue 2026-05-28 21:40:05 +02:00
holger krekel 30729d9be0 fix: core 2.50.0 does not have delete_server_after config anymore. 2026-05-28 21:40:05 +02:00
Jagoda Estera Ślązak febfb90510 chore(release): prepare for 0.7.0 (#163)
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-05-26 06:56:00 +02:00
Jagoda Estera Ślązak c84a96ac78 docs(readme): Remove docs for options removed in 664ad57 (#162) 2026-05-26 02:25:51 +09:00
link2xt 4b04aae83b feat: reduce maximal_queue_lifetime from 5d to 2d
If the message is not delivered within 2 days,
it is unlikely to be delivered in 5 days either.
2026-05-20 19:27:58 +00:00
link2xt 0eed92171c fix: reduce maxproc for filtermail-transport LMTP client to 500
This further reduces it from 1000.
For small servers this may be needed if they have low memory.
For large servers may be increased manually for now.
2026-05-20 15:39:11 +00:00
link2xt a5b9a98baa fix: limit the number of LMTP clients for filtermail-transport to 1000
Postfix does not have jitter for deferred mails
and scans the queue periodically every
queue_run_delay (<https://www.postfix.org/postconf.5.html#queue_run_delay>).
As a result it is likely
to try delivering many deferred messages
at the same time.

Normally the number of outgoing connections
should be low even with unreachable destinations,
but after the server downtime
or if admin flushes the queue manually
it is possible that a lot of messages
to the same unreachable destination
expire at once and are moved
from "deferred" into the "active" queue.

Trying to deliver them all at once
may make the server run out of memory
by starting many LMTP clients.
Limiting the number of LMTP processes
turns OOM problem into head of line blocking problem.
Messages sent to reachable destinations
will be delayed as well,
but at least deferred messages will
get distributed over time.

In this case "active" queue may grow
(up to qmgr_message_active_limit defaulting to 20000),
but then admin may notice the problem
and solve it e.g. by making the destinations reachable
or setting up a transport map to route
messages for known dead servers into discard transport.

Eventually the problem should be solved
by filtermail-transport quickly returning temporary errors
for destinations which already have many messages queued,
then we can reduce "maxproc" further.
2026-05-19 22:17:04 +00:00
link2xt ab2d807084 fix: set relay restrictions per smtpd service with default reject
We never want to defer email with a tepporary error when it has destination
that we cannot deliver locally and don't want to relay.
To avoid doing this accidentally, set default action to "reject"
and then override it with the minimal restrictions per smtpd.

Submission ports already had smtpd_relay_restrictions=permit_sasl_authenticated,reject override.

Each smtpd port must have at least one of
reject, reject_unauth_destination, defer, defer_if_permit, defer_unauth_destination
according to <https://www.postfix.org/postconf.5.html#smtpd_relay_restrictions>.

I have set smtpd_relay_restrictions=reject_unauth_destination for port 25 and incoming reinject port,
and smtpd_relay_restrictions=permit_mynetworks,reject for outgoing reinject port.
2026-05-19 15:54:15 +00:00
j4n ce05b26c77 ci: auto-trigger docker build on release tag push
docker-dispatch.yaml previously only fired on push to main and manual
workflow_dispatch, so tagging 1.11.0 did not build the release image.
This change adds matching of X.Y.Z tag.
2026-05-19 14:58:05 +02:00
missytake 77ed93fb7a docs: add scripts/initenv.sh to upgrade instructions 2026-05-18 10:35:25 +02:00
missytake 39d1ecaa03 chore(release): prepare for 1.11.0 2026-05-15 17:13:58 +02:00
link2xt 847e19546b fix: do not crash if accepting new connection fails
If we run out of file descriptors, we cannot do anything
to accept queued connections, so at least don't crash the process.
Fixes <https://github.com/chatmail/filtermail/issues/139>.
2026-05-14 20:59:59 +00:00
holger krekel a266ffd060 fix: fix #972 by increasing file descriptors for filtermail 2026-05-14 22:40:25 +02:00
holger krekel 664ad571b6 feat!: remove passthrough options that allowed unencrypted mail to pass
see https://github.com/chatmail/relay/pull/970/ for the related removal of chatmail.ini options
2026-05-14 20:59:13 +02:00
holger krekel a47bb94143 feat: warn about any unused chatmail.ini parameter at the end of "cmdeploy run" 2026-05-14 20:58:47 +02:00
holger krekel 43ae9fee5c feat!: ignore passthrough_sender and passthrough_recipients to eliminate one more source of unencrypted messages
When running "cmdeploy run" operators will see a warning if their chatmail.ini contains these unused options.
2026-05-14 20:58:47 +02:00
holger krekel 42dc781d7d feat: make turn_socket_path configurable, and cleanup tests and turnserver code.
this is originally motivated by https://github.com/chatmail/relay/pull/840
2026-05-13 21:02:28 +02:00
missytake ed664cd9cd feat(config): load default values from Config(), not chatmail.ini.f (#853)
* config: comment out values in chatmail.ini.f, so defaults take precedence
* config: remove testrun-specific overrides
* config: remove filtermail ports from default ini
2026-05-12 22:44:06 +02:00
holger krekel 26a13fbc26 feat: DKIM-sign bounce messages (mainly "user does not exist")
This was originally based on Jagoda's https://github.com/chatmail/relay/pull/874
but then the postfix config was simplified, and it comes with a simpler and more robust test.
2026-05-12 14:19:11 +02:00
missytake d054fbb5aa docs: document how to upgrade to new version (#965)
Co-authored-by: Jagoda Estera Ślązak <128227338+j-g00da@users.noreply.github.com>
2026-05-12 14:13:28 +02:00
j4n def08c52f4 feat(doc/docker): Introduce docker images in documentation 2026-05-12 13:45:21 +02:00
Jagoda Estera Ślązak 32cfa9c76c chore(deps): Upgrade filtermail to v0.6.6 (#967)
## 0.6.6 - 2026-05-12

### Bug Fixes

- Return HTTP 200 because madmail expects it, and make sure https is immediately retried when SMTP fails

### Features

- Improved SMTP error responses

### Miscellaneous Tasks

- Remove mac and windows from matrix tests
- Run cmlxc tests in all classic/classic-ipv4/madmail combinations

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-05-12 13:24:23 +02:00
Jagoda Estera Ślązak 91d09ab370 chore(release): prepare for 0.6.6 (#156)
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-05-12 13:02:17 +02:00
holger krekel e2be5dc92d ci: run cmlxc tests in all classic/classic-ipv4/madmail combinations 2026-05-12 12:46:20 +02:00
Jagoda Estera Ślązak d74e5b85b3 feat: Improved SMTP error responses (#147)
Fixes: #140

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-05-12 12:37:02 +02:00
Jagoda Estera Ślązak b7a213ff1b ci: Remove mac and windows from matrix tests (#154)
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-05-12 12:20:25 +02:00
holger krekel 3896044f36 fix: return HTTP 200 because madmail expects it, and make sure https is immediately retried when SMTP fails (#153) 2026-05-12 12:13:00 +02:00
Jagoda Estera Ślązak c0b207c320 chore(deps): Upgrade filtermail to v0.6.5 (#966) 2026-05-12 10:37:28 +02:00
Jagoda Estera Ślązak e26cca0260 chore(release): prepare for 0.6.5 (#152)
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-05-12 09:52:13 +02:00
dependabot[bot] 272744115f chore(deps): Bump taiki-e/install-action from 2.75.22 to 2.75.28 (#148)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.75.22 to 2.75.28.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/cf525cb33f51aca27cd6fa02034117ab963ff9f1...51cd0b8c0499559d9a4d75c0f5c67bec3a894ec8)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.75.28
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-12 07:01:30 +02:00
l a65e7a1490 fix: Validate mail data (#150)
Do not allow bare CR and LF etc.
2026-05-12 07:00:21 +02:00
holger krekel 36921dd29f fix: advertise 8BITMIME to prevent conversion after DKIM signing (#149)
for bounces postfix sets Content-Transfer-Encoding: 8bit
which causes a quoted-printable conversion when filtermail SMTP does not offer 8BITMIME,
which in turn breaks DKIM verification.
2026-05-12 06:56:11 +02:00
holger krekel 4ebde2825d feat: support setup without domain, with only an IPv4 address (#963)
* dovecot: enable login names with square brackets

* config: make IPv4-only relays use self-signed TLS certs

* postfix: make delivery for IP-only relays work

* cmdeploy: skip DNS checks for IPv4 only relays

* www: generate dclogin codes for IPv4-only relays

* opendkim: disable DKIM signing on ipv4-only relays

* get delivery working

* get tests working on IPv4 only machine

* doc: document IPv4-only relays

* dns: warn if mail_domain is an IP, instead of checking DNS

* config: validate domains when formatting them

* ci: add cmlxc testing for no-DNS relays

* ci: run no-dns and normal CI in parallel

* retain "config.mail_domain" as the domain part of @ email addresses, so for ipv4 relays  "[1.2.3.4]" and introduce config.ipv4_relay and config.mail_domain_bare helpers.

* ci: migrate from --no-dns to --type ipv4 for cmlxc compatibility

* cleanup dead code, fix docs, fixate cmlxc version

---------

Co-authored-by: missytake <missytake@systemli.org>
2026-05-11 21:52:33 +02:00
holger krekel 6a7e6ce9e7 feat: expose metadata "maxsmtprecipients" value
also add metadata tests and make metadata lookup method more readable by using structural match/case syntax
2026-05-11 20:08:38 +02:00
holger krekel 8db668c037 fix(logging): log all http requests to syslog 2026-05-10 23:32:42 +02:00
holger krekel 45fafa10a9 fix: legacy token metadata storage used list type, but if no new setmetadata happened, the user would not be notified at all. 2026-05-08 21:39:40 +02:00
missytake ee435a7ef7 fix(dns): query correct NS if MNAME server is hidden (#954)
replaces #870
fix #851

* fix(dns): address possible IndexError
* fix(dns): remove redundant docstring
* fix(dns): don't make NS explicit if None
* bump cmlxc to 0.13.5 which fixes a powerdns config issue
* remove the unneccessary SOA mocks, simplify mock tests, and run ruff format

Co-authored-by: holger krekel <holger@merlinux.eu>
2026-05-08 19:34:42 +02:00