It looks like the reason it exists in the default configuration
is an example for <https://www.postfix.org/FILTER_README.html>
Using this filter requires configuration such as
"-o content_filter=filter:dummy" and we don't have it.
Default behavior is processing headers
that are actually part of the message body.
This is unlikely to be needed
as headers such as Received are not added into MIME parts,
but may result in broken DKIM signatures
if the body is modified.
There is a known bug that `quanta` clock
used by default sometimes jumps back:
<https://github.com/metrics-rs/quanta/issues/111>.
This results in `governor` rate limiter
incorrectly rejecting the messages
even when less than the configured burst size
has been sent.
Switched to standard monotonic clock
which does not have this problem.
I tested that this fixed the problem for me,
chatmail relay test `test_exceed_rate_limit`
stopped failing randomly.
Fixes <https://github.com/chatmail/filtermail/issues/166>
With "default" (like it was for SMTP) or not set (like it was for IMAP),
both TLS and STARTTLS are tried.
Trying STARTTLS against TLS port is going to timeout
because in STARTTLS server talks first,
but when connected to TLS port the server
waits for TLS client hello and does not send anything.
Should not actually matter in tests which connect successfully
on the first try because implicit TLS is tried first.
Postfix does not have jitter for deferred mails
and scans the queue periodically every
queue_run_delay (<https://www.postfix.org/postconf.5.html#queue_run_delay>).
As a result it is likely
to try delivering many deferred messages
at the same time.
Normally the number of outgoing connections
should be low even with unreachable destinations,
but after the server downtime
or if admin flushes the queue manually
it is possible that a lot of messages
to the same unreachable destination
expire at once and are moved
from "deferred" into the "active" queue.
Trying to deliver them all at once
may make the server run out of memory
by starting many LMTP clients.
Limiting the number of LMTP processes
turns OOM problem into head of line blocking problem.
Messages sent to reachable destinations
will be delayed as well,
but at least deferred messages will
get distributed over time.
In this case "active" queue may grow
(up to qmgr_message_active_limit defaulting to 20000),
but then admin may notice the problem
and solve it e.g. by making the destinations reachable
or setting up a transport map to route
messages for known dead servers into discard transport.
Eventually the problem should be solved
by filtermail-transport quickly returning temporary errors
for destinations which already have many messages queued,
then we can reduce "maxproc" further.
We never want to defer email with a tepporary error when it has destination
that we cannot deliver locally and don't want to relay.
To avoid doing this accidentally, set default action to "reject"
and then override it with the minimal restrictions per smtpd.
Submission ports already had smtpd_relay_restrictions=permit_sasl_authenticated,reject override.
Each smtpd port must have at least one of
reject, reject_unauth_destination, defer, defer_if_permit, defer_unauth_destination
according to <https://www.postfix.org/postconf.5.html#smtpd_relay_restrictions>.
I have set smtpd_relay_restrictions=reject_unauth_destination for port 25 and incoming reinject port,
and smtpd_relay_restrictions=permit_mynetworks,reject for outgoing reinject port.
This change reverts 06560dd071
Main reason for using the same address for sending
as the one used in DNS is to pass FCrDNS
(forward-confirmed reverse DNS) checks:
IP address used by SMTP client should resolve
to the domain which in turn resolves to the same IP.
chatmail relays don't do check reverse DNS
for incoming connections,
but other email servers may do and reject email
if the check does not pass.
Most chatmail relays only have one IP address per address family,
so this configuration does not change anything.
For chatmail relays that have multiple addresses
and only publishing one IP to DNS,
source address used for outgoing SMTP connections
should be the public IP.
This can be ensured by configuring the source
address in the routing table,
e.g. with the `src` argument
to `ip route add/change/replace` command.
Solving this by binding SMTP client address
on the application level prevents chatmail relays
from configuring alternative routes.
Besides, some chatmail relays are NATed
and NAT is responsible for translating the address to the public one,
in which case using `smtp_bind_address_enforce`
will result in unnecessarily deferring all mails.
doc/README.md was outdated, it did not include sphinxcontrib-mermaid.
Better use scripts/initenv.sh which already installs all dependencies
and is used in CI.
This is aimed at reducing SSD wear level.
SSDs wear out because of writes
according to <https://superuser.com/a/440219/1777696>,
so anything reducing the writes should be helpful.
For online users Maildir format that we use
results in first storing the message in new/
and then moving to cur/ and then maybe even deleting
it immediately for users with a single device
or bots. Syncing all these changes to disk
is unnecessary and wears SSDs.
This change was accidentally added in cf96be2cbb
Relay should not stop validating TLS certificates of other relays
just because it has a self-signed or externally managed certificate.
Externally managed certificate is likely to even be valid.
Similar data is already generated by fsreport
available for the relay operator
and metrics for prometheus are generated by mtail.
Closes <https://github.com/chatmail/relay/issues/431>
This elimitates the problem of acmetool failing
to start when nginx is installed already and uses port 80.
This also makes nginx redirect HTTP requests to HTTPS
for setups that don't have acmetool.
`String::from_utf8_lossy` always copies the data
for the case when it needs to replace invalid UTF-8,
but we don't want invalid UTF-8 anyway as replacing
invalid UTF-8 characters will break DKIM signature.
filtermail rate limiter is using leaky bucket
algorithm (GCRA).
Exceeting the limit requires sending
at least max_user_send_per_minute
messages to exhaust allowed burst,
and then sending messages faster
than the leak rate.
As we don't know how fast is the network
between the server and test runner,
try to send 3 times max_user_send_per_minute
messages to ensure the test does not
fail randomly.
Deployments to test servers will not be cancelled anymore,
but it is not clear if we even want it.
This setup is much simpler because it only depends
on GitHub Actions features and does not allocate
a runner just to sleep there and wait in the queue.
On FreeBSD 127.0.0.2 is not assigned to any interface by default,
so 127.0.0.2 source address hack cannot be used to make OpenDKIM
verify the signature instead of signing.
This change sets InternalHosts to `-` so no IP addresses
make OpenDKIM sign the message. Instead of IP address,
OpenDKIM in the outgoing pipeline is explicitly told
to sign messages by setting `{daemon_name}` macro to `ORIGINATING`.
I am running git-cliff 2.11.0.
Ran `git-cliff --init` to generate `cliff.toml`.
Removed emojis, replaced `doc` with `docs` to match chatmail core
convention.
smtp_tls_mandatory_protocols does not affect port 25
because we require STARTTLS on port 25 since commit
8d7e1dad0e
We don't have any smtpd ports with opportunistic TLS.
Submission ports require TLSv1.3 and starting with this commit
MX port will require TLSv1.2 instead of TLSv1.
I have not managed to connect using TLSv1.1
even without this fix to reproduce the problem,
but I have checked that setting
`-o smtpd_tls_mandatory_protocols=>=TLSv1.3`
does not allow to connect using TLSv1.2 anymore using
`openssl s_client -connect example.org:25 -starttls smtp -tls1_2`.
`smtpd_tls_protocols` setting is removed
because it does not affect anything except the internal ports
and its `git blame` points to the wrong commit.
According to
<https://www.postfix.org/postconf.5.html#smtp_tls_security_level>
for outgoing connections with smtp_tls_security_level
`encrypt` and higher (such as `verify` that we currently use)
the setting `smtp_tls_mandatory_protocols`
is used instead of `smtp_tls_protocols`.
According to `postconf -d`
(and `postconf` because the default is not changed)
current setting value is `smtp_tls_mandatory_protocols = >=TLSv1`.
But we only want to connect outside with TLS 1.2 and TLS 1.3.
`smtp_tls_protocols` which was already set to `>= TLSv1.2`
in commit 0155f32df6
only affected outgoing connections with the `may` level
exception set for nauta.cu domain via `smtp_tls_policy_maps`
which does not support STARTTLS at all.
We already require that outgoing connections
use STARTTLS so other servers need a valid TLS
certificate to accept messages from us.
It is then very unlikely that they cannot use TLS
to send messages to us.
Conversely, if they only can send messages to use without TLS,
it likely does not have STARTLS on its port 25
and then we don't want to accept messages from them
because we will likely not be able to reply.
Stalwart sends `NOTIFY=DELAY,FAILURE`
to request Delivery Status Notifications.
aiosmtpd does not support any parameters,
not just ORCPT, so we have to ignore all of them.
High-security mode could be configured
to handle more connections by increasing process_limit,
but has problems logging in many users at once after
each Dovecot restart or config reload.
Otherwise email providers which allow to bring your own domain
and use the same IP addresses for all customers
send wildcard certificate instead of the correct one
and Postfix refuses to connect with an error
server certificate verification failed for example.org[A.B.C.D]:25: num=62:hostname mismatch
It does not work because of `smtpd_proxy_filter`
forwarding the message to filtermail
and we cleanup the message once
filtermail reinjects it on port 10025.
I tested with -tls1_2 option
of openssl s_client
that TLS 1.2 connections
are no longer possible
on any ports except port 25.
Port 25 requires at least TLS 1.2
for encrypted connections.
`authclean` cleanup server is used by
reinjecting smtpd running on localhost:10025 by default.
It runs after filtermail
and currently removes `Received` header
to avoid leaking IP address.
Can as well be used to replace `Subject` lines
with `Subject: [...]`.
If there are multiple `Subject` lines,
all of them should be replaced.
This allows us to avoid dealing with
localized subjects, including SecureJoin
messages `vc-request` and `vg-request`
which can have Subject lines like
Subject: =?utf-8?q?Nachricht_von_nrn178fi4=40nine=2Etestrun=2Eorg?=
Replace \r\r\n in literal.eml test with \r\n
to make `test_filtermail_no_literal_packets`
actually reach `check_openpgp_payload()`
and make `check_openpgp_payload()` more strict.
unbound-control is not installed out of the box
and even once installed `flush_zone` does not seem
to work reliably.
Instead of trying to flush the cache from unbound,
we now query authoritative nameserver directly using `dig`.
perform_initial_checks may exit early
and not add `acme_account_url` if required DNS
records are not found.
In this case other `cmdeploy run` fails
with KeyError.
To avoid this, `acme_account_url` should always be set.
Unlike DNS checks, running acmetool
may not fail due to network errors,
so it is more reliable and should be checked first.
link2xt
