Commit Graph
100 Commits
Author SHA1 Message Date
link2xt 4bca7891a2 Switch SPF from fail to softfail (~all instead of -all)
This is recommended to prevent SPF failure
from rejecting the message early in case messages
are remailed without breaking DKIM.
2024-03-09 20:02:29 +00:00
link2xtandholger krekel 2e23e743fd dovecot: increase default_client_limit 2024-03-09 14:01:00 +01:00
link2xtandholger krekel edc593586b Implement "iterate" command in metadata server
Otherwise Dovecot times out when trying to iterate over metadata
of the folder. Apparently it happens when attempting to delete
folder from the server over IMAP.
2024-03-08 05:39:59 +01:00
link2xt 42e50b089f Push notification extension
This change adds XDELTAPUSH capability.

Delta Chat clients detecting this capability
can set /private/devicetoken IMAP metadata
on the inbox to subscribe for Apple (APNS)
notifications.

Notifications are implemented in a new
`chatmail-metadata` service
which handles requests to set /private/devicetoken
IMAP metadata from Delta Chat clients
and /private/messagenew requests from
push_notification_lua script.

To avoid sending notifications for
MDNs, webxdc updates and Delta Chat sync messages,
messages with Auto-Submitted header are ignored
by setting $Auto keyword (flag) on them in Sieve script
and skipping such messages in push_notification_lua script.
Outgoing messages are also ignored.
2024-03-06 19:00:04 +00:00
link2xt 14342383cf Generate our own single-line DKIM entry 2024-02-17 09:34:25 +00:00
link2xt ee25d35db1 Fix Python 3.9 support
I installed pyenv and then installed Python 3.9:
$ pyenv install 3.9
$ eval "$(pyenv init -)"
$ pyenv shell 3.9

In a clean repository I ran
$ scripts/cmdeploy init
$ scripts/cmdeploy run
$ scripts/cmdeploy dns
$ scripts/cmdeploy fmt

With the changes made all these commands work.

scripts/cmdeploy test fails some tests
using maildata fixture at
  importlib.resources.files(__package__).joinpath("mail-data")
line but this is not critical.
2024-02-17 09:27:02 +00:00
link2xt ee2115584b Run scripts/cmdeploy fmt 2024-02-15 14:07:10 +00:00
link2xt c8d9f20a48 fix: avoid "Argument list too long" in expunge.cron
Make `find` look for accounts.
2024-02-13 07:37:23 +00:00
link2xtandmissytake 9e9ab80422 Do not subscribe to TLS reports 2024-01-31 14:35:54 +01:00
link2xt 5b9debfbdf Test dict protocol handler as a separate function 2024-01-30 23:49:17 +00:00
link2xt 788309b85a Merge Postfix TLS hardening
https://github.com/deltachat/chatmail/pull/97
2024-01-30 18:45:34 +00:00
link2xt 5bbb3d9b21 Rewrite and document smtpd_tls_exclude_ciphers 2024-01-27 02:10:02 +00:00
link2xt 6bc2186912 postfix: set tls_preempt_cipherlist 2024-01-26 19:45:53 +00:00
link2xt 8d5f91bf98 postfix: use new syntax for TLS version 2024-01-26 19:42:18 +00:00
link2xt 05bdf65996 Add ADSP DNS record
ADSP RFC 5617 is declared historic because of no deployment:
<https://datatracker.ietf.org/doc/status-change-adsp-rfc5617-to-historic/>

However, it is declared as supported by <https://github.com/fastmail/authentication_milter>.

OpenDKIM has a release note from 2014-12-27 saying "Discontinue support for ADSP"
and does not support ADSP anymore.

Anyway, it does not hurt to publish a TXT record
indicating the strictest possible ADSP policy
that we apply to all incoming mail ourselves.
Unlike DMARC which allows either SPF or DKIM to pass,
ADSP requires that DKIM passes.
2024-01-26 15:04:09 +00:00
link2xt 6d6217812d Add missing login map 2024-01-25 23:17:57 +00:00
link2xt ea36e73b8e postfix: require that login matches envelope FROM
Testing that envelope FROM matches From: header
already happens in filtermail
and tested with `test_reject_forged_from`.

The most important part here is
`reject_sender_login_mismatch` check
documented in
<https://www.postfix.org/postconf.5.html#reject_sender_login_mismatch>.
2024-01-25 23:17:57 +00:00
link2xt 5588e13e54 Create opendkim configs before installing 2024-01-24 13:29:24 +00:00
link2xt 7c7f1cad7f Replace rspamd with OpenDKIM
OpenDKIM configuration
has two Lua scripts defining strict DKIM policy.

screen.lua filters out signatures that do not correspond
to the From: domain so they are not even checked.
final.lua rejects mail if it is not outgoing
and has no valid DKIM signatures.

OpenDKIM is configured as a milter on port 25 smtpd
to check DKIM signatures
and on mail reinjecting smtpd
to sign outgoing messages with DKIM signatures.
2024-01-24 13:29:24 +00:00
link2xt a6b333672d Revert "Pin deltachat-rpc-server version"
This reverts commit 3940b9256d.

1.133.2 release has OpenSSL 3.2 downgraded to 3.1 and pass the tests.
2024-01-24 03:53:23 +00:00
link2xt 29857143c9 Dovecot: setup METADATA
There is no dictionary to set additional attributes,
but admin email can already be retrieved:

? GETMETADATA "" (/shared/admin)
* METADATA "" (/shared/admin {27}
mailto:root@c20.testrun.org)
? OK Getmetadata completed (0.001 + 0.000 secs).
2024-01-24 01:55:13 +00:00
link2xtandholger krekel 9f31357a9c Remove postscreen-related entries from Postfix master.cf
All these entries are related to `postscreen` service
which is currently not enabled.

For documentation see https://www.postfix.org/POSTSCREEN_README.html

If we later want to enable it, we can readd uncommented entries
and document it.
2024-01-24 02:08:30 +01:00
link2xtandholger krekel c94ef0379a Update pip and setuptools in scripts/initenv.sh
This is to support Debian 11 which ships setuptools
that do not support `-e` without setup.py
2024-01-24 01:31:48 +01:00
link2xt bc66325d71 Cleanup Received headers after filtermail as well 2024-01-23 21:27:23 +00:00
link2xt 27f44ae911 Cleanup Received headers only on outgoing mail 2024-01-23 20:28:34 +00:00
link2xt 3940b9256d Pin deltachat-rpc-server version 2024-01-22 14:44:39 +00:00
link2xt 4886ff9b86 Do not use redirect on /cgi-bin/newemail.py
Delta Chat does not follow redirects,
so it breaks old QR codes printed on paper
and published on various web pages.
2024-01-21 13:20:00 +00:00
link2xt 86940b2ee1 Stop requesting DMARC reports
Nobody reads these XML reports
and we know our DKIM is valid
when `cmdeploy dns` is happy.
2024-01-17 01:37:54 +00:00
link2xt 24fb9eb65b Nicer /new URL for new accounts and redirect GET requests
If user types in https://nine.testrun.org/new manually
in the browser, at least Firefox and Brave suggest
to open the app after following the redirect.
2024-01-15 13:06:29 +00:00
link2xt 700256c273 Split DKIM checks into separate rules
Now errors distinguish between missing DKIM singature,
missing DNS entry or invalid DKIM signature.
2024-01-15 02:36:10 +00:00
link2xt d575d62b18 rspamd: give the reason to MTA when incoming mail is rejected
This is not secret but makes it easier for mail server admins
to debug why chatmail does not accept their emails.
If the server generates bounce messages, users will also see this
and can redirect to their server support.
It also shows up in /var/log/rspamd/rspamd.log on chatmail server.
2024-01-14 13:12:46 +00:00
link2xt 8cdf8ce376 Merge 'rspamd' branch, replacing OpenDKIM with rspamd
This adds DKIM and SPF checks and replaces OpenDKIM with rspamd for
DKIM signing.
2024-01-14 09:30:31 +00:00
link2xt 7c9abfbde3 Reject on DKIM PERMFAIL and SPF PERMFAIL as well 2024-01-14 09:19:04 +00:00
link2xt 95de87a325 Fixup rspamd disabled.conf deployment message 2024-01-14 08:45:39 +00:00
link2xt 5366df8dc6 Replace rspamd rule weights with a strict rule 2024-01-14 08:45:23 +00:00
link2xt 0a6db5161d Remove unused _configure_opendkim 2024-01-12 19:05:23 +00:00
link2xt 62e25e44fd Disable ratelimit module like other modules 2024-01-12 18:56:11 +00:00
link2xt ce9fe920dc Do not return anything from remove_opendkim() 2024-01-12 18:47:57 +00:00
link2xt c171866faf Actually disable phising, rbl and hfilter 2024-01-12 18:46:07 +00:00
link2xt 542d63888a nginx: redirect www. to non-www 2024-01-12 13:34:23 +00:00
link2xt 449f8a014c Fix indentation in nginx.conf.j2 2024-01-12 13:34:23 +00:00
link2xt 57764d0cf5 dns: require www. subdomain and request TLS certificate for it 2024-01-12 13:34:23 +00:00
link2xt c39a79e26a dns: check mta-sts CNAME directly without resolving to IP 2024-01-12 13:34:23 +00:00
link2xt b6622fc68e chore: run scripts/cmdeploy fmt 2024-01-12 12:18:28 +00:00
link2xt 75b41641f0 doveauth: fix home directory returned from lookup_passdb
It is currently unused, but better have it correct
in case of enabling debugging options such as rawlogs.
2024-01-08 16:40:08 +00:00
link2xtandholger krekel 30a61972fb Update autoconfig XML URL with RFC draft
Old page does not exist anymore and linking to web archive is not nice.
2024-01-08 16:33:04 +01:00
link2xtandmissytake 70b0e9d5e5 postfix: increase compatibility_level to 3.6 2023-12-27 00:29:12 +01:00
link2xt a44ed0aeb3 Use dig +short option to simplify DNS parsing
Without this option parsing of answer was flaky
as for long records like
_submission._tcp.nine.testrun.org.
dig printed the result with a space rather
than tab as a separator and .split("\t") did not work.

This change makes the `dig` command print the answer
in the form we need so there is no need for complex parsing
other than taking the first line.

`-r` option is added to make sure options are not changed by .digrc
in the root home directory.
2023-12-22 21:49:12 +00:00
link2xt f5bfa6bd56 test: test scanning QR code 2023-12-21 22:22:38 +00:00
link2xt 81a6f8808b fix: escape login and password when passed from dovecot to doveauth
This should allow to use / in the password
2023-12-21 22:22:38 +00:00
link2xt be3685519f Document ports 80 and 443 and add more hyperlinks 2023-12-21 16:16:17 +00:00
link2xtandmissytake 85a9183b61 Do not call show_dns with run args 2023-12-19 19:39:49 +01:00
link2xt 3d00ca1672 doveauth: add support for Dovecot 2.3.16 2023-12-18 19:44:11 +00:00
link2xtandholger krekel 485bbb9cbd Let acmetool manage port 80
This avoids circular dependency with nginx.
nginx needs a certificate to start
and getting a certificate requires someone
listening on port 80.
2023-12-18 16:36:36 +01:00
link2xt 92ca3283fd Add metrics 2023-12-14 22:22:10 +00:00
1c0543cb46 Update README.md
Co-authored-by: missytake <missytake@systemli.org>
2023-12-12 23:37:16 +01:00
link2xtandholger krekel dde879c7fc Add scripts/cmdeploy 2023-12-12 23:37:16 +01:00
link2xt cf95dfd49d Setup unbound DNS resolver 2023-12-12 21:52:05 +00:00
link2xt 4a96f19faf s/fuff/ruff/ 2023-12-12 20:29:34 +00:00
link2xt a2a78c0aff Do not attmpt to activate venv from scripts/initenv.sh
If you run it as scripts/initenv.sh,
activating venv is useless as bash will exit immediately afterwards.

If you `source` it as suggested by README.md,
`set -e` will set the flag for the current shell
and your shell will exit as soon as some command returns non-zero status,
e.g. cmdeploy fails or you simply do `ls /foo/bar/baz` and `ls`
complains that `/foo/bar/baz` does not exist.
2023-12-12 20:22:34 +00:00
link2xtandmissytake d7683ed3f7 Move ssl_certificate back to http and fix indentation 2023-11-25 00:39:27 +01:00
link2xt 322bc9a3aa Set critical flag on generated CAA record
This does not really matter as Let's Encrypt
supports current CAA `issue` syntax,
but may be useful if more records are added and this flag is copy-pasted.

For reference: <https://www.rfc-editor.org/rfc/rfc8659#name-critical-flag>
2023-11-13 15:12:32 +00:00
link2xtandholger krekel e4009854dc Add NOTIFY capability
Delta Chat does not use it now,
but should: <https://github.com/deltachat/deltachat-core-rust/issues/4983>
Having no capability will confuse whoever develops it.
2023-11-12 20:41:29 +01:00
link2xt 9e14a741c3 Autoformat tests with black 2023-11-08 20:29:44 +00:00
link2xt 01fcb9ae0e Fix None dereference in benchmarks 2023-11-08 20:29:21 +00:00
link2xt 064f6d36ad Fix path in scripts/bench.sh 2023-11-08 20:23:14 +00:00
link2xt 251aac18fb fix(dictproxy): check that user exists and create it in a transaction
Otherwise user may be already created by another connection
as checking if the user exists happens
in a different read-only transaction.
This happens when Delta Chat connects IMAP and SMTP at the same time.

Also update last_login time on login.
2023-11-08 19:34:17 +00:00
link2xtandholger krekel f46bf2f670 Remove authentication logs from dictproxy
They log the passwords and make it difficult to spot actual exceptions.
2023-11-07 21:04:33 +01:00
link2xt 5633c0612e dovecot: increase number of simultaneous connections handled by imap-login
Otherwise deltachat core CI running fails with "Connection queue full"
error on IMAP connections.
2023-10-29 19:09:33 +00:00
link2xt f75eb0658c Require that passwords are at least 10 characters long 2023-10-28 13:38:15 +00:00
link2xt 7c5ec1e0df Add scripts/generate-dns-zone.sh 2023-10-24 21:23:20 +00:00
link2xtandholger krekel 7ee84b44df Use tox -c option 2023-10-22 14:48:30 +02:00
link2xtandholger krekel 02205246dd Setup deltachat dependency in init.sh 2023-10-22 14:48:30 +02:00
link2xtandholger krekel eb69dd58f7 Setup CI 2023-10-20 15:18:17 +02:00
link2xtandholger krekel 31c45f951d dictproxy: use crypt instead of doveadm pw 2023-10-20 14:05:25 +02:00
link2xt 902f98c9ba Set syslog name for reinject proxy 2023-10-19 03:22:27 +00:00
link2xtandGitHub 89311063f8 Turn filtermail into a beforequeue handler and implement rate limit 2023-10-19 03:04:00 +00:00
link2xt 1cdc5d1351 Revert "open a persistent client between the BeforeQueueHandler and postfix smtpd without content filter"
This reverts commit fb2ea27477.
2023-10-19 02:22:38 +00:00
link2xt 30680cb170 filtermail: port is args[0], not args[1] 2023-10-19 02:22:30 +00:00
link2xt c514fb00a3 Import SMTP from aiosmtpd.lmtp, not aiosmtpd.smtp 2023-10-19 02:22:15 +00:00
link2xtandholger krekel 329b845c79 Configure journald to retain logs for 3 days 2023-10-18 22:54:50 +02:00
link2xtandholger krekel 76765164dc Deploy nginx and autoconfig XML 2023-10-18 18:32:28 +02:00
link2xt 1ea25eb28c Add /etc/mailname 2023-10-17 19:27:57 +00:00
link2xt 5936f7a3be postfix: disable virtual and local MDAs
We do not need them, all mails are delivered to Dovecot over LMTP.
2023-10-16 22:21:39 +00:00
link2xt 983ffa6236 Disable acmetool redirector 2023-10-16 22:12:45 +00:00
link2xt 179c79a052 Allow to send securejoin 2023-10-16 21:15:56 +00:00
link2xt 408da296f1 test.sh: do not run slow tests by default 2023-10-16 20:13:41 +00:00
link2xt cf1be90115 Switch from BLF-CRYPT to SHA512-CRYPT 2023-10-15 21:42:14 +00:00
link2xt 5781d3b04e Make scripts/measure_tls_and_logins.py executable 2023-10-15 21:42:14 +00:00
link2xt 862b09d268 dovecot: enable authentication cache 2023-10-15 21:42:14 +00:00
link2xt 9b438a7a96 Test different users logging in with the same password 2023-10-15 21:42:14 +00:00
link2xt a107fb3cca Avoid reusing accounts between tests
Add time as a prefix.
2023-10-15 21:42:14 +00:00
link2xt 2a59cd4702 Do not apt update more than once a day 2023-10-15 20:31:13 +00:00
link2xt 57df5e254c Require that encrypted messages have "..." as a Subject 2023-10-15 19:10:25 +00:00
link2xt 20b7af9d71 filtermail: more robust check_encrypted() 2023-10-15 19:10:14 +00:00
link2xt b05154b818 Remove chatmaild merging from the plan 2023-10-15 16:09:29 +00:00
link2xt aab0a1f992 Update chatmaild README 2023-10-15 16:08:58 +00:00
link2xt 40ad67dc20 Merge doveauth and filtermail folders 2023-10-15 16:07:26 +00:00
link2xt b548a8ddbd Merge doveauth and filtermail into chatmaild 2023-10-15 15:57:36 +00:00
link2xt 262eb36a5c Rename chatmail-pyinfra into deploy-chatmail 2023-10-15 15:40:17 +00:00