Compare commits

..

35 Commits

Author SHA1 Message Date
holger krekel
4c9eaf3d63 turn chatmail-expire live 2025-10-22 18:51:47 +02:00
holger krekel
7a91aa539d add a bit more date info 2025-10-22 18:51:14 +02:00
holger krekel
4c8ad95244 allow files and directories to vanish while we iterate over them 2025-10-22 18:11:41 +02:00
holger krekel
8d65770c28 fixup after trying out on nine: don't remove large messages after 7 days if they are in the new/ folder (this is old behaviour)
and provide reporting about which mailbox is being scanned.
2025-10-21 21:46:51 +02:00
holger krekel
0931da21b8 make sure fsreport can run on empty mailbox dir 2025-10-21 18:43:37 +02:00
holger krekel
11a8f8cf9e try fix CI 2025-10-21 18:43:37 +02:00
holger krekel
0aa255e3f1 replace expunge mentioning in architecture 2025-10-21 18:43:37 +02:00
holger krekel
6c4764b452 Apply suggestions from code review
fix typo

Co-authored-by: l <link2xt@testrun.org>
2025-10-21 18:43:37 +02:00
holger krekel
c1f08a9afe simplify and beautify formatting and sizes 2025-10-21 18:43:37 +02:00
holger krekel
5c8afb377e also run fsreport 2025-10-21 18:43:37 +02:00
holger krekel
8225a9f398 use systemd timer instead of cron-job for expiry (tested by hand on c2) 2025-10-21 18:43:37 +02:00
holger krekel
eb221ca1af unify K output 2025-10-21 18:43:37 +02:00
holger krekel
93421b317b always use "H" for printing numbers, and make "chatmail.ini" file optional, defaulting to where it is on chatmail relays 2025-10-21 18:43:37 +02:00
holger krekel
777be107f3 fix another invocation 2025-10-21 18:43:37 +02:00
holger krekel
8b81d5b5d6 unify chatmail-fsreport and chatmail-expire to both just require a chatmail.ini file 2025-10-21 18:43:37 +02:00
holger krekel
e6a2906e82 cosmetic: refine summary and fix typo 2025-10-21 18:43:37 +02:00
holger krekel
67ba4ac99e address four review comments from link2xt 2025-10-21 18:43:37 +02:00
holger krekel
8cadf51387 prefix new commands 2025-10-21 18:43:37 +02:00
holger krekel
ce4bb97294 remove superflous totalsize attribute 2025-10-21 18:43:37 +02:00
holger krekel
3a0c629f3b during fsreport (reporting) don't store all mailbxoes but categorize them immediately, provide a few command line options to select 2025-10-21 18:43:37 +02:00
holger krekel
8df53c2655 fix lint issues 2025-10-21 18:43:37 +02:00
holger krekel
3fd3ab1a68 some renaming 2025-10-21 18:43:37 +02:00
holger krekel
d74f792787 remove superflous Stats class 2025-10-21 18:43:37 +02:00
holger krekel
1135372b81 further reduce code 2025-10-21 18:43:37 +02:00
holger krekel
c9f80bffd8 no reporting by default, and adding a summary line 2025-10-21 18:43:37 +02:00
holger krekel
10e53d17e8 don't globally collect files anymore to avoid using growing-with-number-of-mailboxes ram 2025-10-21 18:43:37 +02:00
holger krekel
01ca2a8b91 more streamline 2025-10-21 18:43:37 +02:00
holger krekel
fb01944f0d strike superflous code 2025-10-21 18:43:37 +02:00
holger krekel
a90a651ba0 fix comment 2025-10-21 18:43:37 +02:00
holger krekel
7d74b46502 add argument parsing for reporting 2025-10-21 18:43:37 +02:00
holger krekel
6d3e690653 add basic command line parsing for expire + some streamlining 2025-10-21 18:43:37 +02:00
holger krekel
ed7a70ba31 refactor and write tests for overall expiry/report runs 2025-10-21 18:43:37 +02:00
holger krekel
023116bc91 add summary reporting, rework expiry logic 2025-10-21 18:43:37 +02:00
holger krekel
b13929119b do all expunging in python 2025-10-21 18:43:37 +02:00
holger krekel
a4152140ca move delete_inactive_users to new implementation 2025-10-21 18:43:37 +02:00
12 changed files with 38 additions and 50 deletions

View File

@@ -1 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: Mutual Help Chat Group
url: https://i.delta.chat/#6CBFF8FFD505C0FDEA20A66674F2916EA8FBEE99&a=invitebot%40nine.testrun.org&g=Chatmail%20Mutual%20Help&x=7sFF7Ik50pWv6J1z7RVC5527&i=X69wTFfvCfs3d-JzqP0kVA3i&s=ibp-447dU-wUq-52QanwAtWc
about: If you have troubles setting up the relay server, feel free to ask here.

View File

@@ -2,15 +2,6 @@
## untagged
- acmetool: use ECDSA keys instead of RSA
([#689](https://github.com/chatmail/relay/pull/689))
- Require TLS 1.2 for outgoing SMTP connections
([#685](https://github.com/chatmail/relay/pull/685))
- require STARTTLS for incoming port 25 connections
([#684](https://github.com/chatmail/relay/pull/684))
- filtermail: run CPU-intensive handle_DATA in a thread pool executor
([#676](https://github.com/chatmail/relay/pull/676))
@@ -30,7 +21,7 @@
([#650](https://github.com/chatmail/relay/pull/650))
- filtermail: accept mails from Protonmail
([#616](https://github.com/chatmail/relay/pull/616))
([#616](https://github.com/chatmail/relay/pull/655))
- Ignore all RCPT TO: parameters
([#651](https://github.com/chatmail/relay/pull/651))
@@ -63,7 +54,7 @@
to only do a single iteration over sometimes millions of messages
instead of doing "find" commands that iterate 9 times over the messages.
Provide an "fsreport" CLI for more fine grained analysis of message files.
([#637](https://github.com/chatmail/relay/pull/637))
([#637](https://github.com/chatmail/relay/pull/632))
## 1.7.0 2025-09-11

View File

@@ -180,10 +180,6 @@ The components of chatmail are:
- [Iroh relay](https://www.iroh.computer/docs/concepts/relay)
which helps client devices to establish Peer-to-Peer connections
- [TURN](https://github.com/chatmail/chatmail-turn)
to enable relay users to start webRTC calls
even if a p2p connection can't be established
- and the chatmaild services, explained in the next section:
### chatmaild
@@ -308,8 +304,6 @@ Chatmail address creation will be denied while this file is present.
[Nginx](https://www.nginx.com/) listens on port 8443 (HTTPS-ALT) and 443 (HTTPS).
Port 443 multiplexes HTTPS, IMAP and SMTP using ALPN to redirect connections to ports 8443, 465 or 993.
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (HTTP).
[chatmail-turn](https://github.com/chatmail/chatmail-turn) listens on UDP port 3478 (STUN/TURN),
and temporarily opens UDP ports when users request them. UDP port range is not restricted, any free port may be allocated.
chatmail-core based apps will, however, discover all ports and configurations
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail relay server.

View File

@@ -338,9 +338,9 @@ def _install_dovecot_package(package: str, arch: str):
match (package, arch):
case ("core", "amd64"):
sha256 = "dd060706f52a306fa863d874717210b9fe10536c824afe1790eec247ded5b27d"
sha256 = "43f593332e22ac7701c62d58b575d2ca409e0f64857a2803be886c22860f5587"
case ("core", "arm64"):
sha256 = "e7548e8a82929722e973629ecc40fcfa886894cef3db88f23535149e7f730dc9"
sha256 = "4d21eba1a83f51c100f08f2e49f0c9f8f52f721ebc34f75018e043306da993a7"
case ("imapd", "amd64"):
sha256 = "8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86"
case ("imapd", "arm64"):

View File

@@ -10,9 +10,12 @@ def deploy_acmetool(email="", domains=[]):
packages=["acmetool"],
)
files.file(
path="/etc/cron.d/acmetool",
present=False,
files.put(
src=importlib.resources.files(__package__).joinpath("acmetool.cron").open("rb"),
dest="/etc/cron.d/acmetool",
user="root",
group="root",
mode="644",
)
files.put(

View File

@@ -0,0 +1,4 @@
SHELL=/bin/sh
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
MAILTO=root
20 16 * * * root /usr/bin/acmetool --batch reconcile && systemctl reload dovecot && systemctl reload postfix && systemctl reload nginx

View File

@@ -1,8 +1,7 @@
request:
provider: https://acme-v02.api.letsencrypt.org/directory
key:
type: ecdsa
ecdsa-curve: nistp256
type: rsa
challenge:
webroot-paths:
- /var/www/html/.well-known/acme-challenge

View File

@@ -70,12 +70,6 @@ userdb {
# Mailboxes are stored in the "mail" directory of the vmail user home.
mail_location = maildir:{{ config.mailboxes_dir }}/%u
# index/cache files are not very useful for chatmail relay operations
# but it's not clear how to disable them completely.
# According to https://doc.dovecot.org/2.3/settings/advanced/#core_setting-mail_cache_max_size
# if the cache file becomes larger than the specified size, it is truncated by dovecot
mail_cache_max_size = 500K
namespace inbox {
inbox = yes

View File

@@ -26,7 +26,6 @@ smtp_tls_security_level=verify
smtp_tls_servername = hostname
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_policy_maps = inline:{nauta.cu=may}
smtp_tls_protocols = >=TLSv1.2
smtpd_tls_protocols = >=TLSv1.2
# Disable anonymous cipher suites

View File

@@ -14,7 +14,6 @@ smtp inet n - y - - smtpd -v
{%- else %}
smtp inet n - y - - smtpd
{%- endif %}
-o smtpd_tls_security_level=encrypt
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port_incoming }}
submission inet n - y - 5000 smtpd
-o syslog_name=postfix/submission

View File

@@ -1,5 +1,5 @@
import queue
import smtplib
import socket
import threading
import pytest
@@ -91,23 +91,25 @@ def test_concurrent_logins_same_account(
def test_no_vrfy(chatmail_config):
domain = chatmail_config.mail_domain
s = smtplib.SMTP(domain)
s.starttls()
s.putcmd("vrfy", f"wrongaddress@{chatmail_config.mail_domain}")
result = s.getreply()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(10)
try:
sock.connect((domain, 25))
except socket.timeout:
pytest.skip(f"port 25 not reachable for {domain}")
banner = sock.recv(1024)
print(banner)
sock.send(b"VRFY wrongaddress@%s\r\n" % (chatmail_config.mail_domain.encode(),))
result = sock.recv(1024)
print(result)
s.putcmd("vrfy", f"echo@{chatmail_config.mail_domain}")
result2 = s.getreply()
sock.send(b"VRFY echo@%s\r\n" % (chatmail_config.mail_domain.encode(),))
result2 = sock.recv(1024)
print(result2)
assert result[0] == result2[0] == 252
assert result[1][0:6] == result2[1][0:6] == b"2.0.0 "
s.putcmd("vrfy", "wrongaddress")
result = s.getreply()
assert result[0:10] == result2[0:10]
sock.send(b"VRFY wrongaddress\r\n")
result = sock.recv(1024)
print(result)
s.putcmd("vrfy", "echo")
result2 = s.getreply()
sock.send(b"VRFY echo\r\n")
result2 = sock.recv(1024)
print(result2)
assert result[0] == result2[0] == 252
assert result[1][0:6] == result2[1][0:6] == b"2.0.0 "
assert result[0:10] == result2[0:10] == b"252 2.0.0 "

View File

@@ -143,7 +143,6 @@ def test_reject_missing_dkim(cmsetup, maildata, from_addr):
"encrypted.eml", from_addr=from_addr, to_addr=recipient.addr
).as_string()
conn = smtplib.SMTP(cmsetup.maildomain, 25, timeout=10)
conn.starttls()
with conn as s:
with pytest.raises(smtplib.SMTPDataError, match="No valid DKIM signature"):