Compare commits

..

35 Commits

Author SHA1 Message Date
holger krekel
4c9eaf3d63 turn chatmail-expire live 2025-10-22 18:51:47 +02:00
holger krekel
7a91aa539d add a bit more date info 2025-10-22 18:51:14 +02:00
holger krekel
4c8ad95244 allow files and directories to vanish while we iterate over them 2025-10-22 18:11:41 +02:00
holger krekel
8d65770c28 fixup after trying out on nine: don't remove large messages after 7 days if they are in the new/ folder (this is old behaviour)
and provide reporting about which mailbox is being scanned.
2025-10-21 21:46:51 +02:00
holger krekel
0931da21b8 make sure fsreport can run on empty mailbox dir 2025-10-21 18:43:37 +02:00
holger krekel
11a8f8cf9e try fix CI 2025-10-21 18:43:37 +02:00
holger krekel
0aa255e3f1 replace expunge mentioning in architecture 2025-10-21 18:43:37 +02:00
holger krekel
6c4764b452 Apply suggestions from code review
fix typo

Co-authored-by: l <link2xt@testrun.org>
2025-10-21 18:43:37 +02:00
holger krekel
c1f08a9afe simplify and beautify formatting and sizes 2025-10-21 18:43:37 +02:00
holger krekel
5c8afb377e also run fsreport 2025-10-21 18:43:37 +02:00
holger krekel
8225a9f398 use systemd timer instead of cron-job for expiry (tested by hand on c2) 2025-10-21 18:43:37 +02:00
holger krekel
eb221ca1af unify K output 2025-10-21 18:43:37 +02:00
holger krekel
93421b317b always use "H" for printing numbers, and make "chatmail.ini" file optional, defaulting to where it is on chatmail relays 2025-10-21 18:43:37 +02:00
holger krekel
777be107f3 fix another invocation 2025-10-21 18:43:37 +02:00
holger krekel
8b81d5b5d6 unify chatmail-fsreport and chatmail-expire to both just require a chatmail.ini file 2025-10-21 18:43:37 +02:00
holger krekel
e6a2906e82 cosmetic: refine summary and fix typo 2025-10-21 18:43:37 +02:00
holger krekel
67ba4ac99e address four review comments from link2xt 2025-10-21 18:43:37 +02:00
holger krekel
8cadf51387 prefix new commands 2025-10-21 18:43:37 +02:00
holger krekel
ce4bb97294 remove superflous totalsize attribute 2025-10-21 18:43:37 +02:00
holger krekel
3a0c629f3b during fsreport (reporting) don't store all mailbxoes but categorize them immediately, provide a few command line options to select 2025-10-21 18:43:37 +02:00
holger krekel
8df53c2655 fix lint issues 2025-10-21 18:43:37 +02:00
holger krekel
3fd3ab1a68 some renaming 2025-10-21 18:43:37 +02:00
holger krekel
d74f792787 remove superflous Stats class 2025-10-21 18:43:37 +02:00
holger krekel
1135372b81 further reduce code 2025-10-21 18:43:37 +02:00
holger krekel
c9f80bffd8 no reporting by default, and adding a summary line 2025-10-21 18:43:37 +02:00
holger krekel
10e53d17e8 don't globally collect files anymore to avoid using growing-with-number-of-mailboxes ram 2025-10-21 18:43:37 +02:00
holger krekel
01ca2a8b91 more streamline 2025-10-21 18:43:37 +02:00
holger krekel
fb01944f0d strike superflous code 2025-10-21 18:43:37 +02:00
holger krekel
a90a651ba0 fix comment 2025-10-21 18:43:37 +02:00
holger krekel
7d74b46502 add argument parsing for reporting 2025-10-21 18:43:37 +02:00
holger krekel
6d3e690653 add basic command line parsing for expire + some streamlining 2025-10-21 18:43:37 +02:00
holger krekel
ed7a70ba31 refactor and write tests for overall expiry/report runs 2025-10-21 18:43:37 +02:00
holger krekel
023116bc91 add summary reporting, rework expiry logic 2025-10-21 18:43:37 +02:00
holger krekel
b13929119b do all expunging in python 2025-10-21 18:43:37 +02:00
holger krekel
a4152140ca move delete_inactive_users to new implementation 2025-10-21 18:43:37 +02:00
10 changed files with 29 additions and 47 deletions

View File

@@ -1 +1,5 @@
blank_issues_enabled: true blank_issues_enabled: true
contact_links:
- name: Mutual Help Chat Group
url: https://i.delta.chat/#6CBFF8FFD505C0FDEA20A66674F2916EA8FBEE99&a=invitebot%40nine.testrun.org&g=Chatmail%20Mutual%20Help&x=7sFF7Ik50pWv6J1z7RVC5527&i=X69wTFfvCfs3d-JzqP0kVA3i&s=ibp-447dU-wUq-52QanwAtWc
about: If you have troubles setting up the relay server, feel free to ask here.

View File

@@ -2,15 +2,6 @@
## untagged ## untagged
- acmetool: use ECDSA keys instead of RSA
([#689](https://github.com/chatmail/relay/pull/689))
- Require TLS 1.2 for outgoing SMTP connections
([#685](https://github.com/chatmail/relay/pull/685))
- require STARTTLS for incoming port 25 connections
([#684](https://github.com/chatmail/relay/pull/684))
- filtermail: run CPU-intensive handle_DATA in a thread pool executor - filtermail: run CPU-intensive handle_DATA in a thread pool executor
([#676](https://github.com/chatmail/relay/pull/676)) ([#676](https://github.com/chatmail/relay/pull/676))
@@ -30,7 +21,7 @@
([#650](https://github.com/chatmail/relay/pull/650)) ([#650](https://github.com/chatmail/relay/pull/650))
- filtermail: accept mails from Protonmail - filtermail: accept mails from Protonmail
([#616](https://github.com/chatmail/relay/pull/616)) ([#616](https://github.com/chatmail/relay/pull/655))
- Ignore all RCPT TO: parameters - Ignore all RCPT TO: parameters
([#651](https://github.com/chatmail/relay/pull/651)) ([#651](https://github.com/chatmail/relay/pull/651))
@@ -63,7 +54,7 @@
to only do a single iteration over sometimes millions of messages to only do a single iteration over sometimes millions of messages
instead of doing "find" commands that iterate 9 times over the messages. instead of doing "find" commands that iterate 9 times over the messages.
Provide an "fsreport" CLI for more fine grained analysis of message files. Provide an "fsreport" CLI for more fine grained analysis of message files.
([#637](https://github.com/chatmail/relay/pull/637)) ([#637](https://github.com/chatmail/relay/pull/632))
## 1.7.0 2025-09-11 ## 1.7.0 2025-09-11

View File

@@ -180,10 +180,6 @@ The components of chatmail are:
- [Iroh relay](https://www.iroh.computer/docs/concepts/relay) - [Iroh relay](https://www.iroh.computer/docs/concepts/relay)
which helps client devices to establish Peer-to-Peer connections which helps client devices to establish Peer-to-Peer connections
- [TURN](https://github.com/chatmail/chatmail-turn)
to enable relay users to start webRTC calls
even if a p2p connection can't be established
- and the chatmaild services, explained in the next section: - and the chatmaild services, explained in the next section:
### chatmaild ### chatmaild
@@ -308,8 +304,6 @@ Chatmail address creation will be denied while this file is present.
[Nginx](https://www.nginx.com/) listens on port 8443 (HTTPS-ALT) and 443 (HTTPS). [Nginx](https://www.nginx.com/) listens on port 8443 (HTTPS-ALT) and 443 (HTTPS).
Port 443 multiplexes HTTPS, IMAP and SMTP using ALPN to redirect connections to ports 8443, 465 or 993. Port 443 multiplexes HTTPS, IMAP and SMTP using ALPN to redirect connections to ports 8443, 465 or 993.
[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (HTTP). [acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (HTTP).
[chatmail-turn](https://github.com/chatmail/chatmail-turn) listens on UDP port 3478 (STUN/TURN),
and temporarily opens UDP ports when users request them. UDP port range is not restricted, any free port may be allocated.
chatmail-core based apps will, however, discover all ports and configurations chatmail-core based apps will, however, discover all ports and configurations
automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail relay server. automatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail relay server.

View File

@@ -338,9 +338,9 @@ def _install_dovecot_package(package: str, arch: str):
match (package, arch): match (package, arch):
case ("core", "amd64"): case ("core", "amd64"):
sha256 = "dd060706f52a306fa863d874717210b9fe10536c824afe1790eec247ded5b27d" sha256 = "43f593332e22ac7701c62d58b575d2ca409e0f64857a2803be886c22860f5587"
case ("core", "arm64"): case ("core", "arm64"):
sha256 = "e7548e8a82929722e973629ecc40fcfa886894cef3db88f23535149e7f730dc9" sha256 = "4d21eba1a83f51c100f08f2e49f0c9f8f52f721ebc34f75018e043306da993a7"
case ("imapd", "amd64"): case ("imapd", "amd64"):
sha256 = "8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86" sha256 = "8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86"
case ("imapd", "arm64"): case ("imapd", "arm64"):

View File

@@ -1,8 +1,7 @@
request: request:
provider: https://acme-v02.api.letsencrypt.org/directory provider: https://acme-v02.api.letsencrypt.org/directory
key: key:
type: ecdsa type: rsa
ecdsa-curve: nistp256
challenge: challenge:
webroot-paths: webroot-paths:
- /var/www/html/.well-known/acme-challenge - /var/www/html/.well-known/acme-challenge

View File

@@ -70,12 +70,6 @@ userdb {
# Mailboxes are stored in the "mail" directory of the vmail user home. # Mailboxes are stored in the "mail" directory of the vmail user home.
mail_location = maildir:{{ config.mailboxes_dir }}/%u mail_location = maildir:{{ config.mailboxes_dir }}/%u
# index/cache files are not very useful for chatmail relay operations
# but it's not clear how to disable them completely.
# According to https://doc.dovecot.org/2.3/settings/advanced/#core_setting-mail_cache_max_size
# if the cache file becomes larger than the specified size, it is truncated by dovecot
mail_cache_max_size = 500K
namespace inbox { namespace inbox {
inbox = yes inbox = yes

View File

@@ -25,7 +25,7 @@ smtp_tls_security_level=verify
# <https://www.postfix.org/postconf.5.html#smtp_tls_servername> # <https://www.postfix.org/postconf.5.html#smtp_tls_servername>
smtp_tls_servername = hostname smtp_tls_servername = hostname
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_protocols = >=TLSv1.2 smtp_tls_policy_maps = inline:{nauta.cu=may}
smtpd_tls_protocols = >=TLSv1.2 smtpd_tls_protocols = >=TLSv1.2
# Disable anonymous cipher suites # Disable anonymous cipher suites

View File

@@ -14,7 +14,6 @@ smtp inet n - y - - smtpd -v
{%- else %} {%- else %}
smtp inet n - y - - smtpd smtp inet n - y - - smtpd
{%- endif %} {%- endif %}
-o smtpd_tls_security_level=encrypt
-o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port_incoming }} -o smtpd_proxy_filter=127.0.0.1:{{ config.filtermail_smtp_port_incoming }}
submission inet n - y - 5000 smtpd submission inet n - y - 5000 smtpd
-o syslog_name=postfix/submission -o syslog_name=postfix/submission

View File

@@ -1,5 +1,5 @@
import queue import queue
import smtplib import socket
import threading import threading
import pytest import pytest
@@ -91,23 +91,25 @@ def test_concurrent_logins_same_account(
def test_no_vrfy(chatmail_config): def test_no_vrfy(chatmail_config):
domain = chatmail_config.mail_domain domain = chatmail_config.mail_domain
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s = smtplib.SMTP(domain) sock.settimeout(10)
s.starttls() try:
sock.connect((domain, 25))
s.putcmd("vrfy", f"wrongaddress@{chatmail_config.mail_domain}") except socket.timeout:
result = s.getreply() pytest.skip(f"port 25 not reachable for {domain}")
banner = sock.recv(1024)
print(banner)
sock.send(b"VRFY wrongaddress@%s\r\n" % (chatmail_config.mail_domain.encode(),))
result = sock.recv(1024)
print(result) print(result)
s.putcmd("vrfy", f"echo@{chatmail_config.mail_domain}") sock.send(b"VRFY echo@%s\r\n" % (chatmail_config.mail_domain.encode(),))
result2 = s.getreply() result2 = sock.recv(1024)
print(result2) print(result2)
assert result[0] == result2[0] == 252 assert result[0:10] == result2[0:10]
assert result[1][0:6] == result2[1][0:6] == b"2.0.0 " sock.send(b"VRFY wrongaddress\r\n")
s.putcmd("vrfy", "wrongaddress") result = sock.recv(1024)
result = s.getreply()
print(result) print(result)
s.putcmd("vrfy", "echo") sock.send(b"VRFY echo\r\n")
result2 = s.getreply() result2 = sock.recv(1024)
print(result2) print(result2)
assert result[0] == result2[0] == 252 assert result[0:10] == result2[0:10] == b"252 2.0.0 "
assert result[1][0:6] == result2[1][0:6] == b"2.0.0 "

View File

@@ -143,7 +143,6 @@ def test_reject_missing_dkim(cmsetup, maildata, from_addr):
"encrypted.eml", from_addr=from_addr, to_addr=recipient.addr "encrypted.eml", from_addr=from_addr, to_addr=recipient.addr
).as_string() ).as_string()
conn = smtplib.SMTP(cmsetup.maildomain, 25, timeout=10) conn = smtplib.SMTP(cmsetup.maildomain, 25, timeout=10)
conn.starttls()
with conn as s: with conn as s:
with pytest.raises(smtplib.SMTPDataError, match="No valid DKIM signature"): with pytest.raises(smtplib.SMTPDataError, match="No valid DKIM signature"):