-# Chatmail servers for secure instant messaging
+# Chatmail servers for end-to-end encrypted instant messaging
-Chatmail servers are interoperable email routing machines optimized for:
+Chatmail servers are interoperable e-mail routing machines optimized for
- **Convenience:** Low friction instant onboarding
- **Privacy:** No name, phone numbers, email required or collected
+
+- **End-to-End Encryption enforced**: only OpenPGP messages with metadata minimization allowed
+
- **Instant:** Privacy-preserving push notifications for Apple, Google, and Huawei
- **Speed:** Message delivery in well under a second.
-- **Security:** Strict TLS, DKIM and OpenPGP with metadata-minimization rules enforced.
+- **Transport Security:** Strict TLS and DKIM enforced.
-- **Reliability:** No spam or IP reputation checks, rate-limits suitable for realtime chats.
+- **Reliability:** No spam or IP reputation checks; rate-limits are suitable for realtime chats.
- **Efficiency:** Messages are only stored for transit and removed automatically.
@@ -191,9 +194,9 @@ A short overview:
to authenticate users
to send mails for them.
-- [`filtermail`](https://github.com/chatmail/server/blob/main/chatmaild/src/chatmaild/filtermail.py) prevents
- unencrypted email from leaving the chatmail service
- and is integrated into Postfix's outbound mail pipelines.
+- [`filtermail`](https://github.com/chatmail/server/blob/main/chatmaild/src/chatmaild/filtermail.py)
+ prevents unencrypted email from leaving or entering the chatmail service
+ and is integrated into Postfix's outbound and inbound mail pipelines.
- [`chatmail-metadata`](https://github.com/chatmail/server/blob/main/chatmaild/src/chatmaild/metadata.py) is contacted by a
[dovecot lua script](https://github.com/chatmail/server/blob/main/cmdeploy/src/cmdeploy/dovecot/push_notification.lua)
@@ -237,7 +240,6 @@ by the according markdown `.md` file in the `www/src` directory.
### Refining the web pages
-
```
scripts/cmdeploy webdev
```
@@ -252,6 +254,23 @@ This starts a local live development cycle for chatmail web pages:
- Starts a browser window automatically where you can "refresh" as needed.
+## Mailbox directory layout
+
+Fresh chatmail server addresses have a mailbox directory that contains:
+
+- a `password` file with the salted password required for authenticating
+ whether a login may use the address to send/receive messages.
+ If you modify the password file manually, you effectively block the user.
+
+- `enforceE2EEincoming` is a default-created file with each address.
+ If present the file indicates that this chatmail address rejects incoming cleartext messages.
+ If absent the address accepts incoming cleartext messages.
+
+- `dovecot*`, `cur`, `new` and `tmp` represent IMAP/mailbox state.
+ If the address is only used by one device, the Maildir directories
+ will typically be empty unless the user of that address hasn't been online
+ for a while.
+
## Emergency Commands to disable automatic account creation
diff --git a/chatmaild/src/chatmaild/config.py b/chatmaild/src/chatmaild/config.py
index 670b2042..e5e797a1 100644
--- a/chatmaild/src/chatmaild/config.py
+++ b/chatmaild/src/chatmaild/config.py
@@ -29,7 +29,13 @@ class Config:
self.passthrough_senders = params["passthrough_senders"].split()
self.passthrough_recipients = params["passthrough_recipients"].split()
self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
+ self.filtermail_smtp_port_incoming = int(
+ params["filtermail_smtp_port_incoming"]
+ )
self.postfix_reinject_port = int(params["postfix_reinject_port"])
+ self.postfix_reinject_port_incoming = int(
+ params["postfix_reinject_port_incoming"]
+ )
self.mtail_address = params.get("mtail_address")
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
diff --git a/chatmaild/src/chatmaild/filtermail.py b/chatmaild/src/chatmaild/filtermail.py
index 25d92487..f66188e5 100644
--- a/chatmaild/src/chatmaild/filtermail.py
+++ b/chatmaild/src/chatmaild/filtermail.py
@@ -11,9 +11,12 @@ from email.utils import parseaddr
from smtplib import SMTP as SMTPClient
from aiosmtpd.controller import Controller
+from aiosmtpd.smtp import SMTP
from .config import read_config
+ENCRYPTION_NEEDED_523 = "523 Encryption Needed: Invalid Unencrypted Mail"
+
def check_openpgp_payload(payload: bytes):
"""Checks the OpenPGP payload.
@@ -157,9 +160,14 @@ def check_encrypted(message):
return True
-async def asyncmain_beforequeue(config):
- port = config.filtermail_smtp_port
- Controller(BeforeQueueHandler(config), hostname="127.0.0.1", port=port).start()
+async def asyncmain_beforequeue(config, mode):
+ if mode == "outgoing":
+ port = config.filtermail_smtp_port
+ handler = OutgoingBeforeQueueHandler(config)
+ else:
+ port = config.filtermail_smtp_port_incoming
+ handler = IncomingBeforeQueueHandler(config)
+ HackedController(handler, hostname="127.0.0.1", port=port).start()
def recipient_matches_passthrough(recipient, passthrough_recipients):
@@ -171,7 +179,21 @@ def recipient_matches_passthrough(recipient, passthrough_recipients):
return False
-class BeforeQueueHandler:
+class HackedController(Controller):
+ def factory(self):
+ return SMTPDiscardRCPTO_options(self.handler, **self.SMTP_kwargs)
+
+
+class SMTPDiscardRCPTO_options(SMTP):
+ def _getparams(self, params):
+ # aiosmtpd's SMTP daemon fails to handle a request if there are RCPT TO options
+ # We just ignore them for our incoming filtermail purposes
+ if len(params) == 1 and params[0].startswith("ORCPT"):
+ return {}
+ return super()._getparams(params)
+
+
+class OutgoingBeforeQueueHandler:
def __init__(self, config):
self.config = config
self.send_rate_limiter = SendRateLimiter()
@@ -210,28 +232,85 @@ class BeforeQueueHandler:
_, from_addr = parseaddr(message.get("from").strip())
- logging.info(f"mime-from: {from_addr} envelope-from: {envelope.mail_from!r}")
if envelope.mail_from.lower() != from_addr.lower():
return f"500 Invalid FROM <{from_addr!r}> for <{envelope.mail_from!r}>"
- if mail_encrypted:
- print("Filtering encrypted mail.", file=sys.stderr)
- else:
- print("Filtering unencrypted mail.", file=sys.stderr)
+ if mail_encrypted or is_securejoin(message):
+ print("Outgoing: Filtering encrypted mail.", file=sys.stderr)
+ return
+
+ print("Outgoing: Filtering unencrypted mail.", file=sys.stderr)
if envelope.mail_from in self.config.passthrough_senders:
return
- if mail_encrypted or is_securejoin(message):
- return
+ # allow self-sent Autocrypt Setup Message
+ if envelope.rcpt_tos == [from_addr]:
+ if message.get("subject") == "Autocrypt Setup Message":
+ if message.get_content_type() == "multipart/mixed":
+ return
passthrough_recipients = self.config.passthrough_recipients
for recipient in envelope.rcpt_tos:
if recipient_matches_passthrough(recipient, passthrough_recipients):
continue
+
print("Rejected unencrypted mail.", file=sys.stderr)
- return "500 Invalid unencrypted mail"
+ return ENCRYPTION_NEEDED_523
+
+
+class IncomingBeforeQueueHandler:
+ def __init__(self, config):
+ self.config = config
+
+ async def handle_DATA(self, server, session, envelope):
+ logging.info("handle_DATA before-queue")
+ error = self.check_DATA(envelope)
+ if error:
+ return error
+ logging.info("re-injecting the mail that passed checks")
+
+ # the smtp daemon on reinject_port_incoming gives it to dkim milter
+ # which looks at source address to determine whether to verify or sign
+ client = SMTPClient(
+ "localhost",
+ self.config.postfix_reinject_port_incoming,
+ source_address=("127.0.0.2", 0),
+ )
+ client.sendmail(
+ envelope.mail_from, envelope.rcpt_tos, envelope.original_content
+ )
+ return "250 OK"
+
+ def check_DATA(self, envelope):
+ """the central filtering function for e-mails."""
+ logging.info(f"Processing DATA message from {envelope.mail_from}")
+
+ message = BytesParser(policy=policy.default).parsebytes(envelope.content)
+ mail_encrypted = check_encrypted(message)
+
+ if mail_encrypted or is_securejoin(message):
+ print("Incoming: Filtering encrypted mail.", file=sys.stderr)
+ return
+
+ print("Incoming: Filtering unencrypted mail.", file=sys.stderr)
+
+ # we want cleartext mailer-daemon messages to pass through
+ # chatmail core will typically not display them as normal messages
+ if message.get("auto-submitted"):
+ _, from_addr = parseaddr(message.get("from").strip())
+ if from_addr.lower().startswith("mailer-daemon@"):
+ if message.get_content_type() == "multipart/report":
+ return
+
+ for recipient in envelope.rcpt_tos:
+ user = self.config.get_user(recipient)
+ if user is None or user.is_incoming_cleartext_ok():
+ continue
+
+ print("Rejected unencrypted mail.", file=sys.stderr)
+ return ENCRYPTION_NEEDED_523
class SendRateLimiter:
@@ -250,11 +329,14 @@ class SendRateLimiter:
def main():
args = sys.argv[1:]
- assert len(args) == 1
+ assert len(args) == 2
config = read_config(args[0])
+ mode = args[1]
logging.basicConfig(level=logging.WARN)
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
- task = asyncmain_beforequeue(config)
+ assert mode in ["incoming", "outgoing"]
+ task = asyncmain_beforequeue(config, mode)
loop.create_task(task)
+ logging.info("entering serving loop")
loop.run_forever()
diff --git a/chatmaild/src/chatmaild/ini/chatmail.ini.f b/chatmaild/src/chatmaild/ini/chatmail.ini.f
index 60fe1dcd..d9e5261a 100644
--- a/chatmaild/src/chatmaild/ini/chatmail.ini.f
+++ b/chatmaild/src/chatmaild/ini/chatmail.ini.f
@@ -46,12 +46,14 @@ passthrough_recipients = xstore@testrun.org
# Deployment Details
#
-# where the filtermail SMTP service listens
+# SMTP outgoing filtermail and reinjection
filtermail_smtp_port = 10080
-
-# postfix accepts on the localhost reinject SMTP port
postfix_reinject_port = 10025
+# SMTP incoming filtermail and reinjection
+filtermail_smtp_port_incoming = 10081
+postfix_reinject_port_incoming = 10026
+
# if set to "True" IPv6 is disabled
disable_ipv6 = False
diff --git a/chatmaild/src/chatmaild/tests/mail-data/asm.eml b/chatmaild/src/chatmaild/tests/mail-data/asm.eml
new file mode 100644
index 00000000..d7bf6baa
--- /dev/null
+++ b/chatmaild/src/chatmaild/tests/mail-data/asm.eml
@@ -0,0 +1,56 @@
+From: {from_addr}
+To: {to_addr}
+Autocrypt-Setup-Message: v1
+Subject: Autocrypt Setup Message
+Date: Tue, 22 Jan 2019 12:56:29 +0100
+Content-type: multipart/mixed; boundary="Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ"
+
+--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
+Content-Type: text/plain
+
+This message contains all information to transfer your Autocrypt
+settings along with your secret key securely from your original
+device.
+
+To set up your new device for Autocrypt, please follow the
+instuctions that should be presented by your new device.
+
+You can keep this message and use it as a backup for your secret
+key. If you want to do this, you should write down the Setup Code
+and store it securely.
+--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ
+Content-Type: application/autocrypt-setup
+Content-Disposition: attachment; filename="autocrypt-setup-message.html"
+
+
++This is the Autocrypt setup file used to transfer settings and +keys between clients. You can decrypt it using the Setup Code +presented on your old device, and then import the contained key +into your keyring. +
+ ++-----BEGIN PGP MESSAGE----- +Passphrase-Format: numeric9x4 +Passphrase-Begin: 17 + +jA0EBwMCFAxADoCdzeX/0ukBlqI5+pfpKb751qd/7nLNbkpy3gVcaf1QwRPZYt40 +Ynp08UqRQ2g48ZlnzHLSwlTGOPTuv2Jt8ka+pgZ45xzvJSG2gau03xP4VsC271kR +VmCjdb0Y6Rk96mAwfGzrkbaRQ9Z7fIoL866GOv6h9neiVIkp+JYlTV6ISD0ZQJ4Q +I6dOQkB/TWZyVjtiJDOQHdfNWliA6NtqaLq19wlu9L5xXjuNpY95KwR8EJXWe0+o +Y3d2U/KxOAkXKghP2Qg1GtlPVeGC5T4p03TGI6pzKT+kHX6Rrm9wK6sM9aTquMmF +Vok84Jg1DFnwivWC2RILR81rXi7k/+Y6MUbveFgJ9cQduqpxnmD7TjOblYu7M6zp +YGAUxh8DRKlIMn2QsA++DBYQ6ACZvwuY8qTDLkqPDo4WqM313dsMJbyGjDdVE7EM +PESS+RlABETpZXz8g/ycr6DIUNdlbPcmYlsBfHWDOuR2GFFTwmlv5slWS39dJv38 +E0eIe1CwdxI801Se7t7dUUS/ZF8wb6GlmxOcqGbF8eko1Z0S64IAm7/h13MRQCxI +geQnHfGYVJ2FOimoCMEKwfa9x++RFTDW0u7spDC2uWvK/1viV8OfRppFhLr/kmKb +18lWXuAz80DAjUDUsVqEq2MvJBJGoCJUEyjuRsLkHYRM5jYk4v50LyyR0Om73nWF +nZBqmqNzdr7Xb9PHHdFhnEc0VvoYbrcM0RVYcEMW3YbmejM891j1d6Iv+/n/qND/ +NdebGrfWJMmFLf/iEkzTZ3/v5inW9LpWoRc94ioCjJTaEo8Rib6ARRFaJVIsmNXi +YicFGO98D+zX+a2t9Yz6IpPajVslnOp6ScpmXgts/2XWD7oE+JgxSAqo/dLVsHgP +Ufo= +=pulM +-----END PGP MESSAGE----- ++--Y6fyGi9SoGeH8WwRaEdC6bbBcYOedDzrQ-- diff --git a/chatmaild/src/chatmaild/tests/mail-data/mailer-daemon.eml b/chatmaild/src/chatmaild/tests/mail-data/mailer-daemon.eml new file mode 100644 index 00000000..66e7c7c2 --- /dev/null +++ b/chatmaild/src/chatmaild/tests/mail-data/mailer-daemon.eml @@ -0,0 +1,46 @@ +Date: Fri, 8 Jul 1994 09:21:47 -0400 +From: Mail Delivery Subsystem