Compare commits

..

2 Commits

Author SHA1 Message Date
missytake
a0d5f5dc8d fix sha256 checksums 2025-08-06 08:13:44 +02:00
missytake
72689563f9 ci: testing https://github.com/chatmail/dovecot/pull/4 2025-08-06 08:04:50 +02:00
40 changed files with 160 additions and 1339 deletions

View File

@@ -1,5 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: Mutual Help Chat Group
url: https://i.delta.chat/#6CBFF8FFD505C0FDEA20A66674F2916EA8FBEE99&a=invitebot%40nine.testrun.org&g=Chatmail%20Mutual%20Help&x=7sFF7Ik50pWv6J1z7RVC5527&i=X69wTFfvCfs3d-JzqP0kVA3i&s=ibp-447dU-wUq-52QanwAtWc
url: https://i.delta.chat/#C2846EB4C1CB8DF84B1818F5E3A638FC3FBDC981&a=stalebot1%40nine.testrun.org&g=Chatmail%20Mutual%20Help&x=7sFF7Ik50pWv6J1z7RVC5527&i=d7s1HvOsk5UrSf9AoqRZggg4&s=XmX_9BAW6-g5Ao5E8PyaeKNB
about: If you have troubles setting up the relay server, feel free to ask here.

View File

@@ -70,6 +70,9 @@ jobs:
rsync -avz dkimkeys-restore/dkimkeys root@staging-ipv4.testrun.org:/etc/ || true
ssh -o StrictHostKeyChecking=accept-new -v root@staging-ipv4.testrun.org chown root:root -R /var/lib/acme || true
- name: run formatting checks
run: cmdeploy fmt -v
- name: run deploy-chatmail offline tests
run: pytest --pyargs cmdeploy
@@ -77,7 +80,7 @@ jobs:
cmdeploy init staging-ipv4.testrun.org
sed -i 's#disable_ipv6 = False#disable_ipv6 = True#' chatmail.ini
- run: cmdeploy run --verbose --skip-dns-check
- run: cmdeploy run
- name: set DNS entries
run: |

View File

@@ -70,12 +70,15 @@ jobs:
rsync -avz dkimkeys-restore/dkimkeys root@staging2.testrun.org:/etc/ || true
ssh -o StrictHostKeyChecking=accept-new -v root@staging2.testrun.org chown root:root -R /var/lib/acme || true
- name: run formatting checks
run: cmdeploy fmt -v
- name: run deploy-chatmail offline tests
run: pytest --pyargs cmdeploy
- run: cmdeploy init staging2.testrun.org
- run: cmdeploy run --verbose --skip-dns-check
- run: cmdeploy run --verbose
- name: set DNS entries
run: |

6
.gitignore vendored
View File

@@ -164,9 +164,3 @@ cython_debug/
#.idea/
chatmail.zone
# docker
/data/
/custom/
docker-compose.yaml
.env

View File

@@ -1,50 +0,0 @@
This diagram shows components of the chatmail server; this is a draft
overview as of mid-August 2025:
```mermaid
graph LR;
cmdeploy --- sshd;
letsencrypt --- |80|acmetool-redirector;
acmetool-redirector --- |443|nginx-right(["`nginx
(external)`"]);
nginx-external --- |465|postfix;
nginx-external(["`nginx
(external)`"]) --- |8443|nginx-internal["`nginx
(internal)`"];
nginx-internal --- website["`Website
/var/www/html`"];
nginx-internal --- newemail.py;
nginx-internal --- autoconfig.xml;
certs-nginx[("`TLS certs
/var/lib/acme`")] --> nginx-internal;
cron --- chatmail-metrics;
cron --- acmetool;
cron --- expunge;
chatmail-metrics --- website;
acmetool --> certs[("`TLS certs
/var/lib/acme`")];
nginx-external --- |993|dovecot;
autoconfig.xml --- postfix;
autoconfig.xml --- dovecot;
postfix --- echobot;
postfix --- |10080,10081|filtermail;
postfix --- users["`User data
home/vmail/mail`"];
postfix --- |doveauth.socket|doveauth;
dovecot --- |doveauth.socket|doveauth;
dovecot --- users;
dovecot --- |metadata.socket|chatmail-metadata;
doveauth --- users;
expunge --- users;
chatmail-metadata --- iroh-relay;
certs-nginx --> postfix;
certs-nginx --> dovecot;
style certs fill:#ff6;
style certs-nginx fill:#ff6;
style nginx-external fill:#fc9;
style nginx-right fill:#fc9;
```
The edges in this graph should not be taken too literally; they
reflect some sort of communication path or dependency relationship
between components of the chatmail server.

View File

@@ -2,58 +2,6 @@
## untagged
- Setup TURN server
([#621](https://github.com/chatmail/relay/pull/621))
- cmdeploy: make --ssh-host work with localhost
([#659](https://github.com/chatmail/relay/pull/659))
- Update iroh-relay to 0.35.0
([#650](https://github.com/chatmail/relay/pull/650))
- filtermail: accept mails from Protonmail
([#616](https://github.com/chatmail/relay/pull/655))
- Ignore all RCPT TO: parameters
([#651](https://github.com/chatmail/relay/pull/651))
- Add config parameter for Let's Encrypt ACME email
([#663](https://github.com/chatmail/relay/pull/663))
- Use max username length in newemail.py, not min
([#648](https://github.com/chatmail/relay/pull/648))
- Add startup for `fcgiwrap.service` because sometimes it did not start automatically.
([#657](https://github.com/chatmail/relay/pull/657))
- Add `cmdeploy init --force` command for recreating chatmail.ini
([#656](https://github.com/chatmail/relay/pull/656))
- Increase maxproc for reinjecting ports from 10 to 100
([#646](https://github.com/chatmail/relay/pull/646))
- Allow ports 143 and 993 to be used by `dovecot` process
([#639](https://github.com/chatmail/relay/pull/639))
- Add `--skip-dns-check` argument to `cmdeploy run` command, which disables DNS record checking before installation.
([#661](https://github.com/chatmail/relay/pull/661))
- Add installation via docker compose (MVP 1). The instructions, known issues and limitations are located in `/docs`
([#614](https://github.com/chatmail/relay/pull/614))
- Add configuration parameters
([#614](https://github.com/chatmail/relay/pull/614)):
- `change_kernel_settings` - Whether to change kernel parameters during installation (default: `True`)
- `fs_inotify_max_user_instances_and_watchers` - Value for kernel parameters `fs.inotify.max_user_instances` and `fs.inotify.max_user_watches` (default: `65535`)
## 1.7.0 2025-09-11
- Make www upload path configurable
([#618](https://github.com/chatmail/relay/pull/618))
- Check whether GCC is installed in initenv.sh
([#608](https://github.com/chatmail/relay/pull/608))
- Expire push notification tokens after 90 days
([#583](https://github.com/chatmail/relay/pull/583))
@@ -78,9 +26,6 @@
- filtermail: respect config message size limit
([#572](https://github.com/chatmail/relay/pull/572))
- Don't deploy if one of the ports used for chatmail relay services is occupied by an unexpected process
([#568](https://github.com/chatmail/relay/pull/568))
- Add config value after how many days large files are deleted
([#555](https://github.com/chatmail/relay/pull/555))

View File

@@ -74,23 +74,22 @@ Please substitute it with your own domain.
```
git clone https://github.com/chatmail/relay
cd relay
```
### Manual installation
1. On your local PC, create chatmail configuration file `chatmail.ini`:
```
scripts/initenv.sh
```
3. On your local PC, create chatmail configuration file `chatmail.ini`:
```
scripts/cmdeploy init chat.example.org # <-- use your domain
```
2. Verify that SSH root login to your remote server works:
4. Verify that SSH root login to your remote server works:
```
ssh root@chat.example.org # <-- use your domain
```
3. From your local PC, deploy the remote chatmail relay server:
5. From your local PC, deploy the remote chatmail relay server:
```
scripts/cmdeploy run
@@ -100,12 +99,6 @@ Please substitute it with your own domain.
which you should configure at your DNS provider
(it can take some time until they are public).
### Docker installation
We have experimental support for [docker compose](./docs/DOCKER_INSTALLATION_EN.md),
but it is not covered by automated tests yet,
so don't expect everything to work.
### Other helpful commands
To check the status of your remotely running chatmail service:
@@ -262,18 +255,6 @@ This starts a local live development cycle for chatmail web pages:
- Starts a browser window automatically where you can "refresh" as needed.
#### Custom web pages
You can skip uploading a web page
by setting `www_folder=disabled` in `chatmail.ini`.
If you want to manage your web pages outside this git repository,
you can set `www_folder` in `chatmail.ini` to a custom directory on your computer.
`cmdeploy run` will upload it as the server's home page,
and if it contains a `src/index.md` file,
will build it with hugo.
## Mailbox directory layout
Fresh chatmail addresses have a mailbox directory that contains:

View File

@@ -29,7 +29,6 @@ echobot = "chatmaild.echo:main"
chatmail-metrics = "chatmaild.metrics:main"
delete_inactive_users = "chatmaild.delete_inactive_users:main"
lastlogin = "chatmaild.lastlogin:main"
turnserver = "chatmaild.turnserver:main"
[project.entry-points.pytest11]
"chatmaild.testplugin" = "chatmaild.tests.plugin"

View File

@@ -33,7 +33,6 @@ class Config:
self.password_min_length = int(params["password_min_length"])
self.passthrough_senders = params["passthrough_senders"].split()
self.passthrough_recipients = params["passthrough_recipients"].split()
self.www_folder = params.get("www_folder", "")
self.filtermail_smtp_port = int(params["filtermail_smtp_port"])
self.filtermail_smtp_port_incoming = int(
params["filtermail_smtp_port_incoming"]
@@ -44,13 +43,6 @@ class Config:
)
self.mtail_address = params.get("mtail_address")
self.disable_ipv6 = params.get("disable_ipv6", "false").lower() == "true"
self.acme_email = params.get("acme_email", "")
self.change_kernel_settings = (
params.get("change_kernel_settings", "true").lower() == "true"
)
self.fs_inotify_max_user_instances_and_watchers = int(
params["fs_inotify_max_user_instances_and_watchers"]
)
self.imap_rawlog = params.get("imap_rawlog", "false").lower() == "true"
if "iroh_relay" not in params:
self.iroh_relay = "https://" + params["mail_domain"]

View File

@@ -83,14 +83,8 @@ def check_openpgp_payload(payload: bytes):
return False
def check_armored_payload(payload: str, outgoing: bool):
"""Check the armored PGP message for invalid content.
:param payload: the armored PGP message
:param outgoing: whether the message is outgoing or incoming
:return: whether the message is a valid PGP message
"""
prefix = "-----BEGIN PGP MESSAGE-----\r\n"
def check_armored_payload(payload: str):
prefix = "-----BEGIN PGP MESSAGE-----\r\n\r\n"
if not payload.startswith(prefix):
return False
payload = payload.removeprefix(prefix)
@@ -102,17 +96,6 @@ def check_armored_payload(payload: str, outgoing: bool):
return False
payload = payload.removesuffix(suffix)
# Disallow comments in outgoing messages
version_comment = "Version: "
if payload.startswith(version_comment):
version_line = payload.splitlines()[0]
payload = payload.removeprefix(version_line)
if outgoing:
return False
while payload.startswith("\r\n"):
payload = payload.removeprefix("\r\n")
# Remove CRC24.
payload = payload.rpartition("=")[0]
@@ -148,7 +131,7 @@ def is_securejoin(message):
return True
def check_encrypted(message, outgoing=True):
def check_encrypted(message):
"""Check that the message is an OpenPGP-encrypted message.
MIME structure of the message must correspond to <https://www.rfc-editor.org/rfc/rfc3156>.
@@ -175,7 +158,7 @@ def check_encrypted(message, outgoing=True):
if part.get_content_type() != "application/octet-stream":
return False
if not check_armored_payload(part.get_payload(), outgoing=outgoing):
if not check_armored_payload(part.get_payload()):
return False
else:
return False
@@ -214,13 +197,11 @@ class HackedController(Controller):
class SMTPDiscardRCPTO_options(SMTP):
def _getparams(self, params):
# Ignore RCPT TO parameters.
#
# Otherwise parameters such as `ORCPT=...`
# or `NOTIFY=DELAY,FAILURE` (generated by Stalwart)
# make aiosmtpd reject the message here:
# <https://github.com/aio-libs/aiosmtpd/blob/98f578389ae86e5345cc343fa4e5a17b21d9c96d/aiosmtpd/smtp.py#L1379-L1384>
return {}
# aiosmtpd's SMTP daemon fails to handle a request if there are RCPT TO options
# We just ignore them for our incoming filtermail purposes
if len(params) == 1 and params[0].startswith("ORCPT"):
return {}
return super()._getparams(params)
class OutgoingBeforeQueueHandler:
@@ -258,7 +239,7 @@ class OutgoingBeforeQueueHandler:
logging.info(f"Processing DATA message from {envelope.mail_from}")
message = BytesParser(policy=policy.default).parsebytes(envelope.content)
mail_encrypted = check_encrypted(message, outgoing=True)
mail_encrypted = check_encrypted(message)
_, from_addr = parseaddr(message.get("from").strip())
@@ -318,7 +299,7 @@ class IncomingBeforeQueueHandler:
logging.info(f"Processing DATA message from {envelope.mail_from}")
message = BytesParser(policy=policy.default).parsebytes(envelope.content)
mail_encrypted = check_encrypted(message, outgoing=False)
mail_encrypted = check_encrypted(message)
if mail_encrypted or is_securejoin(message):
print("Incoming: Filtering encrypted mail.", file=sys.stderr)

View File

@@ -45,9 +45,6 @@ passthrough_senders =
# (space-separated, item may start with "@" to whitelist whole recipient domains)
passthrough_recipients = xstore@testrun.org echo@{mail_domain}
# path to www directory - documented here: https://github.com/chatmail/relay/#custom-web-pages
#www_folder = www
#
# Deployment Details
#
@@ -63,19 +60,6 @@ postfix_reinject_port_incoming = 10026
# if set to "True" IPv6 is disabled
disable_ipv6 = False
# Your email adress, which will be used in acmetool to manage Let's Encrypt SSL certificates
acme_email =
#
# Kernel settings
#
# if you set "True", the kernel settings will be configured according to the values below
change_kernel_settings = True
# change fs.inotify.max_user_instances and fs.inotify.max_user_watches kernel settings
fs_inotify_max_user_instances_and_watchers = 65535
# Defaults to https://iroh.{{mail_domain}} and running `iroh-relay` on the chatmail
# service.
# If you set it to anything else, the service will be disabled

View File

@@ -7,7 +7,6 @@ from .config import read_config
from .dictproxy import DictProxy
from .filedict import FileDict
from .notifier import Notifier
from .turnserver import turn_credentials
def _is_valid_token_timestamp(timestamp, now):
@@ -76,12 +75,11 @@ class Metadata:
class MetadataDictProxy(DictProxy):
def __init__(self, notifier, metadata, iroh_relay=None, turn_hostname=None):
def __init__(self, notifier, metadata, iroh_relay=None):
super().__init__()
self.notifier = notifier
self.metadata = metadata
self.iroh_relay = iroh_relay
self.turn_hostname = turn_hostname
def handle_lookup(self, parts):
# Lpriv/43f5f508a7ea0366dff30200c15250e3/devicetoken\tlkj123poi@c2.testrun.org
@@ -100,11 +98,6 @@ class MetadataDictProxy(DictProxy):
):
# Handle `GETMETADATA "" /shared/vendor/deltachat/irohrelay`
return f"O{self.iroh_relay}\n"
elif keyname == "vendor/vendor.dovecot/pvt/server/vendor/deltachat/turn":
res = turn_credentials()
port = 3478
return f"O{self.turn_hostname}:{port}:{res}\n"
logging.warning(f"lookup ignored: {parts!r}")
return "N\n"
@@ -128,7 +121,6 @@ def main():
config = read_config(config_path)
iroh_relay = config.iroh_relay
mail_domain = config.mail_domain
vmail_dir = config.mailboxes_dir
if not vmail_dir.exists():
@@ -142,10 +134,7 @@ def main():
notifier.start_notification_threads(metadata.remove_token_from_addr)
dictproxy = MetadataDictProxy(
notifier=notifier,
metadata=metadata,
iroh_relay=iroh_relay,
turn_hostname=mail_domain,
notifier=notifier, metadata=metadata, iroh_relay=iroh_relay
)
dictproxy.serve_forever_from_socket(socket)

View File

@@ -15,7 +15,7 @@ ALPHANUMERIC_PUNCT = string.ascii_letters + string.digits + string.punctuation
def create_newemail_dict(config: Config):
user = "".join(random.choices(ALPHANUMERIC, k=config.username_max_length))
user = "".join(random.choices(ALPHANUMERIC, k=config.username_min_length))
password = "".join(
secrets.choice(ALPHANUMERIC_PUNCT)
for _ in range(config.password_min_length + 3)

View File

@@ -241,9 +241,8 @@ def test_cleartext_passthrough_senders(gencreds, handler, maildata):
def test_check_armored_payload():
prefix = "-----BEGIN PGP MESSAGE-----\r\n"
comment = "Version: ProtonMail\r\n"
payload = """\r
payload = """-----BEGIN PGP MESSAGE-----\r
\r
wU4DSqFx0d1yqAoSAQdAYkX/ZN/Az4B0k7X47zKyWrXxlDEdS3WOy0Yf2+GJTFgg\r
Zk5ql0mLG8Ze+ZifCS0XMO4otlemSyJ0K1ZPdFMGzUDBTgNqzkFabxXoXRIBB0AM\r
755wlX41X6Ay3KhnwBq7yEqSykVH6F3x11iHPKraLCAGZoaS8bKKNy/zg5slda1X\r
@@ -279,25 +278,16 @@ UN4fiB0KR9JyG2ayUdNJVkXZSZLnHyRgiaadlpUo16LVvw==\r
\r
"""
commented_payload = prefix + comment + payload
assert check_armored_payload(commented_payload, outgoing=False) == True
assert check_armored_payload(commented_payload, outgoing=True) == False
payload = prefix + payload
assert check_armored_payload(payload, outgoing=False) == True
assert check_armored_payload(payload, outgoing=True) == True
assert check_armored_payload(payload) == True
payload = payload.removesuffix("\r\n")
assert check_armored_payload(payload, outgoing=False) == True
assert check_armored_payload(payload, outgoing=True) == True
assert check_armored_payload(payload) == True
payload = payload.removesuffix("\r\n")
assert check_armored_payload(payload, outgoing=False) == True
assert check_armored_payload(payload, outgoing=True) == True
assert check_armored_payload(payload) == True
payload = payload.removesuffix("\r\n")
assert check_armored_payload(payload, outgoing=False) == True
assert check_armored_payload(payload, outgoing=True) == True
assert check_armored_payload(payload) == True
payload = """-----BEGIN PGP MESSAGE-----\r
\r
@@ -305,8 +295,7 @@ HELLOWORLD
-----END PGP MESSAGE-----\r
\r
"""
assert check_armored_payload(payload, outgoing=False) == False
assert check_armored_payload(payload, outgoing=True) == False
assert check_armored_payload(payload) == False
payload = """-----BEGIN PGP MESSAGE-----\r
\r
@@ -314,8 +303,7 @@ HELLOWORLD
-----END PGP MESSAGE-----\r
\r
"""
assert check_armored_payload(payload, outgoing=False) == False
assert check_armored_payload(payload, outgoing=True) == False
assert check_armored_payload(payload) == False
# Test payload using partial body length
# as generated by GopenPGP.
@@ -357,5 +345,4 @@ myLbG7cJB787QjplEyVe2P/JBO6xYvbkJLf9Q+HaviTO25rugRSrYsoKMDfO8VlQ\r
=6iHb\r
-----END PGP MESSAGE-----\r
"""
assert check_armored_payload(payload, outgoing=False) == True
assert check_armored_payload(payload, outgoing=True) == True
assert check_armored_payload(payload) == True

View File

@@ -1,9 +0,0 @@
#!/usr/bin/env python3
import socket
def turn_credentials() -> str:
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client_socket:
client_socket.connect("/run/chatmail-turn/turn.socket")
with client_socket.makefile("rb") as file:
return file.readline().decode("utf-8")

View File

@@ -11,9 +11,9 @@ from io import StringIO
from pathlib import Path
from chatmaild.config import Config, read_config
from pyinfra import facts, host, logger
from pyinfra import facts, host
from pyinfra.api import FactBase
from pyinfra.facts.files import File, Sha256File
from pyinfra.facts.files import File
from pyinfra.facts.server import Sysctl
from pyinfra.facts.systemd import SystemdEnabled
from pyinfra.operations import apt, files, pip, server, systemd
@@ -128,7 +128,6 @@ def _install_remote_venv_with_chatmaild(config) -> None:
"echobot",
"chatmail-metadata",
"lastlogin",
"turnserver",
):
execpath = fn if fn != "filtermail-incoming" else "filtermail"
params = dict(
@@ -322,14 +321,14 @@ def _configure_postfix(config: Config, debug: bool = False) -> bool:
def _install_dovecot_package(package: str, arch: str):
arch = "amd64" if arch == "x86_64" else arch
arch = "arm64" if arch == "aarch64" else arch
url = f"https://download.delta.chat/dovecot/dovecot-{package}_2.3.21%2Bdfsg1-3_{arch}.deb"
url = f"https://download.delta.chat/dovecot/staging-drop-libicu/dovecot-{package}_2.3.21%2Bdfsg1-3_{arch}.deb"
deb_filename = "/root/" + url.split("/")[-1]
match (package, arch):
case ("core", "amd64"):
sha256 = "43f593332e22ac7701c62d58b575d2ca409e0f64857a2803be886c22860f5587"
sha256 = "dd060706f52a306fa863d874717210b9fe10536c824afe1790eec247ded5b27d"
case ("core", "arm64"):
sha256 = "4d21eba1a83f51c100f08f2e49f0c9f8f52f721ebc34f75018e043306da993a7"
sha256 = "e7548e8a82929722e973629ecc40fcfa886894cef3db88f23535149e7f730dc9"
case ("imapd", "amd64"):
sha256 = "8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86"
case ("imapd", "arm64"):
@@ -396,21 +395,20 @@ def _configure_dovecot(config: Config, debug: bool = False) -> bool:
config=config,
)
# as per https://doc.dovecot.org/2.3/configuration_manual/os/
# as per https://doc.dovecot.org/configuration_manual/os/
# it is recommended to set the following inotify limits
if config.change_kernel_settings:
for name in ("max_user_instances", "max_user_watches"):
key = f"fs.inotify.{name}"
if host.get_fact(Sysctl)[key] == config.fs_inotify_max_user_instances_and_watchers:
# Skip updating limits if already sufficient
# (enables running in incus containers where sysctl readonly)
continue
server.sysctl(
name=f"Change {key}",
key=key,
value=config.fs_inotify_max_user_instances_and_watchers,
persist=True,
)
for name in ("max_user_instances", "max_user_watches"):
key = f"fs.inotify.{name}"
if host.get_fact(Sysctl)[key] > 65535:
# Skip updating limits if already sufficient
# (enables running in incus containers where sysctl readonly)
continue
server.sysctl(
name=f"Change {key}",
key=key,
value=65535,
persist=True,
)
timezone_env = files.line(
name="Set TZ environment variable",
@@ -499,56 +497,6 @@ def check_config(config):
return config
def deploy_turn_server(config):
(url, sha256sum) = {
"x86_64": (
"https://github.com/chatmail/chatmail-turn/releases/download/v0.3/chatmail-turn-x86_64-linux",
"841e527c15fdc2940b0469e206188ea8f0af48533be12ecb8098520f813d41e4",
),
"aarch64": (
"https://github.com/chatmail/chatmail-turn/releases/download/v0.3/chatmail-turn-aarch64-linux",
"a5fc2d06d937b56a34e098d2cd72a82d3e89967518d159bf246dc69b65e81b42",
),
}[host.get_fact(facts.server.Arch)]
need_restart = False
existing_sha256sum = host.get_fact(Sha256File, "/usr/local/bin/chatmail-turn")
if existing_sha256sum != sha256sum:
server.shell(
name="Download chatmail-turn",
commands=[
f"(curl -L {url} >/usr/local/bin/chatmail-turn.new && (echo '{sha256sum} /usr/local/bin/chatmail-turn.new' | sha256sum -c) && mv /usr/local/bin/chatmail-turn.new /usr/local/bin/chatmail-turn)",
"chmod 755 /usr/local/bin/chatmail-turn",
],
)
need_restart = True
source_path = importlib.resources.files(__package__).joinpath(
"service", "turnserver.service.f"
)
content = source_path.read_text().format(mail_domain=config.mail_domain).encode()
systemd_unit = files.put(
name="Upload turnserver.service",
src=io.BytesIO(content),
dest="/etc/systemd/system/turnserver.service",
user="root",
group="root",
mode="644",
)
need_restart |= systemd_unit.changed
systemd.service(
name="Setup turnserver service",
service="turnserver.service",
running=True,
enabled=True,
restarted=need_restart,
daemon_reload=systemd_unit.changed,
)
def deploy_mtail(config):
# Uninstall mtail package, we are going to install a static binary.
apt.packages(name="Uninstall mtail", packages=["mtail"], present=False)
@@ -607,12 +555,12 @@ def deploy_mtail(config):
def deploy_iroh_relay(config) -> None:
(url, sha256sum) = {
"x86_64": (
"https://github.com/n0-computer/iroh/releases/download/v0.35.0/iroh-relay-v0.35.0-x86_64-unknown-linux-musl.tar.gz",
"45c81199dbd70f8c4c30fef7f3b9727ca6e3cea8f2831333eeaf8aa71bf0fac1",
"https://github.com/n0-computer/iroh/releases/download/v0.28.1/iroh-relay-v0.28.1-x86_64-unknown-linux-musl.tar.gz",
"2ffacf7c0622c26b67a5895ee8e07388769599f60e5f52a3bd40a3258db89b2c",
),
"aarch64": (
"https://github.com/n0-computer/iroh/releases/download/v0.35.0/iroh-relay-v0.35.0-aarch64-unknown-linux-musl.tar.gz",
"f8ef27631fac213b3ef668d02acd5b3e215292746a3fc71d90c63115446008b1",
"https://github.com/n0-computer/iroh/releases/download/v0.28.1/iroh-relay-v0.28.1-aarch64-unknown-linux-musl.tar.gz",
"b915037bcc1ff1110cc9fcb5de4a17c00ff576fd2f568cd339b3b2d54c420dc4",
),
}[host.get_fact(facts.server.Arch)]
@@ -621,18 +569,15 @@ def deploy_iroh_relay(config) -> None:
packages=["curl"],
)
need_restart = False
server.shell(
name="Download iroh-relay",
commands=[
f"(echo '{sha256sum} /usr/local/bin/iroh-relay' | sha256sum -c) || (curl -L {url} | gunzip | tar -x -f - ./iroh-relay -O >/usr/local/bin/iroh-relay.new && mv /usr/local/bin/iroh-relay.new /usr/local/bin/iroh-relay)",
"chmod 755 /usr/local/bin/iroh-relay",
],
)
existing_sha256sum = host.get_fact(Sha256File, "/usr/local/bin/iroh-relay")
if existing_sha256sum != sha256sum:
server.shell(
name="Download iroh-relay",
commands=[
f"(curl -L {url} | gunzip | tar -x -f - ./iroh-relay -O >/usr/local/bin/iroh-relay.new && (echo '{sha256sum} /usr/local/bin/iroh-relay.new' | sha256sum -c) && mv /usr/local/bin/iroh-relay.new /usr/local/bin/iroh-relay)",
"chmod 755 /usr/local/bin/iroh-relay",
],
)
need_restart = True
need_restart = False
systemd_unit = files.put(
name="Upload iroh-relay systemd unit",
@@ -673,7 +618,7 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
check_config(config)
mail_domain = config.mail_domain
from .www import build_webpages, get_paths
from .www import build_webpages
server.group(name="Create vmail group", group="vmail", system=True)
server.user(name="Create vmail user", user="vmail", group="vmail", system=True)
@@ -725,39 +670,15 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
packages=["rsync"],
)
deploy_turn_server(config)
# Run local DNS resolver `unbound`.
# `resolvconf` takes care of setting up /etc/resolv.conf
# to use 127.0.0.1 as the resolver.
from cmdeploy.cmdeploy import Out
port_services = [
(["master", "smtpd"], 25),
("unbound", 53),
("acmetool", 80),
(["imap-login", "dovecot"], 143),
("nginx", 443),
(["master", "smtpd"], 465),
(["master", "smtpd"], 587),
(["imap-login", "dovecot"], 993),
("iroh-relay", 3340),
("nginx", 8443),
(["master", "smtpd"], config.postfix_reinject_port),
(["master", "smtpd"], config.postfix_reinject_port_incoming),
("filtermail", config.filtermail_smtp_port),
("filtermail", config.filtermail_smtp_port_incoming),
]
for service, port in port_services:
print(f"Checking if port {port} is available for {service}...")
running_service = host.get_fact(Port, port=port)
if running_service:
if running_service not in service:
Out().red(
f"Deploy failed: port {port} is occupied by: {running_service}"
)
exit(1)
process_on_53 = host.get_fact(Port, port=53)
if process_on_53 not in (None, "unbound"):
Out().red(f"Can't install unbound: port 53 is occupied by: {process_on_53}")
exit(1)
apt.packages(
name="Install unbound",
packages=["unbound", "unbound-anchor", "dnsutils"],
@@ -781,7 +702,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
# Deploy acmetool to have TLS certificates.
tls_domains = [mail_domain, f"mta-sts.{mail_domain}", f"www.{mail_domain}"]
deploy_acmetool(
email=config.acme_email,
domains=tls_domains,
)
@@ -811,16 +731,12 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
packages=["fcgiwrap"],
)
www_path, src_dir, build_dir = get_paths(config)
# if www_folder was set to a non-existing folder, skip upload
if not www_path.is_dir():
logger.warning("Building web pages is disabled in chatmail.ini, skipping")
else:
# if www_folder is a hugo page, build it
if build_dir:
www_path = build_webpages(src_dir, build_dir, config)
# if it is not a hugo page, upload it as is
files.rsync(f"{www_path}/", "/var/www/html", flags=["-avz", "--chown=www-data"])
www_path = importlib.resources.files(__package__).joinpath("../../../www").resolve()
build_dir = www_path.joinpath("build")
src_dir = www_path.joinpath("src")
build_webpages(src_dir, build_dir, config)
files.rsync(f"{build_dir}/", "/var/www/html", flags=["-avz"])
_install_remote_venv_with_chatmaild(config)
debug = False
@@ -868,19 +784,6 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
restarted=nginx_need_restart,
)
systemd.service(
name="Start and enable fcgiwrap",
service="fcgiwrap.service",
running=True,
enabled=True,
)
systemd.service(
name="Restart echobot if postfix and dovecot were just started",
service="echobot.service",
restarted=postfix_need_restart and dovecot_need_restart,
)
# This file is used by auth proxy.
# https://wiki.debian.org/EtcMailName
server.shell(
@@ -913,14 +816,8 @@ def deploy_chatmail(config_path: Path, disable_mail: bool) -> None:
name="Ensure cron is installed",
packages=["cron"],
)
try:
git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode()
except Exception:
git_hash = "unknown\n"
try:
git_diff = subprocess.check_output(["git", "diff"]).decode()
except Exception:
git_diff = ""
git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode()
git_diff = subprocess.check_output(["git", "diff"]).decode()
files.put(
name="Upload chatmail relay git commiit hash",
src=StringIO(git_hash + git_diff),

View File

@@ -1,5 +1,7 @@
import importlib.resources
from pyinfra import host
from pyinfra.facts.systemd import SystemdStatus
from pyinfra.operations import apt, files, server, systemd
@@ -52,6 +54,12 @@ def deploy_acmetool(email="", domains=[]):
group="root",
mode="644",
)
if host.get_fact(SystemdStatus).get("nginx.service"):
systemd.service(
name="Stop nginx service to free port 80",
service="nginx",
running=False,
)
systemd.service(
name="Setup acmetool-redirector service",

View File

@@ -19,7 +19,7 @@ from packaging import version
from termcolor import colored
from . import dns, remote
from .sshexec import SSHExec, LocalExec
from .sshexec import SSHExec
#
# cmdeploy sub commands and options
@@ -32,30 +32,17 @@ def init_cmd_options(parser):
action="store",
help="fully qualified DNS domain name for your chatmail instance",
)
parser.add_argument(
"--force",
dest="recreate_ini",
action="store_true",
help="force reacreate ini file",
)
def init_cmd(args, out):
"""Initialize chatmail config file."""
mail_domain = args.chatmail_domain
inipath = args.inipath
if args.inipath.exists():
if not args.recreate_ini:
print(f"[WARNING] Path exists, not modifying: {inipath}")
return 1
else:
print(
f"[WARNING] Force argument was provided, deleting config file: {inipath}"
)
inipath.unlink()
write_initial_config(inipath, mail_domain, overrides={})
out.green(f"created config file for {mail_domain} in {inipath}")
print(f"Path exists, not modifying: {args.inipath}")
return 1
else:
write_initial_config(args.inipath, mail_domain, overrides={})
out.green(f"created config file for {mail_domain} in {args.inipath}")
def run_cmd_options(parser):
@@ -72,24 +59,20 @@ def run_cmd_options(parser):
help="install/upgrade the server, but disable postfix & dovecot for now",
)
parser.add_argument(
"--skip-dns-check",
dest="dns_check_disabled",
action="store_true",
help="disable checks nslookup for dns",
"--ssh-host",
dest="ssh_host",
help="specify an SSH host to deploy to; uses mail_domain from chatmail.ini by default",
)
add_ssh_host_option(parser)
def run_cmd(args, out):
"""Deploy chatmail services on the remote server."""
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
sshexec = get_sshexec(ssh_host)
sshexec = args.get_sshexec()
require_iroh = args.config.enable_iroh_relay
if not args.dns_check_disabled:
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
if not dns.check_initial_remote_data(remote_data, print=out.red):
return 1
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
if not dns.check_initial_remote_data(remote_data, print=out.red):
return 1
env = os.environ.copy()
env["CHATMAIL_INI"] = args.inipath
@@ -97,11 +80,8 @@ def run_cmd(args, out):
env["CHATMAIL_REQUIRE_IROH"] = "True" if require_iroh else ""
deploy_path = importlib.resources.files(__package__).joinpath("deploy.py").resolve()
pyinf = "pyinfra --dry" if args.dry_run else "pyinfra"
ssh_host = args.config.mail_domain if not args.ssh_host else args.ssh_host
cmd = f"{pyinf} --ssh-user root {ssh_host} {deploy_path} -y"
if ssh_host in ["localhost", "@docker"]:
cmd = f"{pyinf} @local {deploy_path} -y"
if version.parse(pyinfra.__version__) < version.parse("3"):
out.red("Please re-run scripts/initenv.sh to update pyinfra to version 3.")
return 1
@@ -109,18 +89,6 @@ def run_cmd(args, out):
try:
retcode = out.check_call(cmd, env=env)
if retcode == 0:
if not args.disable_mail:
print("\nYou can try out the relay by talking to this echo bot: ")
sshexec = SSHExec(args.config.mail_domain, verbose=args.verbose)
print(
sshexec(
call=remote.rshell.shell,
kwargs=dict(command="cat /var/lib/echobot/invite-link.txt"),
)
)
server_deployed_message = f"Chatmail server started: https://{args.config.mail_domain}/"
delimiter_line = "=" * len(server_deployed_message)
out.green(f"{delimiter_line}\n{server_deployed_message}\n{delimiter_line}")
out.green("Deploy completed, call `cmdeploy dns` next.")
elif not remote_data["acme_account_url"]:
out.red("Deploy completed but letsencrypt not configured")
@@ -142,13 +110,11 @@ def dns_cmd_options(parser):
default=None,
help="write out a zonefile",
)
add_ssh_host_option(parser)
def dns_cmd(args, out):
"""Check DNS entries and optionally generate dns zone file."""
ssh_host = args.ssh_host if args.ssh_host else args.config.mail_domain
sshexec = get_sshexec(ssh_host, verbose=args.verbose)
sshexec = args.get_sshexec()
remote_data = dns.get_initial_remote_data(sshexec, args.config.mail_domain)
if not remote_data:
return 1
@@ -302,15 +268,6 @@ class Out:
return proc.returncode
def add_ssh_host_option(parser):
parser.add_argument(
"--ssh-host",
dest="ssh_host",
help="Run commands on 'localhost', via '@docker', or on a specific SSH host "
"instead of chatmail.ini's mail_domain.",
)
def add_config_option(parser):
parser.add_argument(
"--config",
@@ -366,16 +323,6 @@ def get_parser():
return parser
def get_sshexec(ssh_host: str, verbose=True):
if ssh_host in ["localhost", "@local"]:
return LocalExec(verbose, docker=False)
elif ssh_host == "@docker":
return LocalExec(verbose, docker=True)
if verbose:
print(f"[ssh] login to {ssh_host}")
return SSHExec(ssh_host, verbose=verbose)
def main(args=None):
"""Provide main entry point for 'cmdeploy' CLI invocation."""
parser = get_parser()
@@ -383,6 +330,12 @@ def main(args=None):
if not hasattr(args, "func"):
return parser.parse_args(["-h"])
def get_sshexec():
print(f"[ssh] login to {args.config.mail_domain}")
return SSHExec(args.config.mail_domain, verbose=args.verbose)
args.get_sshexec = get_sshexec
out = Out()
kwargs = {}
if args.func.__name__ not in ("init_cmd", "fmt_cmd"):

View File

@@ -45,7 +45,8 @@ def check_full_zone(sshexec, remote_data, out, zonefile) -> int:
and return (exitcode, remote_data) tuple."""
required_diff, recommended_diff = sshexec.logged(
remote.rdns.check_zonefile, kwargs=dict(zonefile=zonefile, verbose=False),
remote.rdns.check_zonefile,
kwargs=dict(zonefile=zonefile, mail_domain=remote_data["mail_domain"]),
)
returncode = 0

View File

@@ -1,11 +1,5 @@
enable_relay = true
http_bind_addr = "[::]:3340"
# Disable built-in STUN server in iroh-relay 0.35
# as we deploy our own TURN server instead.
# STUN server is going to be removed in iroh-relay 1.0
# and this line can be removed after upgrade.
enable_stun = false
enable_stun = true
enable_metrics = false
metrics_bind_addr = "127.0.0.1:9092"

View File

@@ -77,13 +77,13 @@ scache unix - - y - 1 scache
postlog unix-dgram n - n - 1 postlogd
filter unix - n n - - lmtp
# Local SMTP server for reinjecting outgoing filtered mail.
127.0.0.1:{{ config.postfix_reinject_port }} inet n - n - 100 smtpd
127.0.0.1:{{ config.postfix_reinject_port }} inet n - n - 10 smtpd
-o syslog_name=postfix/reinject
-o smtpd_milters=unix:opendkim/opendkim.sock
-o cleanup_service_name=authclean
# Local SMTP server for reinjecting incoming filtered mail
127.0.0.1:{{ config.postfix_reinject_port_incoming }} inet n - n - 100 smtpd
127.0.0.1:{{ config.postfix_reinject_port_incoming }} inet n - n - 10 smtpd
-o syslog_name=postfix/reinject_incoming
-o smtpd_milters=unix:opendkim/opendkim.sock

View File

@@ -12,23 +12,23 @@ All functions of this module
import re
from .rshell import CalledProcessError, shell, log_progress
from .rshell import CalledProcessError, shell
def perform_initial_checks(mail_domain, pre_command=""):
def perform_initial_checks(mail_domain):
"""Collecting initial DNS settings."""
assert mail_domain
if not shell("dig", fail_ok=True, print=log_progress):
shell("apt-get update && apt-get install -y dnsutils", print=log_progress)
if not shell("dig", fail_ok=True):
shell("apt-get update && apt-get install -y dnsutils")
A = query_dns("A", mail_domain)
AAAA = query_dns("AAAA", mail_domain)
MTA_STS = query_dns("CNAME", f"mta-sts.{mail_domain}")
WWW = query_dns("CNAME", f"www.{mail_domain}")
res = dict(mail_domain=mail_domain, A=A, AAAA=AAAA, MTA_STS=MTA_STS, WWW=WWW)
res["acme_account_url"] = shell(pre_command + "acmetool account-url", fail_ok=True, print=log_progress)
res["acme_account_url"] = shell("acmetool account-url", fail_ok=True)
res["dkim_entry"], res["web_dkim_entry"] = get_dkim_entry(
mail_domain, pre_command, dkim_selector="opendkim"
mail_domain, dkim_selector="opendkim"
)
if not MTA_STS or not WWW or (not A and not AAAA):
@@ -40,12 +40,11 @@ def perform_initial_checks(mail_domain, pre_command=""):
return res
def get_dkim_entry(mail_domain, pre_command, dkim_selector):
def get_dkim_entry(mail_domain, dkim_selector):
try:
dkim_pubkey = shell(
f"{pre_command}openssl rsa -in /etc/dkimkeys/{dkim_selector}.private "
"-pubout 2>/dev/null | awk '/-/{next}{printf(\"%s\",$0)}'",
print=log_progress
f"openssl rsa -in /etc/dkimkeys/{dkim_selector}.private "
"-pubout 2>/dev/null | awk '/-/{next}{printf(\"%s\",$0)}'"
)
except CalledProcessError:
return
@@ -62,7 +61,7 @@ def query_dns(typ, domain):
# Get autoritative nameserver from the SOA record.
soa_answers = [
x.split()
for x in shell(f"dig -r -q {domain} -t SOA +noall +authority +answer", print=log_progress).split(
for x in shell(f"dig -r -q {domain} -t SOA +noall +authority +answer").split(
"\n"
)
]
@@ -72,13 +71,13 @@ def query_dns(typ, domain):
ns = soa[0][4]
# Query authoritative nameserver directly to bypass DNS cache.
res = shell(f"dig @{ns} -r -q {domain} -t {typ} +short", print=log_progress)
res = shell(f"dig @{ns} -r -q {domain} -t {typ} +short")
if res:
return res.split("\n")[0]
return ""
def check_zonefile(zonefile, verbose=True):
def check_zonefile(zonefile, mail_domain):
"""Check expected zone file entries."""
required = True
required_diff = []
@@ -90,7 +89,7 @@ def check_zonefile(zonefile, verbose=True):
continue
if not zf_line.strip() or zf_line.startswith(";"):
continue
print(f"dns-checking {zf_line!r}") if verbose else log_progress("")
print(f"dns-checking {zf_line!r}")
zf_domain, zf_typ, zf_value = zf_line.split(maxsplit=2)
zf_domain = zf_domain.rstrip(".")
zf_value = zf_value.strip()

View File

@@ -1,14 +1,7 @@
import sys
from subprocess import DEVNULL, CalledProcessError, check_output
def log_progress(data):
sys.stderr.write(".")
sys.stderr.flush()
def shell(command, fail_ok=False, print=print):
def shell(command, fail_ok=False):
print(f"$ {command}")
args = dict(shell=True)
if fail_ok:

View File

@@ -1,16 +0,0 @@
[Unit]
Description=A wrapper for the TURN server
After=network.target
[Service]
Type=simple
Restart=always
ExecStart=/usr/local/bin/chatmail-turn --realm {mail_domain} --socket /run/chatmail-turn/turn.socket
# Create /run/chatmail-turn
RuntimeDirectory=chatmail-turn
User=vmail
Group=vmail
[Install]
WantedBy=multi-user.target

View File

@@ -42,7 +42,6 @@ def bootstrap_remote(gateway, remote=remote):
def print_stderr(item="", end="\n"):
print(item, file=sys.stderr, end=end)
sys.stderr.flush()
class SSHExec:
@@ -71,6 +70,10 @@ class SSHExec:
raise self.FuncError(data)
def logged(self, call, kwargs):
def log_progress(data):
sys.stderr.write(".")
sys.stderr.flush()
title = call.__doc__
if not title:
title = call.__name__
@@ -79,22 +82,6 @@ class SSHExec:
return self(call, kwargs, log_callback=print_stderr)
else:
print_stderr(title, end="")
res = self(call, kwargs, log_callback=remote.rshell.log_progress)
res = self(call, kwargs, log_callback=log_progress)
print_stderr()
return res
class LocalExec:
def __init__(self, verbose=False, docker=False):
self.verbose = verbose
self.docker = docker
def logged(self, call, kwargs: dict):
where = "locally"
if self.docker:
if call == remote.rdns.perform_initial_checks:
kwargs['pre_command'] = "docker exec chatmail "
where = "in docker"
if self.verbose:
print(f"Running {where}: {call.__name__}(**{kwargs})")
return call(**kwargs)

View File

@@ -2,7 +2,6 @@ import datetime
import smtplib
import socket
import subprocess
import time
import pytest
@@ -10,37 +9,10 @@ from cmdeploy import remote
from cmdeploy.sshexec import SSHExec
class FuncError(Exception):
pass
class DockerExec:
FuncError = FuncError
def __init__(self, pre_command):
self.pre_command = pre_command
def __call__(self, call, kwargs=None):
if kwargs is None:
kwargs = {}
return call(**kwargs)
def logged(self, call, kwargs):
title = call.__doc__
if not title:
title = call.__name__
print("[ssh] " + title)
return self(call, kwargs)
class TestSSHExecutor:
@pytest.fixture(scope="class")
def sshexec(self, sshdomain):
try:
sshexec = SSHExec(sshdomain)
except FileNotFoundError:
sshexec = DockerExec("docker exec chatmail ")
return sshexec
return SSHExec(sshdomain)
def test_ls(self, sshexec):
out = sshexec(call=remote.rdns.shell, kwargs=dict(command="ls"))
@@ -54,15 +26,12 @@ class TestSSHExecutor:
assert res["A"] or res["AAAA"]
def test_logged(self, sshexec, maildomain, capsys):
if isinstance(sshexec, DockerExec):
pytest.skip("This test only works via SSH")
sshexec.logged(
remote.rdns.perform_initial_checks, kwargs=dict(mail_domain=maildomain)
)
out, err = capsys.readouterr()
assert err.startswith("Collecting")
# XXX could not figure out how capturing can be made to work properly
#assert err.endswith("....\n")
assert err.endswith("....\n")
assert err.count("\n") == 1
sshexec.verbose = True
@@ -71,8 +40,7 @@ class TestSSHExecutor:
)
out, err = capsys.readouterr()
lines = err.split("\n")
# XXX could not figure out how capturing can be made to work properly
#assert len(lines) > 4
assert len(lines) > 4
assert remote.rdns.perform_initial_checks.__doc__ in lines[0]
def test_exception(self, sshexec, capsys):
@@ -84,8 +52,6 @@ class TestSSHExecutor:
except sshexec.FuncError as e:
assert "rdns.py" in str(e)
assert "AssertionError" in str(e)
except AssertionError:
assert isinstance(sshexec, DockerExec)
else:
pytest.fail("didn't raise exception")
@@ -103,7 +69,7 @@ def test_timezone_env(remote):
for line in remote.iter_output("env"):
print(line)
if line == "tz=:/etc/localtime":
return
return True
pytest.fail("TZ is not set")
@@ -180,16 +146,6 @@ def test_reject_missing_dkim(cmsetup, maildata, from_addr):
s.sendmail(from_addr=from_addr, to_addrs=recipient.addr, msg=msg)
def try_n_times(n, f):
for _ in range(n - 1):
try:
return f()
except Exception:
time.sleep(1)
return f()
def test_rewrite_subject(cmsetup, maildata):
"""Test that subject gets replaced with [...]."""
user1, user2 = cmsetup.gen_users(2)
@@ -202,8 +158,7 @@ def test_rewrite_subject(cmsetup, maildata):
).as_string()
user1.smtp.sendmail(from_addr=user1.addr, to_addrs=[user2.addr], msg=sent_msg)
# The message may need some time to get delivered by postfix.
messages = try_n_times(5, user2.imap.fetch_all_messages)
messages = user2.imap.fetch_all_messages()
assert len(messages) == 1
rcvd_msg = messages[0]
assert "Subject: [...]" not in sent_msg

View File

@@ -337,14 +337,10 @@ class Remote:
def iter_output(self, logcmd=""):
getjournal = "journalctl -f" if not logcmd else logcmd
try:
self.popen = subprocess.Popen(
["ssh", f"root@{self.sshdomain}", getjournal],
stdout=subprocess.PIPE,
)
except FileNotFoundError:
# inside docker container, run locally
self.popen = subprocess.Popen([getjournal], stdout=subprocess.PIPE)
self.popen = subprocess.Popen(
["ssh", f"root@{self.sshdomain}", getjournal],
stdout=subprocess.PIPE,
)
while 1:
line = self.popen.stdout.readline()
res = line.decode().strip().lower()

View File

@@ -1,10 +1,8 @@
import importlib
import os
import pytest
from cmdeploy.cmdeploy import get_parser, main
from cmdeploy.www import get_paths
@pytest.fixture(autouse=True)
@@ -26,36 +24,6 @@ class TestCmdline:
def test_init_not_overwrite(self, capsys):
assert main(["init", "chat.example.org"]) == 0
capsys.readouterr()
assert main(["init", "chat.example.org"]) == 1
out, err = capsys.readouterr()
assert "path exists" in out.lower()
assert main(["init", "chat.example.org", "--force"]) == 0
out, err = capsys.readouterr()
assert "deleting config file" in out.lower()
def test_www_folder(example_config, tmp_path):
reporoot = importlib.resources.files(__package__).joinpath("../../../../").resolve()
assert not example_config.www_folder
www_path, src_dir, build_dir = get_paths(example_config)
assert www_path.absolute() == reporoot.joinpath("www").absolute()
assert src_dir == reporoot.joinpath("www").joinpath("src")
assert build_dir == reporoot.joinpath("www").joinpath("build")
example_config.www_folder = "disabled"
www_path, _, _ = get_paths(example_config)
assert not www_path.is_dir()
example_config.www_folder = str(tmp_path)
www_path, src_dir, build_dir = get_paths(example_config)
assert www_path == tmp_path
assert not src_dir.exists()
assert not build_dir
src_path = tmp_path.joinpath("src")
os.mkdir(src_path)
with open(src_path / "index.md", "w") as f:
f.write("# Test")
www_path, src_dir, build_dir = get_paths(example_config)
assert www_path == tmp_path
assert src_dir == src_path
assert build_dir == tmp_path.joinpath("build")

View File

@@ -89,14 +89,18 @@ class TestZonefileChecks:
def test_check_zonefile_all_ok(self, cm_data, mockdns_base):
zonefile = cm_data.get("zftest.zone")
parse_zonefile_into_dict(zonefile, mockdns_base)
required_diff, recommended_diff = remote.rdns.check_zonefile(zonefile)
required_diff, recommended_diff = remote.rdns.check_zonefile(
zonefile, "some.domain"
)
assert not required_diff and not recommended_diff
def test_check_zonefile_recommended_not_set(self, cm_data, mockdns_base):
zonefile = cm_data.get("zftest.zone")
zonefile_mocked = zonefile.split("; Recommended")[0]
parse_zonefile_into_dict(zonefile_mocked, mockdns_base)
required_diff, recommended_diff = remote.rdns.check_zonefile(zonefile)
required_diff, recommended_diff = remote.rdns.check_zonefile(
zonefile, "some.domain"
)
assert not required_diff
assert len(recommended_diff) == 8

View File

@@ -3,7 +3,6 @@ import importlib.resources
import time
import traceback
import webbrowser
from pathlib import Path
import markdown
from chatmaild.config import read_config
@@ -31,25 +30,9 @@ def prepare_template(source):
return render_vars, page_layout
def get_paths(config) -> (Path, Path, Path):
reporoot = importlib.resources.files(__package__).joinpath("../../../").resolve()
www_path = Path(config.www_folder)
# if www_folder was not set, use default directory
if config.www_folder == "":
www_path = reporoot.joinpath("www")
src_dir = www_path.joinpath("src")
# if www_folder is a hugo page, build it
if src_dir.joinpath("index.md").is_file():
build_dir = www_path.joinpath("build")
# if it is not a hugo page, upload it as is
else:
build_dir = None
return www_path, src_dir, build_dir
def build_webpages(src_dir, build_dir, config) -> Path:
def build_webpages(src_dir, build_dir, config):
try:
return _build_webpages(src_dir, build_dir, config)
_build_webpages(src_dir, build_dir, config)
except Exception:
print(traceback.format_exc())
@@ -123,11 +106,15 @@ def main():
config = read_config(inipath)
config.webdev = True
assert config.mail_domain
www_path = reporoot.joinpath("www")
src_path = www_path.joinpath("src")
stats = None
build_dir = www_path.joinpath("build")
src_dir = www_path.joinpath("src")
index_path = build_dir.joinpath("index.html")
# start web page generation, open a browser and wait for changes
www_path, src_path, build_dir = get_paths(config)
build_dir = build_webpages(src_path, build_dir, config)
index_path = build_dir.joinpath("index.html")
build_webpages(src_dir, build_dir, config)
webbrowser.open(str(index_path))
stats = snapshot_dir_stats(src_path)
print(f"\nOpened URL: file://{index_path.resolve()}\n")
@@ -148,7 +135,7 @@ def main():
changenum += 1
stats = newstats
build_webpages(src_path, build_dir, config)
build_webpages(src_dir, build_dir, config)
print(f"[{changenum}] regenerated web pages at: {index_path}")
print(f"URL: file://{index_path.resolve()}\n\n")
count = 0

View File

@@ -1,83 +0,0 @@
FROM jrei/systemd-debian:12 AS base
ENV LANG=en_US.UTF-8
RUN echo 'APT::Install-Recommends "0";' > /etc/apt/apt.conf.d/01norecommend && \
echo 'APT::Install-Suggests "0";' >> /etc/apt/apt.conf.d/01norecommend && \
apt-get update && \
apt-get install -y \
ca-certificates && \
DEBIAN_FRONTEND=noninteractive \
TZ=Europe/London \
apt-get install -y tzdata && \
apt-get install -y locales && \
sed -i -e "s/# $LANG.*/$LANG UTF-8/" /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=$LANG \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && \
apt-get install -y \
git \
python3 \
python3-venv \
python3-virtualenv \
gcc \
python3-dev \
opendkim \
opendkim-tools \
curl \
rsync \
unbound \
unbound-anchor \
dnsutils \
postfix \
acl \
nginx \
libnginx-mod-stream \
fcgiwrap \
cron \
&& for pkg in core imapd lmtpd; do \
case "$pkg" in \
core) sha256="43f593332e22ac7701c62d58b575d2ca409e0f64857a2803be886c22860f5587" ;; \
imapd) sha256="8d8dc6fc00bbb6cdb25d345844f41ce2f1c53f764b79a838eb2a03103eebfa86" ;; \
lmtpd) sha256="2f69ba5e35363de50962d42cccbfe4ed8495265044e244007d7ccddad77513ab" ;; \
esac; \
url="https://download.delta.chat/dovecot/dovecot-${pkg}_2.3.21%2Bdfsg1-3_amd64.deb"; \
file="/tmp/$(basename "$url")"; \
curl -fsSL "$url" -o "$file"; \
echo "$sha256 $file" | sha256sum -c -; \
apt-get install -y "$file"; \
rm -f "$file"; \
done \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /opt/chatmail
ARG SETUP_CHATMAIL_SERVICE_PATH=/lib/systemd/system/setup_chatmail.service
COPY ./files/setup_chatmail.service "$SETUP_CHATMAIL_SERVICE_PATH"
RUN ln -sf "$SETUP_CHATMAIL_SERVICE_PATH" "/etc/systemd/system/multi-user.target.wants/setup_chatmail.service"
COPY --chmod=555 ./files/setup_chatmail_docker.sh /setup_chatmail_docker.sh
COPY --chmod=555 ./files/update_ini.sh /update_ini.sh
COPY --chmod=555 ./files/entrypoint.sh /entrypoint.sh
## TODO: add git clone.
## Problem: how correct save only required files inside container....
# RUN git clone https://github.com/chatmail/relay.git -b master . \
# && ./scripts/initenv.sh
# EXPOSE 443 25 587 143 993
VOLUME ["/sys/fs/cgroup", "/home"]
STOPSIGNAL SIGRTMIN+3
ENTRYPOINT ["/entrypoint.sh"]
CMD [ "--default-standard-output=journal+console", \
"--default-standard-error=journal+console" ]
## TODO: Add installation and configuration of chatmaild inside the Dockerfile.
## This is required to ensure repeatable deployment.
## In the current MVP, the chatmaild server is updated on every container restart.

View File

@@ -1,59 +0,0 @@
services:
chatmail:
build:
context: ./docker
dockerfile: chatmail_relay.dockerfile
tags:
- chatmail-relay:latest
image: chatmail-relay:latest
restart: unless-stopped
container_name: chatmail
cgroup: host # required for systemd
tty: true # required for logs
tmpfs: # required for systemd
- /tmp
- /run
- /run/lock
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
environment:
MAIL_DOMAIN: $MAIL_DOMAIN
CHANGE_KERNEL_SETTINGS: "False"
ACME_EMAIL: $ACME_EMAIL
# RECREATE_VENV: "false"
# MAX_MESSAGE_SIZE: "50M"
# DEBUG_COMMANDS_ENABLED: "true"
# FORCE_REINIT_INI_FILE: "true"
# USE_FOREIGN_CERT_MANAGER: "True"
# ENABLE_CERTS_MONITORING: "true"
# CERTS_MONITORING_TIMEOUT: 10
# IS_DEVELOPMENT_INSTANCE: "True"
ports:
- "80:80"
- "443:443"
- "25:25"
- "587:587"
- "143:143"
- "465:465"
- "993:993"
volumes:
## system
- /sys/fs/cgroup:/sys/fs/cgroup:rw # required for systemd
- ./:/opt/chatmail
## data
- ./data/chatmail:/home
- ./data/chatmail-dkimkeys:/etc/dkimkeys
- ./data/chatmail-echobot:/run/echobot
- ./data/chatmail-acme:/var/lib/acme
## custom resources
# - ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
## debug
# - ./docker/files/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
# - ./docker/files/entrypoint.sh:/entrypoint.sh
# - ./docker/files/update_ini.sh:/update_ini.sh

View File

@@ -1 +0,0 @@
MAIL_DOMAIN="chat.example.com"

View File

@@ -1,11 +0,0 @@
#!/bin/bash
set -eo pipefail
unlink /etc/nginx/sites-enabled/default || true
SETUP_CHATMAIL_SERVICE_PATH="${SETUP_CHATMAIL_SERVICE_PATH:-/lib/systemd/system/setup_chatmail.service}"
env_vars=$(printenv | cut -d= -f1 | xargs)
sed -i "s|<envs_list>|$env_vars|g" $SETUP_CHATMAIL_SERVICE_PATH
exec /lib/systemd/systemd $@

View File

@@ -1,14 +0,0 @@
[Unit]
Description=Run container setup commands
After=multi-user.target
ConditionPathExists=/setup_chatmail_docker.sh
[Service]
Type=oneshot
ExecStart=/bin/bash /setup_chatmail_docker.sh
RemainAfterExit=true
WorkingDirectory=/opt/chatmail
PassEnvironment=<envs_list>
[Install]
WantedBy=multi-user.target

View File

@@ -1,78 +0,0 @@
#!/bin/bash
set -eo pipefail
export INI_FILE="${INI_FILE:-chatmail.ini}"
export ENABLE_CERTS_MONITORING="${ENABLE_CERTS_MONITORING:-true}"
export CERTS_MONITORING_TIMEOUT="${CERTS_MONITORING_TIMEOUT:-60}"
export PATH_TO_SSL="${PATH_TO_SSL:-/var/lib/acme/live/${MAIL_DOMAIN}}"
export CHANGE_KERNEL_SETTINGS=${CHANGE_KERNEL_SETTINGS:-"False"}
export RECREATE_VENV=${RECREATE_VENV:-"false"}
if [ -z "$MAIL_DOMAIN" ]; then
echo "ERROR: Environment variable 'MAIL_DOMAIN' must be set!" >&2
exit 1
fi
debug_commands() {
echo "Executing debug commands"
# git config --global --add safe.directory /opt/chatmail
# ./scripts/initenv.sh
}
calculate_hash() {
find "$PATH_TO_SSL" -type f -exec sha1sum {} \; | sort | sha1sum | awk '{print $1}'
}
monitor_certificates() {
if [ "$ENABLE_CERTS_MONITORING" != "true" ]; then
echo "Certs monitoring disabled."
exit 0
fi
current_hash=$(calculate_hash)
previous_hash=$current_hash
while true; do
current_hash=$(calculate_hash)
if [[ "$current_hash" != "$previous_hash" ]]; then
# TODO: add an option to restart at a specific time interval
echo "[INFO] Certificate's folder hash was changed, reloading nginx, dovecot and postfix services."
systemctl reload nginx.service
systemctl reload dovecot.service
systemctl reload postfix.service
previous_hash=$current_hash
fi
sleep $CERTS_MONITORING_TIMEOUT
done
}
### MAIN
if [ "$DEBUG_COMMANDS_ENABLED" == "true" ]; then
debug_commands
fi
if [ "$FORCE_REINIT_INI_FILE" == "true" ]; then
INI_CMD_ARGS=--force
fi
/usr/sbin/opendkim-genkey -D /etc/dkimkeys -d $MAIL_DOMAIN -s opendkim
chown opendkim:opendkim /etc/dkimkeys/opendkim.private
chown opendkim:opendkim /etc/dkimkeys/opendkim.txt
# TODO: Move to debug_commands after git clone is moved to dockerfile.
git config --global --add safe.directory /opt/chatmail
if [ "$RECREATE_VENV" == "true" ]; then
rm -rf venv
fi
./scripts/initenv.sh
./scripts/cmdeploy init --config "${INI_FILE}" $INI_CMD_ARGS $MAIL_DOMAIN
bash /update_ini.sh
./scripts/cmdeploy run --ssh-host docker
echo "ForwardToConsole=yes" >> /etc/systemd/journald.conf
systemctl restart systemd-journald
monitor_certificates &

View File

@@ -1,79 +0,0 @@
#!/bin/bash
set -eo pipefail
INI_FILE="${INI_FILE:-chatmail.ini}"
if [ ! -f "$INI_FILE" ]; then
echo "Error: file $INI_FILE not found." >&2
exit 1
fi
TMP_FILE="$(mktemp)"
convert_to_bytes() {
local value="$1"
if [[ "$value" =~ ^([0-9]+)([KkMmGgTt])$ ]]; then
local num="${BASH_REMATCH[1]}"
local unit="${BASH_REMATCH[2]}"
case "$unit" in
[Kk]) echo $((num * 1024)) ;;
[Mm]) echo $((num * 1024 * 1024)) ;;
[Gg]) echo $((num * 1024 * 1024 * 1024)) ;;
[Tt]) echo $((num * 1024 * 1024 * 1024 * 1024)) ;;
esac
elif [[ "$value" =~ ^[0-9]+$ ]]; then
echo "$value"
else
echo "Error: incorrect size format: $value." >&2
return 1
fi
}
process_specific_params() {
local key=$1
local value=$2
local destination_file=$3
if [[ "$key" == "max_message_size" ]]; then
converted=$(convert_to_bytes "$value") || exit 1
if grep -q -e "## .* = .* bytes" "$destination_file"; then
sed "s|## .* = .* bytes|## $value = $converted bytes|g" "$destination_file";
else
echo "## $value = $converted bytes" >> "$destination_file"
fi
echo "$key = $converted" >> "$destination_file"
else
echo "$key = $value" >> "$destination_file"
fi
}
while IFS= read -r line; do
if [[ "$line" =~ ^[[:space:]]*#.* || "$line" =~ ^[[:space:]]*$ ]]; then
echo "$line" >> "$TMP_FILE"
continue
fi
if [[ "$line" =~ ^([a-z0-9_]+)[[:space:]]*=[[:space:]]*(.*)$ ]]; then
key="${BASH_REMATCH[1]}"
current_value="${BASH_REMATCH[2]}"
env_var_name=$(echo "$key" | tr 'a-z' 'A-Z')
env_value="${!env_var_name}"
if [[ -n "$env_value" ]]; then
process_specific_params "$key" "$env_value" "$TMP_FILE"
else
echo "$line" >> "$TMP_FILE"
fi
else
echo "$line" >> "$TMP_FILE"
fi
done < "$INI_FILE"
PERMS=$(stat -c %a "$INI_FILE")
OWNER=$(stat -c %u "$INI_FILE")
GROUP=$(stat -c %g "$INI_FILE")
chmod "$PERMS" "$TMP_FILE"
chown "$OWNER":"$GROUP" "$TMP_FILE"
mv "$TMP_FILE" "$INI_FILE"

View File

@@ -1,197 +0,0 @@
# Known issues and limitations
- Chatmail will be reinstalled every time the container is started (longer the first time, faster on subsequent starts). This is how the original installer works because it wasnt designed for Docker. At the end of the documentation, theres a [proposed solution](#locking-the-chatmail-version).
- Requires cgroups v2 configured in the system. Operation with cgroups v1 has not been tested.
- Yes, of course, using systemd inside a container is a hack, and it would be better to split it into several services, but since this is an MVP, it turned out to be easier to do it this way initially than to rewrite the entire deployment system.
- The Docker image is only suitable for amd64. If you need to run it on a different architecture, try modifying the Dockerfile (specifically the part responsible for installing dovecot).
# Docker installation
This section provides instructions for installing Chatmail using docker-compose.
## Preliminary setup
We use `chat.example.org` as the Chatmail domain in the following steps.
Please substitute it with your own domain.
1. Setup the initial DNS records.
The following is an example in the familiar BIND zone file format with
a TTL of 1 hour (3600 seconds).
Please substitute your domain and IP addresses.
```
chat.example.com. 3600 IN A 198.51.100.5
chat.example.com. 3600 IN AAAA 2001:db8::5
www.chat.example.com. 3600 IN CNAME chat.example.com.
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
```
2. clone the repository on your server.
```shell
git clone https://github.com/chatmail/relay
cd relay
```
## Installation
1. Copy the file `./docker/docker-compose-default.yaml` to `docker-compose.yaml`. This is necessary because `docker-compose.yaml` is in `.gitignore` and wont cause conflicts when updating the git repository.
```shell
cp ./docker/docker-compose-default.yaml docker-compose.yaml
```
3. Configure environment variables in the `.env` file. These variables are used in the `docker-compose.yaml` file to pass repeated values.
4. Configure kernel parameters because they cannot be changed inside the container, specifically `fs.inotify.max_user_instances` and `fs.inotify.max_user_watches`. Run the following:
```shell
echo "fs.inotify.max_user_instances=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
echo "fs.inotify.max_user_watches=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
sudo sysctl --system
```
5. Configure container environment variables. Below is the list of variables used during deployment:
- `MAIL_DOMAIN` The domain name of the future server. (required)
- `DEBUG_COMMANDS_ENABLED` Run debug commands before installation. (default: `false`)
- `FORCE_REINIT_INI_FILE` Recreate the ini configuration file on startup. (default: `false`)
- `USE_FOREIGN_CERT_MANAGER` Use a third-party certificate manager. (default: `false`)
- `RECREATE_VENV` - Recreate the virtual environment (venv). If set to `true`, the environment will be recreated when the container starts, which will increase the startup time of the service but can help avoid certain errors. (default: `false`)
- `INI_FILE` Path to the ini configuration file. (default: `./chatmail.ini`)
- `PATH_TO_SSL` Path to where the certificates are stored. (default: `/var/lib/acme/live/${MAIL_DOMAIN}`)
- `ENABLE_CERTS_MONITORING` Enable certificate monitoring if `USE_FOREIGN_CERT_MANAGER=true`. If certificates change, services will be automatically restarted. (default: `false`)
- `CERTS_MONITORING_TIMEOUT` Interval in seconds to check if certificates have changed. (default: `'60'`)
You can also use any variables from the [ini configuration file](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/ini/chatmail.ini.f); they must be in uppercase.
Mandatory variables for deployment via Docker:
- `CHANGE_KERNEL_SETTINGS` Change kernel settings (`fs.inotify.max_user_instances` and `fs.inotify.max_user_watches`) on startup. Changing kernel settings inside the container is not possible! (default: `False`)
6. Build the Docker image:
```shell
docker compose build chatmail
```
7. Start docker compose and wait for the installation to finish:
```shell
docker compose up -d # start service
docker compose logs -f chatmail # view container logs, press CTRL+C to exit
```
8. After installation is complete, you can open `https://<your_domain_name>` in your browser.
## Using custom files
When using Docker, you can apply modified configuration files to make the installation more personalized. This is usually needed for the `www/src` section so that the Chatmail landing page is customized to your taste, but it can be used for any other cases as well.
To replace files correctly:
1. Create the `./custom` directory. It is in `.gitignore`, so it wont cause conflicts when updating.
```shell
mkdir -p ./custom
```
2. Modify the required file. For example, `index.md`:
```shell
mkdir -p ./custom/www/src
nano ./custom/www/src/index.md
```
3. In `docker-compose.yaml`, add the file mount in the `volumes` section:
```yaml
services:
chatmail:
volumes:
...
## custom resources
- ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
```
4. Restart the service:
```shell
docker compose down
docker compose up -d
```
## Locking the Chatmail version
> [!note]
> These steps are optional and should only be done if you are not satisfied that the service is installed each time the container starts.
Since the current Docker version installs the Chatmail service every time the container starts, you can lock the container version after installation as follows:
1. Commit the current state of the configured container:
```shell
docker container commit chatmail configured-chatmail:$(date +'%Y-%m-%d')
docker image ls | grep configured-chatmail
```
2. Change the entrypoint for the container in `docker-compose.yaml` to:
```yaml
services:
chatmail:
image: <image name from step 1>
volumes:
...
## custom resources
- ./custom/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
```
3. Create the file `./custom/setup_chatmail_docker.sh` with the new configuration:
```shell
mkdir -p ./custom
cat > ./custom/setup_chatmail_docker.sh << 'EOF'
#!/bin/bash
set -eo pipefail
export ENABLE_CERTS_MONITORING="${ENABLE_CERTS_MONITORING:-true}"
export CERTS_MONITORING_TIMEOUT="${CERTS_MONITORING_TIMEOUT:-60}"
export PATH_TO_SSL="${PATH_TO_SSL:-/var/lib/acme/live/${MAIL_DOMAIN}}"
calculate_hash() {
find "$PATH_TO_SSL" -type f -exec sha1sum {} \; | sort | sha1sum | awk '{print $1}'
}
monitor_certificates() {
if [ "$ENABLE_CERTS_MONITORING" != "true" ]; then
echo "Certs monitoring disabled."
exit 0
fi
current_hash=$(calculate_hash)
previous_hash=$current_hash
while true; do
current_hash=$(calculate_hash)
if [[ "$current_hash" != "$previous_hash" ]]; then
# TODO: add an option to restart at a specific time interval
echo "[INFO] Certificate's folder hash was changed, reloading nginx, dovecot and postfix services."
systemctl reload nginx.service
systemctl reload dovecot.service
systemctl reload postfix.service
previous_hash=$current_hash
fi
sleep $CERTS_MONITORING_TIMEOUT
done
}
monitor_certificates &
EOF
```
4. Restart the service:
```shell
docker compose down
docker compose up -d
```

View File

@@ -1,177 +0,0 @@
# Известные проблемы и ограничения
- Chatmail будет переустановлен при каждом запуске контейнера (при первом - долго, при последующих быстрее). Так устроен изначальный установщик, потому что он не был заточен под docker. В конце документации [представлено](#фиксирование-версии-chatmail) возможное решение
- Требуется настроенный в системе cgroups v2. Работа с cgroups v1 не тестировалась.
- Да, понятно дело что systemd использовать в контейнере костыль и надо это всё разнести на несколько сервисов, но это MVP и в первом приближении оказалось сделать проще так, чем переписывать всю систему развертывания.
- docker образ подходит только для amd64, если нужно запустить на другой архитектуре, попробуйте изменить dockerfile (конкретно ту часть что ответсвенна за установку dovecot)
# Docker installation
Здесь представлена инструкция по установке chatmail с помощью docker-compose.
## Предварительная настройка
We use `chat.example.org` as the chatmail domain in the following steps.
Please substitute it with your own domain.
1. Настройте начальные записи DNS.Ниже приведен пример в привычном формате файла зоны BIND сTTL 1 час (3600 секунд).
Замените домен и IP-адреса на свои.
```
chat.example.com. 3600 IN A 198.51.100.5
chat.example.com. 3600 IN AAAA 2001:db8::5
www.chat.example.com. 3600 IN CNAME chat.example.com.
mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.
```
2. Склонируйте репозиторий на свой сервер.
```shell
git clone https://github.com/chatmail/relay
cd relay
```
## Installation
1. Скопировать файл `./docker/docker-compose-default.yaml` в `docker-compose.yaml`. Это нужно потому что `docker-compose.yaml` находится в `.gitignore` и не будет создавать конфликты при обновлении гит репозитория.
```shell
cp ./docker/docker-compose-default.yaml docker-compose.yaml
```
3. Настроить переменные окружения в `.env` файле. Эти переменные используются в `docker-compose.yaml` файле, чтобы передавать повторяющиеся значения.
4. Настроить параметры ядра, потому что внутри контейнера их нельзя изменить, а конкретно `fs.inotify.max_user_instances` и `fs.inotify.max_user_watches`. Для этого выполнить следующее:
```shell
echo "fs.inotify.max_user_instances=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
echo "fs.inotify.max_user_watches=65536" | sudo tee -a /etc/sysctl.d/99-inotify.conf
sudo sysctl --system
```
5. Настроить переменные окружения контейнера. Ниже перечислен список переменных учавствующих при развертывании.
- `MAIL_DOMAIN` - Доменное имя будущего сервера. (required)
- `DEBUG_COMMANDS_ENABLED` - Выполнить debug команды перед установкой. (default: `false`)
- `FORCE_REINIT_INI_FILE` - Пересоздавать ini файл конфигурации при запуске. (default: `false`)
- `USE_FOREIGN_CERT_MANAGER` - Использовать сторонний менеджер сертификатов. (default: `false`)
- `RECREATE_VENV` - Пересоздать виртуальное окружение (venv). Если выставлено `true`, то окружение будет пересоздано при запуске контейнера, из-за чего включение сервиса займет больше времени, но поможет избежать ряда ошибок. (default: `false`)
- `INI_FILE` - путь к ini файлу конфигурации. (default: `./chatmail.ini`)
- `PATH_TO_SSL` - Путь где располагаются сертификаты. (default: `/var/lib/acme/live/${MAIL_DOMAIN}`)
- `ENABLE_CERTS_MONITORING` - Включить мониторинг сертификатов, если `USE_FOREIGN_CERT_MANAGER=true`. Если сертфикаты изменятся сервисы будут автоматически перезапущены. (default: `false`)
- `CERTS_MONITORING_TIMEOUT` - Раз во сколько секунд проверять что изменились сертификаты. (default: `'60'`)
Также могут быть использованы все переменные из [ini файла конфигурации](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/ini/chatmail.ini.f), они обязаны быть в uppercase формате.
Ниже перечислены переменные, которые обязательны быть выставлены при развертывании через docker:
- `CHANGE_KERNEL_SETTINGS` - Менять настройки ядра (`fs.inotify.max_user_instances` и `fs.inotify.max_user_watches`) при запуске. При запуске в контейнере смена настроек ядра не может быть выполнена! (default: `False`)
6. Собрать docker образ
```shell
docker compose build chatmail
```
7. Запустить docker compose и дождаться завершения установки
```shell
docker compose up -d # запуск сервиса
docker compose logs -f chatmail # просмотр логов контейнера. Для выхода нажать CTRL+C
```
8. По окончанию установки можно открыть в браузер `https://<your_domain_name>`
## Использование кастомных файлов
При использовании docker есть возможность использовать измененые файлы конфигурации, чтобы сделать установку более персонализированной. Обычно это требуется для секции `www/src`, чтобы ознакомительная страница Chatmail была сделана на ваш вкус. Но также это можно использовать и для любых других случаев.
Для того чтобы корректно выполнить подмену файлов необходимо
1. создать каталог `./custom`, он находится в `.gitignore`, поэтому при обновлении не вызовет конфликтов.
```shell
mkdir -p ./custom
```
2. Изменить нужный файл. Для примера возьмем `index.md`
```shell
mkdir -p ./custom/www/src
nano ./custom/www/src/index.md
```
3. В `docker-compose.yaml` добавить монтирование файла с помощью секции `volumes`
```yaml
services:
chatmail:
volumes:
...
## custom resources
- ./custom/www/src/index.md:/opt/chatmail/www/src/index.md
```
4. Перезапустить сервис
```shell
docker compose down
docker compose up -d
```
## Фиксирование версии Chatmail
> [!note]
> Это опциональные шаги, их делать требуется только если вас не устраивает что сервис устанавливается каждый раз при запуске
Поскольку в текущей версии docker chatmail сервис устанавливается каждый раз запуске контейнера, чтобы этого не происходило можно зафиксировать версию контейнера после установки. Делается это следующим образом:
1. Зафиксировать текущее состояние сконфигурированного контейнера
```shell
docker container commit chatmail configured-chatmail:$(date +'%Y-%m-%d')
docker image ls | grep configured-chatmail
```
2. Изменить entrypoint для контейнера в `docker-compose.yaml` на
```yaml
services:
chatmail:
image: <image name from step 1>
volumes:
...
## custom resources
- ./custom/setup_chatmail_docker.sh:/setup_chatmail_docker.sh
```
3. Создать файл `./custom/setup_chatmail_docker.sh` с новым файлом конфигурации
```shell
mkdir -p ./custom
cat > ./custom/setup_chatmail_docker.sh << 'EOF'
#!/bin/bash
set -eo pipefail
export ENABLE_CERTS_MONITORING="${ENABLE_CERTS_MONITORING:-true}"
export CERTS_MONITORING_TIMEOUT="${CERTS_MONITORING_TIMEOUT:-60}"
export PATH_TO_SSL="${PATH_TO_SSL:-/var/lib/acme/live/${MAIL_DOMAIN}}"
calculate_hash() {
find "$PATH_TO_SSL" -type f -exec sha1sum {} \; | sort | sha1sum | awk '{print $1}'
}
monitor_certificates() {
if [ "$ENABLE_CERTS_MONITORING" != "true" ]; then
echo "Certs monitoring disabled."
exit 0
fi
current_hash=$(calculate_hash)
previous_hash=$current_hash
while true; do
current_hash=$(calculate_hash)
if [[ "$current_hash" != "$previous_hash" ]]; then
# TODO: add an option to restart at a specific time interval
echo "[INFO] Certificate's folder hash was changed, reloading nginx, dovecot and postfix services."
systemctl reload nginx.service
systemctl reload dovecot.service
systemctl reload postfix.service
previous_hash=$current_hash
fi
sleep $CERTS_MONITORING_TIMEOUT
done
}
monitor_certificates &
EOF
```
4. Перезапустить сервис
```shell
docker compose down
docker compose up -d
```

View File

@@ -9,11 +9,6 @@ if command -v lsb_release 2>&1 >/dev/null; then
echo "You need to install python3-dev for installing the other dependencies."
exit 1
fi
if ! gcc --version 2>&1 >/dev/null
then
echo "You need to install gcc for building Python dependencies."
exit 1
fi
;;
esac
fi