From 41650478bc3d79a14fc07770238719260ac9d7bd Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Sat, 10 Oct 2026 15:44:55 +0300 Subject: [PATCH] Add the system cleanup role Adds ansible-role-cleanup from the MASH project. It can remove superseded Docker images and stopped containers on every playbook run (unlike `docker image prune -a`, keeping the images of stopped services), vacuum old logs, remove paths and, on Debian-based distributions, upgrade packages and purge old kernels. Everything is off by default. The role runs after systemd_service_manager, once every service is running, so the Docker cleanup can tell which images are still needed. Co-Authored-By: Claude Opus 5.5 (1M context) --- CHANGELOG.md | 8 +++ README.md | 1 + docs/configuring-playbook-system-cleanup.md | 58 +++++++++++++++++++++ docs/configuring-playbook.md | 2 + docs/maintenance-and-troubleshooting.md | 2 + group_vars/matrix_servers | 19 +++++++ requirements.yml | 3 ++ setup.yml | 5 ++ 8 files changed, 98 insertions(+) create mode 100644 docs/configuring-playbook-system-cleanup.md diff --git a/CHANGELOG.md b/CHANGELOG.md index f8cbafc5d..b02273a88 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,11 @@ +# 2026-10-10 + +## Optional system cleanup, including safe removal of superseded Docker images + +The playbook now includes the [system cleanup](./docs/configuring-playbook-system-cleanup.md) role from [the MASH project](https://github.com/mother-of-all-self-hosting/ansible-role-cleanup). It can remove superseded Docker images and stopped containers on every playbook run, keeping the images that services you have stopped still need (unlike `docker image prune -a`, which `just run-tags run-docker-prune` uses). It can also vacuum old logs, remove arbitrary paths and, on Debian-based distributions, upgrade packages and purge old kernels. + +Everything is off by default. To enable the Docker cleanup, add `system_cleanup_docker: true` to your `vars.yml` file and re-run the playbook (`just setup-all`). See the [documentation](./docs/configuring-playbook-system-cleanup.md) for the other options. + # 2026-09-22 ## LiveKit's TURN over TLS works for clients that advertise an ALPN protocol diff --git a/README.md b/README.md index 2f8cc1a20..b08d7f7a7 100644 --- a/README.md +++ b/README.md @@ -165,6 +165,7 @@ Services that help you in administrating and monitoring your Matrix installation | [Ketesa](https://github.com/etkecc/ketesa) | ❌ | Fully-featured web UI for administrating your Matrix homeserver — users, rooms, media, sessions, and more | [Link](docs/configuring-playbook-ketesa.md) | | Metrics and Graphs | ❌ | Consists of the [Prometheus](https://prometheus.io) time-series database server, the Prometheus [node-exporter](https://prometheus.io/docs/guides/node-exporter/) host metrics exporter, and the [Grafana](https://grafana.com/) web UI, with [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) being available too | [Link](docs/configuring-playbook-prometheus-grafana.md) (for [prometheus-nginxlog-exporter](docs/configuring-playbook-prometheus-grafana.md#enable-metrics-and-graphs-for-nginx-logs-optional)) | | [Borg](https://borgbackup.org) | ❌ | Backups | [Link](docs/configuring-playbook-backup-borg.md) | +| [System cleanup](https://github.com/mother-of-all-self-hosting/ansible-role-cleanup) | ❌ | Removes superseded Docker images, old logs, etc. | [Link](docs/configuring-playbook-system-cleanup.md) | | [rageshake](https://github.com/matrix-org/rageshake) | ❌ | Bug report server | [Link](docs/configuring-playbook-rageshake.md) | | [synapse-usage-exporter](https://github.com/loelkes/synapse-usage-exporter) | ❌ | Export the usage statistics of a Synapse homeserver to be scraped by Prometheus. | [Link](docs/configuring-playbook-synapse-usage-exporter.md) | diff --git a/docs/configuring-playbook-system-cleanup.md b/docs/configuring-playbook-system-cleanup.md new file mode 100644 index 000000000..a1f5231e1 --- /dev/null +++ b/docs/configuring-playbook-system-cleanup.md @@ -0,0 +1,58 @@ + + +# Setting up system cleanup (optional) + +The playbook can do some housekeeping on the server: removing Docker leftovers, old logs and arbitrary paths, and (on Debian-based distributions) upgrading packages and purging old kernels. Everything is off by default. + +The [Ansible role for system cleanup](https://github.com/mother-of-all-self-hosting/ansible-role-cleanup) is developed and maintained by [the MASH (mother-of-all-self-hosting) project](https://github.com/mother-of-all-self-hosting). For all options, see the role's [`defaults/main.yml`](https://github.com/mother-of-all-self-hosting/ansible-role-cleanup/blob/main/defaults/main.yml). + +## Adjusting the playbook configuration + +To enable the cleanup tasks you'd like, add the following configuration to your `inventory/host_vars/matrix.example.com/vars.yml` file: + +```yaml +# Removes stopped containers and Docker images which a newer image of the same repository superseded +# at least 3 days ago (and which no container uses). Unlike `docker image prune -a`, unused images +# without a newer version (e.g. of services you have stopped) are kept. +system_cleanup_docker: true + +# Installs a systemd timer which runs `journalctl --vacuum-time=7d` daily. +# journald already limits the space logs take up on its own, so this is only needed for keeping logs for a shorter time. +system_cleanup_logs: true + +# Absolute paths to remove on each run +system_cleanup_paths: [] + +# The following options only have an effect on Debian-based distributions. + +# Runs a safe upgrade, `apt-get autoremove`, `apt-get clean`, etc. +system_cleanup_apt: true + +# WARNING: very dangerous! Purges old Linux kernels, and their modules +system_cleanup_kernels: false +``` + +The Docker cleanup runs when the playbook is run with the `start` tag (e.g. `just install-all` or `just setup-all`), after all services have been started. To also run it daily via a systemd timer (useful for removing superseded images without re-running the playbook), add `system_cleanup_docker_timer_enabled: true`. + +## Installing + +After configuring the playbook, run it with [playbook tags](playbook-tags.md) as below: + + +```sh +ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start +``` + +The shortcut commands with the [`just` program](just.md) are also available: `just setup-all` + +## Usage + +The Docker cleanup is done by a script, which you can also run by hand on the server. Pass `--dry-run` to only see what it would remove: + +```sh +/matrix/system-cleanup/bin/cleanup-docker --dry-run +``` diff --git a/docs/configuring-playbook.md b/docs/configuring-playbook.md index 859f4fd5d..35bd3858a 100644 --- a/docs/configuring-playbook.md +++ b/docs/configuring-playbook.md @@ -224,6 +224,8 @@ Services that help you in administrating and monitoring your Matrix installation - [Enabling synapse-usage-exporter for Synapse usage statistics](configuring-playbook-synapse-usage-exporter.md) +- [Setting up system cleanup](configuring-playbook-system-cleanup.md) — removing superseded Docker images, old logs, etc. + - Backups: - [Setting up BorgBackup](configuring-playbook-backup-borg.md) — a full Matrix server backup solution, including the Postgres database diff --git a/docs/maintenance-and-troubleshooting.md b/docs/maintenance-and-troubleshooting.md index bd3e4b84c..01e667be4 100644 --- a/docs/maintenance-and-troubleshooting.md +++ b/docs/maintenance-and-troubleshooting.md @@ -38,6 +38,8 @@ ansible-playbook -i inventory/hosts setup.yml --tags=run-docker-prune The shortcut command with `just` program is also available: `just run-tags run-docker-prune` +Note that this also removes the images of services you have stopped, which then need to be pulled again. To have the playbook remove only the Docker images which newer ones have superseded, on every run, see [Setting up system cleanup](configuring-playbook-system-cleanup.md). + ### Postgres See the dedicated [PostgreSQL maintenance](maintenance-postgres.md) documentation page. diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index a35865bc3..06c88c56d 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -967,6 +967,25 @@ devture_systemd_service_manager_services_list_auto: | ######################################################################## +######################################################################## +# # +# cleanup # +# # +######################################################################## + +# All cleanup tasks are off by default. See docs/configuring-playbook-system-cleanup.md + +system_cleanup_identifier: matrix-system-cleanup + +system_cleanup_base_path: "{{ matrix_base_data_path }}/system-cleanup" + +######################################################################## +# # +# /cleanup # +# # +######################################################################## + + ######################################################################## # # # com.devture.ansible.role.timesync # diff --git a/requirements.yml b/requirements.yml index 29bb05022..14e5f6725 100644 --- a/requirements.yml +++ b/requirements.yml @@ -9,6 +9,9 @@ - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-cinny.git version: v4.12.7-1 name: cinny +- src: git+https://github.com/mother-of-all-self-hosting/ansible-role-cleanup.git + version: v2.0.0-1 + name: cleanup - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-container-socket-proxy.git version: v0.5.0-2 name: container_socket_proxy diff --git a/setup.yml b/setup.yml index 09667ad94..219a2e87b 100644 --- a/setup.yml +++ b/setup.yml @@ -151,6 +151,11 @@ - when: devture_systemd_service_manager_enabled | bool role: galaxy/systemd_service_manager + # This role needs to run after the services have been (re)started. + # By then, every image the services need has been pulled and is in use by a running container, + # so the Docker cleanup can tell which images are still needed and which have been superseded. + - role: galaxy/cleanup + # This is pretty much last, because we want it to better serve as a "last known good configuration". # See: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/2217#issuecomment-1301487601 - when: devture_playbook_state_preserver_enabled | bool