Files
Slavi PantaleevandClaude Opus 5 85f80a3c7e Test the Molecule scenarios against Postgres rather than sqlite
`group_vars/matrix_servers` selects postgres whenever postgres is enabled, which
is the default, so postgres is what essentially every deployment runs. The
scenarios were testing sqlite - a path almost nobody is on.

How little that path is used is not a guess: the mautrix-meta bridges could not
start at all under sqlite, and nobody reported it. Testing the engine users are
actually on is worth more than keeping coverage of the one they are not, so no
scenario is left behind on sqlite.

Four of the eight scenarios have a database and are converted; the other four
have none and are untouched.

molecule-shared/tasks/postgres.yml stands Postgres up on the scenario's network,
with the data directory on a tmpfs since it is thrown away with the container.
The image is pinned at the major the postgres role deploys to new installations
and left to Renovate: when a new major lands, the PR bumping that pin runs every
scenario against it, which is the earliest warning we get that a component does
not cope.

Each scenario gives its database and user names that differ from the role's
defaults, so the component reaching the database proves the role built its
connection string out of them. The assertions moved from "a file appeared at the
path we configured" to "these tables exist", which is strictly stronger: tables
can only appear once the component has resolved the hostname, authenticated with
the credentials the role rendered, and run its migrations to completion.

Costs about 10 seconds per affected scenario (115s to 125s locally for
mautrix-whatsapp), on jobs that run in parallel.

Gotcha worth recording: since Postgres 18 the image puts PGDATA in a versioned
subdirectory and refuses to start if it finds a mount at the old
/var/lib/postgresql/data, so the tmpfs is mounted at /var/lib/postgresql.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SEH3vxYSQ5SV4N5z61eyGT
2026-08-27 18:02:53 +03:00

226 lines
12 KiB
YAML

# SPDX-FileCopyrightText: 2026 Slavi Pantaleev
#
# SPDX-License-Identifier: AGPL-3.0-or-later
---
# Proves matrix-reminder-bot starts on the configuration the role rendered, logs in as the
# user the role gave it, opens the database at the path the role gave it, and is the version
# the role pins.
#
# The bot has no HTTP surface to probe, so the evidence is what it says about itself in the
# journal plus what it left on disk. It does NOT set real reminders. See docs/molecule-testing.md.
- name: Verify matrix-reminder-bot
hosts: all
become: true
vars_files:
- "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/vars.yml"
- "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/playbook-context.yml"
gather_facts: false
vars:
matrix_bot_matrix_reminder_bot_molecule_user_id: "@{{ matrix_bot_matrix_reminder_bot_matrix_user_id_localpart }}:{{ matrix_domain }}"
matrix_bot_matrix_reminder_bot_molecule_container_user: "{{ matrix_user_uid }}:{{ matrix_user_gid }}"
tasks:
# Read from the role's own defaults rather than pinned in molecule.yml, so the version
# assertion compares the running image against what defaults/main.yml ships.
# Pinning it here would make that assertion compare the scenario with itself.
- name: Load the role's defaults under a separate name
ansible.builtin.include_vars:
file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/defaults/main.yml"
name: matrix_bot_matrix_reminder_bot_role_defaults
- name: Wait for the matrix-reminder-bot service to become active
ansible.builtin.systemd_service:
name: matrix-bot-matrix-reminder-bot.service
register: matrix_bot_matrix_reminder_bot_service
until: matrix_bot_matrix_reminder_bot_service.status.ActiveState == 'active'
retries: 30
delay: 5
failed_when: false
# `Restart=always` means a bot crash-looping on unreadable config still reports `active`,
# so the restart counter is checked too. The config file is parsed before the bot's own
# catch-all retry loop starts, so anything wrong in what the role rendered shows up here
# as restarts. Asserted `is defined` because `| int` turns a missing property into 0.
- name: Assert the service is active and has not been restarting
ansible.builtin.assert:
that:
- matrix_bot_matrix_reminder_bot_service.status.ActiveState == 'active'
- matrix_bot_matrix_reminder_bot_service.status.NRestarts is defined
- matrix_bot_matrix_reminder_bot_service.status.NRestarts | int == 0
fail_msg: >-
matrix-bot-matrix-reminder-bot.service is
{{ matrix_bot_matrix_reminder_bot_service.status.ActiveState | default('unknown') }}
after {{ matrix_bot_matrix_reminder_bot_service.status.NRestarts | default('?') }}
automatic restart(s)
success_msg: "matrix-bot-matrix-reminder-bot.service is active and has not restarted"
# The unit runs `docker start --attach`, so the container's output is in the journal
# despite `--log-driver=none`. It is the only thing this bot reports about itself.
#
# Filtered rather than tailed: startup lines are the OLDEST in the journal, so a
# `--lines=N` tail loses them behind anything logged later, and reading it whole pulls
# unbounded text into a variable. The filter keeps the failure line too, so the
# "did not fail to log in" assertion below still has something to see.
- name: Wait for the bot to report that it finished starting up
ansible.builtin.shell:
cmd: >-
set -o pipefail;
journalctl --unit=matrix-bot-matrix-reminder-bot.service --no-pager --output=cat --lines=all
| grep -E 'Logged in as|Startup complete|Failed to login|Database initialization' | head -n 50 || true
executable: /bin/bash
register: matrix_bot_matrix_reminder_bot_journal
changed_when: false
until: "'Startup complete' in matrix_bot_matrix_reminder_bot_journal.stdout"
retries: 24
delay: 5
failed_when: false
# "Logged in as ..." is only reached once the login call returned something other than a
# LoginError, so this covers the whole chain at once: homeserver URL, user ID and password
# were all good enough for a real login round-trip.
- name: Assert the bot logged in as the user the role configured
ansible.builtin.assert:
that:
- "'Startup complete' in matrix_bot_matrix_reminder_bot_journal.stdout"
- "'Logged in as ' + matrix_bot_matrix_reminder_bot_molecule_user_id in matrix_bot_matrix_reminder_bot_journal.stdout"
- "'Failed to login' not in matrix_bot_matrix_reminder_bot_journal.stdout"
fail_msg: >-
The bot did not log in as {{ matrix_bot_matrix_reminder_bot_molecule_user_id }}
and reach startup
success_msg: >-
The bot logged in as {{ matrix_bot_matrix_reminder_bot_molecule_user_id }} and finished starting up
# The role picks the storage engine by building the connection string the bot parses,
# and the bot names the type it settled on once the database is open.
- name: Assert the bot opened the database engine the role selected
ansible.builtin.assert:
that:
- "\"Database initialization of type 'postgres' complete\" in matrix_bot_matrix_reminder_bot_journal.stdout"
fail_msg: "The bot did not report a completed Postgres database initialization"
success_msg: "The bot initialized the Postgres database the role pointed it at"
- name: Read the configuration file the role rendered
ansible.builtin.slurp:
src: "{{ matrix_bot_matrix_reminder_bot_config_path }}/config.yaml"
register: matrix_bot_matrix_reminder_bot_config_file
# Every one differs from both the role's defaults and the bot's own fallbacks, so their
# presence means the role rendered this file rather than coinciding with it.
- name: Assert the rendered configuration carries this scenario's values
ansible.builtin.assert:
that:
- matrix_bot_matrix_reminder_bot_molecule_user_id in matrix_bot_matrix_reminder_bot_config_rendered
- matrix_bot_matrix_reminder_bot_matrix_user_password in matrix_bot_matrix_reminder_bot_config_rendered
- matrix_bot_matrix_reminder_bot_matrix_homeserver_url in matrix_bot_matrix_reminder_bot_config_rendered
- matrix_bot_matrix_reminder_bot_reminders_timezone in matrix_bot_matrix_reminder_bot_config_rendered
- matrix_bot_matrix_reminder_bot_command_prefix in matrix_bot_matrix_reminder_bot_config_rendered
- matrix_bot_matrix_reminder_bot_database_username in matrix_bot_matrix_reminder_bot_config_rendered
- matrix_bot_matrix_reminder_bot_database_name in matrix_bot_matrix_reminder_bot_config_rendered
- matrix_bot_matrix_reminder_bot_database_hostname in matrix_bot_matrix_reminder_bot_config_rendered
- "'@molecule-allowed:molecule.local' in matrix_bot_matrix_reminder_bot_config_rendered"
- "'.*:blocked.molecule.local' in matrix_bot_matrix_reminder_bot_config_rendered"
fail_msg: "The rendered configuration does not carry the scenario's settings"
success_msg: "The rendered configuration carries the scenario's settings"
vars:
matrix_bot_matrix_reminder_bot_config_rendered: "{{ matrix_bot_matrix_reminder_bot_config_file.content | b64decode }}"
# `device_name` is hardcoded in the role's template, so this value can only be here if
# `..._configuration_extension_yaml` was merged over it rather than ignored.
- name: Assert the configuration extension was merged over the template
ansible.builtin.assert:
that:
- "'device_name: Molecule Reminder Bot' in matrix_bot_matrix_reminder_bot_config_rendered"
- "'device_name: Reminder Bot' not in matrix_bot_matrix_reminder_bot_config_rendered"
fail_msg: >-
The configuration extension did not override the device name the
role's template hardcodes
success_msg: "The configuration extension was merged over the role's template"
vars:
matrix_bot_matrix_reminder_bot_config_rendered: "{{ matrix_bot_matrix_reminder_bot_config_file.content | b64decode }}"
# With no HTTP surface, the schema in Postgres is the evidence that the storage
# configuration reached the running process rather than only the file on disk. The bot can
# only have created tables by resolving the hostname, authenticating with the credentials
# the role rendered, and running its migrations.
- name: List the tables the bot created in Postgres
ansible.builtin.command:
argv:
- docker
- exec
- matrix-postgres-molecule
- psql
- --username={{ matrix_bot_matrix_reminder_bot_database_username }}
- --dbname={{ matrix_bot_matrix_reminder_bot_database_name }}
- --tuples-only
- --no-align
- --command=SELECT tablename FROM pg_tables WHERE schemaname = 'public'
register: matrix_bot_matrix_reminder_bot_tables
changed_when: false
- name: Assert the bot created its schema in the database the role pointed it at
ansible.builtin.assert:
that:
- matrix_bot_matrix_reminder_bot_tables.rc == 0
- matrix_bot_matrix_reminder_bot_table_names | length > 0
fail_msg: >-
The bot created no tables in {{ matrix_bot_matrix_reminder_bot_database_name }}
success_msg: "The bot created its schema in the database the role pointed it at"
vars:
matrix_bot_matrix_reminder_bot_table_names: "{{ matrix_bot_matrix_reminder_bot_tables.stdout_lines | select | list }}"
# matrix-nio writes its encryption store here once login succeeds, so a populated directory
# means the bot could use the store path the role created inside a read-only container.
- name: List the encryption store the role created
ansible.builtin.find:
paths: "{{ matrix_bot_matrix_reminder_bot_data_store_path }}"
file_type: file
register: matrix_bot_matrix_reminder_bot_store_files
- name: Assert the bot wrote its encryption store where the role put it
ansible.builtin.assert:
that:
- matrix_bot_matrix_reminder_bot_store_files.matched | int > 0
fail_msg: >-
{{ matrix_bot_matrix_reminder_bot_data_store_path }} is empty, so the bot never
got far enough to open its encryption store
success_msg: "The bot wrote its encryption store under the role's data path"
- name: Read the running container's user and environment
ansible.builtin.command:
argv:
- docker
- container
- inspect
- matrix-bot-matrix-reminder-bot
- --format
- "{{ '{{' }} .Config.User {{ '}}' }} {{ '{{' }} json .Config.Env {{ '}}' }} {{ '{{' }} .Config.Image {{ '}}' }}"
register: matrix_bot_matrix_reminder_bot_container
changed_when: false
# The timezone reaches the container twice, through the config file checked above and
# through TZ on the unit. The uid/gid come from the playbook context, not from the image.
- name: Assert the container runs as the role's user with the configured timezone
ansible.builtin.assert:
that:
- "'\"TZ=' + matrix_bot_matrix_reminder_bot_reminders_timezone + '\"' in matrix_bot_matrix_reminder_bot_container.stdout"
- matrix_bot_matrix_reminder_bot_molecule_container_user in matrix_bot_matrix_reminder_bot_container.stdout
fail_msg: >-
The container does not run as {{ matrix_user_uid }}:{{ matrix_user_gid }} with
TZ={{ matrix_bot_matrix_reminder_bot_reminders_timezone }}
({{ matrix_bot_matrix_reminder_bot_container.stdout }})
success_msg: >-
The container runs as {{ matrix_user_uid }}:{{ matrix_user_gid }} with
TZ={{ matrix_bot_matrix_reminder_bot_reminders_timezone }}
- name: Assert the running container is the version defaults/main.yml pins
ansible.builtin.assert:
that:
- matrix_bot_matrix_reminder_bot_role_defaults.matrix_bot_matrix_reminder_bot_version | string in matrix_bot_matrix_reminder_bot_container.stdout
fail_msg: >-
The running container is {{ matrix_bot_matrix_reminder_bot_container.stdout }}, which
does not carry the pinned version
{{ matrix_bot_matrix_reminder_bot_role_defaults.matrix_bot_matrix_reminder_bot_version }}
success_msg: "The running container is the version defaults/main.yml pins"