diff --git a/docs/stores/exec.md b/docs/stores/exec.md index 7533d1a..138d89c 100644 --- a/docs/stores/exec.md +++ b/docs/stores/exec.md @@ -1,60 +1,470 @@ # Exec Identity Store -This Identity Store lets you run arbitrary commands to handle the various requests in each support feature. - -This is the most versatile Identity store of mxisd, allowing you to connect any kind of logic in any language/scripting. +- [Features](#features) +- [Overview](#overview) +- [Configuration](#configuration) + - [Global](#global) + - [Tokens](#tokens) + - [Executable](#executable) + - [Input](#input) + - [Output](#output) + - [Examples](#examples) + - [Per-Feature](#per-feature) +- [Authentication](#authentication) + - [Tokens](#tokens-1) + - [Input](#input-1) + - [Output](#output-1) +- [Directory](#directory) + - [Tokens](#tokens-2) + - [Input](#input-2) + - [Output](#output-2) +- [Identity](#identity) + - [Single Lookup](#single-lookup) + - [Tokens](#tokens-3) + - [Input](#input-3) + - [Output](#output-3) + - [Bulk Lookup](#bulk-lookup) + - [Tokens](#tokens-4) + - [Input](#input-4) + - [Output](#output-4) +- [Profile](#profile) + - [Tokens](#tokens-5) + - [Input](#input-5) + - [Output](#output-5) + +--- ## Features -| Name | Supported? | -|----------------|---------------| -| Authentication | Yes | -| Directory | *In Progress* | -| Identity | *In Progress* | -| Profile | *In Progress* | +| Name | Supported | +|-------------------------------------------------|-----------| +| [Authentication](../features/authentication.md) | Yes | +| [Directory](../features/directory.md) | Yes | +| [Identity](../features/identity.md) | Yes | +| [Profile](#profile) | Yes | + +This Identity Store lets you run arbitrary commands to handle the various requests in each support feature. +It is the most versatile Identity store of mxisd, allowing you to connect any kind of logic with any executable/script. ## Overview Each request can be mapping to a fully customizable command configuration. The various parameters can be provided via any combination of: -- Standard Input -- Command line arguments -- Environment variables +- [Standard Input](https://en.wikipedia.org/wiki/Standard_streams#Standard_input_(stdin)) +- [Command-line arguments](https://en.wikipedia.org/wiki/Command-line_interface#Arguments) +- [Environment variables](https://en.wikipedia.org/wiki/Environment_variable) Each of those supports a set of customizable token which will be replaced prior to running the command, allowing to provide the input values in any number of ways. -Success and data will be provided via [Exit status](https://en.wikipedia.org/wiki/Exit_status) and Standard Output, both -supporting a set of options. +Success and data will be provided via any combination of: +- [Exit status](https://en.wikipedia.org/wiki/Exit_status) +- [Standard Output](https://en.wikipedia.org/wiki/Standard_streams#Standard_output_(stdout)) + +Each of those supports a set of configuration item to decide how to process the value and/or in which format. + +All values, inputs and outputs are UTF-8 encoded. ## Configuration +Each feature comes with a set of possible lookup/action which is mapped to a generic configuration item block. +We will use the term `Executable` for each lookup/action and `Processor` for each configuration block. + +### Global ```yaml exec.enabled: ``` -Enable/disable the Identity store at a global/default level. Each feature can still be enabled/disabled specifically. +Enable/disable the Identity store at a global/default level. Each feature can still be individually enabled/disabled. -*TBC* +#### Tokens +The following options allow to globally set tokens for value replacement across all features and processors config. +Not all features use all tokens, and each feature might also have its own specific tokens. See each feature documentation. + +They can be set within the following scope: -## Use-case examples +```yaml +exec.token.: '' +``` + +--- + +The following tokens and default values are available: +```yaml +localpart: '{localpart}' +``` +Localpart of Matrix User IDs + +```yaml +domain: '{domain}' +``` +Domain of Matrix User IDs + +```yaml +mxid: '{mxid}' +``` +Full representation of Matrix User IDs + +```yaml +medium: '{medium}' +``` +Medium of 3PIDs + +```yaml +address: '{address}' +``` +Address of 3PIDs + +```yaml +type: '{type}' +``` +Type of query + +```yaml +query: '{query}' +``` +Query value + +### Executable +*Executable*s have the following options: +```yaml +command: '/path/to/executableOrScript' + +``` +Set the executable (relative or absolute) path to be executed. If no command is given, the action will return a "neutral" +result if possible or be skipped altogether. + +--- + +Command line arguments can be given via a list via both YAML formats: +```yaml +args: + - '-t' + - '{token}' + - '-v' + - 'value' +``` +or +```yaml +args: ['-t', '{token}', '-v', 'value] +``` +Each argument will be processed for token replacement. + +--- + +Environment variables can be given as key/value pairs: +```yaml +env: + ENV_VAR_1: 'value' + ENV_VAR_2: '{token}' +``` +Each variable value will be processed for token replacement. + +#### Input +Standard input can be configured in the namespaces `input` with: +- `type`: The format to use +- `template`: The full or partial template with tokens to be used when generating the input + +Not all features and *Executable*s allow for a template to be provided. +Templates for listed-based input are not supported at this time. +Default templates may be provided per *Executable*. + +The following types are available: +- `json`: Use JSON format, shared with the [REST Identity Store](rest.md) +- `plain`: Use a custom multi-lines, optionally tab-separated input + +#### Output +Standard output can be configured in the namespaces `output` with: +- `type`: The format to use +- `template`: The full or partial template with tokens to be used when processing the output + +Not all features and *Executable*s allow for a template to be provided. +Templates for listed-based output are not supported at this time. +Default templates may be provided per *Executable*. + +The following types are available: +- `json`: Use JSON format, shared with the [REST Identity Store](rest.md) +- `plain`: Use a custom multi-lines, optionally tab-separated output + +### Examples +#### Basic +```yaml +exec.auth.enabled: true +exec.auth.command: '/opt/mxisd-exec/auth.sh' +exec.auth.args: ['{localpart}'] +exec.auth.input.type: 'plain' +exec.auth.input.template: '{password}' +exec.auth.env: + DOMAIN: '{domain}' +``` +With Authentication enabled, run `/opt/mxisd-exec/auth.sh` when validating credentials, providing: +- A single command-line argument to provide the `localoart` as username +- A plain text string with the password token for standard input, which will be replaced by the password to check +- A single environment variable `DOMAIN` containing Matrix ID domain, if given + +The command will use the default values for: +- Success exit status of `0` +- Failure exit status of `1` +- Any other exit status considered as error +- The standard output processing as not processed + +#### Advanced +Given the fictional `placeholder` feature: ```yaml exec.enabled: true +exec.token.mxid: '{matrixId}' -exec.auth.command: '/path/to/auth/executable' -exec.auth.args: ['-u', '{localpart}'] -exec.auth.env: - PASSWORD: '{password}' +exec.placeholder.token.localpart: '{username}' +exec.placeholder.command: '/path/to/executable' +exec.placeholder.args: + - '-u' + - '{username}' +exec.placeholder.env: MATRIX_DOMAIN: '{domain}' - MATRIX_USER_ID: '{mxid}' + MATRIX_USER_ID: '{matrixId}' + +exec.placeholder.output.type: 'json' +exec.placeholder.exit.success: [0, 128] +exec.placeholder.exit.failure: [1, 129] ``` -This will run `/path/to/auth/executable` with: -- The extracted Matrix User ID `localpart` provided as the second command line argument, the first one being `-u` -- The password, the extract Matrix `domain` and the full User ID as arbitrary environment variables, respectively `PASSWORD`, `MATRIX_DOMAIN` and `MATRIX_USER_ID` +With: +- The Identity store enabled for all features +- A global specific token `{matrixId}` for Matrix User IDs, replacing the default `{mxid}` +Running `/path/to/executable` providing: +- A custom token for localpart, `{username}`, used as a 2nd command-line argument +- An extracted Matrix User ID `localpart` provided as the second command line argument, the first one being `-u` +- A password, the extracted Matrix `domain` and the full User ID as arbitrary environment variables, respectively + `PASSWORD`, `MATRIX_DOMAIN` and `MATRIX_USER_ID` + +After execution: +- Process stdout as [JSON](https://en.wikipedia.org/wiki/JSON) +- Consider exit status `0` and `128` as success and try to process the stdout for data +- Consider exit status `1` and `129` as failure and try to process the stdout for error code and message + +### Per Feature +See each dedicated [Feature](#features) section. + +## Authentication +The Authentication feature can be enabled/disabled using: ```yaml -## Few more available config items -# -# exec.token.domain: '{matrixDomain}' # This sets the default replacement token for the Matrix Domain of the User ID, across all features. -# exec.auth.token.domain: '{matrixDomainForAuth}' # We can also set another token specific to a feature. -# exec.auth.input: 'json' # This is not supported yet. -# exec.auth.exit.success: [0] # Exit status that will consider the request successful. This is already the default. -# exec.auth.exit.failure: [1,2,3] # Exist status that will consider the request failed. Anything else than success or failure statuses will throw an exception. -# exec.auth.output: 'json' # Required if stdout should be read on success. This uses the same output as the REST Identity store for Auth. +exec.auth.enabled: ``` -*TBC* + +--- + +This feature provides a single *Executable* under the namespace: +```yaml +exec.auth: + ... +``` + +### Tokens +The following tokens/default values are specific to this feature: +```yaml +password: '{password}' +``` +The provided password + +### Input +Supported input types and default templates: + +#### JSON (`json`) +Same as the [REST Identity Store](rest.md); + +#### Plain (`plain`) +Default template: +``` +{localpart} +{domain} +{mxid} +{password} +``` + +### Output +Supported output types and default templates: + +#### JSON (`json`) +Same as the [REST Identity Store](rest.md); + +#### Plain (`plain`) +**NOTE:** This has limited support. Use the JSON type for full support. + +Default template: +``` +[success status, true or 1 are interpreted as success] +[display name of the user] +``` + +## Directory +The Directory feature can be enabled/disabled using: +```yaml +exec.directory.enabled: +``` + +--- + +Two search types configuration namespace are available, using the same input/output formats and templates: + +By name: +```yaml +exec.directory.search.byName: + ... +``` +By 3PID: +```yaml +exec.directory.search.byThreepid: + ... +``` + +#### Tokens +No specific tokens are available. + +#### Input +Supported input types and default templates: + +##### JSON (`json`) +Same as the [REST Identity Store](rest.md); + +##### Plain (`plain`) +Default template: +``` +[type of search, following the REST Identity store format] +[query string] +``` + +#### Output +Supported output types and default templates: + +##### JSON (`json`) +Same as the [REST Identity Store](rest.md); + +##### Plain (`plain`) +**Not supported at this time.** Use the JSON type. + +## Identity +The Identity feature can be enabled/disabled using: +```yaml +exec.identity.enabled: +``` + +### Single lookup +Configuration namespace: +```yaml +exec.identity.lookup.single: + ... +``` + +#### Tokens +No specific tokens are available. + +#### Input +Supported input types and default templates: + +##### JSON (`json`) +Same as the [REST Identity Store](rest.md); + +##### Plain (`plain`) +Default template: +``` +{medium} +{address} +``` + +#### Output +Supported output types and default templates: + +##### JSON (`json`) +Same as the [REST Identity Store](rest.md); + +##### Plain (`plain`) +Default template: +``` +[User ID type, as documented in the REST Identity Store] +[User ID value] +``` + +The User ID type will default to `localpart` if: +- Only one line is returned +- The first line is empty + +### Bulk lookup +Configuration namespace: +```yaml +exec.identity.lookup.bulk: + ... +``` + +#### Tokens +No specific tokens are available. + +#### Input +Supported input types and default templates: + +##### JSON (`json`) +**NOTE:** Custom Templates are not supported. + +Same as the [REST Identity Store](rest.md). + +##### Plain (`plain`) +**Not supported at this time.** Use the JSON type. + +#### Output +Supported output types and default templates: + +##### JSON (`json`) +**NOTE:** Custom Templates are not supported. + +Same as the [REST Identity Store](rest.md). + +##### Plain (`plain`) +**Not supported at this time.** Use the JSON type. + +## Profile +The Profile feature can be enabled/disabled using: +```yaml +exec.profile.enabled: +``` + +--- + +The following *Executable*s namespace are available, share the same input/output formats and templates: + +Get Display name: +```yaml +exec.profile.displayName: + ... +``` + +Get 3PIDs: +```yaml +exec.profile.threePid: + ... +``` + +Get Roles: +```yaml +exec.profile.role: + ... +``` + + +### Tokens +No specific tokens are available. + +### Input +Supported input types and default templates: + +#### JSON (`json`) +Same as the [REST Identity Store](rest.md); + +#### Plain (`plain`) +Default template: +``` +{localpart} +{domain} +{mxid} +``` +### Output +Supported output types and default templates: + +#### JSON (`json`) +Same as the [REST Identity Store](rest.md); + +#### Plain (`plain`) +**Not supported at this time.** Use the JSON type. diff --git a/src/main/java/io/kamax/mxisd/backend/exec/ExecAuthStore.java b/src/main/java/io/kamax/mxisd/backend/exec/ExecAuthStore.java index 997ee92..a920ab4 100644 --- a/src/main/java/io/kamax/mxisd/backend/exec/ExecAuthStore.java +++ b/src/main/java/io/kamax/mxisd/backend/exec/ExecAuthStore.java @@ -81,7 +81,7 @@ public class ExecAuthStore extends ExecStore implements AuthenticatorProvider { json.setPassword(tokens.getPassword()); return json; }); - p.addInputTemplate(MultilinesType, tokens -> tokens.getLocalpart() + System.lineSeparator() + + p.addInputTemplate(PlainType, tokens -> tokens.getLocalpart() + System.lineSeparator() + tokens.getDomain() + System.lineSeparator() + tokens.getMxid() + System.lineSeparator() + tokens.getPassword() + System.lineSeparator() @@ -102,7 +102,7 @@ public class ExecAuthStore extends ExecStore implements AuthenticatorProvider { return result; }); - p.addSuccessMapper(MultilinesType, output -> { + p.addSuccessMapper(PlainType, output -> { String[] lines = output.split("\\R"); if (lines.length > 2) { throw new InternalServerError("Exec auth command returned more than 2 lines (" + lines.length + ")"); diff --git a/src/main/java/io/kamax/mxisd/backend/exec/ExecDirectoryStore.java b/src/main/java/io/kamax/mxisd/backend/exec/ExecDirectoryStore.java index 4a0bb51..a036fe9 100644 --- a/src/main/java/io/kamax/mxisd/backend/exec/ExecDirectoryStore.java +++ b/src/main/java/io/kamax/mxisd/backend/exec/ExecDirectoryStore.java @@ -60,7 +60,7 @@ public class ExecDirectoryStore extends ExecStore implements IDirectoryProvider Processor p = new Processor<>(cfg); p.addJsonInputTemplate(tokens -> new UserDirectorySearchRequest(tokens.getType(), tokens.getQuery())); - p.addInputTemplate(MultilinesType, tokens -> tokens.getType() + System.lineSeparator() + tokens.getQuery()); + p.addInputTemplate(PlainType, tokens -> tokens.getType() + System.lineSeparator() + tokens.getQuery()); p.addTokenMapper(cfg.getToken().getType(), request::getBy); p.addTokenMapper(cfg.getToken().getQuery(), request::getSearchTerm); diff --git a/src/main/java/io/kamax/mxisd/backend/exec/ExecIdentityStore.java b/src/main/java/io/kamax/mxisd/backend/exec/ExecIdentityStore.java index 8415745..ea5a1ba 100644 --- a/src/main/java/io/kamax/mxisd/backend/exec/ExecIdentityStore.java +++ b/src/main/java/io/kamax/mxisd/backend/exec/ExecIdentityStore.java @@ -111,7 +111,7 @@ public class ExecIdentityStore extends ExecStore implements IThreePidProvider { p.addTokenMapper(getSingleCfg().getToken().getAddress(), request::getThreePid); p.addJsonInputTemplate(tokens -> new ThreePid(tokens.getMedium(), tokens.getAddress())); - p.addInputTemplate(MultilinesType, tokens -> tokens.getMedium() + p.addInputTemplate(PlainType, tokens -> tokens.getMedium() + System.lineSeparator() + tokens.getAddress() ); @@ -128,7 +128,7 @@ public class ExecIdentityStore extends ExecStore implements IThreePidProvider { .map(mxId -> new SingleLookupReply(request, mxId)); }); - p.addSuccessMapper(MultilinesType, output -> { + p.addSuccessMapper(PlainType, output -> { String[] lines = output.split("\\R"); if (lines.length > 2) { throw new InternalServerError("Exec auth command returned more than 2 lines (" + lines.length + ")"); @@ -138,7 +138,7 @@ public class ExecIdentityStore extends ExecStore implements IThreePidProvider { return Optional.empty(); } - String type = StringUtils.trimToEmpty(lines.length == 1 ? "uid" : lines[0]); + String type = StringUtils.trimToEmpty(lines.length == 1 ? UserIdType.Localpart.getId() : lines[0]); String value = StringUtils.trimToEmpty(lines.length == 2 ? lines[1] : lines[0]); if (UserIdType.Localpart.is(type)) { @@ -168,7 +168,7 @@ public class ExecIdentityStore extends ExecStore implements IThreePidProvider { .collect(Collectors.toList())); return GsonUtil.get().toJson(GsonUtil.makeObj("lookup", tpids)); }); - p.addInput(MultilinesType, () -> { + p.addInput(PlainType, () -> { StringBuilder input = new StringBuilder(); for (ThreePidMapping mapping : mappings) { input.append(mapping.getMedium()).append("\t").append(mapping.getValue()).append(System.lineSeparator()); diff --git a/src/main/java/io/kamax/mxisd/backend/exec/ExecProfileStore.java b/src/main/java/io/kamax/mxisd/backend/exec/ExecProfileStore.java index 10007d3..5066330 100644 --- a/src/main/java/io/kamax/mxisd/backend/exec/ExecProfileStore.java +++ b/src/main/java/io/kamax/mxisd/backend/exec/ExecProfileStore.java @@ -58,7 +58,7 @@ public class ExecProfileStore extends ExecStore implements ProfileProvider { Processor> p = new Processor<>(cfg); p.addJsonInputTemplate(tokens -> new JsonProfileRequest(tokens.getLocalpart(), tokens.getDomain(), tokens.getMxid())); - p.addInputTemplate(MultilinesType, tokens -> tokens.getLocalpart() + System.lineSeparator() + p.addInputTemplate(PlainType, tokens -> tokens.getLocalpart() + System.lineSeparator() + tokens.getDomain() + System.lineSeparator() + tokens.getMxid() + System.lineSeparator() ); diff --git a/src/main/java/io/kamax/mxisd/backend/exec/ExecStore.java b/src/main/java/io/kamax/mxisd/backend/exec/ExecStore.java index cd6dfc2..8219e7b 100644 --- a/src/main/java/io/kamax/mxisd/backend/exec/ExecStore.java +++ b/src/main/java/io/kamax/mxisd/backend/exec/ExecStore.java @@ -41,7 +41,7 @@ import java.util.stream.Collectors; public class ExecStore { public static final String JsonType = "json"; - public static final String MultilinesType = "multilines"; + public static final String PlainType = "plain"; protected static String toJson(Object o) { return GsonUtil.get().toJson(o); diff --git a/src/test/java/io/kamax/mxisd/backend/exec/auth/input/ExecAuthInputMultilinesTest.java b/src/test/java/io/kamax/mxisd/backend/exec/auth/input/ExecAuthInputMultilinesTest.java index 1216e39..0b4b7ae 100644 --- a/src/test/java/io/kamax/mxisd/backend/exec/auth/input/ExecAuthInputMultilinesTest.java +++ b/src/test/java/io/kamax/mxisd/backend/exec/auth/input/ExecAuthInputMultilinesTest.java @@ -32,7 +32,7 @@ public class ExecAuthInputMultilinesTest extends ExecAuthStoreTest { @Override protected void setValidInput() { - cfg.getAuth().getInput().setType(ExecStore.MultilinesType); + cfg.getAuth().getInput().setType(ExecStore.PlainType); cfg.getAuth().getInput().setTemplate(null); }