Research code for #14

- Integration with AS and a fallback user to decline expired invites (#120)
- Rework of the AS feature to make it more independent/re-usable
- Skeleton for admin interface via bot to manage invites (#138)

README.md

@@ -74,6 +74,9 @@ Also, check [our FAQ entry](docs/faq.md#what-kind-of-setup-is-mxisd-really-desig
 See the [dedicated document](docs/getting-started.md)
 
 # Support
+## Troubleshooting
+A basic troubleshooting guide is available [here](docs/troubleshooting.md).
+
 ## Community
 Over Matrix: [#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) ([Preview](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/))
 

build.gradle

@@ -145,6 +145,9 @@ dependencies {
 
     // HTTP server
     compile 'io.undertow:undertow-core:2.0.16.Final'
+    
+    // Command parser for AS interface
+    implementation 'commons-cli:commons-cli:1.4'
 
     testCompile 'junit:junit:4.12'
     testCompile 'com.github.tomakehurst:wiremock:2.8.0'
@@ -152,6 +155,14 @@ dependencies {
     testCompile 'com.icegreen:greenmail:1.5.9'
 }
 
+jar {
+    manifest {
+        attributes(
+            'Implementation-Version': mxisdVersion()
+        )
+    }
+}
+
 shadowJar {
     baseName = project.name
     classifier = null
@@ -190,13 +201,13 @@ task debBuild(dependsOn: shadowJar) {
         ant.replaceregexp( // FIXME adapt to new config format
                 file: "${debBuildConfPath}/${debConfFileName}",
                 match: "key:\\R  path:(.*)",
-                replace: "key:\n  path: '${debDataPath}/signing.key'"
+                replace: "key:\n  path: '${debDataPath}/keys'"
         )
 
         ant.replaceregexp( // FIXME adapt to new config format
                 file: "${debBuildConfPath}/${debConfFileName}",
                 match: "storage:\\R  provider:\\R    sqlite:\\R      database:(.*)",
-                replace: "storage:\n  provider:\n    sqlite:\n      database: '${debDataPath}/mxisd.db'"
+                replace: "storage:\n  provider:\n    sqlite:\n      database: '${debDataPath}/store.db'"
         )
 
         copy {

docs/features/authentication.md

@@ -21,7 +21,7 @@ It allows to use Identity stores configured in mxisd to authenticate users on yo
 
 Authentication is divided into two parts:
 - [Basic](#basic): authenticate with a regular username.
-- [Advanced](#advanced): same as basic with extra ability to authenticate using a 3PID.
+- [Advanced](#advanced): same as basic with extra abilities like authenticate using a 3PID or do username rewrite.
 
 ## Basic
 Authentication by username is possible by linking synapse and mxisd together using a specific module for synapse, also
@@ -145,7 +145,49 @@ Your VirtualHost should now look similar to:
 </VirtualHost>
 ```
 
+##### nginx
+
+The specific configuration to add under the relevant `server`:
+
+```nginx
+location /_matrix/client/r0/login {
+    proxy_pass http://localhost:8090;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $remote_addr;
+}
+```
+
+Your `server` section should now look similar to:
+
+```nginx
+server {
+    listen 443 ssl;
+    server_name matrix.example.org;
+    
+    # ...
+    
+    location /_matrix/client/r0/login {
+        proxy_pass http://localhost:8090;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix/identity {
+        proxy_pass http://localhost:8090/_matrix/identity;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix {
+        proxy_pass http://localhost:8008/_matrix;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+}
+```
+
 #### DNS Overwrite
+
 Just like you need to configure a reverse proxy to send client requests to mxisd, you also need to configure mxisd with
 the internal IP of the Homeserver so it can talk to it directly to integrate its directory search.
 
@@ -165,6 +207,12 @@ In case the hostname is the same as your Matrix domain and `server.name` is not
 
 `value` is the base internal URL of the Homeserver, without any `/_matrix/..` or trailing `/`.
 
+### Optional features
+
+The following features are available after you have a working Advanced setup:
+
+- Username rewrite: Allows you to rewrite the username of a regular login/pass authentication to a 3PID, that then gets resolved using the regular lookup process. Most common use case is to allow login with numerical usernames on synapse, which is not possible out of the box.
+
 #### Username rewrite
 In mxisd config:
 ```yaml

docs/features/experimental/application-service.md

@@ -26,7 +26,7 @@ synapseSql:
   connection: '<DB CONNECTION URL>'
 ```
 
-The `synapseSql` section is used to retrieve display names which are not directly accessible in this mode.
+The `synapseSql` section is optional. It is used to retrieve display names which are not directly accessible in this mode.
 For details about `type` and `connection`, see the [relevant documentation](../../stores/synapse.md).
 If you do not configure it, some placeholders will not be available in the notification, like the Room name.
 

docs/features/federation.md

@@ -46,15 +46,6 @@ lookup:
 invite:
   resolution:
     recursive: false
-session:
-  policy:
-    validation:
-      forLocal:
-        toRemote:
-          enabled: false
-      forRemote:
-        toRemote:
-          enabled: false
 ``` 
 
 There is currently no way to selectively disable federation towards specific servers, but this feature is planned.

docs/features/identity.md

@@ -1,6 +1,4 @@
 # Identity
-**WARNING**: This document is incomplete and can be misleading.
-
 Implementation of the [Identity Service API r0.1.0](https://matrix.org/docs/spec/identity_service/r0.1.0.html).
 
 ## Lookups

docs/getting-started.md

@@ -144,7 +144,8 @@ by the relevant hostname which you configured in your reverse proxy.
 **NOTE:** You might not see a suggestion for the e-mail address, which is normal. Still proceed with the invite.
   
 If it worked, it means you are up and running and can enjoy mxisd in its basic mode! Congratulations!  
-If it did not work, [get in touch](../README.md#support) and we'll do our best to get you started.
+If it did not work, read the basic [troubleshooting guide](troubleshooting.md), [get in touch](../README.md#support) and
+we'll do our best to get you started.
 
 ## Next steps
 Once your mxisd server is up and running, there are several ways you can enhance and integrate further with your

docs/stores/exec.md

@@ -212,21 +212,29 @@ The command will use the default values for:
 #### Advanced
 Given the fictional `placeholder` feature:
 ```yaml
-exec.enabled: true
+exec:
-exec.token.mxid: '{matrixId}'
+  enabled: true
-
+  token:
-exec.placeholder.token.localpart: '{username}'
+    mxid: '{matrixId}'
-exec.placeholder.command: '/path/to/executable'
+  auth:
-exec.placeholder.args:
+    token:
-  - '-u'
+      localpart: '{username}'
-  - '{username}'
+    command: '/path/to/executable'
-exec.placeholder.env:
+    args:
-  MATRIX_DOMAIN: '{domain}'
+      - '-u'
-  MATRIX_USER_ID: '{matrixId}'
+      - '{username}'
-  
+    env:
-exec.placeholder.output.type: 'json'
+      MATRIX_DOMAIN: '{domain}'
-exec.placeholder.exit.success: [0, 128]
+      MATRIX_USER_ID: '{matrixId}'
-exec.placeholder.exit.failure: [1, 129]
+    output:
+      type: 'json'
+    exit:
+      success:
+        - 0
+        - 128
+      failure:
+        - 1
+        - 129
 ```
 With:
 - The Identity store enabled for all features

docs/threepids/medium/email/smtp-connector.md

@@ -1,6 +1,4 @@
 # Email notifications - SMTP connector
-Enabled by default.
-
 Connector ID: `smtp`
 
 ## Configuration

docs/threepids/medium/msisdn/twilio-connector.md

@@ -1,6 +1,4 @@
 # SMS notifications - Twilio connector
-Enabled by default.
-
 Connector ID: `twilio`
 
 ## Configuration

docs/threepids/notification/template-generator.md

@@ -51,7 +51,7 @@ This template is used when someone is invited into a room using an email address
 | `%ROOM_NAME%`         | The Name of the room in which the invite took place. If not available/set, empty         |
 | `%ROOM_NAME_OR_ID%`   | The Name of the room in which the invite took place. If not available/set, its Matrix ID |
 
-### Local validation of 3PID Session
+### Validation of 3PID Session
 This template is used when to user which added their 3PID address to their profile/settings and the session policy
 allows at least local sessions.  
 
@@ -59,17 +59,5 @@ allows at least local sessions.
 | Placeholder          | Purpose                                                                              |
 |----------------------|--------------------------------------------------------------------------------------|
 | `%VALIDATION_LINK%`  | URL, including token, to validate the 3PID session.                                  |
-| `%VALIDATION_TOKEN%` | The token needed to validate the local session, in case the user cannot use the link |
+| `%VALIDATION_TOKEN%` | The token needed to validate the session, in case the user cannot use the link.      |
-
+| `%NEXT_URL%`         | URL to redirect to after the sessions has been validated.                            |
-### Remote validation of 3PID Session
-This template is used when to user which added their 3PID address to their profile/settings and the session policy only
-allows remote sessions.
-
-**NOTE:** 3PID session always require local validation of a token, even if a remote session is enforced.  
-One cannot bind a Matrix ID to the session until both local and remote sessions have been validated.
-
-#### Placeholders
-| Placeholder          | Purpose                                                |
-|----------------------|--------------------------------------------------------|
-| `%VALIDATION_TOKEN%` | The token needed to validate the session               |
-| `%NEXT_URL%`         | URL to continue with remote validation of the session. |

docs/troubleshooting.md

@@ -0,0 +1,53 @@
+# Troubleshooting
+- [Purpose](#purpose)
+- [Logs](#logs)
+  - [Locations](#locations)
+  - [Reading Them](#reading-them)
+  - [Common issues](#common-issues)
+- [Submit an issue](#submit-an-issue)
+
+## Purpose
+This document describes basic troubleshooting steps for mxisd.
+
+## Logs
+### Locations
+mxisd logs to `STDOUT` (Standard Output) and `STDERR` (Standard Error) only, which gets redirected
+to log file(s) depending on your system.
+
+If you use the [Debian package](install/debian.md), this goes to `syslog`.  
+If you use the [Docker image](install/docker.md), this goes to the container logs.  
+
+For any other platform, please refer to your package maintainer.
+
+### Reading them
+Before reporting an issue, it is important to produce clean and complete logs so they can be understood.
+
+It is usually useless to try to troubleshoot an issue based on a single log line. Any action or API request
+in mxisd would trigger more than one log lines, and those would be considered necessary context to
+understand what happened.
+
+You may also find things called *stacktraces*. Those are important to pin-point bugs and the likes and should
+always be included in any report. They also tend to be very specific about the issue at hand.
+
+Example of a stacktrace:
+```
+Exception in thread "main" java.lang.NullPointerException
+        at com.example.myproject.Book.getTitle(Book.java:16)
+        at com.example.myproject.Author.getBookTitles(Author.java:25)
+        at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
+```
+
+### Common issues
+#### Internal Server Error
+`Contact your administrator with reference Transaction #123456789`
+
+This is a generic message produced in case of an unknown error. The transaction reference allows to easily find
+the location in the logs to look for an error.
+
+**IMPORTANT:** That line alone does not tell you anything about the error. You'll need the log lines before and after,
+usually including a stacktrace, to know what happened. Please take the time to read the surround output to get
+context about the issue at hand.
+
+## Submit an issue
+In case the logs do not allow you to understand the issue at hand, please submit clean and complete logs
+as explained [here](#reading-them) in a new issue on the repository, or [get in touch](../README.md#contact).

mxisd.example.yaml

@@ -1,6 +1,11 @@
 # Sample configuration file explaining the minimum required keys to be set to run mxisd
 #
 # For a complete list of options, see https://github.com/kamax-matrix/mxisd/docs/README.md
+#
+# Please follow the Getting Started guide if this is your first time using/configuring mxisd
+#
+#  -- https://github.com/kamax-matrix/mxisd/blob/master/docs/getting-started.md#getting-started
+#
 
 #######################
 # Matrix config items #
@@ -9,6 +14,11 @@
 # NOTE: in Synapse Homeserver, the Matrix domain is defined as 'server_name' in configuration file.
 #
 # This is used to build the various identifiers in all the features.
+#
+# If the hostname of the public URL used to reach your Matrix services is different from your Matrix domain,
+# per example matrix.domain.tld vs domain.tld, then use the server.name configuration option.
+# See the "Configure" section of the Getting Started guide for more info.
+#
 matrix:
   domain: ''
 
@@ -16,26 +26,27 @@ matrix:
 ################
 # Signing keys #
 ################
-# Absolute path for the Identity Server signing key.
+# Absolute path for the Identity Server signing keys database.
-# This is **NOT** your homeserver key.
+# /!\ THIS MUST **NOT** BE YOUR HOMESERVER KEYS FILE /!\
-# The signing key is auto-generated during execution time if not present.
+# If this path does not exist, it will be auto-generated.
 #
-# During testing, /var/tmp/mxisd.key is a possible value
+# During testing, /var/tmp/mxisd/keys is a possible value
 # For production, recommended location shall be one of the following:
-#   - /var/opt/mxisd/sign.key
+#   - /var/lib/mxisd/keys
-#   - /var/local/mxisd/sign.key
+#   - /var/opt/mxisd/keys
-#   - /var/lib/mxisd/sign.key
+#   - /var/local/mxisd/keys
 #
 key:
   path: ''
 
 
 # Path to the SQLite DB file for mxisd internal storage
+# /!\ THIS MUST **NOT** BE YOUR HOMESERVER DATABASE /!\
 #
 # Examples:
-#  - /var/opt/mxisd/mxisd.db
+#  - /var/opt/mxisd/store.db
-#  - /var/local/mxisd/mxisd.db
+#  - /var/local/mxisd/store.db
-#  - /var/lib/mxisd/mxisd.db
+#  - /var/lib/mxisd/store.db
 #
 storage:
   provider:
@@ -43,48 +54,31 @@ storage:
       database: '/path/to/mxisd.db'
 
 
-####################
+###################
-# Fallback servers #
+# Identity Stores #
-####################
+###################
+# If you are using synapse standalone and do not have an Identity store,
+# see https://github.com/kamax-matrix/mxisd/blob/master/docs/stores/synapse.md#synapse-identity-store
 #
-# Root/Central servers to be used as final fallback when performing lookups.
-# By default, for privacy reasons, matrix.org servers are not enabled.
-# See the following issue: https://github.com/kamax-matrix/mxisd/issues/76
-#
-# If you would like to use them and trade away your privacy for convenience, uncomment the following option:
-#
-#forward:
-#  servers: ['matrix-org']
-
-
-################
-# LDAP Backend #
-################
 # If you would like to integrate with your AD/Samba/LDAP server,
 # see https://github.com/kamax-matrix/mxisd/blob/master/docs/stores/ldap.md
-
+#
-
+# For any other Identity store, or to simply discover them,
-###############
+# see https://github.com/kamax-matrix/mxisd/blob/master/docs/stores/README.md
-# SQL Backend #
-###############
-# If you would like to integrate with a MySQL/MariaDB/PostgreQL/SQLite DB,
-# see https://github.com/kamax-matrix/mxisd/blob/master/docs/stores/sql.md
-
-
-################
-# REST Backend #
-################
-# If you would like to integrate with an existing web service/webapp,
-# see https://github.com/kamax-matrix/mxisd/blob/master/docs/stores/rest.md
 
 
 #################################################
 # Notifications for invites/addition to profile #
 #################################################
-# If you would like to change the content,
+# This is mandatory to deal with anything e-mail related.
+#
+# For an introduction to sessions, invites and 3PIDs in general,
+# see https://github.com/kamax-matrix/mxisd/blob/master/docs/threepids/session/session.md#3pid-sessions
+#
+# If you would like to change the content of the notifications,
 # see https://github.com/kamax-matrix/mxisd/blob/master/docs/threepids/notification/template-generator.md
 #
-#### E-mail invite sender
+#### E-mail connector
 threepid:
   medium:
     email:
@@ -100,12 +94,13 @@ threepid:
           # SMTP port
           port: 587
 
-          # TLS mode for the connection.
+          # STARTLS mode for the connection.
+          # SSL/TLS is currently not supported. See https://github.com/kamax-matrix/mxisd/issues/125
           #
           # Possible values:
-          #  0    Disable TLS entirely
+          #  0    Disable any kind of TLS entirely
-          #  1    Enable TLS if supported by server (default)
+          #  1    Enable STARTLS if supported by server (default)
-          #  2    Force TLS and fail if not available
+          #  2    Force STARTLS and fail if not available
           #
           tls: 1
 

src/main/java/io/kamax/mxisd/HttpMxisd.java

@@ -21,23 +21,30 @@
 package io.kamax.mxisd;
 
 import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.http.undertow.handler.InternalInfoHandler;
 import io.kamax.mxisd.http.undertow.handler.OptionsHandler;
 import io.kamax.mxisd.http.undertow.handler.SaneHandler;
 import io.kamax.mxisd.http.undertow.handler.as.v1.AsNotFoundHandler;
 import io.kamax.mxisd.http.undertow.handler.as.v1.AsTransactionHandler;
+import io.kamax.mxisd.http.undertow.handler.as.v1.AsUserHandler;
 import io.kamax.mxisd.http.undertow.handler.auth.RestAuthHandler;
 import io.kamax.mxisd.http.undertow.handler.auth.v1.LoginGetHandler;
 import io.kamax.mxisd.http.undertow.handler.auth.v1.LoginHandler;
 import io.kamax.mxisd.http.undertow.handler.auth.v1.LoginPostHandler;
 import io.kamax.mxisd.http.undertow.handler.directory.v1.UserDirectorySearchHandler;
 import io.kamax.mxisd.http.undertow.handler.identity.v1.*;
+import io.kamax.mxisd.http.undertow.handler.invite.v1.RoomInviteHandler;
 import io.kamax.mxisd.http.undertow.handler.profile.v1.InternalProfileHandler;
 import io.kamax.mxisd.http.undertow.handler.profile.v1.ProfileHandler;
+import io.kamax.mxisd.http.undertow.handler.register.v1.Register3pidRequestTokenHandler;
 import io.kamax.mxisd.http.undertow.handler.status.StatusHandler;
+import io.kamax.mxisd.http.undertow.handler.status.VersionHandler;
 import io.undertow.Handlers;
 import io.undertow.Undertow;
 import io.undertow.server.HttpHandler;
 
+import java.util.Objects;
+
 public class HttpMxisd {
 
     // Core
@@ -46,6 +53,12 @@ public class HttpMxisd {
     // I/O
     private Undertow httpSrv;
 
+    static {
+        // Used in XNIO package, dependency of Undertow
+        // We switch to slf4j
+        System.setProperty("org.jboss.logging.provider", "slf4j");
+    }
+
     public HttpMxisd(MxisdConfig cfg) {
         m = new Mxisd(cfg);
     }
@@ -54,9 +67,12 @@ public class HttpMxisd {
         m.start();
 
         HttpHandler helloHandler = SaneHandler.around(new HelloHandler());
-        HttpHandler asNotFoundHandler = SaneHandler.around(new AsNotFoundHandler(m.getAs()));
+
+        HttpHandler asUserHandler = SaneHandler.around(new AsUserHandler(m.getAs()));
         HttpHandler asTxnHandler = SaneHandler.around(new AsTransactionHandler(m.getAs()));
-        HttpHandler storeInvHandler = SaneHandler.around(new StoreInviteHandler(m.getConfig().getServer(), m.getInvitationManager(), m.getKeyManager()));
+        HttpHandler asNotFoundHandler = SaneHandler.around(new AsNotFoundHandler(m.getAs()));
+
+        HttpHandler storeInvHandler = SaneHandler.around(new StoreInviteHandler(m.getConfig().getServer(), m.getInvite(), m.getKeyManager()));
         HttpHandler sessValidateHandler = SaneHandler.around(new SessionValidateHandler(m.getSession(), m.getConfig().getServer(), m.getConfig().getView()));
 
         httpSrv = Undertow.builder().addHttpListener(m.getConfig().getServer().getPort(), "0.0.0.0").setHandler(Handlers.routing()
@@ -65,6 +81,7 @@ public class HttpMxisd {
 
                 // Status endpoints
                 .get(StatusHandler.Path, SaneHandler.around(new StatusHandler()))
+                .get(VersionHandler.Path, SaneHandler.around(new VersionHandler()))
 
                 // Authentication endpoints
                 .get(LoginHandler.Path, SaneHandler.around(new LoginGetHandler(m.getAuth(), m.getHttpClient())))
@@ -77,40 +94,53 @@ public class HttpMxisd {
                 // Key endpoints
                 .get(KeyGetHandler.Path, SaneHandler.around(new KeyGetHandler(m.getKeyManager())))
                 .get(RegularKeyIsValidHandler.Path, SaneHandler.around(new RegularKeyIsValidHandler(m.getKeyManager())))
-                .get(EphemeralKeyIsValidHandler.Path, SaneHandler.around(new EphemeralKeyIsValidHandler()))
+                .get(EphemeralKeyIsValidHandler.Path, SaneHandler.around(new EphemeralKeyIsValidHandler(m.getKeyManager())))
 
                 // Identity endpoints
                 .get(HelloHandler.Path, helloHandler)
                 .get(HelloHandler.Path + "/", helloHandler) // Be lax with possibly trailing slash
-                .get(SingleLookupHandler.Path, SaneHandler.around(new SingleLookupHandler(m.getIdentity(), m.getSign())))
+                .get(SingleLookupHandler.Path, SaneHandler.around(new SingleLookupHandler(m.getConfig(), m.getIdentity(), m.getSign())))
                 .post(BulkLookupHandler.Path, SaneHandler.around(new BulkLookupHandler(m.getIdentity())))
                 .post(StoreInviteHandler.Path, storeInvHandler)
                 .post(SessionStartHandler.Path, SaneHandler.around(new SessionStartHandler(m.getSession())))
                 .get(SessionValidateHandler.Path, sessValidateHandler)
                 .post(SessionValidateHandler.Path, sessValidateHandler)
                 .get(SessionTpidGetValidatedHandler.Path, SaneHandler.around(new SessionTpidGetValidatedHandler(m.getSession())))
-                .post(SessionTpidBindHandler.Path, SaneHandler.around(new SessionTpidBindHandler(m.getSession(), m.getInvitationManager())))
+                .post(SessionTpidBindHandler.Path, SaneHandler.around(new SessionTpidBindHandler(m.getSession(), m.getInvite())))
                 .post(SessionTpidUnbindHandler.Path, SaneHandler.around(new SessionTpidUnbindHandler(m.getSession())))
+                .post(SignEd25519Handler.Path, SaneHandler.around(new SignEd25519Handler(m.getConfig(), m.getInvite(), m.getSign())))
 
                 // Profile endpoints
                 .get(ProfileHandler.Path, SaneHandler.around(new ProfileHandler(m.getProfile())))
                 .get(InternalProfileHandler.Path, SaneHandler.around(new InternalProfileHandler(m.getProfile())))
 
+                // Registration endpoints
+                .post(Register3pidRequestTokenHandler.Path, SaneHandler.around(new Register3pidRequestTokenHandler(m.getReg(), m.getClientDns(), m.getHttpClient())))
+
+                // Invite endpoints
+                .post(RoomInviteHandler.Path, SaneHandler.around(new RoomInviteHandler(m.getHttpClient(), m.getClientDns(), m.getInvite())))
+
                 // Application Service endpoints
-                .get("/_matrix/app/v1/users/**", asNotFoundHandler)
+                .get(AsUserHandler.Path, asUserHandler)
-                .get("/users/**", asNotFoundHandler) // Legacy endpoint
                 .get("/_matrix/app/v1/rooms/**", asNotFoundHandler)
-                .get("/rooms/**", asNotFoundHandler) // Legacy endpoint
                 .put(AsTransactionHandler.Path, asTxnHandler)
+
+                .get("/users/{" + AsUserHandler.ID + "}", asUserHandler) // Legacy endpoint
+                .get("/rooms/**", asNotFoundHandler) // Legacy endpoint
                 .put("/transactions/{" + AsTransactionHandler.ID + "}", asTxnHandler) // Legacy endpoint
 
+                // Banned endpoints
+                .get(InternalInfoHandler.Path, SaneHandler.around(new InternalInfoHandler()))
+
         ).build();
 
         httpSrv.start();
     }
 
     public void stop() {
-        httpSrv.stop();
+        // Because it might have never been initialized if an exception is thrown early
+        if (Objects.nonNull(httpSrv)) httpSrv.stop();
+
         m.stop();
     }
 

src/main/java/io/kamax/mxisd/Mxisd.java

@@ -20,8 +20,6 @@
 
 package io.kamax.mxisd;
 
-import io.kamax.matrix.crypto.KeyManager;
-import io.kamax.matrix.crypto.SignatureManager;
 import io.kamax.mxisd.as.AppSvcManager;
 import io.kamax.mxisd.auth.AuthManager;
 import io.kamax.mxisd.auth.AuthProviders;
@@ -29,6 +27,9 @@ import io.kamax.mxisd.backend.IdentityStoreSupplier;
 import io.kamax.mxisd.backend.sql.synapse.Synapse;
 import io.kamax.mxisd.config.MxisdConfig;
 import io.kamax.mxisd.crypto.CryptoFactory;
+import io.kamax.mxisd.crypto.KeyManager;
+import io.kamax.mxisd.crypto.SignatureManager;
+import io.kamax.mxisd.crypto.ed25519.Ed25519KeyManager;
 import io.kamax.mxisd.directory.DirectoryManager;
 import io.kamax.mxisd.directory.DirectoryProviders;
 import io.kamax.mxisd.dns.ClientDnsOverwrite;
@@ -46,9 +47,11 @@ import io.kamax.mxisd.notification.NotificationHandlers;
 import io.kamax.mxisd.notification.NotificationManager;
 import io.kamax.mxisd.profile.ProfileManager;
 import io.kamax.mxisd.profile.ProfileProviders;
+import io.kamax.mxisd.registration.RegistrationManager;
 import io.kamax.mxisd.session.SessionManager;
 import io.kamax.mxisd.storage.IStorage;
 import io.kamax.mxisd.storage.ormlite.OrmLiteSqlStorage;
+import org.apache.commons.lang.StringUtils;
 import org.apache.http.impl.client.CloseableHttpClient;
 import org.apache.http.impl.client.HttpClients;
 
@@ -56,6 +59,10 @@ import java.util.ServiceLoader;
 
 public class Mxisd {
 
+    public static final String Name = StringUtils.defaultIfBlank(Mxisd.class.getPackage().getImplementationTitle(), "mxisd");
+    public static final String Version = StringUtils.defaultIfBlank(Mxisd.class.getPackage().getImplementationVersion(), "UNKNOWN");
+    public static final String Agent = Name + "/" + Version;
+
     private MxisdConfig cfg;
 
     private CloseableHttpClient httpClient;
@@ -63,8 +70,9 @@ public class Mxisd {
 
     private IStorage store;
 
-    private KeyManager keyMgr;
+    private Ed25519KeyManager keyMgr;
     private SignatureManager signMgr;
+    private ClientDnsOverwrite clientDns;
 
     // Features
     private AuthManager authMgr;
@@ -75,6 +83,10 @@ public class Mxisd {
     private AppSvcManager asHander;
     private SessionManager sessMgr;
     private NotificationManager notifMgr;
+    private RegistrationManager regMgr;
+
+    // HS-specific classes
+    private Synapse synapse;
 
     public Mxisd(MxisdConfig cfg) {
         this.cfg = cfg.build();
@@ -82,7 +94,7 @@ public class Mxisd {
 
     private void build() {
         httpClient = HttpClients.custom()
-                .setUserAgent("mxisd")
+                .setUserAgent(Agent)
                 .setMaxConnPerRoute(Integer.MAX_VALUE)
                 .setMaxConnTotal(Integer.MAX_VALUE)
                 .build();
@@ -92,10 +104,10 @@ public class Mxisd {
 
         store = new OrmLiteSqlStorage(cfg);
         keyMgr = CryptoFactory.getKeyManager(cfg.getKey());
-        signMgr = CryptoFactory.getSignatureManager(keyMgr, cfg.getServer());
+        signMgr = CryptoFactory.getSignatureManager(keyMgr);
-        ClientDnsOverwrite clientDns = new ClientDnsOverwrite(cfg.getDns().getOverwrite());
+        clientDns = new ClientDnsOverwrite(cfg.getDns().getOverwrite());
         FederationDnsOverwrite fedDns = new FederationDnsOverwrite(cfg.getDns().getOverwrite());
-        Synapse synapse = new Synapse(cfg.getSynapseSql());
+        synapse = new Synapse(cfg.getSynapseSql());
         BridgeFetcher bridgeFetcher = new BridgeFetcher(cfg.getLookup().getRecursive().getBridge(), srvFetcher);
 
         ServiceLoader.load(IdentityStoreSupplier.class).iterator().forEachRemaining(p -> p.accept(this));
@@ -105,10 +117,11 @@ public class Mxisd {
         pMgr = new ProfileManager(ProfileProviders.get(), clientDns, httpClient);
         notifMgr = new NotificationManager(cfg.getNotification(), NotificationHandlers.get());
         sessMgr = new SessionManager(cfg.getSession(), cfg.getMatrix(), store, notifMgr, idStrategy, httpClient);
-        invMgr = new InvitationManager(cfg.getInvite(), store, idStrategy, signMgr, fedDns, notifMgr);
+        invMgr = new InvitationManager(cfg, store, idStrategy, keyMgr, signMgr, fedDns, notifMgr, pMgr);
         authMgr = new AuthManager(cfg, AuthProviders.get(), idStrategy, invMgr, clientDns, httpClient);
         dirMgr = new DirectoryManager(cfg.getDirectory(), clientDns, httpClient, DirectoryProviders.get());
-        asHander = new AppSvcManager(cfg, store, pMgr, notifMgr, synapse);
+        regMgr = new RegistrationManager(cfg.getRegister(), httpClient, clientDns, invMgr);
+        asHander = new AppSvcManager(this);
     }
 
     public MxisdConfig getConfig() {
@@ -119,6 +132,10 @@ public class Mxisd {
         return httpClient;
     }
 
+    public ClientDnsOverwrite getClientDns() {
+        return clientDns;
+    }
+
     public IRemoteIdentityServerFetcher getServerFetcher() {
         return srvFetcher;
     }
@@ -127,7 +144,7 @@ public class Mxisd {
         return keyMgr;
     }
 
-    public InvitationManager getInvitationManager() {
+    public InvitationManager getInvite() {
         return invMgr;
     }
 
@@ -155,6 +172,10 @@ public class Mxisd {
         return signMgr;
     }
 
+    public RegistrationManager getReg() {
+        return regMgr;
+    }
+
     public AppSvcManager getAs() {
         return asHander;
     }
@@ -163,6 +184,14 @@ public class Mxisd {
         return notifMgr;
     }
 
+    public IStorage getStore() {
+        return store;
+    }
+
+    public Synapse getSynapse() {
+        return synapse;
+    }
+
     public void start() {
         build();
     }

src/main/java/io/kamax/mxisd/MxisdStandaloneExec.java

@@ -22,48 +22,64 @@ package io.kamax.mxisd;
 
 import io.kamax.mxisd.config.MxisdConfig;
 import io.kamax.mxisd.config.YamlConfigLoader;
+import io.kamax.mxisd.exception.ConfigurationException;
 import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
-import java.io.IOException;
 import java.util.Arrays;
 import java.util.Iterator;
 import java.util.Objects;
 
 public class MxisdStandaloneExec {
 
-    private static final Logger log = LoggerFactory.getLogger("");
+    private static final Logger log = LoggerFactory.getLogger("App");
-
-    public static void main(String[] args) throws IOException {
-        log.info("------------- mxisd starting -------------");
-        MxisdConfig cfg = null;
-
-        Iterator<String> argsIt = Arrays.asList(args).iterator();
-        while (argsIt.hasNext()) {
-            String arg = argsIt.next();
-            if (StringUtils.equals("-c", arg)) {
-                String cfgFile = argsIt.next();
-                cfg = YamlConfigLoader.loadFromFile(cfgFile);
-            } else {
-                log.info("Invalid argument: {}", arg);
-                System.exit(1);
-            }
-        }
-
-        if (Objects.isNull(cfg)) {
-            cfg = YamlConfigLoader.tryLoadFromFile("mxisd.yaml").orElseGet(MxisdConfig::new);
-        }
 
+    public static void main(String[] args) {
         try {
+            MxisdConfig cfg = null;
+            Iterator<String> argsIt = Arrays.asList(args).iterator();
+            while (argsIt.hasNext()) {
+                String arg = argsIt.next();
+                if (StringUtils.equalsAny(arg, "-h", "--help", "-?", "--usage")) {
+                    System.out.println("Available arguments:" + System.lineSeparator());
+                    System.out.println("  -h, --help       Show this help message");
+                    System.out.println("  --version        Print the version then exit");
+                    System.out.println("  -c, --config     Set the configuration file location");
+                    System.out.println(" ");
+                    System.exit(0);
+                } else if (StringUtils.equalsAny(arg, "-c", "--config")) {
+                    String cfgFile = argsIt.next();
+                    cfg = YamlConfigLoader.loadFromFile(cfgFile);
+                } else if (StringUtils.equals("--version", arg)) {
+                    System.out.println(Mxisd.Version);
+                    System.exit(0);
+                } else {
+                    System.err.println("Invalid argument: " + arg);
+                    System.err.println("Try '--help' for available arguments");
+                    System.exit(1);
+                }
+            }
+
+            log.info("mxisd starting");
+            log.info("Version: {}", Mxisd.Version);
+
+            if (Objects.isNull(cfg)) {
+                cfg = YamlConfigLoader.tryLoadFromFile("mxisd.yaml").orElseGet(MxisdConfig::new);
+            }
+
             HttpMxisd mxisd = new HttpMxisd(cfg);
             Runtime.getRuntime().addShutdownHook(new Thread(() -> {
                 mxisd.stop();
-                log.info("------------- mxisd stopped -------------");
+                log.info("mxisd stopped");
             }));
             mxisd.start();
 
-            log.info("------------- mxisd started -------------");
+            log.info("mxisd started");
+        } catch (ConfigurationException e) {
+            log.error(e.getDetailedMessage());
+            log.error(e.getMessage());
+            System.exit(2);
         } catch (Throwable t) {
             t.printStackTrace();
             System.exit(1);

src/main/java/io/kamax/mxisd/as/AppSvcManager.java

@@ -22,76 +22,183 @@ package io.kamax.mxisd.as;
 
 import com.google.gson.JsonObject;
 import io.kamax.matrix.MatrixID;
-import io.kamax.matrix.ThreePidMedium;
 import io.kamax.matrix._MatrixID;
-import io.kamax.matrix._ThreePid;
+import io.kamax.matrix.client.MatrixClientContext;
+import io.kamax.matrix.client.as.MatrixApplicationServiceClient;
 import io.kamax.matrix.event.EventKey;
 import io.kamax.matrix.json.GsonUtil;
-import io.kamax.mxisd.backend.sql.synapse.Synapse;
+import io.kamax.mxisd.Mxisd;
-import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.as.processor.event.EventTypeProcessor;
+import io.kamax.mxisd.as.processor.event.MembershipEventProcessor;
+import io.kamax.mxisd.as.processor.event.MessageEventProcessor;
+import io.kamax.mxisd.as.registration.SynapseRegistrationYaml;
+import io.kamax.mxisd.config.AppServiceConfig;
 import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.exception.ConfigurationException;
 import io.kamax.mxisd.exception.HttpMatrixException;
 import io.kamax.mxisd.exception.NotAllowedException;
-import io.kamax.mxisd.notification.NotificationManager;
-import io.kamax.mxisd.profile.ProfileManager;
 import io.kamax.mxisd.storage.IStorage;
 import io.kamax.mxisd.storage.ormlite.dao.ASTransactionDao;
 import io.kamax.mxisd.util.GsonParser;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.ObjectUtils;
+import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
+import org.yaml.snakeyaml.Yaml;
+import org.yaml.snakeyaml.introspector.BeanAccess;
+import org.yaml.snakeyaml.representer.Representer;
 
+import java.io.FileOutputStream;
+import java.io.IOException;
 import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
 import java.time.Instant;
 import java.util.*;
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.ConcurrentHashMap;
-import java.util.stream.Collectors;
 
 public class AppSvcManager {
 
-    private transient final Logger log = LoggerFactory.getLogger(AppSvcManager.class);
+    private static final Logger log = LoggerFactory.getLogger(AppSvcManager.class);
 
-    private final GsonParser parser;
+    private final AppServiceConfig cfg;
+    private final IStorage store;
+    private final GsonParser parser = new GsonParser();
 
-    private MatrixConfig cfg;
+    private MatrixApplicationServiceClient client;
-    private IStorage store;
+    private Map<String, EventTypeProcessor> processors = new HashMap<>();
-    private ProfileManager profiler;
+    private Map<String, CompletableFuture<String>> transactionsInProgress = new ConcurrentHashMap<>();
-    private NotificationManager notif;
-    private Synapse synapse;
 
-    private Map<String, CompletableFuture<String>> transactionsInProgress;
+    public AppSvcManager(Mxisd m) {
+        this.cfg = m.getConfig().getAppsvc();
+        this.store = m.getStore();
 
-    public AppSvcManager(MxisdConfig cfg, IStorage store, ProfileManager profiler, NotificationManager notif, Synapse synapse) {
+        /*
-        this.cfg = cfg.getMatrix();
+        We process the configuration to make sure all is fine and setting default values if needed
-        this.store = store;
+         */
-        this.profiler = profiler;
-        this.notif = notif;
-        this.synapse = synapse;
 
-        parser = new GsonParser();
+        // By default, the feature is enabled
-        transactionsInProgress = new ConcurrentHashMap<>();
+        cfg.setEnabled(ObjectUtils.defaultIfNull(cfg.isEnabled(), false));
+
+        if (!cfg.isEnabled()) {
+            return;
+        }
+
+        if (Objects.isNull(cfg.getEndpoint().getToAS().getUrl())) {
+            throw new ConfigurationException("App Service: Endpoint: To AS: URL");
+        }
+
+        if (Objects.isNull(cfg.getEndpoint().getToAS().getToken())) {
+            throw new ConfigurationException("App Service: Endpoint: To AS: Token", "Must be set, even if to an empty string");
+        }
+
+        if (Objects.isNull(cfg.getEndpoint().getToHS().getUrl())) {
+            throw new ConfigurationException("App Service: Endpoint: To HS: URL");
+        }
+
+        if (Objects.isNull(cfg.getEndpoint().getToHS().getToken())) {
+            throw new ConfigurationException("App Service: Endpoint: To HS: Token", "Must be set, even if to an empty string");
+        }
+
+        // We set a default status for each feature individually
+        cfg.getFeature().getAdmin().setEnabled(ObjectUtils.defaultIfNull(cfg.getFeature().getAdmin().getEnabled(), cfg.isEnabled()));
+        cfg.getFeature().setCleanExpiredInvite(ObjectUtils.defaultIfNull(cfg.getFeature().getCleanExpiredInvite(), cfg.isEnabled()));
+        cfg.getFeature().setInviteById(ObjectUtils.defaultIfNull(cfg.getFeature().getInviteById(), false));
+
+        if (cfg.getFeature().getAdmin().getEnabled()) {
+            if (StringUtils.isBlank(cfg.getUser().getMain())) {
+                throw new ConfigurationException("App Service admin feature is enabled, but no main user configured");
+            }
+
+            if (cfg.getUser().getMain().startsWith("@") || cfg.getUser().getMain().contains(":")) {
+                throw new ConfigurationException("App Service: Users: Main ID: Is not a localpart");
+            }
+        }
+
+        if (cfg.getFeature().getCleanExpiredInvite()) {
+            if (StringUtils.isBlank(cfg.getUser().getInviteExpired())) {
+                throw new ConfigurationException("App Service user for Expired Invite is not set");
+            }
+
+            if (cfg.getUser().getMain().startsWith("@") || cfg.getUser().getMain().contains(":")) {
+                throw new ConfigurationException("App Service: Users: Expired Invite ID: Is not a localpart");
+            }
+        }
+
+        MatrixClientContext mxContext = new MatrixClientContext();
+        mxContext.setDomain(m.getConfig().getMatrix().getDomain());
+        mxContext.setToken(cfg.getEndpoint().getToHS().getToken());
+        mxContext.setHsBaseUrl(cfg.getEndpoint().getToHS().getUrl());
+        client = new MatrixApplicationServiceClient(mxContext);
+
+        processors.put("m.room.member", new MembershipEventProcessor(client, m));
+        processors.put("m.room.message", new MessageEventProcessor(m, client));
+
+        processSynapseConfig(m.getConfig());
+    }
+
+    private void processSynapseConfig(MxisdConfig cfg) {
+        String synapseRegFile = cfg.getAppsvc().getRegistration().getSynapse().getFile();
+
+        if (StringUtils.isBlank(synapseRegFile)) {
+            log.info("No synapse registration file path given - skipping generation...");
+            return;
+        }
+
+        SynapseRegistrationYaml syncCfg = SynapseRegistrationYaml.parse(cfg.getAppsvc(), cfg.getMatrix().getDomain());
+
+        Representer rep = new Representer();
+        rep.getPropertyUtils().setBeanAccess(BeanAccess.FIELD);
+        Yaml yaml = new Yaml(rep);
+
+        // SnakeYAML set the type of object on the first line, which can fail to be parsed on synapse
+        // We therefore need to split the resulting string, remove the first line, and then write it
+        List<String> lines = new ArrayList<>(Arrays.asList(yaml.dump(syncCfg).split("\\R+")));
+        if (StringUtils.equals(lines.get(0), "!!" + SynapseRegistrationYaml.class.getCanonicalName())) {
+            lines.remove(0);
+        }
+
+        try (FileOutputStream os = new FileOutputStream(synapseRegFile)) {
+            IOUtils.writeLines(lines, System.lineSeparator(), os, StandardCharsets.UTF_8);
+        } catch (IOException e) {
+            throw new RuntimeException("Unable to write synapse appservice registration file", e);
+        }
+    }
+
+    private void ensureEnabled() {
+        if (!cfg.isEnabled()) {
+            throw new HttpMatrixException(503, "M_NOT_AVAILABLE", "This feature is disabled");
+        }
     }
 
     public AppSvcManager withToken(String token) {
+        ensureEnabled();
+
         if (StringUtils.isBlank(token)) {
             throw new HttpMatrixException(401, "M_UNAUTHORIZED", "No HS token");
         }
 
-        if (!StringUtils.equals(cfg.getListener().getToken().getHs(), token)) {
+        if (!StringUtils.equals(cfg.getEndpoint().getToAS().getToken(), token)) {
             throw new NotAllowedException("Invalid HS token");
         }
 
         return this;
     }
 
+    public void processUser(String userId) {
+        client.createUser(MatrixID.asAcceptable(userId).getLocalPart());
+    }
+
     public CompletableFuture<String> processTransaction(String txnId, InputStream is) {
+        ensureEnabled();
+
         if (StringUtils.isEmpty(txnId)) {
             throw new IllegalArgumentException("Transaction ID cannot be empty");
         }
 
         synchronized (this) {
-            Optional<ASTransactionDao> dao = store.getTransactionResult(cfg.getListener().getLocalpart(), txnId);
+            Optional<ASTransactionDao> dao = store.getTransactionResult(cfg.getUser().getMain(), txnId);
             if (dao.isPresent()) {
                 log.info("AS Transaction {} already processed - returning computed result", txnId);
                 return CompletableFuture.completedFuture(dao.get().getResult());
@@ -122,7 +229,7 @@ public class AppSvcManager {
 
             try {
                 log.info("Saving transaction details to store");
-                store.insertTransactionResult(cfg.getListener().getLocalpart(), txnId, end, result);
+                store.insertTransactionResult(cfg.getUser().getMain(), txnId, end, result);
             } finally {
                 log.debug("Removing CompletedFuture from transaction map");
                 transactionsInProgress.remove(txnId);
@@ -139,7 +246,7 @@ public class AppSvcManager {
         return future;
     }
 
-    public void processTransaction(List<JsonObject> eventsJson) {
+    private void processTransaction(List<JsonObject> eventsJson) {
         log.info("Processing transaction events: start");
 
         eventsJson.forEach(ev -> {
@@ -165,54 +272,14 @@ public class AppSvcManager {
             _MatrixID sender = MatrixID.asAcceptable(senderId);
             log.debug("Sender: {}", senderId);
 
-            if (!StringUtils.equals("m.room.member", GsonUtil.getStringOrNull(ev, "type"))) {
+            String evType = StringUtils.defaultIfBlank(EventKey.Type.getStringOrNull(ev), "<EMPTY/MISSING>");
-                log.debug("This is not a room membership event, skipping");
+            EventTypeProcessor p = processors.get(evType);
+            if (Objects.isNull(p)) {
+                log.debug("No event processor for type {}, skipping", evType);
                 return;
             }
 
-            if (!StringUtils.equals("invite", GsonUtil.getStringOrNull(ev, "membership"))) {
+            p.process(ev, sender, roomId);
-                log.debug("This is not an invite event, skipping");
-                return;
-            }
-
-            String inviteeId = EventKey.StateKey.getStringOrNull(ev);
-            if (StringUtils.isBlank(inviteeId)) {
-                log.warn("Invalid event: No invitee ID, skipping");
-                return;
-            }
-
-            _MatrixID invitee = MatrixID.asAcceptable(inviteeId);
-            if (!StringUtils.equals(invitee.getDomain(), cfg.getDomain())) {
-                log.debug("Ignoring invite for {}: not a local user");
-                return;
-            }
-
-            log.info("Got invite from {} to {}", senderId, inviteeId);
-
-            boolean wasSent = false;
-            List<_ThreePid> tpids = profiler.getThreepids(invitee).stream()
-                    .filter(tpid -> ThreePidMedium.Email.is(tpid.getMedium()))
-                    .collect(Collectors.toList());
-            log.info("Found {} email(s) in identity store for {}", tpids.size(), inviteeId);
-
-            for (_ThreePid tpid : tpids) {
-                log.info("Found Email to notify about room invitation: {}", tpid.getAddress());
-                Map<String, String> properties = new HashMap<>();
-                profiler.getDisplayName(sender).ifPresent(name -> properties.put("sender_display_name", name));
-                try {
-                    synapse.getRoomName(roomId).ifPresent(name -> properties.put("room_name", name));
-                } catch (RuntimeException e) {
-                    log.warn("Could not fetch room name", e);
-                    log.info("Unable to fetch room name: Did you integrate your Homeserver as documented?");
-                }
-
-                IMatrixIdInvite inv = new MatrixIdInvite(roomId, sender, invitee, tpid.getMedium(), tpid.getAddress(), properties);
-                notif.sendForInvite(inv);
-                log.info("Notification for invite of {} sent to {}", inviteeId, tpid.getAddress());
-                wasSent = true;
-            }
-
-            log.info("Was notification sent? {}", wasSent);
 
             log.debug("Event {}: processing end", evId);
         });

src/main/java/io/kamax/mxisd/as/processor/command/CommandProcessor.java

@@ -0,0 +1,32 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as.processor.command;
+
+import io.kamax.matrix.client._MatrixClient;
+import io.kamax.matrix.hs._MatrixRoom;
+import io.kamax.mxisd.Mxisd;
+import org.apache.commons.cli.CommandLine;
+
+public interface CommandProcessor {
+
+    void process(Mxisd m, _MatrixClient client, _MatrixRoom room, CommandLine cmdLine);
+
+}

src/main/java/io/kamax/mxisd/as/processor/command/InviteCommandProcessor.java

@@ -0,0 +1,117 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as.processor.command;
+
+import io.kamax.matrix.client._MatrixClient;
+import io.kamax.matrix.hs._MatrixRoom;
+import io.kamax.mxisd.Mxisd;
+import io.kamax.mxisd.invitation.IThreePidInviteReply;
+import org.apache.commons.cli.CommandLine;
+import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang.text.StrBuilder;
+
+import java.util.List;
+
+public class InviteCommandProcessor implements CommandProcessor {
+
+    public static final String Command = "invite";
+
+    @Override
+    public void process(Mxisd m, _MatrixClient client, _MatrixRoom room, CommandLine cmdLine) {
+        if (cmdLine.getArgs().length < 2) {
+            room.sendNotice(buildHelp());
+        } else {
+            String arg = cmdLine.getArgList().get(1);
+            String response;
+            if (StringUtils.equals("list", arg)) {
+
+                StrBuilder b = new StrBuilder();
+
+                List<IThreePidInviteReply> invites = m.getInvite().listInvites();
+                if (invites.isEmpty()) {
+                    b.appendln("No invites!");
+                    response = b.toString();
+                } else {
+                    b.appendln("Invites:");
+
+
+                    for (IThreePidInviteReply invite : invites) {
+                        b.appendNewLine().append("ID: ").append(invite.getId());
+                        b.appendNewLine().append("Room: ").append(invite.getInvite().getRoomId());
+                        b.appendNewLine().append("Medium: ").append(invite.getInvite().getMedium());
+                        b.appendNewLine().append("Address: ").append(invite.getInvite().getAddress());
+                        b.appendNewLine();
+                    }
+
+                    response = b.appendNewLine().append("Total: " + invites.size()).toString();
+                }
+            } else if (StringUtils.equals("show", arg)) {
+                if (cmdLine.getArgList().size() < 3) {
+                    response = buildHelp();
+                } else {
+                    String id = cmdLine.getArgList().get(2);
+                    IThreePidInviteReply invite = m.getInvite().getInvite(id);
+                    StrBuilder b = new StrBuilder();
+                    b.appendln("Details for Invitation #" + id);
+                    b.appendNewLine().append("Room: ").append(invite.getInvite().getRoomId());
+                    b.appendNewLine().append("Sender: ").append(invite.getInvite().getSender().toString());
+                    b.appendNewLine().append("Medium: ").append(invite.getInvite().getMedium());
+                    b.appendNewLine().append("Address: ").append(invite.getInvite().getAddress());
+                    b.appendNewLine().append("Display name: ").append(invite.getDisplayName());
+                    b.appendNewLine().appendNewLine().append("Properties:");
+                    invite.getInvite().getProperties().forEach((k, v) -> {
+                        b.appendNewLine().append("\t").append(k).append("=").append(v);
+                    });
+                    b.appendNewLine();
+
+                    response = b.toString();
+                }
+            } else if (StringUtils.equals("revoke", arg)) {
+                if (cmdLine.getArgList().size() < 3) {
+                    response = buildHelp();
+                } else {
+                    m.getInvite().expireInvite(cmdLine.getArgList().get(2));
+                    response = "OK";
+                }
+            } else {
+                response = buildError("Unknown invite action: " + arg, true);
+            }
+
+            room.sendNotice(response);
+        }
+    }
+
+    private String buildError(String message, boolean showHelp) {
+        if (showHelp) {
+            message = message + "\n\n" + buildHelp();
+        }
+
+        return message;
+    }
+
+    private String buildHelp() {
+        return "Available actions:\n\n" +
+                "list - List invites\n" +
+                "show ID - Show detailed info about a specific invite\n" +
+                "revoke ID - Revoke a pending invite by resolving it to the configured Expiration user\n";
+    }
+
+}

src/main/java/io/kamax/mxisd/as/processor/command/LookupCommandProcessor.java

@@ -0,0 +1,78 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as.processor.command;
+
+import io.kamax.matrix.client._MatrixClient;
+import io.kamax.matrix.hs._MatrixRoom;
+import io.kamax.mxisd.Mxisd;
+import io.kamax.mxisd.lookup.SingleLookupReply;
+import org.apache.commons.cli.CommandLine;
+import org.apache.commons.lang.text.StrBuilder;
+import org.apache.commons.lang3.StringUtils;
+
+import java.util.Optional;
+
+public class LookupCommandProcessor implements CommandProcessor {
+
+    public static final String Command = "lookup";
+
+    @Override
+    public void process(Mxisd m, _MatrixClient client, _MatrixRoom room, CommandLine cmdLine) {
+        if (cmdLine.getArgList().size() != 3) {
+            room.sendNotice(getUsage());
+            return;
+        }
+
+        String medium = cmdLine.getArgList().get(1);
+        String address = cmdLine.getArgList().get(2);
+        if (StringUtils.isAnyBlank(medium, address)) {
+            room.sendNotice(getUsage());
+            return;
+        }
+
+        room.sendNotice("Processing...");
+        Optional<SingleLookupReply> r = m.getIdentity().find(medium, address, true);
+        if (!r.isPresent()) {
+            room.sendNotice("No result");
+            return;
+        }
+
+        SingleLookupReply lookup = r.get();
+        StrBuilder b = new StrBuilder();
+        b.append("Result for 3PID lookup of ").append(medium).append(" ").appendln(address).appendNewLine();
+        b.append("Matrix ID: ").appendln(lookup.getMxid().getId());
+        b.appendln("Validity:")
+                .append("  Not Before: ").appendln(lookup.getNotBefore())
+                .append("  Not After: ").appendln(lookup.getNotAfter());
+        b.appendln("Signatures:");
+        lookup.getSignatures().forEach((host, signs) -> {
+            b.append("  ").append(host).appendln(":");
+            signs.forEach((key, sign) -> b.append("    ").append(key).append(" -> ").appendln("OK"));
+        });
+
+        room.sendNotice(b.toString());
+    }
+
+    public String getUsage() {
+        return "lookup MEDIUM ADDRESS";
+    }
+
+}

src/main/java/io/kamax/mxisd/as/processor/command/PingCommandProcessor.java

@@ -0,0 +1,37 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as.processor.command;
+
+import io.kamax.matrix.client._MatrixClient;
+import io.kamax.matrix.hs._MatrixRoom;
+import io.kamax.mxisd.Mxisd;
+import org.apache.commons.cli.CommandLine;
+
+public class PingCommandProcessor implements CommandProcessor {
+
+    public static final String Command = "ping";
+
+    @Override
+    public void process(Mxisd m, _MatrixClient client, _MatrixRoom room, CommandLine cmdLine) {
+        room.sendNotice("Pong!");
+    }
+
+}

src/main/java/io/kamax/mxisd/as/processor/event/EventTypeProcessor.java

@@ -0,0 +1,30 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as.processor.event;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix._MatrixID;
+
+public interface EventTypeProcessor {
+
+    void process(JsonObject ev, _MatrixID sender, String roomId);
+
+}

src/main/java/io/kamax/mxisd/as/processor/event/MembershipEventProcessor.java

@@ -0,0 +1,172 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as.processor.event;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePidMedium;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix._ThreePid;
+import io.kamax.matrix.client.as.MatrixApplicationServiceClient;
+import io.kamax.matrix.event.EventKey;
+import io.kamax.matrix.hs._MatrixRoom;
+import io.kamax.mxisd.Mxisd;
+import io.kamax.mxisd.backend.sql.synapse.Synapse;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.invitation.IMatrixIdInvite;
+import io.kamax.mxisd.invitation.MatrixIdInvite;
+import io.kamax.mxisd.notification.NotificationManager;
+import io.kamax.mxisd.profile.ProfileManager;
+import org.apache.commons.lang.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class MembershipEventProcessor implements EventTypeProcessor {
+
+    private final static Logger log = LoggerFactory.getLogger(MembershipEventProcessor.class);
+
+    private MatrixApplicationServiceClient client;
+
+    private final MxisdConfig cfg;
+    private ProfileManager profiler;
+    private NotificationManager notif;
+    private Synapse synapse;
+
+    public MembershipEventProcessor(
+            MatrixApplicationServiceClient client,
+            Mxisd m
+    ) {
+        this.client = client;
+        this.cfg = m.getConfig();
+        this.profiler = m.getProfile();
+        this.notif = m.getNotif();
+        this.synapse = m.getSynapse();
+    }
+
+    @Override
+    public void process(JsonObject ev, _MatrixID sender, String roomId) {
+        JsonObject content = EventKey.Content.findObj(ev).orElseGet(() -> {
+            log.debug("No content found, falling back to full object");
+            return ev;
+        });
+
+        String targetId = EventKey.StateKey.getStringOrNull(ev);
+        if (StringUtils.isBlank(targetId)) {
+            log.warn("Invalid event: No invitee ID, skipping");
+            return;
+        }
+
+        _MatrixID target = MatrixID.asAcceptable(targetId);
+        if (!StringUtils.equals(target.getDomain(), cfg.getMatrix().getDomain())) {
+            log.debug("Ignoring invite for {}: not a local user");
+            return;
+        }
+
+        log.info("Got membership event from {} to {} for room {}", sender.getId(), targetId, roomId);
+
+        boolean isForMainUser = StringUtils.equals(target.getLocalPart(), cfg.getAppsvc().getUser().getMain());
+        boolean isForExpInvUser = StringUtils.equals(target.getLocalPart(), cfg.getAppsvc().getUser().getInviteExpired());
+        boolean isUs = isForMainUser || isForExpInvUser;
+
+        if (StringUtils.equals("join", EventKey.Membership.getStringOrNull(content))) {
+            if (!isForMainUser) {
+                log.warn("We joined the room {} for another identity as the main user, which is not supported. Leaving...", roomId);
+
+                client.getUser(target.getLocalPart()).getRoom(roomId).tryLeave().ifPresent(err -> {
+                    log.warn("Could not decline invite to room {}: {} - {}", roomId, err.getErrcode(), err.getError());
+                });
+            }
+        } else if (StringUtils.equals("invite", EventKey.Membership.getStringOrNull(content))) {
+            if (isForMainUser) {
+                processForMainUser(roomId, sender);
+            } else if (isForExpInvUser) {
+                processForExpiredInviteUser(roomId, target);
+            } else {
+                processForUserIdInvite(roomId, sender, target);
+            }
+        } else if (StringUtils.equals("leave", EventKey.Membership.getStringOrNull(content))) {
+            _MatrixRoom room = client.getRoom(roomId);
+            if (!isUs && room.getJoinedUsers().size() == 1) {
+                // TODO we need to find out if this is only us remaining and leave the room if so, using the right client for it
+            }
+        } else {
+            log.debug("This is not an supported type of membership event, skipping");
+        }
+    }
+
+    private void processForMainUser(String roomId, _MatrixID sender) {
+        boolean isAllowed = profiler.hasAnyRole(sender, cfg.getAppsvc().getFeature().getAdmin().getAllowedRoles());
+        if (!isAllowed) {
+            log.info("Sender does not have any of the required roles, denying");
+            client.getRoom(roomId).tryLeave().ifPresent(err -> {
+                log.warn("Could not decline invite to room {}: {} - {}", roomId, err.getErrcode(), err.getError());
+            });
+        } else {
+            client.getRoom(roomId).tryJoin().ifPresent(err -> {
+                log.warn("Could not join room {}: {} - {}", roomId, err.getErrcode(), err.getError());
+                client.getRoom(roomId).tryLeave().ifPresent(err1 -> {
+                    log.warn("Could not decline invite to room {} after failed join: {} - {}", roomId, err1.getErrcode(), err1.getError());
+                });
+            });
+        }
+    }
+
+    private void processForExpiredInviteUser(String roomId, _MatrixID invitee) {
+        client.getUser(invitee.getLocalPart()).getRoom(roomId).tryLeave().ifPresent(err -> {
+            log.warn("Could not decline invite to room {}: {} - {}", roomId, err.getErrcode(), err.getError());
+        });
+    }
+
+    private void processForUserIdInvite(String roomId, _MatrixID sender, _MatrixID invitee) {
+        String inviteeId = invitee.getId();
+
+        boolean wasSent = false;
+        List<_ThreePid> tpids = profiler.getThreepids(invitee).stream()
+                .filter(tpid -> ThreePidMedium.Email.is(tpid.getMedium()))
+                .collect(Collectors.toList());
+        log.info("Found {} email(s) in identity store for {}", tpids.size(), inviteeId);
+
+        for (_ThreePid tpid : tpids) {
+            log.info("Found Email to notify about room invitation: {}", tpid.getAddress());
+            Map<String, String> properties = new HashMap<>();
+            profiler.getDisplayName(sender).ifPresent(name -> properties.put("sender_display_name", name));
+            try {
+                synapse.getRoomName(roomId).ifPresent(name -> properties.put("room_name", name));
+            } catch (RuntimeException e) {
+                log.warn("Could not fetch room name", e);
+                log.info("Unable to fetch room name: Did you integrate your Homeserver as documented?");
+            }
+
+            IMatrixIdInvite inv = new MatrixIdInvite(roomId, sender, invitee, tpid.getMedium(), tpid.getAddress(), properties);
+            notif.sendForInvite(inv);
+            log.info("Notification for invite of {} sent to {}", inviteeId, tpid.getAddress());
+            wasSent = true;
+        }
+
+        log.info("Was notification sent? {}", wasSent);
+    }
+
+}

src/main/java/io/kamax/mxisd/as/processor/event/MessageEventProcessor.java

@@ -0,0 +1,127 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as.processor.event;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix._MatrixUserProfile;
+import io.kamax.matrix.client.as.MatrixApplicationServiceClient;
+import io.kamax.matrix.hs._MatrixRoom;
+import io.kamax.matrix.json.event.MatrixJsonRoomMessageEvent;
+import io.kamax.mxisd.Mxisd;
+import io.kamax.mxisd.as.processor.command.CommandProcessor;
+import io.kamax.mxisd.as.processor.command.InviteCommandProcessor;
+import io.kamax.mxisd.as.processor.command.LookupCommandProcessor;
+import io.kamax.mxisd.as.processor.command.PingCommandProcessor;
+import org.apache.commons.cli.*;
+import org.apache.commons.lang.text.StrBuilder;
+import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+public class MessageEventProcessor implements EventTypeProcessor {
+
+    private static final Logger log = LoggerFactory.getLogger(MessageEventProcessor.class);
+
+    private final Mxisd m;
+    private final MatrixApplicationServiceClient client;
+    private Map<String, CommandProcessor> processors;
+
+    public MessageEventProcessor(Mxisd m, MatrixApplicationServiceClient client) {
+        this.m = m;
+        this.client = client;
+
+        processors = new HashMap<>();
+        processors.put("?", (m1, client1, room, cmdLine) -> room.sendNotice(getHelp()));
+        processors.put("help", (m1, client1, room, cmdLine) -> room.sendNotice(getHelp()));
+        processors.put(PingCommandProcessor.Command, new PingCommandProcessor());
+        processors.put(InviteCommandProcessor.Command, new InviteCommandProcessor());
+        processors.put(LookupCommandProcessor.Command, new LookupCommandProcessor());
+    }
+
+    @Override
+    public void process(JsonObject ev, _MatrixID sender, String roomId) {
+        MatrixJsonRoomMessageEvent msgEv = new MatrixJsonRoomMessageEvent(ev);
+        if (StringUtils.equals("m.notice", msgEv.getBodyType())) {
+            log.info("Ignoring automated message");
+            return;
+        }
+
+        _MatrixRoom room = client.getRoom(roomId);
+
+        if (!m.getProfile().hasAnyRole(sender, m.getConfig().getAppsvc().getFeature().getAdmin().getAllowedRoles())) {
+            room.sendNotice("You are not allowed to interact with me.");
+            return;
+        }
+
+        List<_MatrixID> joinedUsers = room.getJoinedUsers().stream().map(_MatrixUserProfile::getId).collect(Collectors.toList());
+        boolean joinedWithMainUser = joinedUsers.contains(client.getWhoAmI());
+        boolean isAdminPrivate = joinedWithMainUser && joinedUsers.size() == 2;
+
+        if (!StringUtils.equals("m.text", msgEv.getBodyType())) {
+            log.info("Unsupported message event type: {}", msgEv.getBodyType());
+            return;
+        }
+
+        String command = msgEv.getBody();
+        if (!isAdminPrivate) {
+            if (!StringUtils.startsWith(command, "!" + Mxisd.Name + " ")) {
+                // Not for us
+                return;
+            }
+
+            command = command.substring(("!" + Mxisd.Name + " ").length());
+        }
+
+        try {
+            CommandLineParser p = new DefaultParser();
+            CommandLine cmdLine = p.parse(new Options(), command.split(" ", 0));
+            String cmd = cmdLine.getArgList().get(0);
+
+            CommandProcessor cp = processors.get(cmd);
+            if (Objects.isNull(cp)) {
+                room.sendNotice("Unknown command: " + command + "\n\n" + getHelp());
+            } else {
+                cp.process(m, client, room, cmdLine);
+            }
+        } catch (ParseException e) {
+            room.sendNotice("Invalid input" + "\n\n" + getHelp());
+        } catch (RuntimeException e) {
+            room.sendNotice("Error when running command: " + e.getMessage());
+        }
+    }
+
+    public String getHelp() {
+        StrBuilder builder = new StrBuilder();
+        builder.appendln("Available commands:");
+        for (String cmd : processors.keySet()) {
+            builder.append("\t").appendln(cmd);
+        }
+        return builder.toString();
+    }
+
+}

src/main/java/io/kamax/mxisd/as/registration/SynapseRegistrationYaml.java

@@ -0,0 +1,176 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as.registration;
+
+import io.kamax.mxisd.config.AppServiceConfig;
+
+import java.net.URL;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+
+public class SynapseRegistrationYaml {
+
+    public static SynapseRegistrationYaml parse(AppServiceConfig cfg, String domain) {
+        SynapseRegistrationYaml yaml = new SynapseRegistrationYaml();
+
+        yaml.setId(cfg.getRegistration().getSynapse().getId());
+        yaml.setUrl(cfg.getEndpoint().getToAS().getUrl());
+        yaml.setAsToken(cfg.getEndpoint().getToHS().getToken());
+        yaml.setHsToken(cfg.getEndpoint().getToAS().getToken());
+        yaml.setSenderLocalpart(cfg.getUser().getMain());
+
+        if (cfg.getFeature().getCleanExpiredInvite()) {
+            Namespace ns = new Namespace();
+            ns.setExclusive(true);
+            ns.setRegex("@" + cfg.getUser().getInviteExpired() + ":" + domain);
+            yaml.getNamespaces().getUsers().add(ns);
+        }
+
+        if (cfg.getFeature().getInviteById()) {
+            Namespace ns = new Namespace();
+            ns.setExclusive(false);
+            ns.setRegex("@*:" + domain);
+            yaml.getNamespaces().getUsers().add(ns);
+        }
+
+        return yaml;
+    }
+
+    public static class Namespace {
+
+        private String regex;
+        private boolean exclusive;
+
+        public String getRegex() {
+            return regex;
+        }
+
+        public void setRegex(String regex) {
+            this.regex = regex;
+        }
+
+        public boolean isExclusive() {
+            return exclusive;
+        }
+
+        public void setExclusive(boolean exclusive) {
+            this.exclusive = exclusive;
+        }
+
+    }
+
+    public static class Namespaces {
+
+        private List<Namespace> users = new ArrayList<>();
+        private List<Namespace> aliases = new ArrayList<>();
+        private List<Namespace> rooms = new ArrayList<>();
+
+        public List<Namespace> getUsers() {
+            return users;
+        }
+
+        public void setUsers(List<Namespace> users) {
+            this.users = users;
+        }
+
+        public List<Namespace> getAliases() {
+            return aliases;
+        }
+
+        public void setAliases(List<Namespace> aliases) {
+            this.aliases = aliases;
+        }
+
+        public List<Namespace> getRooms() {
+            return rooms;
+        }
+
+        public void setRooms(List<Namespace> rooms) {
+            this.rooms = rooms;
+        }
+
+    }
+
+    private String id;
+    private String url;
+    private String as_token;
+    private String hs_token;
+    private String sender_localpart;
+    private Namespaces namespaces = new Namespaces();
+
+    public String getId() {
+        return id;
+    }
+
+    public void setId(String id) {
+        this.id = id;
+    }
+
+    public String getUrl() {
+        return url;
+    }
+
+    public void setUrl(String url) {
+        this.url = url;
+    }
+
+    public void setUrl(URL url) {
+        if (Objects.isNull(url)) {
+            this.url = null;
+        } else {
+            this.url = url.toString();
+        }
+    }
+
+    public String getAsToken() {
+        return as_token;
+    }
+
+    public void setAsToken(String as_token) {
+        this.as_token = as_token;
+    }
+
+    public String getHsToken() {
+        return hs_token;
+    }
+
+    public void setHsToken(String hs_token) {
+        this.hs_token = hs_token;
+    }
+
+    public String getSenderLocalpart() {
+        return sender_localpart;
+    }
+
+    public void setSenderLocalpart(String sender_localpart) {
+        this.sender_localpart = sender_localpart;
+    }
+
+    public Namespaces getNamespaces() {
+        return namespaces;
+    }
+
+    public void setNamespaces(Namespaces namespaces) {
+        this.namespaces = namespaces;
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/exec/ExecAuthStore.java

@@ -44,6 +44,7 @@ public class ExecAuthStore extends ExecStore implements AuthenticatorProvider {
     private ExecConfig.Auth cfg;
 
     public ExecAuthStore(ExecConfig cfg) {
+        super(cfg);
         this.cfg = Objects.requireNonNull(cfg.getAuth());
     }
 

src/main/java/io/kamax/mxisd/backend/exec/ExecDirectoryStore.java

@@ -36,11 +36,12 @@ public class ExecDirectoryStore extends ExecStore implements DirectoryProvider {
     private MatrixConfig mxCfg;
 
     public ExecDirectoryStore(MxisdConfig cfg) {
-        this(cfg.getExec().getDirectory(), cfg.getMatrix());
+        this(cfg.getExec(), cfg.getMatrix());
     }
 
-    public ExecDirectoryStore(ExecConfig.Directory cfg, MatrixConfig mxCfg) {
+    public ExecDirectoryStore(ExecConfig cfg, MatrixConfig mxCfg) {
-        this.cfg = cfg;
+        super(cfg);
+        this.cfg = cfg.getDirectory();
         this.mxCfg = mxCfg;
     }
 

src/main/java/io/kamax/mxisd/backend/exec/ExecIdentityStore.java

@@ -55,11 +55,8 @@ public class ExecIdentityStore extends ExecStore implements IThreePidProvider {
     private final MatrixConfig mxCfg;
 
     public ExecIdentityStore(ExecConfig cfg, MatrixConfig mxCfg) {
-        this(cfg.getIdentity(), mxCfg);
+        super(cfg);
-    }
+        this.cfg = cfg.getIdentity();
-
-    public ExecIdentityStore(ExecConfig.Identity cfg, MatrixConfig mxCfg) {
-        this.cfg = cfg;
         this.mxCfg = mxCfg;
     }
 

src/main/java/io/kamax/mxisd/backend/exec/ExecProfileStore.java

@@ -38,11 +38,8 @@ public class ExecProfileStore extends ExecStore implements ProfileProvider {
     private ExecConfig.Profile cfg;
 
     public ExecProfileStore(ExecConfig cfg) {
-        this(cfg.getProfile());
+        super(cfg);
-    }
+        this.cfg = cfg.getProfile();
-
-    public ExecProfileStore(ExecConfig.Profile cfg) {
-        this.cfg = cfg;
     }
 
     private Optional<JsonProfileResult> getFull(_MatrixID userId, ExecConfig.Process cfg) {

src/main/java/io/kamax/mxisd/backend/exec/ExecStore.java

@@ -43,14 +43,19 @@ public class ExecStore {
     public static final String JsonType = "json";
     public static final String PlainType = "plain";
 
+    private static final Logger log = LoggerFactory.getLogger(ExecStore.class);
+
     protected static String toJson(Object o) {
         return GsonUtil.get().toJson(o);
     }
 
-    private transient final Logger log = LoggerFactory.getLogger(ExecStore.class);
+    private final ExecConfig cfg;
-
     private Supplier<ProcessExecutor> executorSupplier = () -> new ProcessExecutor().readOutput(true);
 
+    public ExecStore(ExecConfig cfg) {
+        this.cfg = cfg;
+    }
+
     public void setExecutorSupplier(Supplier<ProcessExecutor> supplier) {
         executorSupplier = supplier;
     }
@@ -64,7 +69,7 @@ public class ExecStore {
         private Function<String, String> inputUnknownTypeMapper;
         private Map<String, Supplier<String>> inputTypeSuppliers;
 
-        private Map<String, Function<ExecConfig.TokenOverride, String>> inputTypeTemplates;
+        private Map<String, Function<ExecConfig.Token, String>> inputTypeTemplates;
         private Supplier<String> inputTypeNoTemplateHandler;
         private Map<String, Supplier<String>> tokenMappers;
         private Function<String, String> tokenHandler;
@@ -156,11 +161,11 @@ public class ExecStore {
             inputTypeSuppliers.put(type, handler);
         }
 
-        protected void addInputTemplate(String type, Function<ExecConfig.TokenOverride, String> template) {
+        protected void addInputTemplate(String type, Function<ExecConfig.Token, String> template) {
             inputTypeTemplates.put(type, template);
         }
 
-        public void addJsonInputTemplate(Function<ExecConfig.TokenOverride, Object> template) {
+        public void addJsonInputTemplate(Function<ExecConfig.Token, Object> template) {
             inputTypeTemplates.put(JsonType, token -> GsonUtil.get().toJson(template.apply(token)));
         }
 

src/main/java/io/kamax/mxisd/backend/ldap/LdapBackend.java

@@ -72,7 +72,11 @@ public abstract class LdapBackend {
     }
 
     protected synchronized LdapConnection getConn() {
-        return new LdapNetworkConnection(cfg.getConnection().getHost(), cfg.getConnection().getPort(), cfg.getConnection().isTls());
+        return getConn(cfg.getConnection().getHost());
+    }
+
+    protected synchronized LdapConnection getConn(String host) {
+        return new LdapNetworkConnection(host, cfg.getConnection().getPort(), cfg.getConnection().isTls());
     }
 
     protected void bind(LdapConnection conn) throws LdapException {

src/main/java/io/kamax/mxisd/backend/ldap/LdapThreePidProvider.java

@@ -28,6 +28,7 @@ import io.kamax.mxisd.lookup.SingleLookupRequest;
 import io.kamax.mxisd.lookup.ThreePidMapping;
 import io.kamax.mxisd.lookup.provider.IThreePidProvider;
 import io.kamax.mxisd.util.GsonUtil;
+import org.apache.commons.lang.StringUtils;
 import org.apache.directory.api.ldap.model.cursor.CursorException;
 import org.apache.directory.api.ldap.model.cursor.CursorLdapReferralException;
 import org.apache.directory.api.ldap.model.cursor.EntryCursor;
@@ -37,8 +38,10 @@ import org.apache.directory.api.ldap.model.message.SearchScope;
 import org.apache.directory.ldap.client.api.LdapConnection;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
+import org.xbill.DNS.*;
 
 import java.io.IOException;
+import java.net.URI;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Optional;
@@ -91,7 +94,10 @@ public class LdapThreePidProvider extends LdapBackend implements IThreePidProvid
                     return Optional.of(buildMatrixIdFromUid(data.get()));
                 }
             } catch (CursorLdapReferralException e) {
-                log.warn("3PID {} is only available via referral, skipping", value);
+                log.info("Got referral info: {}", e.getReferralInfo());
+
+                return followReferral(medium, value, e.getReferralInfo());
+                //log.warn("3PID {} is only available via referral, skipping", value);
             } catch (IOException | LdapException | CursorException e) {
                 throw new InternalServerError(e);
             }
@@ -104,12 +110,50 @@ public class LdapThreePidProvider extends LdapBackend implements IThreePidProvid
     public Optional<SingleLookupReply> find(SingleLookupRequest request) {
         log.info("Performing LDAP lookup {} of type {}", request.getThreePid(), request.getType());
 
-        try (LdapConnection conn = getConn()) {
+        List<String> hosts = new ArrayList<>();
-            bind(conn);
+
-            return lookup(conn, request.getType(), request.getThreePid()).map(id -> new SingleLookupReply(request, id));
+        String domain = getCfg().getConnection().getDomain();
-        } catch (LdapException | IOException e) {
+        if (StringUtils.isNotBlank(domain)) {
-            throw new InternalServerError(e);
+            try {
+                Record[] records = new Lookup("_ldap._tcp.DomainDnsZones." + domain, Type.SRV).run();
+                if (records == null || records.length == 0) {
+                    log.warn("No LDAP server found for domain {}", domain);
+                    return Optional.empty();
+                }
+
+                for (Record record : records) {
+                    if (record instanceof SRVRecord) {
+                        SRVRecord srvRec = (SRVRecord) record;
+                        hosts.add(srvRec.getTarget().toString(true));
+                    }
+                }
+
+                if (hosts.isEmpty()) {
+                    return Optional.empty();
+                }
+            } catch (TextParseException e) {
+                throw new RuntimeException(e);
+            }
+        } else {
+            hosts.add(getCfg().getConnection().getHost());
         }
+
+        for (String host : hosts) {
+            log.info("Trying host {}", host);
+            try (LdapConnection conn = getConn(host)) {
+                bind(conn);
+                Optional<SingleLookupReply> reply = lookup(conn, request.getType(), request.getThreePid()).map(id -> new SingleLookupReply(request, id));
+                if (reply.isPresent()) return reply;
+            } catch (LdapException | IOException e) {
+                if (hosts.size() == 1) {
+                    throw new InternalServerError(e);
+                } else {
+                    log.warn("Unable to query {}: {}", host, e.getMessage());
+                }
+            }
+        }
+
+        return Optional.empty();
     }
 
     @Override
@@ -137,4 +181,51 @@ public class LdapThreePidProvider extends LdapBackend implements IThreePidProvid
         return mappingsFound;
     }
 
+    private Optional<String> followReferral(String medium, String value, String ref) {
+        URI uri = URI.create(ref);
+
+        Optional<String> tPidQueryOpt = getCfg().getIdentity().getQuery(medium);
+        if (!tPidQueryOpt.isPresent()) {
+            log.warn("{} is not a configured 3PID type for LDAP lookup", medium);
+            return Optional.empty();
+        }
+
+        LdapConnection conn = getConn(uri.getHost());
+        try {
+            bind(conn);
+        } catch (LdapException e) {
+            throw new RuntimeException(e);
+        }
+
+        // we merge 3PID specific query with global/specific filter, if one exists.
+        String tPidQuery = tPidQueryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
+        String searchQuery = buildWithFilter(tPidQuery, getCfg().getIdentity().getFilter());
+        log.debug("Query: {}", searchQuery);
+        log.debug("Attributes: {}", GsonUtil.build().toJson(getUidAtt()));
+
+        try (EntryCursor cursor = conn.search(uri.getPath().substring(1), searchQuery, SearchScope.SUBTREE, getUidAtt())) {
+            while (cursor.next()) {
+                Entry entry = cursor.get();
+                log.info("Found possible match, DN: {}", entry.getDn().getName());
+
+                Optional<String> data = getAttribute(entry, getUidAtt());
+                if (!data.isPresent()) {
+                    continue;
+                }
+
+                log.info("DN {} is a valid match", entry.getDn().getName());
+                return Optional.of(buildMatrixIdFromUid(data.get()));
+            }
+
+            return Optional.empty();
+        } catch (CursorLdapReferralException e) {
+            log.info("Got referral info: {}", e.getReferralInfo());
+
+            return followReferral(medium, value, e.getReferralInfo());
+            //log.warn("3PID {} is only available via referral, skipping", value);
+        } catch (IOException | LdapException | CursorException e) {
+            throw new InternalServerError(e);
+        }
+    }
+
 }

src/main/java/io/kamax/mxisd/backend/rest/RestProfileProvider.java

@@ -32,7 +32,7 @@ import io.kamax.mxisd.profile.JsonProfileRequest;
 import io.kamax.mxisd.profile.JsonProfileResult;
 import io.kamax.mxisd.profile.ProfileProvider;
 import org.apache.commons.io.IOUtils;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang3.StringUtils;
 import org.apache.http.client.methods.CloseableHttpResponse;
 import org.apache.http.client.methods.HttpPost;
 import org.apache.http.client.utils.URIBuilder;
@@ -49,7 +49,7 @@ import java.util.function.Function;
 
 public class RestProfileProvider extends RestProvider implements ProfileProvider {
 
-    private transient final Logger log = LoggerFactory.getLogger(RestProfileProvider.class);
+    private static final Logger log = LoggerFactory.getLogger(RestProfileProvider.class);
 
     public RestProfileProvider(RestBackendConfig cfg) {
         super(cfg);
@@ -60,64 +60,71 @@ public class RestProfileProvider extends RestProvider implements ProfileProvider
             Function<RestBackendConfig.ProfileEndpoints, Optional<String>> endpoint,
             Function<JsonProfileResult, Optional<T>> value
     ) {
-        return cfg.getEndpoints().getProfile()
+        Optional<String> url = endpoint.apply(cfg.getEndpoints().getProfile());
-                // We get the endpoint
+        if (!url.isPresent()) {
-                .flatMap(endpoint)
+            return Optional.empty();
-                // We only continue if there is a value
+        }
-                .filter(StringUtils::isNotBlank)
-                // We use the endpoint
-                .flatMap(url -> {
-                    try {
-                        URIBuilder builder = new URIBuilder(url);
-                        HttpPost req = new HttpPost(builder.build());
-                        req.setEntity(new StringEntity(GsonUtil.get().toJson(new JsonProfileRequest(userId)), ContentType.APPLICATION_JSON));
-                        try (CloseableHttpResponse res = client.execute(req)) {
-                            int sc = res.getStatusLine().getStatusCode();
-                            if (sc == 404) {
-                                log.info("Got 404 - No result found");
-                                return Optional.empty();
-                            }
 
-                            if (sc != 200) {
+        try {
-                                throw new InternalServerError("Unexpected backed status code: " + sc);
+            URIBuilder builder = new URIBuilder(url.get());
-                            }
+            HttpPost req = new HttpPost(builder.build());
+            req.setEntity(new StringEntity(GsonUtil.get().toJson(new JsonProfileRequest(userId)), ContentType.APPLICATION_JSON));
+            try (CloseableHttpResponse res = client.execute(req)) {
+                int sc = res.getStatusLine().getStatusCode();
+                if (sc == 404) {
+                    log.info("Got 404 - No result found");
+                    return Optional.empty();
+                }
 
-                            String body = IOUtils.toString(res.getEntity().getContent(), StandardCharsets.UTF_8);
+                if (sc != 200) {
-                            if (StringUtils.isBlank(body)) {
+                    throw new InternalServerError("Unexpected backed status code: " + sc);
-                                log.warn("Backend response body is empty/blank, expected JSON object with profile key");
+                }
-                                return Optional.empty();
-                            }
 
-                            Optional<JsonObject> pJson = GsonUtil.findObj(GsonUtil.parseObj(body), "profile");
+                String body = IOUtils.toString(res.getEntity().getContent(), StandardCharsets.UTF_8);
-                            if (!pJson.isPresent()) {
+                if (StringUtils.isBlank(body)) {
-                                log.warn("Backend response body is invalid, expected JSON object with profile key");
+                    log.warn("Backend response body is empty/blank, expected JSON object with profile key");
-                                return Optional.empty();
+                    return Optional.empty();
-                            }
+                }
 
-                            JsonProfileResult profile = gson.fromJson(pJson.get(), JsonProfileResult.class);
+                Optional<JsonObject> pJson = GsonUtil.findObj(GsonUtil.parseObj(body), "profile");
-                            return value.apply(profile);
+                if (!pJson.isPresent()) {
-                        }
+                    log.warn("Backend response body is invalid, expected JSON object with profile key");
-                    } catch (JsonSyntaxException | InvalidJsonException e) {
+                    return Optional.empty();
-                        log.error("Unable to parse backend response as JSON", e);
+                }
-                        throw new InternalServerError(e);
+
-                    } catch (URISyntaxException e) {
+                JsonProfileResult profile = gson.fromJson(pJson.get(), JsonProfileResult.class);
-                        log.error("Unable to build a valid request URL", e);
+                return value.apply(profile);
-                        throw new InternalServerError(e);
+            }
-                    } catch (IOException e) {
+        } catch (JsonSyntaxException | InvalidJsonException e) {
-                        log.error("I/O Error during backend request", e);
+            log.error("Unable to parse backend response as JSON", e);
-                        throw new InternalServerError();
+            throw new InternalServerError(e);
-                    }
+        } catch (URISyntaxException e) {
-                });
+            log.error("Unable to build a valid request URL", e);
+            throw new InternalServerError(e);
+        } catch (IOException e) {
+            log.error("I/O Error during backend request", e);
+            throw new InternalServerError();
+        }
     }
 
     @Override
     public Optional<String> getDisplayName(_MatrixID userId) {
-        return doRequest(userId, p -> Optional.ofNullable(p.getDisplayName()), profile -> Optional.ofNullable(profile.getDisplayName()));
+        return doRequest(userId, p -> {
+            if (StringUtils.isBlank(p.getDisplayName())) {
+                return Optional.empty();
+            }
+            return Optional.ofNullable(p.getDisplayName());
+        }, profile -> Optional.ofNullable(profile.getDisplayName()));
     }
 
     @Override
     public List<_ThreePid> getThreepids(_MatrixID userId) {
-        return doRequest(userId, p -> Optional.ofNullable(p.getThreepids()), profile -> {
+        return doRequest(userId, p -> {
+            if (StringUtils.isBlank(p.getThreepids())) {
+                return Optional.empty();
+            }
+            return Optional.ofNullable(p.getThreepids());
+        }, profile -> {
             List<_ThreePid> t = new ArrayList<>();
             if (Objects.nonNull(profile.getThreepids())) {
                 t.addAll(profile.getThreepids());
@@ -128,7 +135,12 @@ public class RestProfileProvider extends RestProvider implements ProfileProvider
 
     @Override
     public List<String> getRoles(_MatrixID userId) {
-        return doRequest(userId, p -> Optional.ofNullable(p.getRoles()), profile -> {
+        return doRequest(userId, p -> {
+            if (StringUtils.isBlank(p.getRoles())) {
+                return Optional.empty();
+            }
+            return Optional.ofNullable(p.getRoles());
+        }, profile -> {
             List<String> t = new ArrayList<>();
             if (Objects.nonNull(profile.getRoles())) {
                 t.addAll(profile.getRoles());

src/main/java/io/kamax/mxisd/backend/sql/SqlProfileProvider.java

@@ -23,7 +23,9 @@ package io.kamax.mxisd.backend.sql;
 import io.kamax.matrix.ThreePid;
 import io.kamax.matrix._MatrixID;
 import io.kamax.matrix._ThreePid;
+import io.kamax.mxisd.UserIdType;
 import io.kamax.mxisd.config.sql.SqlConfig;
+import io.kamax.mxisd.exception.InternalServerError;
 import io.kamax.mxisd.profile.ProfileProvider;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -33,16 +35,14 @@ import java.sql.PreparedStatement;
 import java.sql.ResultSet;
 import java.sql.SQLException;
 import java.util.ArrayList;
-import java.util.Collections;
 import java.util.List;
 import java.util.Optional;
 
 public abstract class SqlProfileProvider implements ProfileProvider {
 
-    private transient final Logger log = LoggerFactory.getLogger(SqlProfileProvider.class);
+    private static final Logger log = LoggerFactory.getLogger(SqlProfileProvider.class);
 
     private SqlConfig.Profile cfg;
-
     private SqlConnectionPool pool;
 
     public SqlProfileProvider(SqlConfig cfg) {
@@ -50,6 +50,12 @@ public abstract class SqlProfileProvider implements ProfileProvider {
         this.pool = new SqlConnectionPool(cfg);
     }
 
+    private void setParameters(PreparedStatement stmt, String value) throws SQLException {
+        for (int i = 1; i <= stmt.getParameterMetaData().getParameterCount(); i++) {
+            stmt.setString(i, value);
+        }
+    }
+
     @Override
     public Optional<String> getDisplayName(_MatrixID user) {
         String stmtSql = cfg.getDisplayName().getQuery();
@@ -94,7 +100,33 @@ public abstract class SqlProfileProvider implements ProfileProvider {
 
     @Override
     public List<String> getRoles(_MatrixID user) {
-        return Collections.emptyList();
+        log.info("Querying roles for {}", user.getId());
+
+        List<String> roles = new ArrayList<>();
+
+        String stmtSql = cfg.getRole().getQuery();
+        try (Connection conn = pool.get()) {
+            PreparedStatement stmt = conn.prepareStatement(stmtSql);
+            if (UserIdType.Localpart.is(cfg.getRole().getType())) {
+                setParameters(stmt, user.getLocalPart());
+            } else if (UserIdType.MatrixID.is(cfg.getRole().getType())) {
+                setParameters(stmt, user.getId());
+            } else {
+                throw new InternalServerError("Unsupported user type in SQL Role fetching: " + cfg.getRole().getType());
+            }
+
+            ResultSet rSet = stmt.executeQuery();
+            while (rSet.next()) {
+                String role = rSet.getString(1);
+                roles.add(role);
+                log.debug("Found role {}", role);
+            }
+
+            log.info("Got {} roles", roles.size());
+            return roles;
+        } catch (SQLException e) {
+            throw new RuntimeException(e);
+        }
     }
 
 }

src/main/java/io/kamax/mxisd/backend/sql/generic/GenericSqlStoreSupplier.java

@@ -32,7 +32,7 @@ public class GenericSqlStoreSupplier implements IdentityStoreSupplier {
     @Override
     public void accept(Mxisd mxisd) {
         if (mxisd.getConfig().getSql().getAuth().isEnabled()) {
-            AuthProviders.register(() -> new GenericSqlAuthProvider(mxisd.getConfig().getSql(), mxisd.getInvitationManager()));
+            AuthProviders.register(() -> new GenericSqlAuthProvider(mxisd.getConfig().getSql(), mxisd.getInvite()));
         }
 
         if (mxisd.getConfig().getSql().getDirectory().isEnabled()) {

src/main/java/io/kamax/mxisd/backend/sql/synapse/SynapseQueries.java

@@ -43,6 +43,10 @@ public class SynapseQueries {
         return "SELECT medium, address FROM user_threepids WHERE user_id = ?";
     }
 
+    public static String getRoles() {
+        return "SELECT DISTINCT(group_id) FROM group_users WHERE user_id = ?";
+    }
+
     public static String findByDisplayName(String type, String domain) {
         if (StringUtils.equals("sqlite", type)) {
             return "select " + getUserId(type, domain) + ", displayname from profiles p where displayname like ?";

src/main/java/io/kamax/mxisd/backend/sql/synapse/SynapseSqlStoreSupplier.java

@@ -26,9 +26,13 @@ import io.kamax.mxisd.config.MxisdConfig;
 import io.kamax.mxisd.directory.DirectoryProviders;
 import io.kamax.mxisd.lookup.ThreePidProviders;
 import io.kamax.mxisd.profile.ProfileProviders;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 public class SynapseSqlStoreSupplier implements IdentityStoreSupplier {
 
+    private static final Logger log = LoggerFactory.getLogger(SynapseSqlStoreSupplier.class);
+
     @Override
     public void accept(Mxisd mxisd) {
         accept(mxisd.getConfig());
@@ -44,6 +48,7 @@ public class SynapseSqlStoreSupplier implements IdentityStoreSupplier {
         }
 
         if (cfg.getSynapseSql().getProfile().isEnabled()) {
+            log.debug("Profile is enabled, registering provider");
             ProfileProviders.register(() -> new SynapseSqlProfileProvider(cfg.getSynapseSql()));
         }
     }

src/main/java/io/kamax/mxisd/config/AppServiceConfig.java

@@ -0,0 +1,287 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config;
+
+import io.kamax.mxisd.Mxisd;
+import io.kamax.mxisd.exception.ConfigurationException;
+
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+
+public class AppServiceConfig {
+
+    public static class Users {
+
+        private String main = "mxisd";
+        private String inviteExpired = "_mxisd_invite-expired";
+
+        public String getMain() {
+            return main;
+        }
+
+        public void setMain(String main) {
+            this.main = main;
+        }
+
+        public String getInviteExpired() {
+            return inviteExpired;
+        }
+
+        public void setInviteExpired(String inviteExpired) {
+            this.inviteExpired = inviteExpired;
+        }
+
+        public void build() {
+            // no-op
+        }
+
+    }
+
+    public static class Endpoint {
+
+        private String url;
+        private String token;
+
+        private transient URL cUrl;
+
+        public URL getUrl() {
+            return cUrl;
+        }
+
+        public void setUrl(String url) {
+            this.url = url;
+        }
+
+        public String getToken() {
+            return token;
+        }
+
+        public void setToken(String token) {
+            this.token = token;
+        }
+
+        public void build() {
+            if (Objects.isNull(url)) {
+                return;
+            }
+
+            try {
+                cUrl = new URL(url);
+            } catch (MalformedURLException e) {
+                throw new ConfigurationException("AppService endpoint(s) URL definition");
+            }
+        }
+
+    }
+
+    public static class Endpoints {
+
+        private Endpoint toAS = new Endpoint();
+        private Endpoint toHS = new Endpoint();
+
+        public Endpoint getToAS() {
+            return toAS;
+        }
+
+        public void setToAS(Endpoint toAS) {
+            this.toAS = toAS;
+        }
+
+        public Endpoint getToHS() {
+            return toHS;
+        }
+
+        public void setToHS(Endpoint toHS) {
+            this.toHS = toHS;
+        }
+
+        public void build() {
+            toAS.build();
+            toHS.build();
+        }
+
+    }
+
+    public static class Synapse {
+
+        private String id = "appservice-" + Mxisd.Name;
+        private String file;
+
+        public String getId() {
+            return id;
+        }
+
+        public void setId(String id) {
+            this.id = id;
+        }
+
+        public String getFile() {
+            return file;
+        }
+
+        public void setFile(String file) {
+            this.file = file;
+        }
+
+        public void build() {
+            // no-op
+        }
+
+    }
+
+    public static class Registration {
+
+        private Synapse synapse = new Synapse();
+
+        public Synapse getSynapse() {
+            return synapse;
+        }
+
+        public void setSynapse(Synapse synapse) {
+            this.synapse = synapse;
+        }
+
+        public void build() {
+            synapse.build();
+        }
+
+    }
+
+    public static class AdminFeature {
+
+        private Boolean enabled;
+        private List<String> allowedRoles = new ArrayList<>();
+
+        public Boolean getEnabled() {
+            return enabled;
+        }
+
+        public void setEnabled(Boolean enabled) {
+            this.enabled = enabled;
+        }
+
+        public List<String> getAllowedRoles() {
+            return allowedRoles;
+        }
+
+        public void setAllowedRoles(List<String> allowedRoles) {
+            this.allowedRoles = allowedRoles;
+        }
+
+        public void build() {
+            // no-op
+        }
+
+    }
+
+    public static class Features {
+
+        private AdminFeature admin = new AdminFeature();
+        private Boolean inviteById;
+        private Boolean cleanExpiredInvite;
+
+        public AdminFeature getAdmin() {
+            return admin;
+        }
+
+        public void setAdmin(AdminFeature admin) {
+            this.admin = admin;
+        }
+
+        public Boolean getInviteById() {
+            return inviteById;
+        }
+
+        public void setInviteById(Boolean inviteById) {
+            this.inviteById = inviteById;
+        }
+
+        public Boolean getCleanExpiredInvite() {
+            return cleanExpiredInvite;
+        }
+
+        public void setCleanExpiredInvite(Boolean cleanExpiredInvite) {
+            this.cleanExpiredInvite = cleanExpiredInvite;
+        }
+
+        public void build() {
+            admin.build();
+        }
+
+    }
+
+    private Boolean enabled;
+    private Features feature = new Features();
+    private Endpoints endpoint = new Endpoints();
+    private Registration registration = new Registration();
+    private Users user = new Users();
+
+    public Boolean isEnabled() {
+        return enabled;
+    }
+
+    public void setEnabled(boolean enabled) {
+        this.enabled = enabled;
+    }
+
+    public Features getFeature() {
+        return feature;
+    }
+
+    public void setFeature(Features feature) {
+        this.feature = feature;
+    }
+
+    public Endpoints getEndpoint() {
+        return endpoint;
+    }
+
+    public void setEndpoint(Endpoints endpoint) {
+        this.endpoint = endpoint;
+    }
+
+    public Registration getRegistration() {
+        return registration;
+    }
+
+    public void setRegistration(Registration registration) {
+        this.registration = registration;
+    }
+
+    public Users getUser() {
+        return user;
+    }
+
+    public void setUser(Users user) {
+        this.user = user;
+    }
+
+    public void build() {
+        endpoint.build();
+        feature.build();
+        registration.build();
+        user.build();
+    }
+
+}

src/main/java/io/kamax/mxisd/config/DirectoryConfig.java

@@ -36,9 +36,8 @@ public class DirectoryConfig {
             return homeserver;
         }
 
-        public Exclude setHomeserver(boolean homeserver) {
+        public void setHomeserver(boolean homeserver) {
             this.homeserver = homeserver;
-            return this;
         }
 
         public boolean getThreepid() {

src/main/java/io/kamax/mxisd/config/ExecConfig.java

@@ -20,13 +20,11 @@
 
 package io.kamax.mxisd.config;
 
-import org.apache.commons.lang3.StringUtils;
-
 import java.util.*;
 
 public class ExecConfig {
 
-    public class IO {
+    public static class IO {
 
         private String type;
         private String template;
@@ -49,7 +47,7 @@ public class ExecConfig {
 
     }
 
-    public class Exit {
+    public static class Exit {
 
         private List<Integer> success = Collections.singletonList(0);
         private List<Integer> failure = Collections.singletonList(1);
@@ -72,84 +70,7 @@ public class ExecConfig {
 
     }
 
-    public class TokenOverride {
+    public static class Token {
-
-        private String localpart;
-        private String domain;
-        private String mxid;
-        private String password;
-        private String medium;
-        private String address;
-        private String type;
-        private String query;
-
-        public String getLocalpart() {
-            return StringUtils.defaultIfEmpty(localpart, getToken().getLocalpart());
-        }
-
-        public void setLocalpart(String localpart) {
-            this.localpart = localpart;
-        }
-
-        public String getDomain() {
-            return StringUtils.defaultIfEmpty(domain, getToken().getDomain());
-        }
-
-        public void setDomain(String domain) {
-            this.domain = domain;
-        }
-
-        public String getMxid() {
-            return StringUtils.defaultIfEmpty(mxid, getToken().getMxid());
-        }
-
-        public void setMxid(String mxid) {
-            this.mxid = mxid;
-        }
-
-        public String getPassword() {
-            return StringUtils.defaultIfEmpty(password, getToken().getPassword());
-        }
-
-        public void setPassword(String password) {
-            this.password = password;
-        }
-
-        public String getMedium() {
-            return StringUtils.defaultIfEmpty(medium, getToken().getMedium());
-        }
-
-        public void setMedium(String medium) {
-            this.medium = medium;
-        }
-
-        public String getAddress() {
-            return StringUtils.defaultIfEmpty(address, getToken().getAddress());
-        }
-
-        public void setAddress(String address) {
-            this.address = address;
-        }
-
-        public String getType() {
-            return StringUtils.defaultIfEmpty(type, getToken().getType());
-        }
-
-        public void setType(String type) {
-            this.type = type;
-        }
-
-        public String getQuery() {
-            return StringUtils.defaultIfEmpty(query, getToken().getQuery());
-        }
-
-        public void setQuery(String query) {
-            this.query = query;
-        }
-
-    }
-
-    public class Token {
 
         private String localpart = "{localpart}";
         private String domain = "{domain}";
@@ -226,9 +147,9 @@ public class ExecConfig {
 
     }
 
-    public class Process {
+    public static class Process {
 
-        private TokenOverride token = new TokenOverride();
+        private Token token = new Token();
         private String command;
 
         private List<String> args = new ArrayList<>();
@@ -238,11 +159,11 @@ public class ExecConfig {
         private Exit exit = new Exit();
         private IO output = new IO();
 
-        public TokenOverride getToken() {
+        public Token getToken() {
             return token;
         }
 
-        public void setToken(TokenOverride token) {
+        public void setToken(Token token) {
             this.token = token;
         }
 
@@ -300,7 +221,7 @@ public class ExecConfig {
 
     }
 
-    public class Auth extends Process {
+    public static class Auth extends Process {
 
         private Boolean enabled;
 
@@ -314,9 +235,9 @@ public class ExecConfig {
 
     }
 
-    public class Directory {
+    public static class Directory {
 
-        public class Search {
+        public static class Search {
 
             private Process byName = new Process();
             private Process byThreepid = new Process();
@@ -360,7 +281,7 @@ public class ExecConfig {
 
     }
 
-    public class Lookup {
+    public static class Lookup {
 
         private Process single = new Process();
         private Process bulk = new Process();
@@ -383,7 +304,7 @@ public class ExecConfig {
 
     }
 
-    public class Identity {
+    public static class Identity {
 
         private Boolean enabled;
         private int priority;
@@ -415,7 +336,7 @@ public class ExecConfig {
 
     }
 
-    public class Profile {
+    public static class Profile {
 
         private Boolean enabled;
         private Process displayName = new Process();

src/main/java/io/kamax/mxisd/config/InvitationConfig.java

@@ -20,18 +20,53 @@
 
 package io.kamax.mxisd.config;
 
-import io.kamax.mxisd.util.GsonUtil;
+import io.kamax.matrix.json.GsonUtil;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import java.util.ArrayList;
+import java.util.List;
+
 public class InvitationConfig {
 
-    private transient final Logger log = LoggerFactory.getLogger(InvitationConfig.class);
+    private static final Logger log = LoggerFactory.getLogger(InvitationConfig.class);
+
+    public static class Expiration {
+
+        private Boolean enabled;
+        private long after;
+        private String resolveTo;
+
+        public Boolean isEnabled() {
+            return enabled;
+        }
+
+        public void setEnabled(boolean enabled) {
+            this.enabled = enabled;
+        }
+
+        public long getAfter() {
+            return after;
+        }
+
+        public void setAfter(long after) {
+            this.after = after;
+        }
+
+        public String getResolveTo() {
+            return resolveTo;
+        }
+
+        public void setResolveTo(String resolveTo) {
+            this.resolveTo = resolveTo;
+        }
+
+    }
 
     public static class Resolution {
 
         private boolean recursive = true;
-        private long timer = 1;
+        private long timer = 5;
 
         public boolean isRecursive() {
             return recursive;
@@ -51,7 +86,43 @@ public class InvitationConfig {
 
     }
 
+    public static class SenderPolicy {
+
+        private List<String> hasRole = new ArrayList<>();
+
+        public List<String> getHasRole() {
+            return hasRole;
+        }
+
+        public void setHasRole(List<String> hasRole) {
+            this.hasRole = hasRole;
+        }
+    }
+
+    public static class Policies {
+
+        private SenderPolicy ifSender = new SenderPolicy();
+
+        public SenderPolicy getIfSender() {
+            return ifSender;
+        }
+
+        public void setIfSender(SenderPolicy ifSender) {
+            this.ifSender = ifSender;
+        }
+    }
+
+    private Expiration expiration = new Expiration();
     private Resolution resolution = new Resolution();
+    private Policies policy = new Policies();
+
+    public Expiration getExpiration() {
+        return expiration;
+    }
+
+    public void setExpiration(Expiration expiration) {
+        this.expiration = expiration;
+    }
 
     public Resolution getResolution() {
         return resolution;
@@ -61,9 +132,19 @@ public class InvitationConfig {
         this.resolution = resolution;
     }
 
+    public Policies getPolicy() {
+        return policy;
+    }
+
+    public void setPolicy(Policies policy) {
+        this.policy = policy;
+    }
+
     public void build() {
         log.info("--- Invite config ---");
-        log.info("Resolution: {}", GsonUtil.build().toJson(resolution));
+        log.info("Expiration: {}", GsonUtil.get().toJson(getExpiration()));
+        log.info("Resolution: {}", GsonUtil.get().toJson(getResolution()));
+        log.info("Policies: {}", GsonUtil.get().toJson(getPolicy()));
     }
 
 }

src/main/java/io/kamax/mxisd/config/ListenerConfig.java

@@ -1,107 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2018 Kamax Sarl
- *
- * https://www.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config;
-
-import io.kamax.mxisd.exception.ConfigurationException;
-import org.apache.commons.lang.StringUtils;
-
-import java.net.MalformedURLException;
-import java.net.URL;
-
-public class ListenerConfig {
-
-    public static class Token {
-
-        private String as;
-        private String hs;
-
-        public String getAs() {
-            return as;
-        }
-
-        public void setAs(String as) {
-            this.as = as;
-        }
-
-        public String getHs() {
-            return hs;
-        }
-
-        public void setHs(String hs) {
-            this.hs = hs;
-        }
-
-    }
-
-    private transient URL csUrl;
-    private String url;
-    private String localpart;
-    private Token token = new Token();
-
-    public URL getUrl() {
-        return csUrl;
-    }
-
-    public void setUrl(String url) {
-        this.url = url;
-    }
-
-    public String getLocalpart() {
-        return localpart;
-    }
-
-    public void setLocalpart(String localpart) {
-        this.localpart = localpart;
-    }
-
-    public Token getToken() {
-        return token;
-    }
-
-    public void setToken(Token token) {
-        this.token = token;
-    }
-
-    public void build() {
-        try {
-            if (StringUtils.isBlank(url)) {
-                return;
-            }
-
-            csUrl = new URL(url);
-
-            if (StringUtils.isBlank(getLocalpart())) {
-                throw new IllegalArgumentException("localpart for matrix listener is not set");
-            }
-
-            if (StringUtils.isBlank(getToken().getAs())) {
-                throw new IllegalArgumentException("AS token is not set");
-            }
-
-            if (StringUtils.isBlank(getToken().getHs())) {
-                throw new IllegalArgumentException("HS token is not set");
-            }
-        } catch (MalformedURLException e) {
-            throw new ConfigurationException(e);
-        }
-    }
-
-}

src/main/java/io/kamax/mxisd/config/MatrixConfig.java

@@ -63,7 +63,6 @@ public class MatrixConfig {
 
     private String domain;
     private Identity identity = new Identity();
-    private ListenerConfig listener = new ListenerConfig();
 
     public String getDomain() {
         return domain;
@@ -81,14 +80,6 @@ public class MatrixConfig {
         this.identity = identity;
     }
 
-    public ListenerConfig getListener() {
-        return listener;
-    }
-
-    public void setListener(ListenerConfig listener) {
-        this.listener = listener;
-    }
-
     public void build() {
         log.info("--- Matrix config ---");
 
@@ -99,8 +90,6 @@ public class MatrixConfig {
         log.info("Domain: {}", getDomain());
         log.info("Identity:");
         log.info("\tServers: {}", GsonUtil.get().toJson(identity.getServers()));
-
-        listener.build();
     }
 
 }

src/main/java/io/kamax/mxisd/config/MxisdConfig.java

@@ -83,6 +83,7 @@ public class MxisdConfig {
 
     }
 
+    private AppServiceConfig appsvc = new AppServiceConfig();
     private AuthenticationConfig auth = new AuthenticationConfig();
     private DirectoryConfig directory = new DirectoryConfig();
     private Dns dns = new Dns();
@@ -97,6 +98,7 @@ public class MxisdConfig {
     private MemoryStoreConfig memory = new MemoryStoreConfig();
     private NotificationConfig notification = new NotificationConfig();
     private NetIqLdapConfig netiq = new NetIqLdapConfig();
+    private RegisterConfig register = new RegisterConfig();
     private ServerConfig server = new ServerConfig();
     private SessionConfig session = new SessionConfig();
     private StorageConfig storage = new StorageConfig();
@@ -107,6 +109,14 @@ public class MxisdConfig {
     private ViewConfig view = new ViewConfig();
     private WordpressConfig wordpress = new WordpressConfig();
 
+    public AppServiceConfig getAppsvc() {
+        return appsvc;
+    }
+
+    public void setAppsvc(AppServiceConfig appsvc) {
+        this.appsvc = appsvc;
+    }
+
     public AuthenticationConfig getAuth() {
         return auth;
     }
@@ -219,6 +229,14 @@ public class MxisdConfig {
         this.netiq = netiq;
     }
 
+    public RegisterConfig getRegister() {
+        return register;
+    }
+
+    public void setRegister(RegisterConfig register) {
+        this.register = register;
+    }
+
     public ServerConfig getServer() {
         return server;
     }
@@ -297,6 +315,7 @@ public class MxisdConfig {
             log.debug("server.name is empty, using matrix.domain");
         }
 
+        getAppsvc().build();
         getAuth().build();
         getDirectory().build();
         getExec().build();
@@ -310,6 +329,7 @@ public class MxisdConfig {
         getMemory().build();
         getNetiq().build();
         getNotification().build();
+        getRegister().build();
         getRest().build();
         getSession().build();
         getServer().build();

src/main/java/io/kamax/mxisd/config/RegisterConfig.java

@@ -0,0 +1,201 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config;
+
+import io.kamax.matrix.ThreePidMedium;
+import io.kamax.matrix.json.GsonUtil;
+import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.*;
+import java.util.stream.Collectors;
+
+public class RegisterConfig {
+
+    private static final Logger log = LoggerFactory.getLogger(RegisterConfig.class);
+
+    public static class ThreepidPolicyPattern {
+
+        private List<String> blacklist = new ArrayList<>();
+        private List<String> whitelist = new ArrayList<>();
+
+        public List<String> getBlacklist() {
+            return blacklist;
+        }
+
+        public void setBlacklist(List<String> blacklist) {
+            this.blacklist = blacklist;
+        }
+
+        public List<String> getWhitelist() {
+            return whitelist;
+        }
+
+        public void setWhitelist(List<String> whitelist) {
+            this.whitelist = whitelist;
+        }
+
+    }
+
+    public static class EmailPolicy extends ThreepidPolicy {
+
+        private ThreepidPolicyPattern domain = new ThreepidPolicyPattern();
+
+        public ThreepidPolicyPattern getDomain() {
+            return domain;
+        }
+
+        public void setDomain(ThreepidPolicyPattern domain) {
+            this.domain = domain;
+        }
+
+        private List<String> buildPatterns(List<String> domains) {
+            log.info("Building email policy");
+            return domains.stream().map(d -> {
+                if (StringUtils.startsWith(d, "*")) {
+                    log.info("Found domain and subdomain policy");
+                    d = "(.*)" + d.substring(1);
+                } else if (StringUtils.startsWith(d, ".")) {
+                    log.info("Found subdomain-only policy");
+                    d = "(.*)" + d;
+                } else {
+                    log.info("Found domain-only policy");
+                }
+
+                return "([^@]+)@" + d.replace(".", "\\.");
+            }).collect(Collectors.toList());
+        }
+
+        @Override
+        public void build() {
+            if (Objects.isNull(getDomain())) {
+                return;
+            }
+
+            if (Objects.nonNull(getDomain().getBlacklist())) {
+                if (Objects.isNull(getPattern().getBlacklist())) {
+                    getPattern().setBlacklist(new ArrayList<>());
+                }
+
+                List<String> domains = buildPatterns(getDomain().getBlacklist());
+                getPattern().getBlacklist().addAll(domains);
+            }
+
+            if (Objects.nonNull(getDomain().getWhitelist())) {
+                if (Objects.isNull(getPattern().getWhitelist())) {
+                    getPattern().setWhitelist(new ArrayList<>());
+                }
+
+                List<String> domains = buildPatterns(getDomain().getWhitelist());
+                getPattern().getWhitelist().addAll(domains);
+            }
+
+            setDomain(null);
+        }
+
+    }
+
+    public static class ThreepidPolicy {
+
+        private ThreepidPolicyPattern pattern = new ThreepidPolicyPattern();
+
+        public ThreepidPolicyPattern getPattern() {
+            return pattern;
+        }
+
+        public void setPattern(ThreepidPolicyPattern pattern) {
+            this.pattern = pattern;
+        }
+
+        public void build() {
+            // no-op
+        }
+
+    }
+
+    public static class Policy {
+
+        private boolean allowed;
+        private boolean invite = true;
+        private Map<String, Object> threepid = new HashMap<>();
+
+        public boolean isAllowed() {
+            return allowed;
+        }
+
+        public void setAllowed(boolean allowed) {
+            this.allowed = allowed;
+        }
+
+        public boolean forInvite() {
+            return invite;
+        }
+
+        public void setInvite(boolean invite) {
+            this.invite = invite;
+        }
+
+        public Map<String, Object> getThreepid() {
+            return threepid;
+        }
+
+        public void setThreepid(Map<String, Object> threepid) {
+            this.threepid = threepid;
+        }
+
+    }
+
+    private Policy policy = new Policy();
+
+    public Policy getPolicy() {
+        return policy;
+    }
+
+    public void setPolicy(Policy policy) {
+        this.policy = policy;
+    }
+
+    public void build() {
+        log.info("--- Registration config ---");
+
+        log.info("Before Build");
+        log.info(GsonUtil.getPrettyForLog(this));
+
+        new HashMap<>(getPolicy().getThreepid()).forEach((medium, policy) -> {
+            if (ThreePidMedium.Email.is(medium)) {
+                EmailPolicy pPolicy = GsonUtil.get().fromJson(GsonUtil.get().toJson(policy), EmailPolicy.class);
+                pPolicy.build();
+                policy = GsonUtil.makeObj(pPolicy);
+            } else {
+                ThreepidPolicy pPolicy = GsonUtil.get().fromJson(GsonUtil.get().toJson(policy), ThreepidPolicy.class);
+                pPolicy.build();
+                policy = GsonUtil.makeObj(pPolicy);
+            }
+
+            getPolicy().getThreepid().put(medium, policy);
+        });
+
+        log.info("After Build");
+        log.info(GsonUtil.getPrettyForLog(this));
+    }
+
+}

src/main/java/io/kamax/mxisd/config/YamlConfigLoader.java

@@ -21,12 +21,16 @@
 package io.kamax.mxisd.config;
 
 import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.exception.ConfigurationException;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.yaml.snakeyaml.Yaml;
 import org.yaml.snakeyaml.constructor.Constructor;
+import org.yaml.snakeyaml.introspector.BeanAccess;
+import org.yaml.snakeyaml.parser.ParserException;
 import org.yaml.snakeyaml.representer.Representer;
 
+import java.io.File;
 import java.io.FileInputStream;
 import java.io.FileNotFoundException;
 import java.io.IOException;
@@ -37,17 +41,26 @@ public class YamlConfigLoader {
     private static final Logger log = LoggerFactory.getLogger(YamlConfigLoader.class);
 
     public static MxisdConfig loadFromFile(String path) throws IOException {
-        log.debug("Reading config from {}", path);
+        File f = new File(path).getAbsoluteFile();
+        log.info("Reading config from {}", f.toString());
         Representer rep = new Representer();
+        rep.getPropertyUtils().setBeanAccess(BeanAccess.FIELD);
         rep.getPropertyUtils().setAllowReadOnlyProperties(true);
         rep.getPropertyUtils().setSkipMissingProperties(true);
         Yaml yaml = new Yaml(new Constructor(MxisdConfig.class), rep);
-        try (FileInputStream is = new FileInputStream(path)) {
+        try (FileInputStream is = new FileInputStream(f)) {
-            Object o = yaml.load(is);
+            MxisdConfig raw = yaml.load(is);
             log.debug("Read config in memory from {}", path);
-            MxisdConfig cfg = GsonUtil.get().fromJson(GsonUtil.get().toJson(o), MxisdConfig.class);
+
+            // SnakeYaml set objects to null when there is no value set in the config, even a full sub-tree.
+            // This is problematic for default config values and objects, to avoid NPEs.
+            // Therefore, we'll use Gson to re-parse the data in a way that avoids us checking the whole config for nulls.
+            MxisdConfig cfg = GsonUtil.get().fromJson(GsonUtil.get().toJson(raw), MxisdConfig.class);
+
             log.info("Loaded config from {}", path);
             return cfg;
+        } catch (ParserException t) {
+            throw new ConfigurationException(t.getMessage(), "Could not parse YAML config file - Please check indentation and that the configuration options exist");
         }
     }
 

src/main/java/io/kamax/mxisd/config/ldap/LdapConfig.java

@@ -125,6 +125,7 @@ public abstract class LdapConfig {
 
         private boolean tls = false;
         private String host;
+        private String domain;
         private int port = 389;
         private String bindDn;
         private String bindPassword;
@@ -147,6 +148,14 @@ public abstract class LdapConfig {
             this.host = host;
         }
 
+        public String getDomain() {
+            return domain;
+        }
+
+        public void setDomain(String domain) {
+            this.domain = domain;
+        }
+
         public int getPort() {
             return port;
         }

src/main/java/io/kamax/mxisd/config/rest/RestBackendConfig.java

@@ -28,7 +28,6 @@ import org.slf4j.LoggerFactory;
 import java.net.MalformedURLException;
 import java.net.URL;
 import java.util.Objects;
-import java.util.Optional;
 
 public class RestBackendConfig {
 
@@ -118,8 +117,8 @@ public class RestBackendConfig {
             this.identity = identity;
         }
 
-        public Optional<ProfileEndpoints> getProfile() {
+        public ProfileEndpoints getProfile() {
-            return Optional.ofNullable(profile);
+            return profile;
         }
 
         public void setProfile(ProfileEndpoints profile) {
@@ -128,7 +127,7 @@ public class RestBackendConfig {
 
     }
 
-    private transient final Logger log = LoggerFactory.getLogger(RestBackendConfig.class);
+    private static final Logger log = LoggerFactory.getLogger(RestBackendConfig.class);
 
     private boolean enabled;
     private String host;
@@ -197,6 +196,11 @@ public class RestBackendConfig {
             log.info("Directory endpoint: {}", endpoints.getDirectory());
             log.info("Identity Single endpoint: {}", endpoints.identity.getSingle());
             log.info("Identity Bulk endpoint: {}", endpoints.identity.getBulk());
+
+            log.info("Profile endpoints:");
+            log.info("  - Display name: {}", getEndpoints().getProfile().getDisplayName());
+            log.info("  - 3PIDs: {}", getEndpoints().getProfile().getThreepids());
+            log.info("  - Roles: {}", getEndpoints().getProfile().getRoles());
         }
     }
 

src/main/java/io/kamax/mxisd/config/sql/SqlConfig.java

@@ -21,6 +21,7 @@
 package io.kamax.mxisd.config.sql;
 
 import io.kamax.mxisd.util.GsonUtil;
+import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -192,11 +193,35 @@ public abstract class SqlConfig {
 
     }
 
+    public static class ProfileRoles {
+
+        private String type;
+        private String query;
+
+        public String getType() {
+            return type;
+        }
+
+        public void setType(String type) {
+            this.type = type;
+        }
+
+        public String getQuery() {
+            return query;
+        }
+
+        public void setQuery(String query) {
+            this.query = query;
+        }
+
+    }
+
     public static class Profile {
 
         private Boolean enabled;
         private ProfileDisplayName displayName = new ProfileDisplayName();
         private ProfileThreepids threepid = new ProfileThreepids();
+        private ProfileRoles role = new ProfileRoles();
 
         public Boolean isEnabled() {
             return enabled;
@@ -222,6 +247,14 @@ public abstract class SqlConfig {
             this.threepid = threepid;
         }
 
+        public ProfileRoles getRole() {
+            return role;
+        }
+
+        public void setRole(ProfileRoles role) {
+            this.role = role;
+        }
+
     }
 
     private boolean enabled;
@@ -314,17 +347,19 @@ public abstract class SqlConfig {
         log.info("Enabled: {}", isEnabled());
         if (isEnabled()) {
             log.info("Type: {}", getType());
-            log.info("Connection: {}", getConnection());
+            log.info("Has connection info? {}", !StringUtils.isEmpty(getConnection()));
+            log.debug("Connection: {}", getConnection());
             log.info("Auth enabled: {}", getAuth().isEnabled());
             log.info("Directory queries: {}", GsonUtil.build().toJson(getDirectory().getQuery()));
             log.info("Identity type: {}", getIdentity().getType());
             log.info("3PID mapping query: {}", getIdentity().getQuery());
             log.info("Identity medium queries: {}", GsonUtil.build().toJson(getIdentity().getMedium()));
             log.info("Profile:");
-            log.info("\tEnabled: {}", getProfile().isEnabled());
+            log.info("  Enabled: {}", getProfile().isEnabled());
             if (getProfile().isEnabled()) {
-                log.info("\tDisplay name query: {}", getProfile().getDisplayName().getQuery());
+                log.info("  Display name query: {}", getProfile().getDisplayName().getQuery());
-                log.info("\tProfile 3PID query: {}", getProfile().getThreepid().getQuery());
+                log.info("  Profile 3PID query: {}", getProfile().getThreepid().getQuery());
+                log.info("  Role query: {}", getProfile().getRole().getQuery());
             }
         }
     }

src/main/java/io/kamax/mxisd/config/sql/synapse/SynapseSqlProviderConfig.java

@@ -20,6 +20,7 @@
 
 package io.kamax.mxisd.config.sql.synapse;
 
+import io.kamax.mxisd.UserIdType;
 import io.kamax.mxisd.backend.sql.synapse.SynapseQueries;
 import io.kamax.mxisd.config.sql.SqlConfig;
 import org.apache.commons.lang.StringUtils;
@@ -48,9 +49,17 @@ public class SynapseSqlProviderConfig extends SqlConfig {
             if (StringUtils.isBlank(getProfile().getDisplayName().getQuery())) {
                 getProfile().getDisplayName().setQuery(SynapseQueries.getDisplayName());
             }
+
             if (StringUtils.isBlank(getProfile().getThreepid().getQuery())) {
                 getProfile().getThreepid().setQuery(SynapseQueries.getThreepids());
             }
+
+            if (StringUtils.isBlank(getProfile().getRole().getType())) {
+                getProfile().getRole().setType(UserIdType.MatrixID.getId());
+            }
+            if (StringUtils.isBlank(getProfile().getRole().getQuery())) {
+                getProfile().getRole().setQuery(SynapseQueries.getRoles());
+            }
         }
 
         printConfig();

src/main/java/io/kamax/mxisd/crypto/CryptoFactory.java

@@ -20,9 +20,12 @@
 
 package io.kamax.mxisd.crypto;
 
-import io.kamax.matrix.crypto.*;
 import io.kamax.mxisd.config.KeyConfig;
-import io.kamax.mxisd.config.ServerConfig;
+import io.kamax.mxisd.crypto.ed25519.Ed25519KeyManager;
+import io.kamax.mxisd.crypto.ed25519.Ed25519SignatureManager;
+import io.kamax.mxisd.storage.crypto.FileKeyStore;
+import io.kamax.mxisd.storage.crypto.KeyStore;
+import io.kamax.mxisd.storage.crypto.MemoryKeyStore;
 import org.apache.commons.io.FileUtils;
 import org.apache.commons.lang3.StringUtils;
 
@@ -31,10 +34,10 @@ import java.io.IOException;
 
 public class CryptoFactory {
 
-    public static KeyManager getKeyManager(KeyConfig keyCfg) {
+    public static Ed25519KeyManager getKeyManager(KeyConfig keyCfg) {
-        _KeyStore store;
+        KeyStore store;
         if (StringUtils.equals(":memory:", keyCfg.getPath())) {
-            store = new KeyMemoryStore();
+            store = new MemoryKeyStore();
         } else {
             File keyStore = new File(keyCfg.getPath());
             if (!keyStore.exists()) {
@@ -45,14 +48,14 @@ public class CryptoFactory {
                 }
             }
 
-            store = new KeyFileStore(keyCfg.getPath());
+            store = new FileKeyStore(keyCfg.getPath());
         }
 
-        return new KeyManager(store);
+        return new Ed25519KeyManager(store);
     }
 
-    public static SignatureManager getSignatureManager(KeyManager keyMgr, ServerConfig cfg) {
+    public static SignatureManager getSignatureManager(Ed25519KeyManager keyMgr) {
-        return new SignatureManager(keyMgr, cfg.getName());
+        return new Ed25519SignatureManager(keyMgr);
     }
 
 }

src/main/java/io/kamax/mxisd/crypto/GenericKey.java

@@ -0,0 +1,51 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+public class GenericKey implements Key {
+
+    private final KeyIdentifier id;
+    private final boolean isValid;
+    private final String privKey;
+
+    public GenericKey(KeyIdentifier id, boolean isValid, String privKey) {
+        this.id = new GenericKeyIdentifier(id);
+        this.isValid = isValid;
+        this.privKey = privKey;
+    }
+
+
+    @Override
+    public KeyIdentifier getId() {
+        return id;
+    }
+
+    @Override
+    public boolean isValid() {
+        return isValid;
+    }
+
+    @Override
+    public String getPrivateKeyBase64() {
+        return privKey;
+    }
+
+}

src/main/java/io/kamax/mxisd/crypto/GenericKeyIdentifier.java

@@ -0,0 +1,76 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+import org.apache.commons.lang3.StringUtils;
+
+import java.util.Objects;
+
+public class GenericKeyIdentifier implements KeyIdentifier {
+
+    private final KeyType type;
+    private final String algo;
+    private final String serial;
+
+    public GenericKeyIdentifier(KeyIdentifier id) {
+        this(id.getType(), id.getAlgorithm(), id.getSerial());
+    }
+
+    public GenericKeyIdentifier(KeyType type, String algo, String serial) {
+        if (StringUtils.isAnyBlank(algo, serial)) {
+            throw new IllegalArgumentException("Aglorith and/or Serial cannot be blank");
+        }
+
+        this.type = Objects.requireNonNull(type);
+        this.algo = algo;
+        this.serial = serial;
+    }
+
+    @Override
+    public KeyType getType() {
+        return type;
+    }
+
+    @Override
+    public String getAlgorithm() {
+        return algo;
+    }
+
+    @Override
+    public String getSerial() {
+        return serial;
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (!(o instanceof GenericKeyIdentifier)) return false;
+        GenericKeyIdentifier that = (GenericKeyIdentifier) o;
+        return type == that.type &&
+                algo.equals(that.algo) &&
+                serial.equals(that.serial);
+    }
+
+    @Override
+    public int hashCode() {
+        return Objects.hash(type, algo, serial);
+    }
+}

src/main/java/io/kamax/mxisd/crypto/Key.java

@@ -0,0 +1,44 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+/**
+ * A signing key
+ */
+public interface Key {
+
+    KeyIdentifier getId();
+
+    /**
+     * If the key is currently valid
+     *
+     * @return true if the key is valid, false if not
+     */
+    boolean isValid();
+
+    /**
+     * Get the private key
+     *
+     * @return the private key encoded as Base64
+     */
+    String getPrivateKeyBase64();
+
+}

src/main/java/io/kamax/mxisd/crypto/KeyAlgorithm.java

@@ -0,0 +1,27 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+public interface KeyAlgorithm {
+
+    String Ed25519 = "ed25519";
+
+}

src/main/java/io/kamax/mxisd/crypto/KeyIdentifier.java

@@ -0,0 +1,54 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+/**
+ * Identifying data for a given Key.
+ */
+public interface KeyIdentifier {
+
+    /**
+     * Type of key.
+     *
+     * @return The type of the key
+     */
+    KeyType getType();
+
+    /**
+     * Algorithm of the key. Typically <code>ed25519</code>.
+     *
+     * @return The algorithm of the key
+     */
+    String getAlgorithm();
+
+    /**
+     * Serial of the key, unique for the algorithm.
+     * It is typically made of random alphanumerical characters.
+     *
+     * @return The serial of the key
+     */
+    String getSerial();
+
+    default String getId() {
+        return getAlgorithm().toLowerCase() + ":" + getSerial();
+    }
+
+}

src/main/java/io/kamax/mxisd/crypto/KeyManager.java

@@ -0,0 +1,41 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+import java.util.List;
+
+public interface KeyManager {
+
+    KeyIdentifier generateKey(KeyType type);
+
+    List<KeyIdentifier> getKeys(KeyType type);
+
+    Key getServerSigningKey();
+
+    Key getKey(KeyIdentifier id);
+
+    void disableKey(KeyIdentifier id);
+
+    String getPublicKeyBase64(KeyIdentifier id);
+
+    boolean isValid(KeyType type, String publicKeyBase64);
+
+}

src/main/java/io/kamax/mxisd/crypto/KeyType.java

@@ -0,0 +1,39 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+/**
+ * Types of keys used by an Identity server.
+ * See https://matrix.org/docs/spec/identity_service/r0.1.0.html#key-management
+ */
+public enum KeyType {
+
+    /**
+     * Ephemeral keys are related to 3PID invites and are only valid while the invite is pending.
+     */
+    Ephemeral,
+
+    /**
+     * Regular keys are used by the Identity Server itself to sign requests/responses
+     */
+    Regular
+
+}

src/main/java/io/kamax/mxisd/crypto/RegularKeyIdentifier.java

@@ -0,0 +1,29 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+public class RegularKeyIdentifier extends GenericKeyIdentifier {
+
+    public RegularKeyIdentifier(String algo, String serial) {
+        super(KeyType.Regular, algo, serial);
+    }
+
+}

src/main/java/io/kamax/mxisd/crypto/Signature.java

@@ -0,0 +1,29 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+public interface Signature {
+
+    KeyIdentifier getKey();
+
+    String getSignature();
+
+}

src/main/java/io/kamax/mxisd/crypto/SignatureManager.java

@@ -0,0 +1,64 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto;
+
+import com.google.gson.JsonObject;
+
+import java.nio.charset.StandardCharsets;
+
+public interface SignatureManager {
+
+    /**
+     * Sign the message and produce a <code>signatures</code> object that can directly be added to the object being signed.
+     *
+     * @param domain  The domain under which the signature should be added
+     * @param message The message to sign
+     * @return The <code>signatures</code> object
+     */
+    JsonObject signMessageGson(String domain, String message);
+
+    /**
+     * Sign the canonical form of a JSON object.
+     *
+     * @param obj The JSON object to canonicalize and sign
+     * @return The signature
+     */
+    Signature sign(JsonObject obj);
+
+    /**
+     * Sign the message, using UTF-8 as decoding character set.
+     *
+     * @param message The UTF-8 encoded message
+     * @return The signature
+     */
+    default Signature sign(String message) {
+        return sign(message.getBytes(StandardCharsets.UTF_8));
+    }
+
+    /**
+     * Sign the data.
+     *
+     * @param data The data to sign
+     * @return The signature
+     */
+    Signature sign(byte[] data);
+
+}

src/main/java/io/kamax/mxisd/crypto/ed25519/Ed25519Key.java

@@ -0,0 +1,58 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto.ed25519;
+
+import io.kamax.mxisd.crypto.GenericKeyIdentifier;
+import io.kamax.mxisd.crypto.Key;
+import io.kamax.mxisd.crypto.KeyAlgorithm;
+import io.kamax.mxisd.crypto.KeyIdentifier;
+
+public class Ed25519Key implements Key {
+
+    private KeyIdentifier id;
+    private String privKey;
+
+    public Ed25519Key(KeyIdentifier id, String privKey) {
+        if (!KeyAlgorithm.Ed25519.equals(id.getAlgorithm())) {
+            throw new IllegalArgumentException();
+        }
+
+        this.id = new GenericKeyIdentifier(id);
+        this.privKey = privKey;
+    }
+
+
+    @Override
+    public KeyIdentifier getId() {
+        return id;
+    }
+
+    @Override
+    public boolean isValid() {
+        return true;
+    }
+
+    @Override
+    public String getPrivateKeyBase64() {
+        return privKey;
+    }
+
+}

src/main/java/io/kamax/mxisd/crypto/ed25519/Ed25519KeyManager.java

@@ -0,0 +1,148 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto.ed25519;
+
+import io.kamax.matrix.codec.MxBase64;
+import io.kamax.mxisd.crypto.*;
+import io.kamax.mxisd.storage.crypto.KeyStore;
+import net.i2p.crypto.eddsa.EdDSAPrivateKey;
+import net.i2p.crypto.eddsa.EdDSAPublicKey;
+import net.i2p.crypto.eddsa.KeyPairGenerator;
+import net.i2p.crypto.eddsa.spec.EdDSANamedCurveTable;
+import net.i2p.crypto.eddsa.spec.EdDSAParameterSpec;
+import net.i2p.crypto.eddsa.spec.EdDSAPrivateKeySpec;
+import net.i2p.crypto.eddsa.spec.EdDSAPublicKeySpec;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.lang3.RandomStringUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.nio.ByteBuffer;
+import java.security.KeyPair;
+import java.time.Instant;
+import java.util.List;
+import java.util.stream.Collectors;
+
+public class Ed25519KeyManager implements KeyManager {
+
+    private static final Logger log = LoggerFactory.getLogger(Ed25519KeyManager.class);
+
+    private final EdDSAParameterSpec keySpecs;
+    private final KeyStore store;
+
+    public Ed25519KeyManager(KeyStore store) {
+        this.keySpecs = EdDSANamedCurveTable.getByName(EdDSANamedCurveTable.CURVE_ED25519_SHA512);
+        this.store = store;
+
+        if (!store.getCurrentKey().isPresent()) {
+            List<KeyIdentifier> keys = store.list(KeyType.Regular).stream()
+                    .map(this::getKey)
+                    .filter(Key::isValid)
+                    .map(Key::getId)
+                    .collect(Collectors.toList());
+
+            if (keys.isEmpty()) {
+                keys.add(generateKey(KeyType.Regular));
+            }
+
+            store.setCurrentKey(keys.get(0));
+        }
+    }
+
+    private String generateId() {
+        ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
+        buffer.putLong(Instant.now().toEpochMilli() - 1546297200000L); // TS since 2019-01-01T00:00:00Z to keep IDs short
+        return Base64.encodeBase64URLSafeString(buffer.array()) + RandomStringUtils.randomAlphanumeric(1);
+    }
+
+    private String getPrivateKeyBase64(EdDSAPrivateKey key) {
+        return MxBase64.encode(key.getSeed());
+    }
+
+    EdDSAParameterSpec getKeySpecs() {
+        return keySpecs;
+    }
+
+    @Override
+    public KeyIdentifier generateKey(KeyType type) {
+        KeyIdentifier id;
+        do {
+            id = new GenericKeyIdentifier(type, KeyAlgorithm.Ed25519, generateId());
+        } while (store.has(id));
+
+        KeyPair pair = (new KeyPairGenerator()).generateKeyPair();
+        String keyEncoded = getPrivateKeyBase64((EdDSAPrivateKey) pair.getPrivate());
+
+        Key key = new GenericKey(id, true, keyEncoded);
+        store.add(key);
+
+        return id;
+    }
+
+    @Override
+    public List<KeyIdentifier> getKeys(KeyType type) {
+        return store.list(type);
+    }
+
+    @Override
+    public Key getServerSigningKey() {
+        return store.get(store.getCurrentKey().orElseThrow(IllegalStateException::new));
+    }
+
+    @Override
+    public Key getKey(KeyIdentifier id) {
+        return store.get(id);
+    }
+
+    private EdDSAPrivateKeySpec getPrivateKeySpecs(KeyIdentifier id) {
+        return new EdDSAPrivateKeySpec(Base64.decodeBase64(getKey(id).getPrivateKeyBase64()), keySpecs);
+    }
+
+    EdDSAPrivateKey getPrivateKey(KeyIdentifier id) {
+        return new EdDSAPrivateKey(getPrivateKeySpecs(id));
+    }
+
+    private EdDSAPublicKey getPublicKey(KeyIdentifier id) {
+        EdDSAPrivateKeySpec privKeySpec = getPrivateKeySpecs(id);
+        EdDSAPublicKeySpec pubKeySpec = new EdDSAPublicKeySpec(privKeySpec.getA(), keySpecs);
+        return new EdDSAPublicKey(pubKeySpec);
+    }
+
+    @Override
+    public void disableKey(KeyIdentifier id) {
+        Key key = store.get(id);
+        key = new GenericKey(id, false, key.getPrivateKeyBase64());
+        store.update(key);
+    }
+
+    @Override
+    public String getPublicKeyBase64(KeyIdentifier id) {
+        return MxBase64.encode(getPublicKey(id).getAbyte());
+    }
+
+    @Override
+    public boolean isValid(KeyType type, String publicKeyBase64) {
+        // TODO caching?
+        return getKeys(type).stream().anyMatch(id -> StringUtils.equals(getPublicKeyBase64(id), publicKeyBase64));
+    }
+
+}

src/main/java/io/kamax/mxisd/crypto/ed25519/Ed25519RegularKeyIdentifier.java

@@ -0,0 +1,32 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto.ed25519;
+
+import io.kamax.mxisd.crypto.KeyAlgorithm;
+import io.kamax.mxisd.crypto.RegularKeyIdentifier;
+
+public class Ed25519RegularKeyIdentifier extends RegularKeyIdentifier {
+
+    public Ed25519RegularKeyIdentifier(String serial) {
+        super(KeyAlgorithm.Ed25519, serial);
+    }
+
+}

src/main/java/io/kamax/mxisd/crypto/ed25519/Ed25519SignatureManager.java

@@ -0,0 +1,86 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.crypto.ed25519;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.codec.MxBase64;
+import io.kamax.matrix.json.MatrixJson;
+import io.kamax.mxisd.crypto.KeyIdentifier;
+import io.kamax.mxisd.crypto.Signature;
+import io.kamax.mxisd.crypto.SignatureManager;
+import net.i2p.crypto.eddsa.EdDSAEngine;
+
+import java.security.InvalidKeyException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.security.SignatureException;
+
+public class Ed25519SignatureManager implements SignatureManager {
+
+    private final Ed25519KeyManager keyMgr;
+
+    public Ed25519SignatureManager(Ed25519KeyManager keyMgr) {
+        this.keyMgr = keyMgr;
+    }
+
+    @Override
+    public JsonObject signMessageGson(String domain, String message) {
+        Signature sign = sign(message);
+
+        JsonObject keySignature = new JsonObject();
+        keySignature.addProperty(sign.getKey().getAlgorithm() + ":" + sign.getKey().getSerial(), sign.getSignature());
+        JsonObject signature = new JsonObject();
+        signature.add(domain, keySignature);
+
+        return signature;
+    }
+
+    @Override
+    public Signature sign(JsonObject obj) {
+        return sign(MatrixJson.encodeCanonical(obj));
+    }
+
+    @Override
+    public Signature sign(byte[] data) {
+        try {
+            KeyIdentifier signingKeyId = keyMgr.getServerSigningKey().getId();
+            EdDSAEngine signEngine = new EdDSAEngine(MessageDigest.getInstance(keyMgr.getKeySpecs().getHashAlgorithm()));
+            signEngine.initSign(keyMgr.getPrivateKey(signingKeyId));
+            byte[] signRaw = signEngine.signOneShot(data);
+            String sign = MxBase64.encode(signRaw);
+
+            return new Signature() {
+                @Override
+                public KeyIdentifier getKey() {
+                    return signingKeyId;
+                }
+
+                @Override
+                public String getSignature() {
+                    return sign;
+                }
+            };
+        } catch (NoSuchAlgorithmException | InvalidKeyException | SignatureException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/exception/ConfigurationException.java

@@ -20,11 +20,8 @@
 
 package io.kamax.mxisd.exception;
 
-import java.util.Optional;
-
 public class ConfigurationException extends RuntimeException {
 
-    private String key;
     private String detailedMsg;
 
     public ConfigurationException(String key) {
@@ -40,8 +37,8 @@ public class ConfigurationException extends RuntimeException {
         this.detailedMsg = detailedMsg;
     }
 
-    public Optional<String> getDetailedMessage() {
+    public String getDetailedMessage() {
-        return Optional.ofNullable(detailedMsg);
+        return detailedMsg;
     }
 
 }

src/main/java/io/kamax/mxisd/exception/ObjectNotFoundException.java

@@ -22,8 +22,12 @@ package io.kamax.mxisd.exception;
 
 public class ObjectNotFoundException extends RuntimeException {
 
+    public ObjectNotFoundException(String message) {
+        super(message);
+    }
+
     public ObjectNotFoundException(String type, String id) {
-        super(type + " with ID " + id + " does not exist");
+        this(type + " with ID " + id + " does not exist");
     }
 
 }

src/main/java/io/kamax/mxisd/http/io/identity/SingeLookupReplyJson.java

@@ -22,6 +22,9 @@ package io.kamax.mxisd.http.io.identity;
 
 import io.kamax.mxisd.lookup.SingleLookupReply;
 
+import java.util.HashMap;
+import java.util.Map;
+
 public class SingeLookupReplyJson {
 
     private String address;
@@ -30,6 +33,7 @@ public class SingeLookupReplyJson {
     private long not_after;
     private long not_before;
     private long ts;
+    private Map<String, Map<String, String>> signatures = new HashMap<>();
 
     public SingeLookupReplyJson(SingleLookupReply reply) {
         this.address = reply.getRequest().getThreePid();
@@ -64,4 +68,8 @@ public class SingeLookupReplyJson {
         return ts;
     }
 
+    public Map<String, Map<String, String>> getSignatures() {
+        return signatures;
+    }
+
 }

src/main/java/io/kamax/mxisd/http/undertow/handler/BasicHttpHandler.java

@@ -23,20 +23,29 @@ package io.kamax.mxisd.http.undertow.handler;
 import com.google.gson.JsonElement;
 import com.google.gson.JsonObject;
 import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.exception.AccessTokenNotFoundException;
 import io.kamax.mxisd.exception.HttpMatrixException;
 import io.kamax.mxisd.exception.InternalServerError;
 import io.kamax.mxisd.proxy.Response;
+import io.kamax.mxisd.util.RestClientUtils;
 import io.undertow.server.HttpHandler;
 import io.undertow.server.HttpServerExchange;
 import io.undertow.util.HttpString;
 import org.apache.commons.io.IOUtils;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.Header;
+import org.apache.http.HeaderElement;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.impl.client.CloseableHttpClient;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 import java.io.IOException;
 import java.io.UnsupportedEncodingException;
 import java.net.InetSocketAddress;
+import java.net.URI;
 import java.net.URLDecoder;
 import java.nio.charset.StandardCharsets;
 import java.util.Deque;
@@ -46,7 +55,19 @@ import java.util.Optional;
 
 public abstract class BasicHttpHandler implements HttpHandler {
 
-    private transient final Logger log = LoggerFactory.getLogger(BasicHttpHandler.class);
+    private static final Logger log = LoggerFactory.getLogger(BasicHttpHandler.class);
+
+    protected String getAccessToken(HttpServerExchange exchange) {
+        return Optional.ofNullable(exchange.getRequestHeaders().getFirst("Authorization"))
+                .flatMap(v -> {
+                    if (!v.startsWith("Bearer ")) {
+                        return Optional.empty();
+                    }
+
+                    return Optional.of(v.substring("Bearer ".length()));
+                }).filter(StringUtils::isNotEmpty)
+                .orElseThrow(AccessTokenNotFoundException::new);
+    }
 
     protected String getRemoteHostAddress(HttpServerExchange exchange) {
         return ((InetSocketAddress) exchange.getConnection().getPeerAddress()).getAddress().getHostAddress();
@@ -101,6 +122,10 @@ public abstract class BasicHttpHandler implements HttpHandler {
         return GsonUtil.parseObj(getBodyUtf8(exchange));
     }
 
+    protected void putHeader(HttpServerExchange ex, String name, String value) {
+        ex.getResponseHeaders().put(HttpString.tryFromString(name), value);
+    }
+
     protected void respond(HttpServerExchange ex, int statusCode, JsonElement bodyJson) {
         respondJson(ex, statusCode, GsonUtil.get().toJson(bodyJson));
     }
@@ -145,4 +170,34 @@ public abstract class BasicHttpHandler implements HttpHandler {
         upstream.getHeaders().forEach((key, value) -> exchange.getResponseHeaders().addAll(HttpString.tryFromString(key), value));
         writeBodyAsUtf8(exchange, upstream.getBody());
     }
+
+    protected void proxyPost(HttpServerExchange exchange, JsonObject body, CloseableHttpClient client, ClientDnsOverwrite dns) {
+        String target = dns.transform(URI.create(exchange.getRequestURL())).toString();
+        log.info("Requesting remote: {}", target);
+        HttpPost req = RestClientUtils.post(target, GsonUtil.get(), body);
+
+        exchange.getRequestHeaders().forEach(header -> {
+            header.forEach(v -> {
+                String name = header.getHeaderName().toString();
+                if (!StringUtils.startsWithIgnoreCase(name, "content-")) {
+                    req.addHeader(name, v);
+                }
+            });
+        });
+
+        try (CloseableHttpResponse res = client.execute(req)) {
+            exchange.setStatusCode(res.getStatusLine().getStatusCode());
+            for (Header h : res.getAllHeaders()) {
+                for (HeaderElement el : h.getElements()) {
+                    exchange.getResponseHeaders().add(HttpString.tryFromString(h.getName()), el.getValue());
+                }
+            }
+            res.getEntity().writeTo(exchange.getOutputStream());
+            exchange.endExchange();
+        } catch (IOException e) {
+            log.warn("Unable to make proxy call: {}", e.getMessage(), e);
+            throw new InternalServerError(e);
+        }
+    }
+
 }

src/main/java/io/kamax/mxisd/http/undertow/handler/InternalInfoHandler.java

@@ -0,0 +1,50 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.http.undertow.handler;
+
+import io.undertow.server.HttpServerExchange;
+
+import java.util.concurrent.ThreadLocalRandom;
+
+public class InternalInfoHandler extends BasicHttpHandler {
+
+    /*
+     * This endpoint should never be called as being entierly custom as per instructions of New Vector,
+     * the author of that endpoint.
+     *
+     * Used for the first time at https://github.com/matrix-org/synapse/pull/4681/files#diff-a73c645c44a17da6ab70f256da6b60afR41
+     *
+     * Full context: https://matrix.to/#/!YkZelGRiqijtzXZODa:matrix.org/$15510967621328WMKVu:kamax.io?via=matrix.org
+     * Room name: #matrix-spec
+     * Room alias: #matrix-spec:matrix.org
+     */
+    public static final String Path = "/_matrix/identity/api/{version}/internal-info";
+
+    @Override
+    public void handleRequest(HttpServerExchange exchange) throws Exception {
+        // We will return a random status code in all possible error codes
+        int type = ThreadLocalRandom.current().nextInt(4, 6) * 100; // Random 4 or 5, times 100
+        int status = type + ThreadLocalRandom.current().nextInt(0, 100); // Random 0 to 99
+
+        respond(exchange, status, "M_FORBIDDEN", "This endpoint is under quarantine and possibly wrongfully labeled stable.");
+    }
+
+}

src/main/java/io/kamax/mxisd/http/undertow/handler/SaneHandler.java

@@ -27,8 +27,7 @@ import io.kamax.matrix.json.InvalidJsonException;
 import io.kamax.mxisd.exception.*;
 import io.undertow.server.HttpHandler;
 import io.undertow.server.HttpServerExchange;
-import io.undertow.util.HttpString;
+import org.apache.commons.lang3.StringUtils;
-import org.apache.commons.lang.StringUtils;
 import org.apache.http.HttpStatus;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -37,15 +36,22 @@ import java.time.Instant;
 
 public class SaneHandler extends BasicHttpHandler {
 
+    private static final Logger log = LoggerFactory.getLogger(SaneHandler.class);
+
+    private static final String CorsOriginName = "Access-Control-Allow-Origin";
+    private static final String CorsOriginValue = "*";
+    private static final String CorsMethodsName = "Access-Control-Allow-Methods";
+    private static final String CorsMethodsValue = "GET, POST, PUT, DELETE, OPTIONS";
+    private static final String CorsHeadersName = "Access-Control-Allow-Headers";
+    private static final String CorsHeadersValue = "Origin, X-Requested-With, Content-Type, Accept, Authorization";
+
     public static SaneHandler around(HttpHandler h) {
         return new SaneHandler(h);
     }
 
-    private transient final Logger log = LoggerFactory.getLogger(SaneHandler.class);
+    private final HttpHandler child;
 
-    private HttpHandler child;
+    private SaneHandler(HttpHandler child) {
-
-    public SaneHandler(HttpHandler child) {
         this.child = child;
     }
 
@@ -58,9 +64,9 @@ public class SaneHandler extends BasicHttpHandler {
         } else {
             try {
                 // CORS headers as per spec
-                exchange.getResponseHeaders().put(HttpString.tryFromString("Access-Control-Allow-Origin"), "*");
+                putHeader(exchange, CorsOriginName, CorsOriginValue);
-                exchange.getResponseHeaders().put(HttpString.tryFromString("Access-Control-Allow-Methods"), "GET, POST, PUT, DELETE, OPTIONS");
+                putHeader(exchange, CorsMethodsName, CorsMethodsValue);
-                exchange.getResponseHeaders().put(HttpString.tryFromString("Access-Control-Allow-Headers"), "Origin, X-Requested-With, Content-Type, Accept, Authorization");
+                putHeader(exchange, CorsHeadersName, CorsHeadersValue);
 
                 child.handleRequest(exchange);
             } catch (IllegalArgumentException e) {
@@ -89,9 +95,9 @@ public class SaneHandler extends BasicHttpHandler {
                 handleException(exchange, e);
             } catch (InternalServerError e) {
                 if (StringUtils.isNotBlank(e.getInternalReason())) {
-                    log.error("Reference #{} - {}", e.getReference(), e.getInternalReason());
+                    log.error("Transaction #{} - {}", e.getReference(), e.getInternalReason());
                 } else {
-                    log.error("Reference #{}", e);
+                    log.error("Transaction #{}", e);
                 }
 
                 handleException(exchange, e);
@@ -105,14 +111,11 @@ public class SaneHandler extends BasicHttpHandler {
                 respond(exchange, e.getStatus(), buildErrorBody(exchange, e.getErrorCode(), e.getError()));
             } catch (RuntimeException e) {
                 log.error("Unknown error when handling {}", exchange.getRequestURL(), e);
-                respond(exchange, HttpStatus.SC_INTERNAL_SERVER_ERROR, buildErrorBody(exchange,
+                String message = e.getMessage();
-                        "M_UNKNOWN",
+                if (StringUtils.isBlank(message)) {
-                        StringUtils.defaultIfBlank(
+                    message = "An internal server error occurred. Contact your administrator with reference Transaction #" + Instant.now().toEpochMilli();
-                                e.getMessage(),
+                }
-                                "An internal server error occurred. If this error persists, please contact support with reference #" +
+                respond(exchange, HttpStatus.SC_INTERNAL_SERVER_ERROR, buildErrorBody(exchange, "M_UNKNOWN", message));
-                                        Instant.now().toEpochMilli()
-                        )
-                ));
             } finally {
                 exchange.endExchange();
             }

src/main/java/io/kamax/mxisd/http/undertow/handler/as/v1/AsUserHandler.java

@@ -0,0 +1,46 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.http.undertow.handler.as.v1;
+
+import io.kamax.mxisd.as.AppSvcManager;
+import io.undertow.server.HttpServerExchange;
+
+import java.util.LinkedList;
+
+public class AsUserHandler extends ApplicationServiceHandler {
+
+    public static final String ID = "userId";
+    public static final String Path = "/_matrix/app/v1/users/{" + ID + "}";
+
+    private final AppSvcManager app;
+
+    public AsUserHandler(AppSvcManager app) {
+        this.app = app;
+    }
+
+    @Override
+    public void handleRequest(HttpServerExchange exchange) {
+        String userId = exchange.getQueryParameters().getOrDefault(ID, new LinkedList<>()).peekFirst();
+        app.withToken(getToken(exchange)).processUser(userId);
+        respondJson(exchange, "{}");
+    }
+
+}

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/EphemeralKeyIsValidHandler.java

@@ -20,6 +20,8 @@
 
 package io.kamax.mxisd.http.undertow.handler.identity.v1;
 
+import io.kamax.mxisd.crypto.KeyManager;
+import io.kamax.mxisd.crypto.KeyType;
 import io.kamax.mxisd.http.IsAPIv1;
 import io.undertow.server.HttpServerExchange;
 import org.slf4j.Logger;
@@ -31,11 +33,19 @@ public class EphemeralKeyIsValidHandler extends KeyIsValidHandler {
 
     private transient final Logger log = LoggerFactory.getLogger(EphemeralKeyIsValidHandler.class);
 
+    private KeyManager mgr;
+
+    public EphemeralKeyIsValidHandler(KeyManager mgr) {
+        this.mgr = mgr;
+    }
+
     @Override
     public void handleRequest(HttpServerExchange exchange) {
-        log.warn("Ephemeral key was requested but no ephemeral key are generated, replying not valid");
+        // FIXME process + correctly in query parameter handling
+        String pubKey = getQueryParameter(exchange, "public_key").replace(" ", "+");
+        log.info("Validating ephemeral public key {}", pubKey);
 
-        respondJson(exchange, invalidKey);
+        respondJson(exchange, mgr.isValid(KeyType.Ephemeral, pubKey) ? validKey : invalidKey);
     }
 
 }

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/KeyGetHandler.java

@@ -21,18 +21,20 @@
 package io.kamax.mxisd.http.undertow.handler.identity.v1;
 
 import com.google.gson.JsonObject;
-import io.kamax.matrix.crypto.KeyManager;
+import io.kamax.mxisd.crypto.GenericKeyIdentifier;
-import io.kamax.mxisd.exception.BadRequestException;
+import io.kamax.mxisd.crypto.KeyManager;
+import io.kamax.mxisd.crypto.KeyType;
 import io.kamax.mxisd.http.IsAPIv1;
 import io.kamax.mxisd.http.undertow.handler.BasicHttpHandler;
 import io.undertow.server.HttpServerExchange;
+import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 public class KeyGetHandler extends BasicHttpHandler {
 
     public static final String Key = "key";
-    public static final String Path = IsAPIv1.Base + "/pubkey/{key}";
+    public static final String Path = IsAPIv1.Base + "/pubkey/{" + Key + "}";
 
     private transient final Logger log = LoggerFactory.getLogger(KeyGetHandler.class);
 
@@ -45,17 +47,17 @@ public class KeyGetHandler extends BasicHttpHandler {
     @Override
     public void handleRequest(HttpServerExchange exchange) {
         String key = getQueryParameter(exchange, Key);
-        String[] v = key.split(":", 2);
+        if (StringUtils.isBlank(key)) {
-        String keyType = v[0];
+            throw new IllegalArgumentException("Key ID cannot be empty or blank");
-        int keyId = Integer.parseInt(v[1]);
-
-        if (!"ed25519".contentEquals(keyType)) {
-            throw new BadRequestException("Invalid algorithm: " + keyType);
         }
 
-        log.info("Key {}:{} was requested", keyType, keyId);
+        String[] v = key.split(":", 2); // Maybe use regex?
+        String keyAlgo = v[0];
+        String keyId = v[1];
+
+        log.info("Key {}:{} was requested", keyAlgo, keyId);
         JsonObject obj = new JsonObject();
-        obj.addProperty("public_key", mgr.getPublicKeyBase64(keyId));
+        obj.addProperty("public_key", mgr.getPublicKeyBase64(new GenericKeyIdentifier(KeyType.Regular, keyAlgo, keyId)));
         respond(exchange, obj);
     }
 

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/RegularKeyIsValidHandler.java

@@ -20,10 +20,10 @@
 
 package io.kamax.mxisd.http.undertow.handler.identity.v1;
 
-import io.kamax.matrix.crypto.KeyManager;
+import io.kamax.mxisd.crypto.KeyManager;
+import io.kamax.mxisd.crypto.KeyType;
 import io.kamax.mxisd.http.IsAPIv1;
 import io.undertow.server.HttpServerExchange;
-import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -41,12 +41,11 @@ public class RegularKeyIsValidHandler extends KeyIsValidHandler {
 
     @Override
     public void handleRequest(HttpServerExchange exchange) {
-        String pubKey = getQueryParameter(exchange, "public_key");
+        // FIXME process + correctly in query parameter handling
+        String pubKey = getQueryParameter(exchange, "public_key").replace(" ", "+");
         log.info("Validating public key {}", pubKey);
 
-        // TODO do in manager
+        respondJson(exchange, mgr.isValid(KeyType.Regular, pubKey) ? validKey : invalidKey);
-        boolean valid = StringUtils.equals(pubKey, mgr.getPublicKeyBase64(mgr.getCurrentIndex()));
-        respondJson(exchange, valid ? validKey : invalidKey);
     }
 
 }

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/SignEd25519Handler.java

@@ -0,0 +1,75 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.http.undertow.handler.identity.v1;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.matrix.json.MatrixJson;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.crypto.SignatureManager;
+import io.kamax.mxisd.http.IsAPIv1;
+import io.kamax.mxisd.http.undertow.handler.BasicHttpHandler;
+import io.kamax.mxisd.invitation.IThreePidInviteReply;
+import io.kamax.mxisd.invitation.InvitationManager;
+import io.undertow.server.HttpServerExchange;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class SignEd25519Handler extends BasicHttpHandler {
+
+    public static final String Path = IsAPIv1.Base + "/sign-ed25519";
+
+    private static final Logger log = LoggerFactory.getLogger(SignEd25519Handler.class);
+
+    private final MxisdConfig cfg;
+    private final InvitationManager invMgr;
+    private final SignatureManager signMgr;
+
+    public SignEd25519Handler(MxisdConfig cfg, InvitationManager invMgr, SignatureManager signMgr) {
+        this.cfg = cfg;
+        this.invMgr = invMgr;
+        this.signMgr = signMgr;
+    }
+
+    @Override
+    public void handleRequest(HttpServerExchange exchange) {
+        JsonObject body = parseJsonObject(exchange);
+
+        _MatrixID mxid = MatrixID.asAcceptable(GsonUtil.getStringOrThrow(body, "mxid"));
+        String token = GsonUtil.getStringOrThrow(body, "token");
+        String privKey = GsonUtil.getStringOrThrow(body, "private_key");
+
+        IThreePidInviteReply reply = invMgr.getInvite(token, privKey);
+        _MatrixID sender = reply.getInvite().getSender();
+
+        JsonObject res = new JsonObject();
+        res.addProperty("token", token);
+        res.addProperty("sender", sender.getId());
+        res.addProperty("mxid", mxid.getId());
+        res.add("signatures", signMgr.signMessageGson(cfg.getServer().getName(), MatrixJson.encodeCanonical(res)));
+
+        log.info("Signed data for invite using token {}", token);
+        respondJson(exchange, res);
+    }
+
+}

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/SingleLookupHandler.java

@@ -21,10 +21,12 @@
 package io.kamax.mxisd.http.undertow.handler.identity.v1;
 
 import com.google.gson.JsonObject;
-import io.kamax.matrix.crypto.SignatureManager;
 import io.kamax.matrix.event.EventKey;
 import io.kamax.matrix.json.GsonUtil;
 import io.kamax.matrix.json.MatrixJson;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.config.ServerConfig;
+import io.kamax.mxisd.crypto.SignatureManager;
 import io.kamax.mxisd.http.IsAPIv1;
 import io.kamax.mxisd.http.io.identity.SingeLookupReplyJson;
 import io.kamax.mxisd.lookup.SingleLookupReply;
@@ -42,10 +44,12 @@ public class SingleLookupHandler extends LookupHandler {
 
     private transient final Logger log = LoggerFactory.getLogger(SingleLookupHandler.class);
 
+    private ServerConfig cfg;
     private LookupStrategy strategy;
     private SignatureManager signMgr;
 
-    public SingleLookupHandler(LookupStrategy strategy, SignatureManager signMgr) {
+    public SingleLookupHandler(MxisdConfig cfg, LookupStrategy strategy, SignatureManager signMgr) {
+        this.cfg = cfg.getServer();
         this.strategy = strategy;
         this.signMgr = signMgr;
     }
@@ -72,7 +76,7 @@ public class SingleLookupHandler extends LookupHandler {
 
             // FIXME signing should be done in the business model, not in the controller
             JsonObject obj = GsonUtil.makeObj(new SingeLookupReplyJson(lookup));
-            obj.add(EventKey.Signatures.get(), signMgr.signMessageGson(MatrixJson.encodeCanonical(obj)));
+            obj.add(EventKey.Signatures.get(), signMgr.signMessageGson(cfg.getName(), MatrixJson.encodeCanonical(obj)));
 
             respondJson(exchange, obj);
         }

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/StoreInviteHandler.java

@@ -24,9 +24,9 @@ import com.google.gson.JsonObject;
 import com.google.gson.reflect.TypeToken;
 import io.kamax.matrix.MatrixID;
 import io.kamax.matrix._MatrixID;
-import io.kamax.matrix.crypto.KeyManager;
 import io.kamax.matrix.json.GsonUtil;
 import io.kamax.mxisd.config.ServerConfig;
+import io.kamax.mxisd.crypto.KeyManager;
 import io.kamax.mxisd.exception.BadRequestException;
 import io.kamax.mxisd.http.IsAPIv1;
 import io.kamax.mxisd.http.io.identity.StoreInviteRequest;
@@ -96,7 +96,8 @@ public class StoreInviteHandler extends BasicHttpHandler {
         IThreePidInvite invite = new ThreePidInvite(sender, inv.getMedium(), inv.getAddress(), inv.getRoomId(), parameters);
         IThreePidInviteReply reply = invMgr.storeInvite(invite);
 
-        respondJson(exchange, new ThreePidInviteReplyIO(reply, keyMgr.getPublicKeyBase64(keyMgr.getCurrentIndex()), cfg.getPublicUrl()));
+        // FIXME the key info must be set by the invitation manager in the reply object!
+        respondJson(exchange, new ThreePidInviteReplyIO(reply, keyMgr.getPublicKeyBase64(keyMgr.getServerSigningKey().getId()), cfg.getPublicUrl()));
     }
 
 }

src/main/java/io/kamax/mxisd/http/undertow/handler/invite/v1/RoomInviteHandler.java

@@ -0,0 +1,103 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.http.undertow.handler.invite.v1;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.exception.InternalServerError;
+import io.kamax.mxisd.exception.NotAllowedException;
+import io.kamax.mxisd.exception.RemoteHomeServerException;
+import io.kamax.mxisd.http.undertow.handler.BasicHttpHandler;
+import io.kamax.mxisd.invitation.InvitationManager;
+import io.undertow.server.HttpServerExchange;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Optional;
+
+public class RoomInviteHandler extends BasicHttpHandler {
+
+    public static final String Path = "/_matrix/client/r0/rooms/{roomId}/invite";
+
+    private static final Logger log = LoggerFactory.getLogger(RoomInviteHandler.class);
+
+    private final CloseableHttpClient client;
+    private final ClientDnsOverwrite dns;
+    private final InvitationManager invMgr;
+
+    public RoomInviteHandler(CloseableHttpClient client, ClientDnsOverwrite dns, InvitationManager invMgr) {
+        this.client = client;
+        this.dns = dns;
+        this.invMgr = invMgr;
+    }
+
+    @Override
+    public void handleRequest(HttpServerExchange exchange) {
+        String accessToken = getAccessToken(exchange);
+
+        String whoamiUri = dns.transform(URI.create(exchange.getRequestURL()).resolve(URI.create("/_matrix/client/r0/account/whoami"))).toString();
+        log.info("Who Am I URL: {}", whoamiUri);
+        HttpGet whoAmIReq = new HttpGet(whoamiUri);
+        whoAmIReq.addHeader("Authorization", "Bearer " + accessToken);
+        _MatrixID uId;
+        try (CloseableHttpResponse whoAmIRes = client.execute(whoAmIReq)) {
+            int sc = whoAmIRes.getStatusLine().getStatusCode();
+            String body = EntityUtils.toString(whoAmIRes.getEntity());
+
+            if (sc != 200) {
+                log.warn("Unable to get caller identity from Homeserver - Status code: {}", sc);
+                log.debug("Body: {}", body);
+                throw new RemoteHomeServerException(body);
+            }
+
+            JsonObject json = GsonUtil.parseObj(body);
+            Optional<String> uIdRaw = GsonUtil.findString(json, "user_id");
+            if (!uIdRaw.isPresent()) {
+                throw new RemoteHomeServerException("No User ID provided when checking identity");
+            }
+
+            uId = MatrixID.asAcceptable(uIdRaw.get());
+        } catch (IOException e) {
+            InternalServerError ex = new InternalServerError(e);
+            log.error("Ref {}: Unable to fetch caller identity from Homeserver", ex.getReference());
+            throw ex;
+        }
+
+        log.info("Processing room invite from {}", uId.getId());
+        JsonObject reqBody = parseJsonObject(exchange);
+        if (!invMgr.canInvite(uId, reqBody)) {
+            throw new NotAllowedException("Your account is not allowed to invite that address");
+        }
+
+        log.info("Invite was allowing, relaying to the Homeserver");
+        proxyPost(exchange, reqBody, client, dns);
+    }
+
+}

src/main/java/io/kamax/mxisd/http/undertow/handler/register/v1/Register3pidRequestTokenHandler.java

@@ -0,0 +1,77 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.http.undertow.handler.register.v1;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix.ThreePidMedium;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.exception.NotAllowedException;
+import io.kamax.mxisd.http.io.identity.SessionEmailTokenRequestJson;
+import io.kamax.mxisd.http.io.identity.SessionPhoneTokenRequestJson;
+import io.kamax.mxisd.http.undertow.handler.BasicHttpHandler;
+import io.kamax.mxisd.registration.RegistrationManager;
+import io.undertow.server.HttpServerExchange;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class Register3pidRequestTokenHandler extends BasicHttpHandler {
+
+    public static final String Key = "medium";
+    public static final String Path = "/_matrix/client/r0/register/{" + Key + "}/requestToken";
+
+    private static final Logger log = LoggerFactory.getLogger(Register3pidRequestTokenHandler.class);
+
+    private final RegistrationManager mgr;
+    private final ClientDnsOverwrite dns;
+    private final CloseableHttpClient client;
+
+    public Register3pidRequestTokenHandler(RegistrationManager mgr, ClientDnsOverwrite dns, CloseableHttpClient client) {
+        this.mgr = mgr;
+        this.dns = dns; // FIXME this shouldn't be in here but in the manager
+        this.client = client; // FIXME this shouldn't be in here but in the manager
+    }
+
+    @Override
+    public void handleRequest(HttpServerExchange exchange) {
+        JsonObject body = parseJsonObject(exchange);
+
+        String medium = getPathVariable(exchange, Key);
+        String address = GsonUtil.findString(body, "address").orElse("");
+        if (ThreePidMedium.Email.is(medium)) {
+            address = GsonUtil.get().fromJson(body, SessionEmailTokenRequestJson.class).getValue();
+        } else if (ThreePidMedium.PhoneNumber.is(medium)) {
+            address = GsonUtil.get().fromJson(body, SessionPhoneTokenRequestJson.class).getValue();
+        } else {
+            log.warn("Unsupported 3PID medium. We attempted to extract the address but the call might fail");
+        }
+
+        ThreePid tpid = new ThreePid(medium, address);
+        if (!mgr.isAllowed(tpid)) {
+            throw new NotAllowedException("Your " + medium + " address cannot be used for registration");
+        }
+
+        proxyPost(exchange, body, client, dns);
+    }
+
+}

src/main/java/io/kamax/mxisd/http/undertow/handler/status/StatusHandler.java

@@ -26,7 +26,7 @@ import io.undertow.server.HttpServerExchange;
 
 public class StatusHandler extends BasicHttpHandler {
 
-    public static final String Path = "/_matrix/identity/status";
+    public static final String Path = "/status";
 
     @Override
     public void handleRequest(HttpServerExchange exchange) {

src/main/java/io/kamax/mxisd/http/undertow/handler/status/VersionHandler.java

@@ -0,0 +1,48 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.http.undertow.handler.status;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.Mxisd;
+import io.kamax.mxisd.http.undertow.handler.BasicHttpHandler;
+import io.undertow.server.HttpServerExchange;
+
+public class VersionHandler extends BasicHttpHandler {
+
+    public static final String Path = "/version";
+
+    private final String body;
+
+    public VersionHandler() {
+        JsonObject server = new JsonObject();
+        server.addProperty("name", Mxisd.Name);
+        server.addProperty("version", Mxisd.Version);
+
+        body = GsonUtil.getPrettyForLog(GsonUtil.makeObj("server", server));
+    }
+
+    @Override
+    public void handleRequest(HttpServerExchange exchange) {
+        respondJson(exchange, body);
+    }
+
+}

src/main/java/io/kamax/mxisd/invitation/IMatrixIdInvite.java

@@ -18,10 +18,9 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.as;
+package io.kamax.mxisd.invitation;
 
 import io.kamax.matrix._MatrixID;
-import io.kamax.mxisd.invitation.IThreePidInvite;
 
 public interface IMatrixIdInvite extends IThreePidInvite {
 

src/main/java/io/kamax/mxisd/invitation/IThreePidInviteReply.java

@@ -20,6 +20,8 @@
 
 package io.kamax.mxisd.invitation;
 
+import java.util.List;
+
 public interface IThreePidInviteReply {
 
     String getId();
@@ -30,4 +32,6 @@ public interface IThreePidInviteReply {
 
     String getDisplayName();
 
+    List<String> getPublicKeys();
+
 }

src/main/java/io/kamax/mxisd/invitation/InvitationManager.java

@@ -23,21 +23,29 @@ package io.kamax.mxisd.invitation;
 import com.google.gson.JsonArray;
 import com.google.gson.JsonObject;
 import io.kamax.matrix.MatrixID;
-import io.kamax.matrix.crypto.SignatureManager;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
 import io.kamax.matrix.json.GsonUtil;
 import io.kamax.mxisd.config.InvitationConfig;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.config.ServerConfig;
+import io.kamax.mxisd.crypto.*;
 import io.kamax.mxisd.dns.FederationDnsOverwrite;
 import io.kamax.mxisd.exception.BadRequestException;
+import io.kamax.mxisd.exception.ConfigurationException;
 import io.kamax.mxisd.exception.MappingAlreadyExistsException;
+import io.kamax.mxisd.exception.ObjectNotFoundException;
 import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.ThreePidMapping;
 import io.kamax.mxisd.lookup.strategy.LookupStrategy;
 import io.kamax.mxisd.notification.NotificationManager;
+import io.kamax.mxisd.profile.ProfileManager;
 import io.kamax.mxisd.storage.IStorage;
 import io.kamax.mxisd.storage.ormlite.dao.ThreePidInviteIO;
+import org.apache.commons.codec.binary.Base64;
 import org.apache.commons.io.IOUtils;
-import org.apache.commons.lang.RandomStringUtils;
+import org.apache.commons.lang3.RandomStringUtils;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang3.StringUtils;
 import org.apache.http.client.methods.CloseableHttpResponse;
 import org.apache.http.client.methods.HttpPost;
 import org.apache.http.conn.ssl.NoopHostnameVerifier;
@@ -57,6 +65,8 @@ import java.io.IOException;
 import java.net.MalformedURLException;
 import java.net.URL;
 import java.nio.charset.StandardCharsets;
+import java.time.DateTimeException;
+import java.time.Instant;
 import java.util.*;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ForkJoinPool;
@@ -64,14 +74,20 @@ import java.util.concurrent.TimeUnit;
 
 public class InvitationManager {
 
-    private transient final Logger log = LoggerFactory.getLogger(InvitationManager.class);
+    private static final Logger log = LoggerFactory.getLogger(InvitationManager.class);
+    private static final String CreatedAtPropertyKey = "created_at";
+
+    private final String defaultCreateTs = Long.toString(Instant.now().toEpochMilli());
 
     private InvitationConfig cfg;
+    private ServerConfig srvCfg;
     private IStorage storage;
     private LookupStrategy lookupMgr;
+    private KeyManager keyMgr;
     private SignatureManager signMgr;
     private FederationDnsOverwrite dns;
     private NotificationManager notifMgr;
+    private ProfileManager profileMgr;
 
     private CloseableHttpClient client;
     private Timer refreshTimer;
@@ -79,23 +95,29 @@ public class InvitationManager {
     private Map<String, IThreePidInviteReply> invitations = new ConcurrentHashMap<>();
 
     public InvitationManager(
-            InvitationConfig cfg,
+            MxisdConfig mxisdCfg,
             IStorage storage,
             LookupStrategy lookupMgr,
+            KeyManager keyMgr,
             SignatureManager signMgr,
             FederationDnsOverwrite dns,
-            NotificationManager notifMgr
+            NotificationManager notifMgr,
+            ProfileManager profileMgr
     ) {
-        this.cfg = cfg;
+        this.cfg = requireValid(mxisdCfg);
+        this.srvCfg = mxisdCfg.getServer();
         this.storage = storage;
         this.lookupMgr = lookupMgr;
+        this.keyMgr = keyMgr;
         this.signMgr = signMgr;
         this.dns = dns;
         this.notifMgr = notifMgr;
+        this.profileMgr = profileMgr;
 
         log.info("Loading saved invites");
         Collection<ThreePidInviteIO> ioList = storage.getInvites();
         ioList.forEach(io -> {
+            io.getProperties().putIfAbsent(CreatedAtPropertyKey, defaultCreateTs);
             log.info("Processing invite {}", GsonUtil.get().toJson(io));
             ThreePidInvite invite = new ThreePidInvite(
                     MatrixID.asAcceptable(io.getSender()),
@@ -105,7 +127,7 @@ public class InvitationManager {
                     io.getProperties()
             );
 
-            ThreePidInviteReply reply = new ThreePidInviteReply(getId(invite), invite, io.getToken(), "");
+            ThreePidInviteReply reply = new ThreePidInviteReply(io.getId(), invite, io.getToken(), "", Collections.emptyList());
             invitations.put(reply.getId(), reply);
         });
 
@@ -122,25 +144,69 @@ public class InvitationManager {
 
         log.info("Setting up invitation mapping refresh timer");
         refreshTimer = new Timer();
-        refreshTimer.scheduleAtFixedRate(new TimerTask() {
-            @Override
-            public void run() {
-                try {
-                    lookupMappingsForInvites();
-                } catch (Throwable t) {
-                    log.error("Error when running background mapping refresh", t);
-                }
-            }
-        }, 5000L, TimeUnit.MILLISECONDS.convert(cfg.getResolution().getTimer(), TimeUnit.MINUTES));
 
+        // We add a shutdown hook to cancel the hook and wait for pending resolutions
         Runtime.getRuntime().addShutdownHook(new Thread(() -> {
             refreshTimer.cancel();
             ForkJoinPool.commonPool().awaitQuiescence(1, TimeUnit.MINUTES);
         }));
+
+        // We set the refresh timer for background tasks
+        refreshTimer.scheduleAtFixedRate(new TimerTask() {
+            @Override
+            public void run() {
+                try {
+                    doMaintenance();
+                } catch (Throwable t) {
+                    log.error("Error when running background maintenance", t);
+                }
+            }
+        }, 5000L, TimeUnit.MILLISECONDS.convert(cfg.getResolution().getTimer(), TimeUnit.MINUTES));
     }
 
-    private String getId(IThreePidInvite invite) {
+    private InvitationConfig requireValid(MxisdConfig cfg) {
-        return invite.getSender().getDomain().toLowerCase() + invite.getMedium().toLowerCase() + invite.getAddress().toLowerCase();
+        // This is not configured, we'll apply a default configuration
+        if (Objects.isNull(cfg.getInvite().getExpiration().isEnabled())) {
+            // We compute our own user, so it can be used if we bridge as well
+            String mxId = MatrixID.asAcceptable("_mxisd-expired_invite", cfg.getMatrix().getDomain()).getId();
+
+            // Enabled by default
+            cfg.getInvite().getExpiration().setEnabled(true);
+
+            // We'll resolve to our computed User ID
+            cfg.getInvite().getExpiration().setResolveTo(mxId);
+
+            // One calendar week (60min/1h * 24 = 1d * 7 = 1w)
+            cfg.getInvite().getExpiration().setAfter(60 * 24 * 7);
+        }
+
+        if (cfg.getInvite().getExpiration().isEnabled()) {
+            if (cfg.getInvite().getExpiration().getAfter() < 1) {
+                throw new ConfigurationException("Invitation expiration delay must be greater or equal to 1");
+            }
+
+            if (StringUtils.isBlank(cfg.getInvite().getExpiration().getResolveTo())) {
+                String localpart = cfg.getAppsvc().getUser().getInviteExpired();
+                if (StringUtils.isBlank(localpart)) {
+                    throw new ConfigurationException("Could not compute the Invitation expiration resolution target from App service user: not set");
+                }
+
+                cfg.getInvite().getExpiration().setResolveTo(MatrixID.asAcceptable(localpart, cfg.getMatrix().getDomain()).getId());
+            }
+
+            try {
+                MatrixID.asAcceptable(cfg.getInvite().getExpiration().getResolveTo());
+            } catch (IllegalArgumentException e) {
+                throw new ConfigurationException("Invitation expiration resolution target is not a valid Matrix ID: " + e.getMessage());
+            }
+        }
+
+        return cfg.getInvite();
+    }
+
+    private String computeId(IThreePidInvite invite) {
+        String rawId = invite.getSender().getDomain().toLowerCase() + invite.getMedium().toLowerCase() + invite.getAddress().toLowerCase();
+        return Base64.encodeBase64URLSafeString(rawId.getBytes(StandardCharsets.UTF_8));
     }
 
     private String getIdForLog(IThreePidInviteReply reply) {
@@ -205,19 +271,56 @@ public class InvitationManager {
         return lookupMgr.find(medium, address, cfg.getResolution().isRecursive());
     }
 
+    public List<IThreePidInviteReply> listInvites() {
+        return new ArrayList<>(invitations.values());
+    }
+
+    public IThreePidInviteReply getInvite(String id) {
+        IThreePidInviteReply v = invitations.get(id);
+        if (Objects.isNull(v)) {
+            throw new ObjectNotFoundException("Invite", id);
+        }
+
+        return v;
+    }
+
+    public boolean canInvite(_MatrixID sender, JsonObject request) {
+        if (!request.has("medium")) {
+            log.info("Not a 3PID invite, allowing");
+            return true;
+        }
+        log.info("3PID invite detected, checking policies...");
+
+        List<String> allowedRoles = cfg.getPolicy().getIfSender().getHasRole();
+        if (Objects.isNull(allowedRoles)) {
+            log.info("No allowed role configured for sender, allowing");
+            return true;
+        }
+
+        List<String> userRoles = profileMgr.getRoles(sender);
+        if (Collections.disjoint(userRoles, allowedRoles)) {
+            log.info("Sender does not have any of the required roles, denying");
+            return false;
+        }
+        log.info("Sender has at least one of the required roles");
+
+        log.info("Sender pass all policies to invite, allowing");
+        return true;
+    }
+
     public synchronized IThreePidInviteReply storeInvite(IThreePidInvite invitation) { // TODO better sync
         if (!notifMgr.isMediumSupported(invitation.getMedium())) {
             throw new BadRequestException("Medium type " + invitation.getMedium() + " is not supported");
         }
 
-        String invId = getId(invitation);
+        String invId = computeId(invitation);
         log.info("Handling invite for {}:{} from {} in room {}", invitation.getMedium(), invitation.getAddress(), invitation.getSender(), invitation.getRoomId());
         IThreePidInviteReply reply = invitations.get(invId);
         if (reply != null) {
             log.info("Invite is already pending for {}:{}, returning data", invitation.getMedium(), invitation.getAddress());
             if (!StringUtils.equals(invitation.getRoomId(), reply.getInvite().getRoomId())) {
                 log.info("Sending new notification as new invite room {} is different from the original {}", invitation.getRoomId(), reply.getInvite().getRoomId());
-                notifMgr.sendForReply(new ThreePidInviteReply(reply.getId(), invitation, reply.getToken(), reply.getDisplayName()));
+                notifMgr.sendForReply(new ThreePidInviteReply(reply.getId(), invitation, reply.getToken(), reply.getDisplayName(), reply.getPublicKeys()));
             } else {
                 // FIXME we should check attempt and send if bigger
             }
@@ -232,8 +335,21 @@ public class InvitationManager {
 
         String token = RandomStringUtils.randomAlphanumeric(64);
         String displayName = invitation.getAddress().substring(0, 3) + "...";
+        KeyIdentifier pKeyId = keyMgr.getServerSigningKey().getId();
+        KeyIdentifier eKeyId = keyMgr.generateKey(KeyType.Ephemeral);
 
-        reply = new ThreePidInviteReply(invId, invitation, token, displayName);
+        String pPubKey = keyMgr.getPublicKeyBase64(pKeyId);
+        String ePubKey = keyMgr.getPublicKeyBase64(eKeyId);
+
+        invitation.getProperties().put(CreatedAtPropertyKey, Long.toString(Instant.now().toEpochMilli()));
+        invitation.getProperties().put("p_key_algo", pKeyId.getAlgorithm());
+        invitation.getProperties().put("p_key_serial", pKeyId.getSerial());
+        invitation.getProperties().put("p_key_public", pPubKey);
+        invitation.getProperties().put("e_key_algo", eKeyId.getAlgorithm());
+        invitation.getProperties().put("e_key_serial", eKeyId.getSerial());
+        invitation.getProperties().put("e_key_public", ePubKey);
+
+        reply = new ThreePidInviteReply(invId, invitation, token, displayName, Arrays.asList(pPubKey, ePubKey));
 
         log.info("Performing invite to {}:{}", invitation.getMedium(), invitation.getAddress());
         notifMgr.sendForReply(reply);
@@ -246,6 +362,78 @@ public class InvitationManager {
         return reply;
     }
 
+    public boolean hasInvite(ThreePid tpid) {
+        for (IThreePidInviteReply reply : invitations.values()) {
+            if (!StringUtils.equals(tpid.getMedium(), reply.getInvite().getMedium())) {
+                continue;
+            }
+
+            if (!StringUtils.equals(tpid.getAddress(), reply.getInvite().getAddress())) {
+                continue;
+            }
+
+            return true;
+        }
+
+        return false;
+    }
+
+    private void removeInvite(IThreePidInviteReply reply) {
+        invitations.remove(reply.getId());
+        storage.deleteInvite(reply.getId());
+    }
+
+    /**
+     * Trigger the periodic maintenance tasks
+     */
+    public void doMaintenance() {
+        lookupMappingsForInvites();
+        expireInvites();
+    }
+
+    public void expireInvites() {
+        log.debug("Invite expiration: started");
+
+        if (!cfg.getExpiration().isEnabled()) {
+            log.debug("Invite expiration is disabled, skipping");
+            return;
+        }
+
+        if (invitations.isEmpty()) {
+            log.debug("No invite to expired, skipping");
+            return;
+        }
+
+        String targetMxid = cfg.getExpiration().getResolveTo();
+        for (IThreePidInviteReply reply : invitations.values()) {
+            log.debug("Processing invite {}", reply.getId());
+
+            String tsRaw = reply.getInvite().getProperties().computeIfAbsent(CreatedAtPropertyKey, k -> defaultCreateTs);
+            try {
+                Instant ts = Instant.ofEpochMilli(Long.parseLong(tsRaw));
+                Instant targetTs = ts.plusSeconds(cfg.getExpiration().getAfter() * 60);
+                Instant now = Instant.now();
+                log.debug("Invite {} - Created at {} - Expires at {} - Current time is {}", reply.getId(), ts, targetTs, now);
+                if (targetTs.isAfter(now)) {
+                    log.debug("Invite {} has not expired yet, skipping", reply.getId());
+                    continue;
+                }
+
+                log.info("Invite {} has expired at TS {} - Expiring and resolving to {}", targetTs, targetMxid);
+                publishMapping(reply, targetMxid);
+            } catch (NumberFormatException | DateTimeException e) {
+                log.warn("Invite {} has an invalid creation TS, setting to default value of {}", reply.getId(), defaultCreateTs);
+                reply.getInvite().getProperties().put(CreatedAtPropertyKey, defaultCreateTs);
+            }
+        }
+
+        log.debug("Invite expiration: finished");
+    }
+
+    public void expireInvite(String id) {
+        publishMapping(getInvite(id), cfg.getExpiration().getResolveTo());
+    }
+
     public void lookupMappingsForInvites() {
         if (!invitations.isEmpty()) {
             log.info("Checking for existing mapping for pending invites");
@@ -266,6 +454,28 @@ public class InvitationManager {
         }
     }
 
+    public IThreePidInviteReply getInvite(String token, String privKey) {
+        for (IThreePidInviteReply reply : invitations.values()) {
+            if (StringUtils.equals(reply.getToken(), token)) {
+                String algo = reply.getInvite().getProperties().get("e_key_algo");
+                String serial = reply.getInvite().getProperties().get("e_key_serial");
+
+                if (StringUtils.isAnyBlank(algo, serial)) {
+                    continue;
+                }
+
+                String storedPrivKey = keyMgr.getKey(new GenericKeyIdentifier(KeyType.Ephemeral, algo, serial)).getPrivateKeyBase64();
+                if (!StringUtils.equals(storedPrivKey, privKey)) {
+                    continue;
+                }
+
+                return reply;
+            }
+        }
+
+        throw new ObjectNotFoundException("No invite with such token and/or private key");
+    }
+
     private void publishMapping(IThreePidInviteReply reply, String mxid) {
         String medium = reply.getInvite().getMedium();
         String address = reply.getInvite().getAddress();
@@ -280,7 +490,7 @@ public class InvitationManager {
             JsonObject obj = new JsonObject();
             obj.addProperty("mxid", mxid);
             obj.addProperty("token", reply.getToken());
-            obj.add("signatures", signMgr.signMessageGson(obj.toString()));
+            obj.add("signatures", signMgr.signMessageGson(srvCfg.getName(), obj.toString()));
 
             JsonObject objUp = new JsonObject();
             objUp.addProperty("mxid", mxid);
@@ -298,30 +508,45 @@ public class InvitationManager {
             content.addProperty("address", address);
             content.addProperty("mxid", mxid);
 
-            content.add("signatures", signMgr.signMessageGson(content.toString()));
+            content.add("signatures", signMgr.signMessageGson(srvCfg.getName(), content.toString()));
 
             StringEntity entity = new StringEntity(content.toString(), StandardCharsets.UTF_8);
             entity.setContentType("application/json");
             req.setEntity(entity);
+
+            Instant resolvedAt = Instant.now();
+            boolean couldPublish = false;
+            boolean shouldArchive = true;
             try {
                 log.info("Posting onBind event to {}", req.getURI());
                 CloseableHttpResponse response = client.execute(req);
                 int statusCode = response.getStatusLine().getStatusCode();
                 log.info("Answer code: {}", statusCode);
                 if (statusCode >= 300 && statusCode != 403) {
-                    log.warn("Answer body: {}", IOUtils.toString(response.getEntity().getContent(), StandardCharsets.UTF_8));
+                    log.info("Answer body: {}", IOUtils.toString(response.getEntity().getContent(), StandardCharsets.UTF_8));
-                } else {
+                    log.warn("HS returned an error.");
-                    if (statusCode == 403) {
-                        log.info("Invite was obsolete");
-                    }
 
-                    invitations.remove(getId(reply.getInvite()));
+                    shouldArchive = statusCode != 502;
-                    storage.deleteInvite(reply.getId());
+                    if (shouldArchive) {
-                    log.info("Removed invite from internal store");
+                        log.info("Invite can be found in historical storage for manual re-processing");
+                    }
+                } else {
+                    couldPublish = true;
+                    if (statusCode == 403) {
+                        log.info("Invite is obsolete or no longer under our control");
+                    }
                 }
                 response.close();
             } catch (IOException e) {
                 log.warn("Unable to tell HS {} about invite being mapped", domain, e);
+            } finally {
+                if (shouldArchive) {
+                    synchronized (this) {
+                        storage.insertHistoricalInvite(reply, mxid, resolvedAt, couldPublish);
+                        removeInvite(reply);
+                        log.info("Moved invite {} to historical table", reply.getId());
+                    }
+                }
             }
         }).start();
     }
@@ -337,7 +562,7 @@ public class InvitationManager {
         @Override
         public void run() {
             try {
-                log.info("Searching for mapping created since invite {} was created", getIdForLog(reply));
+                log.info("Searching for mapping created after invite {} was created", getIdForLog(reply));
                 Optional<SingleLookupReply> result = lookup3pid(reply.getInvite().getMedium(), reply.getInvite().getAddress());
                 if (result.isPresent()) {
                     SingleLookupReply lookup = result.get();

src/main/java/io/kamax/mxisd/invitation/MatrixIdInvite.java

@@ -18,7 +18,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.as;
+package io.kamax.mxisd.invitation;
 
 import io.kamax.matrix._MatrixID;
 

src/main/java/io/kamax/mxisd/invitation/ThreePidInviteReply.java

@@ -20,18 +20,24 @@
 
 package io.kamax.mxisd.invitation;
 
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
 public class ThreePidInviteReply implements IThreePidInviteReply {
 
     private String id;
     private IThreePidInvite invite;
     private String token;
     private String displayName;
+    private List<String> publicKeys;
 
-    public ThreePidInviteReply(String id, IThreePidInvite invite, String token, String displayName) {
+    public ThreePidInviteReply(String id, IThreePidInvite invite, String token, String displayName, List<String> publicKeys) {
         this.id = id;
         this.invite = invite;
         this.token = token;
         this.displayName = displayName;
+        this.publicKeys = Collections.unmodifiableList(new ArrayList<>(publicKeys));
     }
 
     @Override
@@ -54,4 +60,9 @@ public class ThreePidInviteReply implements IThreePidInviteReply {
         return displayName;
     }
 
+    @Override
+    public List<String> getPublicKeys() {
+        return publicKeys;
+    }
+
 }

src/main/java/io/kamax/mxisd/lookup/SingleLookupReply.java

@@ -27,6 +27,8 @@ import io.kamax.matrix._MatrixID;
 import io.kamax.mxisd.http.io.identity.SingeLookupReplyJson;
 
 import java.time.Instant;
+import java.util.HashMap;
+import java.util.Map;
 
 public class SingleLookupReply {
 
@@ -39,6 +41,7 @@ public class SingleLookupReply {
     private Instant notBefore;
     private Instant notAfter;
     private Instant timestamp;
+    private Map<String, Map<String, String>> signatures = new HashMap<>();
 
     public static SingleLookupReply fromRecursive(SingleLookupRequest request, String body) {
         SingleLookupReply reply = new SingleLookupReply();
@@ -52,6 +55,7 @@ public class SingleLookupReply {
             reply.notAfter = Instant.ofEpochMilli(json.getNot_after());
             reply.notBefore = Instant.ofEpochMilli(json.getNot_before());
             reply.timestamp = Instant.ofEpochMilli(json.getTs());
+            reply.signatures = new HashMap<>(json.getSignatures());
         } catch (JsonSyntaxException e) {
             // stub - we only want to try, nothing more
         }
@@ -107,4 +111,12 @@ public class SingleLookupReply {
         return timestamp;
     }
 
+    public Map<String, Map<String, String>> getSignatures() {
+        return signatures;
+    }
+
+    public Map<String, String> getSignature(String host) {
+        return signatures.computeIfAbsent(host, k -> new HashMap<>());
+    }
+
 }

src/main/java/io/kamax/mxisd/notification/NotificationHandler.java

@@ -21,7 +21,7 @@
 package io.kamax.mxisd.notification;
 
 import io.kamax.matrix.ThreePid;
-import io.kamax.mxisd.as.IMatrixIdInvite;
+import io.kamax.mxisd.invitation.IMatrixIdInvite;
 import io.kamax.mxisd.invitation.IThreePidInviteReply;
 import io.kamax.mxisd.threepid.session.IThreePidSession;
 

src/main/java/io/kamax/mxisd/notification/NotificationManager.java

@@ -21,9 +21,9 @@
 package io.kamax.mxisd.notification;
 
 import io.kamax.matrix.ThreePid;
-import io.kamax.mxisd.as.IMatrixIdInvite;
 import io.kamax.mxisd.config.threepid.notification.NotificationConfig;
 import io.kamax.mxisd.exception.NotImplementedException;
+import io.kamax.mxisd.invitation.IMatrixIdInvite;
 import io.kamax.mxisd.invitation.IThreePidInviteReply;
 import io.kamax.mxisd.threepid.session.IThreePidSession;
 import org.apache.commons.lang.StringUtils;

src/main/java/io/kamax/mxisd/profile/ProfileManager.java

@@ -37,10 +37,7 @@ import org.slf4j.LoggerFactory;
 
 import java.io.IOException;
 import java.net.URISyntaxException;
-import java.util.ArrayList;
+import java.util.*;
-import java.util.Collection;
-import java.util.List;
-import java.util.Optional;
 import java.util.function.Function;
 import java.util.stream.Collectors;
 
@@ -113,4 +110,8 @@ public class ProfileManager {
         }
     }
 
+    public boolean hasAnyRole(_MatrixID user, List<String> requiredRoles) {
+        return !requiredRoles.isEmpty() || Collections.disjoint(getRoles(user), requiredRoles);
+    }
+
 }

src/main/java/io/kamax/mxisd/registration/RegistrationManager.java

@@ -0,0 +1,142 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.registration;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.config.RegisterConfig;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.exception.NotImplementedException;
+import io.kamax.mxisd.exception.RemoteHomeServerException;
+import io.kamax.mxisd.invitation.InvitationManager;
+import io.kamax.mxisd.util.RestClientUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.Objects;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class RegistrationManager {
+
+    private static final Logger log = LoggerFactory.getLogger(RegistrationManager.class);
+
+    private final RegisterConfig cfg;
+    private final CloseableHttpClient client;
+    private final ClientDnsOverwrite dns;
+    private final InvitationManager invMgr;
+
+    public RegistrationManager(RegisterConfig cfg, CloseableHttpClient client, ClientDnsOverwrite dns, InvitationManager invMgr) {
+        this.cfg = cfg;
+        this.client = client;
+        this.dns = dns;
+        this.invMgr = invMgr;
+    }
+
+    private String resolveProxyUrl(URI target) {
+        URIBuilder builder = dns.transform(target);
+        String urlToLogin = builder.toString();
+        log.info("Proxy resolution: {} to {}", target.toString(), urlToLogin);
+        return urlToLogin;
+    }
+
+    public RegistrationReply execute(URI target, JsonObject request) {
+        HttpPost registerProxyRq = RestClientUtils.post(resolveProxyUrl(target), GsonUtil.get(), request);
+        try (CloseableHttpResponse response = client.execute(registerProxyRq)) {
+            int status = response.getStatusLine().getStatusCode();
+            if (status == 200) {
+                // The user managed to register. We check if it had a session
+                String sessionId = GsonUtil.findObj(request, "auth").flatMap(auth -> GsonUtil.findString(auth, "session")).orElse("");
+                if (StringUtils.isEmpty(sessionId)) {
+                    // No session ID was provided. This is an edge case we do not support for now as investigation is needed
+                    // to ensure how and when this happens.
+
+                    HttpPost newSessReq = RestClientUtils.post(resolveProxyUrl(target), GsonUtil.get(), new JsonObject());
+                    try (CloseableHttpResponse newSessRes = client.execute(newSessReq)) {
+                        RegistrationReply reply = new RegistrationReply();
+                        reply.setStatus(newSessRes.getStatusLine().getStatusCode());
+                        reply.setBody(GsonUtil.parseObj(EntityUtils.toString(newSessRes.getEntity())));
+                        return reply;
+                    }
+                }
+            }
+
+            throw new NotImplementedException("Registration");
+        } catch (IOException e) {
+            throw new RemoteHomeServerException(e.getMessage());
+        }
+    }
+
+    public boolean isAllowed(ThreePid tpid) {
+        // We check if the policy allows registration for invites, and if there is an invite for the 3PID
+        if (cfg.getPolicy().forInvite() && invMgr.hasInvite(tpid)) {
+            log.info("Registration allowed for pending invite");
+            return true;
+        }
+
+        // The following section deals with patterns which can either be built at startup time, or for each invite at runtime.
+        // Registration is a very rare occurrence relatively speaking, so we make the choice to build the patterns each time
+        // at runtime to save on RAM.
+
+        Object policy = cfg.getPolicy().getThreepid().get(tpid.getMedium());
+        if (Objects.nonNull(policy)) {
+            RegisterConfig.ThreepidPolicy tpidPolicy = GsonUtil.get().fromJson(GsonUtil.get().toJson(policy), RegisterConfig.ThreepidPolicy.class);
+            log.info("Found registration policy for {}", tpid.getMedium());
+
+            log.info("Processing pattern blacklist");
+            for (String pattern : tpidPolicy.getPattern().getBlacklist()) {
+                log.info("Processing pattern {}", pattern);
+
+                // We compile the pattern
+                Matcher m = Pattern.compile(pattern).matcher(tpid.getAddress());
+                if (m.matches()) { // We only care about those who match...
+                    log.info("Found matching blacklist entry, denying registration");
+                    return false; // ... and get denied as per blacklist
+                }
+            }
+
+            log.info("Processing pattern whitelist");
+            for (String pattern : tpidPolicy.getPattern().getWhitelist()) {
+                log.info("Processing pattern {}", pattern);
+
+                // We compile the pattern
+                Matcher m = Pattern.compile(pattern).matcher(tpid.getAddress());
+                if (m.matches()) { // We only care about those who match...
+                    log.info("Found matching whitelist entry, allowing registration");
+                    return true; // ... and get accepted as per whitelist
+                }
+            }
+        }
+
+        log.info("Returning default registration policy: {}", cfg.getPolicy().isAllowed());
+        return cfg.getPolicy().isAllowed();
+    }
+
+}

src/main/java/io/kamax/mxisd/registration/RegistrationReply.java

@@ -0,0 +1,46 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.registration;
+
+import com.google.gson.JsonObject;
+
+public class RegistrationReply {
+
+    private int status;
+    private JsonObject body;
+
+    public int getStatus() {
+        return status;
+    }
+
+    public void setStatus(int status) {
+        this.status = status;
+    }
+
+    public JsonObject getBody() {
+        return body;
+    }
+
+    public void setBody(JsonObject body) {
+        this.body = body;
+    }
+
+}

src/main/java/io/kamax/mxisd/storage/IStorage.java

@@ -38,6 +38,8 @@ public interface IStorage {
 
     void deleteInvite(String id);
 
+    void insertHistoricalInvite(IThreePidInviteReply data, String resolvedTo, Instant resolvedAt, boolean couldPublish);
+
     Optional<IThreePidSessionDao> getThreePidSession(String sid);
 
     Optional<IThreePidSessionDao> findThreePidSession(ThreePid tpid, String secret);

src/main/java/io/kamax/mxisd/storage/crypto/FileKeyJson.java

@@ -0,0 +1,63 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.storage.crypto;
+
+import io.kamax.mxisd.crypto.Key;
+
+public class FileKeyJson {
+
+    public static FileKeyJson get(Key key) {
+        FileKeyJson json = new FileKeyJson();
+        json.setVersion("0");
+        json.setKey(key.getPrivateKeyBase64());
+        json.setValid(key.isValid());
+        return json;
+    }
+
+    private String version;
+    private boolean isValid;
+    private String key;
+
+    public String getVersion() {
+        return version;
+    }
+
+    public void setVersion(String version) {
+        this.version = version;
+    }
+
+    public boolean isValid() {
+        return isValid;
+    }
+
+    public void setValid(boolean valid) {
+        isValid = valid;
+    }
+
+    public String getKey() {
+        return key;
+    }
+
+    public void setKey(String key) {
+        this.key = key;
+    }
+
+}

src/main/java/io/kamax/mxisd/storage/crypto/FileKeyStore.java

@@ -0,0 +1,256 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.storage.crypto;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.crypto.KeyFileStore;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.crypto.*;
+import io.kamax.mxisd.exception.ObjectNotFoundException;
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+
+public class FileKeyStore implements KeyStore {
+
+    private static final Logger log = LoggerFactory.getLogger(FileKeyStore.class);
+
+    private final String currentFilename = "current";
+    private final String base;
+
+    public FileKeyStore(String path) {
+        base = new File(path).getAbsoluteFile().toString();
+        File f = new File(base);
+
+        if (!f.exists()) {
+            try {
+                FileUtils.forceMkdir(f);
+            } catch (IOException e) {
+                throw new RuntimeException("Unable to create key store");
+            }
+        } else {
+            if (f.isFile()) {
+                try {
+                    log.info("Found old key store format at {}, migrating...", base);
+                    File oldStorePath = new File(f.toString() + ".backup-before-migration");
+                    FileUtils.moveFile(f, oldStorePath);
+                    FileUtils.forceMkdir(f);
+
+
+                    String privKey = new KeyFileStore(oldStorePath.toString()).load().orElse("");
+                    if (StringUtils.isBlank(privKey)) {
+                        log.info("Empty file, nothing to migrate");
+                    } else {
+                        // We ensure this is valid Base64 data before migrating
+                        Base64.decodeBase64(privKey);
+
+                        // We store the new key
+                        add(new GenericKey(new GenericKeyIdentifier(KeyType.Regular, KeyAlgorithm.Ed25519, "0"), true, privKey));
+
+                        log.info("Store migrated to new directory format");
+                    }
+                } catch (IOException e) {
+                    throw new RuntimeException("Unable to migrate store from old single file format to new directory format", e);
+                }
+            } else {
+                log.info("Key store is already in directory format");
+            }
+        }
+
+        if (!f.isDirectory()) {
+            throw new RuntimeException("Key store path is not a directory: " + f.toString());
+        }
+    }
+
+    private String toDirName(KeyType type) {
+        return type.name().toLowerCase();
+    }
+
+    private Path ensureDirExists(KeyIdentifier id) {
+        File b = Paths.get(base, toDirName(id.getType()), id.getAlgorithm()).toFile();
+
+        if (b.exists()) {
+            if (!b.isDirectory()) {
+                throw new RuntimeException("Key store path already exists but is not a directory: " + b.toString());
+            }
+        } else {
+            try {
+                FileUtils.forceMkdir(b);
+            } catch (IOException e) {
+                throw new RuntimeException("Unable to create key store path at " + b.toString(), e);
+            }
+        }
+
+        return b.toPath();
+    }
+
+    @Override
+    public boolean has(KeyIdentifier id) {
+        return Paths.get(base, toDirName(id.getType()), id.getAlgorithm(), id.getSerial()).toFile().isFile();
+    }
+
+    @Override
+    public List<KeyIdentifier> list() {
+        List<KeyIdentifier> keyIds = new ArrayList<>();
+
+        for (KeyType type : KeyType.values()) {
+            keyIds.addAll(list(type));
+        }
+
+        return keyIds;
+    }
+
+    @Override
+    public List<KeyIdentifier> list(KeyType type) {
+        List<KeyIdentifier> keyIds = new ArrayList<>();
+
+        File algoDir = Paths.get(base, toDirName(type)).toFile();
+        File[] algos = algoDir.listFiles();
+        if (Objects.isNull(algos)) {
+            return keyIds;
+        }
+
+        for (File algo : algos) {
+            File[] serials = algo.listFiles();
+            if (Objects.isNull(serials)) {
+                throw new IllegalStateException("Cannot list stored key serials: was expecting " + algo.toString() + " to be a directory");
+            }
+
+            for (File serial : serials) {
+                keyIds.add(new GenericKeyIdentifier(type, algo.getName(), serial.getName()));
+            }
+        }
+
+        return keyIds;
+    }
+
+    @Override
+    public Key get(KeyIdentifier id) throws ObjectNotFoundException {
+        File keyFile = ensureDirExists(id).resolve(id.getSerial()).toFile();
+        if (!keyFile.exists() || !keyFile.isFile()) {
+            throw new ObjectNotFoundException("Key", id.getId());
+        }
+
+        try (FileInputStream keyIs = new FileInputStream(keyFile)) {
+            FileKeyJson json = GsonUtil.get().fromJson(IOUtils.toString(keyIs, StandardCharsets.UTF_8), FileKeyJson.class);
+            return new GenericKey(id, json.isValid(), json.getKey());
+        } catch (IOException e) {
+            throw new RuntimeException("Unable to read key " + id.getId(), e);
+        }
+    }
+
+    @Override
+    public void add(Key key) throws IllegalStateException {
+        File keyFile = ensureDirExists(key.getId()).resolve(key.getId().getSerial()).toFile();
+        if (keyFile.exists()) {
+            throw new IllegalStateException("Key " + key.getId().getId() + " already exists");
+        }
+
+        FileKeyJson json = FileKeyJson.get(key);
+        try (FileOutputStream keyOs = new FileOutputStream(keyFile, false)) {
+            IOUtils.write(GsonUtil.get().toJson(json), keyOs, StandardCharsets.UTF_8);
+        } catch (IOException e) {
+            throw new RuntimeException("Unable to create key " + key.getId().getId(), e);
+        }
+    }
+
+    @Override
+    public void update(Key key) throws ObjectNotFoundException {
+        File keyFile = ensureDirExists(key.getId()).resolve(key.getId().getSerial()).toFile();
+        if (!keyFile.exists() || !keyFile.isFile()) {
+            throw new ObjectNotFoundException("Key", key.getId().getId());
+        }
+
+        FileKeyJson json = FileKeyJson.get(key);
+        try (FileOutputStream keyOs = new FileOutputStream(keyFile, false)) {
+            IOUtils.write(GsonUtil.get().toJson(json), keyOs, StandardCharsets.UTF_8);
+        } catch (IOException e) {
+            throw new RuntimeException("Unable to create key " + key.getId().getId(), e);
+        }
+    }
+
+    @Override
+    public void delete(KeyIdentifier id) throws ObjectNotFoundException {
+        File keyFile = ensureDirExists(id).resolve(id.getSerial()).toFile();
+        if (!keyFile.exists() || !keyFile.isFile()) {
+            throw new ObjectNotFoundException("Key", id.getId());
+        }
+
+        if (!keyFile.delete()) {
+            throw new RuntimeException("Unable to delete key " + id.getId());
+        }
+    }
+
+    @Override
+    public void setCurrentKey(KeyIdentifier id) throws IllegalArgumentException {
+        if (!has(id)) {
+            throw new IllegalArgumentException("Key " + id.getType() + ":" + id.getAlgorithm() + ":" + id.getSerial() + " is not known to the store");
+        }
+
+        JsonObject json = new JsonObject();
+        json.addProperty("type", id.getType().name());
+        json.addProperty("algo", id.getAlgorithm());
+        json.addProperty("serial", id.getSerial());
+
+        File f = Paths.get(base, currentFilename).toFile();
+
+        try (FileOutputStream keyOs = new FileOutputStream(f, false)) {
+            IOUtils.write(GsonUtil.get().toJson(json), keyOs, StandardCharsets.UTF_8);
+        } catch (IOException e) {
+            throw new RuntimeException("Unable to write to " + f.toString(), e);
+        }
+    }
+
+    @Override
+    public Optional<KeyIdentifier> getCurrentKey() {
+        File f = Paths.get(base, currentFilename).toFile();
+        if (!f.exists()) {
+            return Optional.empty();
+        }
+
+        if (!f.isFile()) {
+            throw new IllegalStateException("Current key file is not a file: " + f.toString());
+        }
+
+        try (FileInputStream keyIs = new FileInputStream(f)) {
+            JsonObject json = GsonUtil.parseObj(IOUtils.toString(keyIs, StandardCharsets.UTF_8));
+            return Optional.of(new GenericKeyIdentifier(KeyType.valueOf(GsonUtil.getStringOrThrow(json, "type")), GsonUtil.getStringOrThrow(json, "algo"), GsonUtil.getStringOrThrow(json, "serial")));
+        } catch (IOException e) {
+            throw new RuntimeException("Unable to read " + f.toString(), e);
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/storage/crypto/KeyStore.java

@@ -0,0 +1,107 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.storage.crypto;
+
+import io.kamax.mxisd.crypto.Key;
+import io.kamax.mxisd.crypto.KeyIdentifier;
+import io.kamax.mxisd.crypto.KeyType;
+import io.kamax.mxisd.exception.ObjectNotFoundException;
+
+import java.util.List;
+import java.util.Optional;
+
+/**
+ * Store to persist signing keys and the identifier for the current long-term signing key
+ */
+public interface KeyStore {
+
+    /**
+     * If a given key is currently stored
+     *
+     * @param id The Identifier elements for the key
+     * @return true if the key is stored, false if not
+     */
+    boolean has(KeyIdentifier id);
+
+    /**
+     * List all keys within the store
+     *
+     * @return The list of key identifiers
+     */
+    List<KeyIdentifier> list();
+
+    /**
+     * List all keys of a given type within the store
+     *
+     * @param type The type to filter on
+     * @return The list of keys identifiers matching the given type
+     */
+    List<KeyIdentifier> list(KeyType type);
+
+    /**
+     * Get the key that relates to the given identifier
+     *
+     * @param id The identifier of the key to get
+     * @return The key
+     * @throws ObjectNotFoundException If no key is found for that identifier
+     */
+    Key get(KeyIdentifier id) throws ObjectNotFoundException;
+
+    /**
+     * Add a key to the store
+     *
+     * @param key The key to store
+     * @throws IllegalStateException If a key already exist for the given identifier data
+     */
+    void add(Key key) throws IllegalStateException;
+
+    /**
+     * Update key properties in the store
+     *
+     * @param key They key to update. <code>getId()</code> will be used to identify the key to update
+     * @throws ObjectNotFoundException If no key is found for that identifier
+     */
+    void update(Key key) throws ObjectNotFoundException;
+
+    /**
+     * Delete a key from the store
+     *
+     * @param id The key identifier of the key to delete
+     * @throws ObjectNotFoundException If no key is found for that identifier
+     */
+    void delete(KeyIdentifier id) throws ObjectNotFoundException;
+
+    /**
+     * Store the information of which key is the current signing key
+     *
+     * @param id The key identifier
+     * @throws IllegalArgumentException If the key is not known to the store
+     */
+    void setCurrentKey(KeyIdentifier id) throws IllegalArgumentException;
+
+    /**
+     * Retrieve the previously stored information of which key is the current signing key, if any
+     *
+     * @return The optional key identifier that was previously stored
+     */
+    Optional<KeyIdentifier> getCurrentKey();
+
+}

src/main/java/io/kamax/mxisd/storage/crypto/MemoryKeyStore.java

@@ -0,0 +1,113 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.storage.crypto;
+
+import io.kamax.mxisd.crypto.*;
+import io.kamax.mxisd.exception.ObjectNotFoundException;
+
+import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class MemoryKeyStore implements KeyStore {
+
+    private Map<KeyType, Map<String, Map<String, FileKeyJson>>> keys = new ConcurrentHashMap<>();
+    private KeyIdentifier current;
+
+    private Map<String, FileKeyJson> getMap(KeyType type, String algo) {
+        return keys.computeIfAbsent(type, k -> new ConcurrentHashMap<>()).computeIfAbsent(algo, k -> new ConcurrentHashMap<>());
+    }
+
+    @Override
+    public boolean has(KeyIdentifier id) {
+        return getMap(id.getType(), id.getAlgorithm()).containsKey(id.getSerial());
+    }
+
+    @Override
+    public List<KeyIdentifier> list() {
+        List<KeyIdentifier> keyIds = new ArrayList<>();
+        keys.forEach((key, value) -> value.forEach((key1, value1) -> value1.forEach((key2, value2) -> keyIds.add(new GenericKeyIdentifier(key, key1, key2)))));
+        return keyIds;
+    }
+
+    @Override
+    public List<KeyIdentifier> list(KeyType type) {
+        List<KeyIdentifier> keyIds = new ArrayList<>();
+        keys.computeIfAbsent(type, t -> new ConcurrentHashMap<>()).forEach((key, value) -> value.forEach((key1, value1) -> keyIds.add(new GenericKeyIdentifier(type, key, key1))));
+        return keyIds;
+    }
+
+    @Override
+    public Key get(KeyIdentifier id) throws ObjectNotFoundException {
+        FileKeyJson data = getMap(id.getType(), id.getAlgorithm()).get(id.getSerial());
+        if (Objects.isNull(data)) {
+            throw new ObjectNotFoundException("Key", id.getType() + ":" + id.getAlgorithm() + ":" + id.getSerial());
+        }
+
+        return new GenericKey(new GenericKeyIdentifier(id), data.isValid(), data.getKey());
+    }
+
+    private void set(Key key) {
+        FileKeyJson data = FileKeyJson.get(key);
+        getMap(key.getId().getType(), key.getId().getAlgorithm()).put(key.getId().getSerial(), data);
+    }
+
+    @Override
+    public void add(Key key) throws IllegalStateException {
+        if (has(key.getId())) {
+            throw new IllegalStateException("Key " + key.getId().getId() + " already exists");
+        }
+
+        set(key);
+    }
+
+    @Override
+    public void update(Key key) throws ObjectNotFoundException {
+        if (!has(key.getId())) {
+            throw new ObjectNotFoundException("Key", key.getId().getType() + ":" + key.getId().getAlgorithm() + ":" + key.getId().getSerial());
+        }
+
+        set(key);
+    }
+
+    @Override
+    public void delete(KeyIdentifier id) throws ObjectNotFoundException {
+        if (!has(id)) {
+            throw new ObjectNotFoundException("Key", id.getType() + ":" + id.getAlgorithm() + ":" + id.getSerial());
+        }
+
+        keys.computeIfAbsent(id.getType(), k -> new ConcurrentHashMap<>()).computeIfAbsent(id.getAlgorithm(), k -> new ConcurrentHashMap<>()).remove(id.getSerial());
+    }
+
+    @Override
+    public void setCurrentKey(KeyIdentifier id) throws IllegalArgumentException {
+        if (!has(id)) {
+            throw new IllegalArgumentException("Key " + id.getType() + ":" + id.getAlgorithm() + ":" + id.getSerial() + " is not known to the store");
+        }
+
+        current = id;
+    }
+
+    @Override
+    public Optional<KeyIdentifier> getCurrentKey() {
+        return Optional.ofNullable(current);
+    }
+
+}

src/main/java/io/kamax/mxisd/storage/ormlite/OrmLiteSqlStorage.java

@@ -34,24 +34,18 @@ import io.kamax.mxisd.invitation.IThreePidInviteReply;
 import io.kamax.mxisd.storage.IStorage;
 import io.kamax.mxisd.storage.dao.IThreePidSessionDao;
 import io.kamax.mxisd.storage.ormlite.dao.ASTransactionDao;
+import io.kamax.mxisd.storage.ormlite.dao.HistoricalThreePidInviteIO;
 import io.kamax.mxisd.storage.ormlite.dao.ThreePidInviteIO;
 import io.kamax.mxisd.storage.ormlite.dao.ThreePidSessionDao;
 import org.apache.commons.lang.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 
 import java.io.IOException;
 import java.sql.SQLException;
 import java.time.Instant;
-import java.util.ArrayList;
+import java.util.*;
-import java.util.Collection;
-import java.util.List;
-import java.util.Optional;
 
 public class OrmLiteSqlStorage implements IStorage {
 
-    private transient final Logger log = LoggerFactory.getLogger(OrmLiteSqlStorage.class);
-
     @FunctionalInterface
     private interface Getter<T> {
 
@@ -67,6 +61,7 @@ public class OrmLiteSqlStorage implements IStorage {
     }
 
     private Dao<ThreePidInviteIO, String> invDao;
+    private Dao<HistoricalThreePidInviteIO, String> expInvDao;
     private Dao<ThreePidSessionDao, String> sessionDao;
     private Dao<ASTransactionDao, String> asTxnDao;
 
@@ -86,6 +81,7 @@ public class OrmLiteSqlStorage implements IStorage {
         withCatcher(() -> {
             ConnectionSource connPool = new JdbcConnectionSource("jdbc:" + backend + ":" + path);
             invDao = createDaoAndTable(connPool, ThreePidInviteIO.class);
+            expInvDao = createDaoAndTable(connPool, HistoricalThreePidInviteIO.class);
             sessionDao = createDaoAndTable(connPool, ThreePidSessionDao.class);
             asTxnDao = createDaoAndTable(connPool, ASTransactionDao.class);
         });
@@ -150,6 +146,24 @@ public class OrmLiteSqlStorage implements IStorage {
         });
     }
 
+    @Override
+    public void insertHistoricalInvite(IThreePidInviteReply data, String resolvedTo, Instant resolvedAt, boolean couldPublish) {
+        withCatcher(() -> {
+            HistoricalThreePidInviteIO io = new HistoricalThreePidInviteIO(data, resolvedTo, resolvedAt, couldPublish);
+            int updated = expInvDao.create(io);
+            if (updated != 1) {
+                throw new RuntimeException("Unexpected row count after DB action: " + updated);
+            }
+
+            // Ugly, but it avoids touching the structure of the historical parent class
+            // and avoid any possible regression at this point.
+            updated = expInvDao.updateId(io, UUID.randomUUID().toString().replace("-", ""));
+            if (updated != 1) {
+                throw new RuntimeException("Unexpected row count after DB action: " + updated);
+            }
+        });
+    }
+
     @Override
     public Optional<IThreePidSessionDao> getThreePidSession(String sid) {
         return withCatcher(() -> Optional.ofNullable(sessionDao.queryForId(sid)));

src/main/java/io/kamax/mxisd/storage/ormlite/dao/HistoricalThreePidInviteIO.java

@@ -0,0 +1,72 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.storage.ormlite.dao;
+
+import com.j256.ormlite.field.DatabaseField;
+import com.j256.ormlite.table.DatabaseTable;
+import io.kamax.mxisd.invitation.IThreePidInviteReply;
+
+import java.time.Instant;
+
+@DatabaseTable(tableName = "invite_3pid_history")
+public class HistoricalThreePidInviteIO extends ThreePidInviteIO {
+
+    @DatabaseField(canBeNull = false)
+    private String resolvedTo;
+
+    @DatabaseField(canBeNull = false)
+    private long resolvedAt;
+
+    @DatabaseField(canBeNull = false)
+    private boolean couldPublish;
+
+    @DatabaseField(canBeNull = false)
+    private long publishAttempts = 1; // Placeholder for retry mechanism, if ever implemented
+
+    public HistoricalThreePidInviteIO() {
+        // Needed for ORMLite
+    }
+
+    public HistoricalThreePidInviteIO(IThreePidInviteReply data, String resolvedTo, Instant resolvedAt, boolean couldPublish) {
+        super(data);
+
+        this.resolvedTo = resolvedTo;
+        this.resolvedAt = resolvedAt.toEpochMilli();
+        this.couldPublish = couldPublish;
+    }
+
+    public String getResolvedTo() {
+        return resolvedTo;
+    }
+
+    public Instant getResolvedAt() {
+        return Instant.ofEpochMilli(resolvedAt);
+    }
+
+    public boolean isCouldPublish() {
+        return couldPublish;
+    }
+
+    public long getPublishAttempts() {
+        return publishAttempts;
+    }
+
+}