Fix #123

- synapse did not check/validate the response as per spec until 0.99.5 it seems
- mxisd was never compliant also

README.md

@@ -53,6 +53,7 @@ As an enhanced Identity service:
   - Central Matrix Identity servers
 - [Session Control](docs/threepids/session/session.md): Extensive control of where 3PIDs are transmitted so they are not
   leaked publicly by users
+- [Registration control](docs/features/registration.md): Control and restrict user registration based on 3PID patterns or criterias, like a pending invite
 - [Authentication](docs/features/authentication.md): Use your Identity stores to perform authentication in [synapse](https://github.com/matrix-org/synapse)
   via the [REST password provider](https://github.com/kamax-io/matrix-synapse-rest-auth)
 - [Directory search](docs/features/directory.md) which allows you to search for users within your organisation,
@@ -80,8 +81,6 @@ A basic troubleshooting guide is available [here](docs/troubleshooting.md).
 ## Community
 Over Matrix: [#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) ([Preview](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/))
 
-For more high-level discussion about the Identity Server architecture/API, go to  [#matrix-identity:kamax.io](https://matrix.to/#/#matrix-identity:kamax.io)
-
 ## Commercial
 If you would prefer professional support/custom development for mxisd and/or for Matrix in general, including other open
 source technologies/products:

build.gradle

@@ -48,6 +48,8 @@ def dockerImageTag = "${dockerImageName}:${mxisdVersion()}"
 
 group = 'io.kamax'
 mainClassName = 'io.kamax.mxisd.MxisdStandaloneExec'
+sourceCompatibility = '1.8'
+targetCompatibility = '1.8'
 
 String mxisdVersion() {
     def versionPattern = Pattern.compile("v(\\d+\\.)?(\\d+\\.)?(\\d+)(-.*)?")

docs/features/experimental/application-service.md

@@ -1,25 +1,107 @@
-# Integration as an Application Service
+# Application Service
 **WARNING:** These features are currently highly experimental. They can be removed or modified without notice.  
-All the features requires a Homeserver capable of connecting Application Services.
+All the features requires a Homeserver capable of connecting [Application Services](https://matrix.org/docs/spec/application_service/r0.1.0.html).
 
-## Email notification for Room invites by Matrix ID
+The following capabilities are provided in this feature:
+- [Admin commands](#admin-commands)
+- [Email Notification about room invites by Matrix IDs](#email-notification-about-room-invites-by-matrix-ids)
+- [Auto-reject of expired 3PID invites](#auto-reject-of-expired-3pid-invites)
+
+## Setup
+> **NOTE:** Make sure you are familiar with [configuration format and rules](../../configure.md).
+
+Integration as an Application service is a three steps process:
+1. Create the baseline mxisd configuration to allow integration.
+2. Integrate with the homeserver.
+3. Configure the specific capabilities, if applicable.
+
+### Configuration
+#### Variables
+Under the `appsvc` namespace:
+
+| Key                   | Type    | Required | Default | Purpose                                                        |
+|-----------------------|---------|----------|---------|----------------------------------------------------------------|
+| `enabled`             | boolean | No       | `false` | Globally enable/disable the feature                            |
+| `user.main`           | string  | No       | `mxisd` | Localpart for the main appservice user                         |
+| `endpoint.toHS.url`   | string  | Yes      | *None*  | Base URL to the Homeserver                                     |
+| `endpoint.toHS.token` | string  | Yes      | *None*  | Token to use when sending requests to the Homeserver           |
+| `endpoint.toAS.url`   | string  | Yes      | *None*  | Base URL to mxisd from the Homeserver                          |
+| `endpoint.toAS.token` | string  | Yes      | *None*  | Token for the Homeserver to use when sending requests to mxisd |
+
+#### Example
+```yaml
+appsvc:
+  enabled: true
+  endpoint:
+    toHS:
+      url: 'http://localhost:8008'
+      token: 'ExampleTokenToHS-ChangeMe!'
+    toAS:
+      url: 'http://localhost:8090'
+      token: 'ExampleTokenToAS-ChangeMe!'
+```
+### Integration
+#### Synapse
+Under the `appsvc.registration.synapse` namespace:
+
+| Key    | Type   | Required | Default            | Purpose                                                                  |
+|--------|--------|----------|--------------------|--------------------------------------------------------------------------|
+| `id`   | string | No       | `appservice-mxisd` | The unique, user-defined ID of this application service. See spec.       |
+| `file` | string | Yes      | *None*             | If defined, the synapse registration file that should be created/updated |
+
+##### Example 
+```yaml
+appsvc:
+  registration:
+    synapse:
+      file: '/etc/matrix-synapse/mxisd-appservice-registration.yaml'
+```
+
+Edit your `homeserver.yaml` and add a new entry to the appservice config file, which should look something like this:
+```yaml
+app_service_config_files:
+  - '/etc/matrix-synapse/mxisd-appservice-registration.yaml'
+  - ...
+```
+
+Restart synapse when done to register mxisd.
+
+#### Others
+See your Homeserver documentation on how to integrate.
+
+## Capabilities
+### Admin commands
+#### Setup
+Min config:
+```yaml
+appsvc:
+  feature:
+    admin:
+      allowedRoles:
+        - '+aMatrixCommunity:example.org'
+        - 'SomeLdapGroup'
+        - 'AnyOtherArbitraryRoleFromIdentityStores'
+```
+
+#### Use
+The following steps assume:
+- `matrix.domain` set to `example.org`
+- `appsvc.user.main` set to `mxisd` or not set
+
+1. Invite `@mxisd:example.org` to a new direct chat
+2. Type `!help` to get all available commands
+
+### Email Notification about room invites by Matrix IDs
 This feature allows for users found in Identity stores to be instantly notified about Room Invites, regardless if their
 account was already provisioned on the Homeserver.
 
-### Requirements
+#### Requirements
 - [Identity store(s)](../../stores/README.md) supporting the Profile feature
 - At least one email entry in the identity store for each user that could be invited.
 
-### Configuration
+#### Configuration
 In your mxisd config file:
 ```yaml
-matrix:
-  listener:
-    url:  '<URL TO THE CS API OF THE HOMESERVER>'
-    localpart: 'appservice-mxisd'
-    token:
-      hs: 'HS_TOKEN_CHANGE_ME'
-
 synapseSql:
   enabled: false ## Do not use this line if Synapse is used as an Identity Store
   type: '<DB TYPE>'
@@ -33,40 +115,8 @@ If you do not configure it, some placeholders will not be available in the notif
 You can also change the default template of the notification using the `generic.matrixId` template option.  
 See [the Template generator documentation](../../threepids/notification/template-generator.md) for more info.
 
-### Homeserver integration
+#### Test
-#### Synapse
-Create a new appservice registration file. Futher config will assume it is in `/etc/matrix-synapse/appservice-mxisd.yaml`
-```yaml
-id: "appservice-mxisd"
-url: "http://127.0.0.1:8090"
-as_token: "AS_TOKEN_CHANGE_ME"
-hs_token: "HS_TOKEN_CHANGE_ME"
-sender_localpart: "appservice-mxisd"
-namespaces:
-  users:
-    - regex: "@*"
-      exclusive: false
-  aliases: []
-  rooms: []
-```
-`id`: An arbitrary unique string to identify the AS.  
-`url`: mxisd to reach mxisd. This ideally should be HTTP and not going through any reverse proxy.  
-`as_token`: Arbitrary value used by mxisd when talking to the HS. Not currently used.  
-`hs_token`: Arbitrary value used by synapse when talking to mxisd. Must match `token.hs` in mxisd config.
-`sender_localpart`: Username for the mxisd itself on the HS. Default configuration should be kept.  
-`namespaces`: To be kept as is.  
-
-Edit your `homeserver.yaml` and add a new entry to the appservice config file, which should look something like this:
-```yaml
-app_service_config_files:
-  - '/etc/matrix-synapse/appservice-mxisd.yaml'
-  - ...
-```
-
-Restart synapse when done to register mxisd.
-
-#### Others
-See your Homeserver documentation on how to integrate.
-
-### Test
 Invite a user which is part of your domain while an appropriate Identity store is used.
+
+### Auto-reject of expired 3PID invites
+*TBC*

docs/features/identity.md

@@ -1,6 +1,13 @@
 # Identity
 Implementation of the [Identity Service API r0.1.0](https://matrix.org/docs/spec/identity_service/r0.1.0.html).
 
+- [Lookups](#lookups)
+- [Invitations](#invitations)
+  - [Expiration](#expiration)
+  - [Policies](#policies)
+  - [Resolution](#resolution)
+- [3PIDs Management](#3pids-management)
+
 ## Lookups
 If you would like to use the central matrix.org Identity server to ensure maximum discovery at the cost of potentially
 leaking all your contacts information, add the following to your configuration:
@@ -12,8 +19,78 @@ forward:
 **NOTE:** You should carefully consider enabling this option, which is discouraged.  
 For more info, see the [relevant issue](https://github.com/kamax-matrix/mxisd/issues/76).
 
-## Room Invitations
+## Invitations
-Resolution can be customized using the following configuration:
+### Expiration
+#### Overview
+Matrix does not provide a mean to remove/cancel pending 3PID invitations with the APIs. The current reference
+implementations also do not provide any mean to do so. This leads to 3PID invites forever stuck in rooms.
+
+To provide this functionality, mxisd uses a workaround: resolve the invite to a dedicated User ID, which can be
+controlled by mxisd or a bot/service that will then reject the invite.
+
+If this dedicated User ID is to be controlled by mxisd, the [Application Service](experimental/application-service.md)
+feature must be configured and integrated with your Homeserver, as well as the *Auto-reject 3PID invite capability*.
+
+#### Configuration
+```yaml
+invite:
+  expiration:
+    enabled: true/false
+    after: 5
+    resolveTo: '@john.doe:example.org'
+```
+`enabled`
+- Purpose: Enable or disable the invite expiration feature.
+- Default: `true`
+
+`after` 
+- Purpose: Amount of minutes before an invitation expires.
+- Default: `10080` (7 days)
+
+`resolveTo`
+- Purpose: Matrix User ID to resolve the expired invitations to.
+- Default: Computed from `appsvc.user.inviteExpired` and `matrix.domain`
+
+### Policies
+3PID invite policies are the companion feature of [Registration](registration.md). While the Registration feature acts on
+requirements for the invitee/register, this feature acts on requirement for the one(s) performing 3PID invites, ensuring
+a coherent system.
+
+It relies on only allowing people with specific [Roles](profile.md) to perform 3PID invites. This would typically allow
+a tight-control on a server setup with is "invite-only" or semi-open (relying on trusted people to invite new members).
+
+It's a middle ground between a closed server, where every user must be created or already exists in an Identity store,
+and an open server, where anyone can register.
+ 
+#### Integration
+Because Identity Servers do not control 3PID invites as per Matrix spec, mxisd needs to intercept a set of Homeserver
+endpoints to apply the policies.
+
+##### Reverse Proxy
+###### nginx
+**IMPORTANT**: Must be placed before your global `/_matrix` entry:
+```nginx
+location ~* ^/_matrix/client/r0/rooms/([^/]+)/invite$ {
+    proxy_pass		    http://127.0.0.1:8090;
+    proxy_set_header	Host $host;
+    proxy_set_header	X-Forwarded-For $remote_addr;
+}
+```
+
+#### Configuration
+The only policy currently available is to restrict 3PID invite to users having a specific (set of) role(s), like so:
+
+```yaml
+invite:
+  policy:
+    ifSender:
+      hasRole:
+        - '<THIS_ROLE>'
+        - '<OR_THIS_ROLE>'
+```
+
+### Resolution
+Resolution of 3PID invitations can be customized using the following configuration:
 
 `invite.resolution.recursive`  
 - Default value: `true`  
@@ -26,5 +103,5 @@ Resolution can be customized using the following configuration:
 - Default value: `1`  
 - Description: How often, in minutes, mxisd should try to resolve pending invites.
 
-## 3PID addition to user profile
+## 3PIDs Management
 See the [3PID session documents](../threepids/session)

docs/features/registration.md

@@ -0,0 +1,111 @@
+# Registration
+- [Overview](#overview)
+- [Integration](#integration)
+  - [Reverse Proxy](#reverse-proxy)
+    - [nginx](#nginx)
+    - [Apache](#apache)
+  - [Homeserver](#homeserver)
+    - [synapse](#synapse)
+- [Configuration](#configuration)
+  - [Example](#example)
+- [Usage](#usage)
+
+## Overview
+**NOTE**: This feature is beta: it is considered stable enough for production but is incomplete and may contain bugs.
+
+Registration is an enhanced feature of mxisd to control registrations involving 3PIDs on a Homeserver based on policies:
+- Match pending 3PID invites on the server
+- Match 3PID pattern, like a specific set of domains for emails
+- In futher releases, use 3PIDs found in Identity stores
+
+It aims to help open or invite-only registration servers control what is possible to do and ensure only approved people
+can register on a given server in a implementation-agnostic manner.
+
+**IMPORTANT:** This feature does not control registration in general. It only acts on endpoints related to 3PIDs during
+the registration process.  
+As such, it relies on the homeserver to require 3PIDs with the registration flows.
+
+This feature is not part of the Matrix Identity Server spec.
+
+## Integration
+mxisd needs to be integrated at several levels for this feature to work:
+- Reverse proxy: intercept the 3PID register endpoints and act on them
+- Homeserver: require 3PID to be part of the registration data
+
+Later version(s) of this feature may directly control registration itself to create a coherent experience
+### Reverse Proxy
+#### nginx
+```nginx
+location ^/_matrix/client/r0/register/[^/]/?$ {
+	proxy_pass		http://127.0.0.1:8090;
+	proxy_set_header	Host $host;
+	proxy_set_header	X-Forwarded-For $remote_addr;
+}
+```
+
+#### apache
+> TBC
+
+### Homeserver
+#### Synapse
+```yaml
+enable_registration: true
+registrations_require_3pid:
+  - email
+```
+
+## Configuration
+See the [Configuration](../configuration.md) introduction doc on how to read the configuration keys.  
+An example of working configuration is avaiable at the end of this section.
+### Enable/Disable
+`register.allowed`, taking a boolean, can be used to enable/disable registration if the attempt is not 3PID-based.  
+`false` is the default value to prevent open registration, as you must allow it on the homeserver side.
+
+### For invites
+`register.invite`, taking a boolean, controls if registration can be made using a 3PID which matches a pending 3PID invite.  
+`true` is the default value.
+
+### 3PID-specific
+At this time, only `email` is supported with 3PID specific configuration with this feature.
+
+#### Email
+**Base key**: `register.threepid.email`
+
+##### Domain whitelist/blacklist
+If you would like to control which domains are allowed to be used when registrating with an email, the following sub-keys
+are available:
+- `domain.whitelist`
+- `domain.blacklist`
+
+The value format is an hybrid between glob patterns and postfix configuration files with the following syntax:
+- `*<domain>` will match the domain and any sub-domain(s)
+- `.<domain>` will only match sub-domain(s)
+- `<domain>` will only match the exact domain
+
+The following table illustrates pattern and maching status against example values:
+
+| Config value   | Matches `example.org` | Matches `sub.example.org` |
+|--------------- |-----------------------|---------------------------|
+| `*example.org` | Yes                   | Yes                       |
+| `.example.org` | No                    | Yes                       |
+| `example.org`  | Yes                   | No                        |
+
+### Example
+For the following example configuration:
+```yaml
+register:
+  policy:
+    threepid:
+      email:
+        domain:
+          whitelist:
+            - '*example.org'
+            - '.example.net'
+            - 'example.com'
+```
+- Users can register using 3PIDs of pending invites, being allowed by default.
+- Users can register using an email from `example.org` and any sub-domain, only sub-domains of `example.net` and `example.com` but not its sub-domains.
+- Otherwise, user registration will be denied.
+
+## Usage
+Nothing special is needed. Register using a regular Matrix client.

docs/getting-started.md

@@ -153,3 +153,5 @@ infrastructure:
 
 - [Enable extra features](features/)
 - [Use your own Identity stores](stores/README.md)
+- [Hardening your mxisd installation](install/security.md)
+- [Learn about day-to-day operations](operations.md)

docs/install/docker.md

@@ -20,3 +20,6 @@ docker run --rm -e MATRIX_DOMAIN=example.org -v /data/mxisd/etc:/etc/mxisd -v /d
 ```
 
 For more info, including the list of possible tags, see [the public repository](https://hub.docker.com/r/kamax/mxisd/)
+
+## Troubleshoot
+Please read the [Troubleshooting guide](../troubleshooting.md).

docs/install/security.md

@@ -0,0 +1,30 @@
+# Security hardening
+## Overview
+This document outlines the various operations you may want to perform to increase the security of your installation and
+avoid leak of credentials/key pairs
+
+## Configuration
+Your config file should have the following ownership:
+- Dedicated user for mxisd, used to run the software
+- Dedicated group for mxisd, used by other applications to access and read configuration files
+
+Your config file should have the following access:
+- Read and write for the mxisd user
+- Read for the mxisd group
+- Nothing for others
+
+This translates into `640` and be applied with `chmod 640 /path/to/config/file.yaml`.
+
+## Data
+The only sensible place is the key store where mxisd's signing keys are stored. You should therefore limit access to only
+the mxisd user, and deny access to anything else.
+
+Your key store should have the following access:
+- Read and write for the mxisd user
+- Nothing for the mxisd group
+- Nothing for others
+
+The identity store can either be a file or a directory, depending on your version. v1.4 and higher are using a directory,
+everything before is using a file.
+- If your version is directory-based, you will want to apply chmod `700` on it.
+- If your version is file-based, you will want to apply chmod `600` on it.

docs/operations.md

@@ -0,0 +1,21 @@
+# Operations Guide
+- [Overview](#overview)
+- [Maintenance](#maintenance)
+- [Backuo](#backup)
+
+## Overview
+This document gives various information for the day-to-day management and operations of mxisd.
+
+## Maintenance
+mxisd does not require any maintenance task to run at optimal performance.
+
+## Backup
+### Run
+mxisd requires all file in its configuration and data directory to be backed up.  
+They are usually located at:
+- `/etc/mxisd`
+- `/var/lib/mxisd`
+
+### Restore
+Reinstall mxisd, restore the two folders above in the appropriate location (depending on your install method) and you
+will be good to go. Simply start mxisd to restore functionality.

docs/stores/ldap.md

@@ -89,7 +89,7 @@ ldap:
 #### 3PIDs
 You can also change the attribute lists for 3PID, like email or phone numbers.
 
-The following example would overwrite the [default list of attributes](../../src/main/resources/application.yaml#L67)
+The following example would overwrite the [default list of attributes](../../src/main/java/io/kamax/mxisd/config/ldap/LdapConfig.java#L64)
 for emails and phone number:
 ```yaml
 ldap:

docs/stores/sql.md

@@ -102,9 +102,41 @@ sql:
 ```
 
 ### Identity
+**NOTE**: Only single lookup is supported. Bulk lookup always returns no mapping. This is a restriction as the Matrix API
+does not allow paging or otherwise limit of results of the API, potentially leading to thousands and thousands 3PIDs at once.
+
 ```yaml
 sql:
   identity:
+    enabled: <boolean>
     type: <string>
     query: <string>
+    medium:
+     mediumTypeExample: <dedicated query>
 ```
+`type` is used to tell mxisd how to process the returned `uid` column containing the User ID:
+- `localpart` will build a full Matrix ID using the `matrix.domain` value.
+- `mxid` will use the ID as-is. If it is not a valid Matrix ID, lookup(s) will fail.
+
+A specific query can also given per 3PID medium type.
+
+### Profile
+```yaml
+sql:
+  profile:
+    enabled: <boolean>
+    displayName:
+      query: <string>
+    threepid:
+      query: <string>
+    role:
+      type: <string>
+      query: <string>
+    
+
+```
+For the `role` query, `type` can be used to tell mxisd how to inject the User ID in the query:
+- `localpart` will extract and set only the localpart.
+- `mxid` will use the ID as-is.
+
+On each query, the first parameter `?` is set as a string with the corresponding ID format.

docs/stores/synapse.md

@@ -1,5 +1,6 @@
 # Synapse Identity Store
-Synapse's Database itself can be used as an Identity store.
+Synapse's Database itself can be used as an Identity store. This identity store is a regular SQL store with
+built-in default queries that matches Synapse DB.
 
 ## Features
 |                       Name                      | Supported |
@@ -9,7 +10,8 @@ Synapse's Database itself can be used as an Identity store.
 | [Identity](../features/identity.md)             | Yes       |
 | [Profile](../features/profile.md)               | Yes       |
 
-Authentication is done by Synapse itself.
+- Authentication is done by Synapse itself.
+- Roles are mapped to communities. The Role name/ID uses the community ID in the form `+id:domain.tld`
 
 ## Configuration
 ### Basic

docs/threepids/medium/email/smtp-connector.md

@@ -12,8 +12,8 @@ threepid:
       connectors:
         smtp:
           host: 'smtpHostname'
-          port: 587
+          tls: 1 # 0 = no STARTLS, 1 = try, 2 = force, 3 = TLS/SSL
-          tls: 1 # 0 = no STARTLS, 1 = try, 2 = force
+          port: 587 # Set appropriate value depending on your TLS setting
           login: 'smtpLogin'
           password: 'smtpPassword'
 ```

docs/threepids/medium/msisdn/twilio-connector.md

@@ -8,7 +8,7 @@ threepid:
     msisdn:
       connectors:
         twilio:
-          accountSid: 'myAccountSid'
+          account_sid: 'myAccountSid'
-          authToken: 'myAuthToken'
+          auth_token: 'myAuthToken'
           number: '+123456789'
 ```

docs/threepids/notification/template-generator.md

@@ -1,63 +1,113 @@
-# Notifications: Generate from templates
+# Notifications: Template generator
-To create notification content, you can use the `template` generator if supported for the 3PID medium which will read
+Most of the Identity actions will trigger a notification of some kind, informing the user of some confirmation, next step
-content from configured files.
+or just informing them about the current state of things.
 
-Placeholders can be integrated into the templates to dynamically populate such content with relevant information like
+Those notifications are by default generated from templates and by replacing placeholder tokens in them with the relevant
-the 3PID that was requested, the domain of your Identity server, etc.
+values of the notification. It is possible to customize the value of some placeholders, making easy to set values in the builtin templates, and/or
+provide your own custom templates.
 
-Templates can be configured for each event that would send a notification to the end user. Events share a set of common
+Templates for the following events/actions are available:
-placeholders and also have their own individual set of placeholders.
+- [3PID invite](../../features/identity.md)
+- [3PID session: validation](../session/session.md)
+- [3PID session: fraudulent unbind](https://github.com/kamax-matrix/mxisd/wiki/mxisd-and-your-privacy#improving-your-privacy-one-commit-at-the-time)
+- [Matrix ID invite](../../features/experimental/application-service.md#email-notification-about-room-invites-by-matrix-ids)
+
+## Placeholders
+All placeholders **MUST** be surrounded with `%` in the template. Per example, the `DOMAIN` placeholder would become
+`%DOMAIN%` within the template. This ensures replacement doesn't happen on non-placeholder strings.
+
+### Global
+The following placeholders are available in every template:
+
+| Placeholder                     | Purpose                                                                      |
+|---------------------------------|------------------------------------------------------------------------------|
+| `DOMAIN`                        | Identity server authoritative domain, as configured in `matrix.domain`       |
+| `DOMAIN_PRETTY`                 | Same as `DOMAIN` with the first letter upper case and all other lower case   |
+| `FROM_EMAIL`                    | Email address configured in `threepid.medium.<3PID medium>.identity.from`    |
+| `FROM_NAME`                     | Name configured in `threepid.medium.<3PID medium>.identity.name`             |
+| `RECIPIENT_MEDIUM`              | The 3PID medium, like `email` or `msisdn`                                    |
+| `RECIPIENT_MEDIUM_URL_ENCODED`  | URL encoded value of `RECIPIENT_MEDIUM`                                      |
+| `RECIPIENT_ADDRESS`             | The address to which the notification is sent                                |
+| `RECIPIENT_ADDRESS_URL_ENCODED` | URL encoded value of `RECIPIENT_ADDRESS`                                     |
+
+### Room invitation
+Specific placeholders:
+
+| Placeholder                  | Purpose                                                                           |
+|------------------------------|-----------------------------------------------------------------------------------|
+| `SENDER_ID`                  | Matrix ID of the user who made the invite                                         |
+| `SENDER_NAME`                | Display name of the user who made the invite, if not available/set, empty         |
+| `SENDER_NAME_OR_ID`          | Display name of the user who made the invite. If not available/set, its Matrix ID |
+| `INVITE_MEDIUM`              | The 3PID medium for the invite.                                                   |
+| `INVITE_MEDIUM_URL_ENCODED`  | URL encoded value of `INVITE_MEDIUM`                                              |
+| `INVITE_ADDRESS`             | The 3PID address for the invite.                                                  |
+| `INVITE_ADDRESS_URL_ENCODED` | URL encoded value of `INVITE_ADDRESS`                                             |
+| `ROOM_ID`                    | The Matrix ID of the Room in which the invite took place                          |
+| `ROOM_NAME`                  | The Name of the room in which the invite took place. If not available/set, empty  |
+| `ROOM_NAME_OR_ID`            | The Name of the room in which the invite took place. If not available/set, its ID |
+| `REGISTER_URL`               | The URL to provide to the user allowing them to register their account, if needed |
+
+### Validation of 3PID Session
+Specific placeholders:
+
+| Placeholder        | Purpose                                                                              |
+|--------------------|--------------------------------------------------------------------------------------|
+| `VALIDATION_LINK`  | URL, including token, to validate the 3PID session.                                  |
+| `VALIDATION_TOKEN` | The token needed to validate the session, in case the user cannot use the link.      |
+| `NEXT_URL`         | URL to redirect to after the sessions has been validated.                            |
+
+## Templates
+mxisd comes with a set of builtin templates to easily get started. Those templates can be found
+[in the repository](https://github.com/kamax-matrix/mxisd/tree/master/src/main/resources/threepids). If you want to use
+customized templates, we recommend using the builtin templates as a starting point.
+
+> **NOTE**: The link above point to the latest version of the built-in templates. Those might be different from your
+version. Be sure to view the repo at the current tag.
 
 ## Configuration
+All configuration is specific to [3PID mediums](https://matrix.org/docs/spec/appendices.html#pid-types) and happen
+under the namespace `threepid.medium.<medium>.generators.template`.
+
+Under such namespace, the following keys are available:
+- `invite`: Path to the 3PID invite notification template
+- `session.validation`: Path to the 3PID session validation notification template
+- `session.unbind.fraudulent`: Path to the 3PID session fraudulent unbind notification template
+- `generic.matrixId`: Path to the Matrix ID invite notification template
+- `placeholder`: Map of key/values to set static values for some placeholders.
+
+The `placeholder` map supports the following keys, mapped to their respective template placeholders:
+- `REGISTER_URL`
+
+### Example
+#### Simple
+```yaml
+threepid:
+  medium:
+    email:
+      generators:
+        template:
+          placeholder:
+            REGISTER_URL: 'https://matrix-client.example.org'
+```
+In this configuration, the builtin templates are used and a static value for the `REGISTER_URL` is set, allowing to point
+a newly invited user to a webapp allowing the creation of its account on the server.
+
+#### Advanced
 To configure paths to the various templates:
 ```yaml
 threepid:
   medium:
-    <YOUR 3PID MEDIUM HERE>:
+    email:
       generators:
         template:
           invite: '/path/to/invite-template.eml'
           session:
             validation: '/path/to/validate-template.eml'
             unbind:
-              frandulent: '/path/to/unbind-fraudulent-template.eml'
+              fraudulent: '/path/to/unbind-fraudulent-template.eml'
           generic:
             matrixId: '/path/to/mxid-invite-template.eml'
+          placeholder:
+            REGISTER_URL: 'https://matrix-client.example.org'
 ```
-The `template` generator is usually the default, so no further configuration is needed.
+In this configuration, a custom template is used for each event and a static value for the `REGISTER_URL` is set.
-
-##  Global placeholders
-| Placeholder           | Purpose                                                                      |
-|-----------------------|------------------------------------------------------------------------------|
-| `%DOMAIN%`            | Identity server authoritative domain, as configured in `matrix.domain`       |
-| `%DOMAIN_PRETTY%`     | Same as `%DOMAIN%` with the first letter upper case and all other lower case |
-| `%FROM_EMAIL%`        | Email address configured in `threepid.medium.<3PID medium>.identity.from`    |
-| `%FROM_NAME%`         | Name configured in `threepid.medium.<3PID medium>.identity.name`             |
-| `%RECIPIENT_MEDIUM%`  | The 3PID medium, like `email` or `msisdn`                                    |
-| `%RECIPIENT_ADDRESS%` | The address to which the notification is sent                                |
-
-## Events
-### Room invitation
-This template is used when someone is invited into a room using an email address which has no known bind to a Matrix ID.
-#### Placeholders
-| Placeholder           | Purpose                                                                                  |
-|-----------------------|------------------------------------------------------------------------------------------|
-| `%SENDER_ID%`         | Matrix ID of the user who made the invite                                                |
-| `%SENDER_NAME%`       | Display name of the user who made the invite, if not available/set, empty                |
-| `%SENDER_NAME_OR_ID%` | Display name of the user who made the invite. If not available/set, its Matrix ID        |
-| `%INVITE_MEDIUM%`     | The 3PID medium for the invite.                                                          |
-| `%INVITE_ADDRESS%`    | The 3PID address for the invite.                                                         |
-| `%ROOM_ID%`           | The Matrix ID of the Room in which the invite took place                                 |
-| `%ROOM_NAME%`         | The Name of the room in which the invite took place. If not available/set, empty         |
-| `%ROOM_NAME_OR_ID%`   | The Name of the room in which the invite took place. If not available/set, its Matrix ID |
-
-### Validation of 3PID Session
-This template is used when to user which added their 3PID address to their profile/settings and the session policy
-allows at least local sessions.  
-
-#### Placeholders
-| Placeholder          | Purpose                                                                              |
-|----------------------|--------------------------------------------------------------------------------------|
-| `%VALIDATION_LINK%`  | URL, including token, to validate the 3PID session.                                  |
-| `%VALIDATION_TOKEN%` | The token needed to validate the session, in case the user cannot use the link.      |
-| `%NEXT_URL%`         | URL to redirect to after the sessions has been validated.                            |

docs/troubleshooting.md

@@ -19,6 +19,11 @@ If you use the [Docker image](install/docker.md), this goes to the container log
 
 For any other platform, please refer to your package maintainer.
 
+### Increase verbosity
+To increase log verbosity and better track issues, the following means are available:
+- Add the `-v` command line parameter
+- Use the environment variable and value `MXISD_LOG_LEVEL=debug`
+
 ### Reading them
 Before reporting an issue, it is important to produce clean and complete logs so they can be understood.
 

gradle/wrapper/gradle-wrapper.properties

@@ -1,6 +1,5 @@
-#Fri Aug 11 17:19:02 CEST 2017
 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-5.3.1-bin.zip
 zipStoreBase=GRADLE_USER_HOME
 zipStorePath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-4.0.2-bin.zip

gradlew

@@ -1,5 +1,21 @@
 #!/usr/bin/env sh
 
+#
+# Copyright 2015 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
 ##############################################################################
 ##
 ##  Gradle start up script for UN*X
@@ -28,7 +44,7 @@ APP_NAME="Gradle"
 APP_BASE_NAME=`basename "$0"`
 
 # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
-DEFAULT_JVM_OPTS=""
+DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
 
 # Use the maximum available, or set MAX_FD != -1 to use that value.
 MAX_FD="maximum"

gradlew.bat

@@ -1,3 +1,19 @@
+@rem
+@rem Copyright 2015 the original author or authors.
+@rem
+@rem Licensed under the Apache License, Version 2.0 (the "License");
+@rem you may not use this file except in compliance with the License.
+@rem You may obtain a copy of the License at
+@rem
+@rem      http://www.apache.org/licenses/LICENSE-2.0
+@rem
+@rem Unless required by applicable law or agreed to in writing, software
+@rem distributed under the License is distributed on an "AS IS" BASIS,
+@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+@rem See the License for the specific language governing permissions and
+@rem limitations under the License.
+@rem
+
 @if "%DEBUG%" == "" @echo off
 @rem ##########################################################################
 @rem
@@ -14,7 +30,7 @@ set APP_BASE_NAME=%~n0
 set APP_HOME=%DIRNAME%
 
 @rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
-set DEFAULT_JVM_OPTS=
+set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
 
 @rem Find java.exe
 if defined JAVA_HOME goto findJavaFromJavaHome

mxisd.example.yaml

@@ -91,19 +91,19 @@ threepid:
           # SMTP host
           host: "smtp.example.org"
 
-          # SMTP port
+          # TLS mode for the connection
-          port: 587
-
-          # STARTLS mode for the connection.
-          # SSL/TLS is currently not supported. See https://github.com/kamax-matrix/mxisd/issues/125
-          #
           # Possible values:
           #  0    Disable any kind of TLS entirely
           #  1    Enable STARTLS if supported by server (default)
           #  2    Force STARTLS and fail if not available
+          #  3    Use full TLS/SSL instead of STARTLS
           #
           tls: 1
 
+          # SMTP port
+          # Be sure to adapt depending on your TLS choice, if changed from default
+          port: 587
+
           # Login for SMTP
           login: "matrix-identity@example.org"
 

src/debian/control

@@ -3,5 +3,7 @@ Maintainer: Kamax.io <foss@kamax.io>
 Homepage: https://github.com/kamax-matrix/mxisd
 Description: Federated Matrix Identity Server
 Architecture: all
+Section: net
+Priority: optional
 Depends: openjdk-8-jre | openjdk-8-jre-headless | openjdk-8-jdk | openjdk-8-jdk-headless
 Version: 0

src/main/java/io/kamax/mxisd/HttpMxisd.java

@@ -73,7 +73,6 @@ public class HttpMxisd {
         HttpHandler asNotFoundHandler = SaneHandler.around(new AsNotFoundHandler(m.getAs()));
 
         HttpHandler storeInvHandler = SaneHandler.around(new StoreInviteHandler(m.getConfig().getServer(), m.getInvite(), m.getKeyManager()));
-        HttpHandler sessValidateHandler = SaneHandler.around(new SessionValidateHandler(m.getSession(), m.getConfig().getServer(), m.getConfig().getView()));
 
         httpSrv = Undertow.builder().addHttpListener(m.getConfig().getServer().getPort(), "0.0.0.0").setHandler(Handlers.routing()
 
@@ -103,10 +102,10 @@ public class HttpMxisd {
                 .post(BulkLookupHandler.Path, SaneHandler.around(new BulkLookupHandler(m.getIdentity())))
                 .post(StoreInviteHandler.Path, storeInvHandler)
                 .post(SessionStartHandler.Path, SaneHandler.around(new SessionStartHandler(m.getSession())))
-                .get(SessionValidateHandler.Path, sessValidateHandler)
+                .get(SessionValidateHandler.Path, SaneHandler.around(new SessionValidationGetHandler(m.getSession(), m.getConfig())))
-                .post(SessionValidateHandler.Path, sessValidateHandler)
+                .post(SessionValidateHandler.Path, SaneHandler.around(new SessionValidationPostHandler(m.getSession())))
                 .get(SessionTpidGetValidatedHandler.Path, SaneHandler.around(new SessionTpidGetValidatedHandler(m.getSession())))
-                .post(SessionTpidBindHandler.Path, SaneHandler.around(new SessionTpidBindHandler(m.getSession(), m.getInvite())))
+                .post(SessionTpidBindHandler.Path, SaneHandler.around(new SessionTpidBindHandler(m.getSession(), m.getInvite(), m.getSign())))
                 .post(SessionTpidUnbindHandler.Path, SaneHandler.around(new SessionTpidUnbindHandler(m.getSession())))
                 .post(SignEd25519Handler.Path, SaneHandler.around(new SignEd25519Handler(m.getConfig(), m.getInvite(), m.getSign())))
 

src/main/java/io/kamax/mxisd/Mxisd.java

@@ -41,6 +41,7 @@ import io.kamax.mxisd.lookup.provider.BridgeFetcher;
 import io.kamax.mxisd.lookup.provider.RemoteIdentityServerFetcher;
 import io.kamax.mxisd.lookup.strategy.LookupStrategy;
 import io.kamax.mxisd.lookup.strategy.RecursivePriorityLookupStrategy;
+import io.kamax.mxisd.matrix.HomeserverFederationResolver;
 import io.kamax.mxisd.matrix.IdentityServerUtils;
 import io.kamax.mxisd.notification.NotificationHandlerSupplier;
 import io.kamax.mxisd.notification.NotificationHandlers;
@@ -99,25 +100,26 @@ public class Mxisd {
                 .setMaxConnTotal(Integer.MAX_VALUE)
                 .build();
 
+        FederationDnsOverwrite fedDns = new FederationDnsOverwrite(cfg.getDns().getOverwrite());
+        HomeserverFederationResolver resolver = new HomeserverFederationResolver(fedDns, httpClient);
         IdentityServerUtils.setHttpClient(httpClient);
         srvFetcher = new RemoteIdentityServerFetcher(httpClient);
 
         store = new OrmLiteSqlStorage(cfg);
         keyMgr = CryptoFactory.getKeyManager(cfg.getKey());
-        signMgr = CryptoFactory.getSignatureManager(keyMgr);
+        signMgr = CryptoFactory.getSignatureManager(cfg, keyMgr);
         clientDns = new ClientDnsOverwrite(cfg.getDns().getOverwrite());
-        FederationDnsOverwrite fedDns = new FederationDnsOverwrite(cfg.getDns().getOverwrite());
+
         synapse = new Synapse(cfg.getSynapseSql());
         BridgeFetcher bridgeFetcher = new BridgeFetcher(cfg.getLookup().getRecursive().getBridge(), srvFetcher);
-
         ServiceLoader.load(IdentityStoreSupplier.class).iterator().forEachRemaining(p -> p.accept(this));
         ServiceLoader.load(NotificationHandlerSupplier.class).iterator().forEachRemaining(p -> p.accept(this));
 
         idStrategy = new RecursivePriorityLookupStrategy(cfg.getLookup(), ThreePidProviders.get(), bridgeFetcher);
         pMgr = new ProfileManager(ProfileProviders.get(), clientDns, httpClient);
         notifMgr = new NotificationManager(cfg.getNotification(), NotificationHandlers.get());
-        sessMgr = new SessionManager(cfg.getSession(), cfg.getMatrix(), store, notifMgr, idStrategy, httpClient);
+        sessMgr = new SessionManager(cfg.getSession(), cfg.getMatrix(), store, notifMgr, idStrategy);
-        invMgr = new InvitationManager(cfg, store, idStrategy, keyMgr, signMgr, fedDns, notifMgr, pMgr);
+        invMgr = new InvitationManager(cfg, store, idStrategy, keyMgr, signMgr, resolver, notifMgr, pMgr);
         authMgr = new AuthManager(cfg, AuthProviders.get(), idStrategy, invMgr, clientDns, httpClient);
         dirMgr = new DirectoryManager(cfg.getDirectory(), clientDns, httpClient, DirectoryProviders.get());
         regMgr = new RegistrationManager(cfg.getRegister(), httpClient, clientDns, invMgr);

src/main/java/io/kamax/mxisd/MxisdStandaloneExec.java

@@ -36,6 +36,11 @@ public class MxisdStandaloneExec {
     private static final Logger log = LoggerFactory.getLogger("App");
 
     public static void main(String[] args) {
+        String logLevel = System.getenv("MXISD_LOG_LEVEL");
+        if (StringUtils.isNotBlank(logLevel)) {
+            System.setProperty("org.slf4j.simpleLogger.log.io.kamax.mxisd", logLevel);
+        }
+
         try {
             MxisdConfig cfg = null;
             Iterator<String> argsIt = Arrays.asList(args).iterator();
@@ -46,8 +51,14 @@ public class MxisdStandaloneExec {
                     System.out.println("  -h, --help       Show this help message");
                     System.out.println("  --version        Print the version then exit");
                     System.out.println("  -c, --config     Set the configuration file location");
+                    System.out.println("  -v               Increase log level (log more info)");
+                    System.out.println("  -vv              Further increase log level");
                     System.out.println(" ");
                     System.exit(0);
+                } else if (StringUtils.equals(arg, "-v")) {
+                    System.setProperty("org.slf4j.simpleLogger.log.io.kamax.mxisd", "debug");
+                } else if (StringUtils.equals(arg, "-vv")) {
+                    System.setProperty("org.slf4j.simpleLogger.log.io.kamax.mxisd", "trace");
                 } else if (StringUtils.equalsAny(arg, "-c", "--config")) {
                     String cfgFile = argsIt.next();
                     cfg = YamlConfigLoader.loadFromFile(cfgFile);
@@ -61,13 +72,13 @@ public class MxisdStandaloneExec {
                 }
             }
 
-            log.info("mxisd starting");
-            log.info("Version: {}", Mxisd.Version);
-
             if (Objects.isNull(cfg)) {
                 cfg = YamlConfigLoader.tryLoadFromFile("mxisd.yaml").orElseGet(MxisdConfig::new);
             }
 
+            log.info("mxisd starting");
+            log.info("Version: {}", Mxisd.Version);
+
             HttpMxisd mxisd = new HttpMxisd(cfg);
             Runtime.getRuntime().addShutdownHook(new Thread(() -> {
                 mxisd.stop();

src/main/java/io/kamax/mxisd/as/AppSvcManager.java

@@ -176,10 +176,12 @@ public class AppSvcManager {
         ensureEnabled();
 
         if (StringUtils.isBlank(token)) {
+            log.info("Denying request without a HS token");
             throw new HttpMatrixException(401, "M_UNAUTHORIZED", "No HS token");
         }
 
         if (!StringUtils.equals(cfg.getEndpoint().getToAS().getToken(), token)) {
+            log.info("Denying request with an invalid HS token");
             throw new NotAllowedException("Invalid HS token");
         }
 

src/main/java/io/kamax/mxisd/as/processor/event/MembershipEventProcessor.java

@@ -27,7 +27,6 @@ import io.kamax.matrix._MatrixID;
 import io.kamax.matrix._ThreePid;
 import io.kamax.matrix.client.as.MatrixApplicationServiceClient;
 import io.kamax.matrix.event.EventKey;
-import io.kamax.matrix.hs._MatrixRoom;
 import io.kamax.mxisd.Mxisd;
 import io.kamax.mxisd.backend.sql.synapse.Synapse;
 import io.kamax.mxisd.config.MxisdConfig;
@@ -81,7 +80,7 @@ public class MembershipEventProcessor implements EventTypeProcessor {
 
         _MatrixID target = MatrixID.asAcceptable(targetId);
         if (!StringUtils.equals(target.getDomain(), cfg.getMatrix().getDomain())) {
-            log.debug("Ignoring invite for {}: not a local user");
+            log.debug("Ignoring invite for {}: not a local user", targetId);
             return;
         }
 
@@ -89,10 +88,9 @@ public class MembershipEventProcessor implements EventTypeProcessor {
 
         boolean isForMainUser = StringUtils.equals(target.getLocalPart(), cfg.getAppsvc().getUser().getMain());
         boolean isForExpInvUser = StringUtils.equals(target.getLocalPart(), cfg.getAppsvc().getUser().getInviteExpired());
-        boolean isUs = isForMainUser || isForExpInvUser;
 
         if (StringUtils.equals("join", EventKey.Membership.getStringOrNull(content))) {
-            if (!isForMainUser) {
+            if (isForExpInvUser) {
                 log.warn("We joined the room {} for another identity as the main user, which is not supported. Leaving...", roomId);
 
                 client.getUser(target.getLocalPart()).getRoom(roomId).tryLeave().ifPresent(err -> {
@@ -108,10 +106,7 @@ public class MembershipEventProcessor implements EventTypeProcessor {
                 processForUserIdInvite(roomId, sender, target);
             }
         } else if (StringUtils.equals("leave", EventKey.Membership.getStringOrNull(content))) {
-            _MatrixRoom room = client.getRoom(roomId);
-            if (!isUs && room.getJoinedUsers().size() == 1) {
             // TODO we need to find out if this is only us remaining and leave the room if so, using the right client for it
-            }
         } else {
             log.debug("This is not an supported type of membership event, skipping");
         }

src/main/java/io/kamax/mxisd/auth/AuthManager.java

@@ -64,6 +64,8 @@ import java.util.Objects;
 
 public class AuthManager {
 
+    private static final Logger log = LoggerFactory.getLogger(AuthManager.class);
+
     private static final String TypeKey = "type";
     private static final String UserKey = "user";
     private static final String IdentifierKey = "identifier";
@@ -72,7 +74,6 @@ public class AuthManager {
     private static final String UserIdTypeValue = "m.id.user";
     private static final String ThreepidTypeValue = "m.id.thirdparty";
 
-    private transient final Logger log = LoggerFactory.getLogger(AuthManager.class);
     private final Gson gson = GsonUtil.get(); // FIXME replace
 
     private List<AuthenticatorProvider> providers;
@@ -138,6 +139,12 @@ public class AuthManager {
                     invMgr.publishMappingIfInvited(new ThreePidMapping(pid, mxId));
                 }
 
+                try {
+                    MatrixID.asValid(mxId);
+                } catch (IllegalArgumentException e) {
+                    log.warn("The returned User ID {} is not a valid Matrix ID. Login might fail at the Homeserver level", mxId);
+                }
+
                 invMgr.lookupMappingsForInvites();
 
                 return authResult;

src/main/java/io/kamax/mxisd/config/InvitationConfig.java

@@ -34,7 +34,7 @@ public class InvitationConfig {
     public static class Expiration {
 
         private Boolean enabled;
-        private long after;
+        private long after = 60 * 24 * 7; // One calendar week (60min/1h * 24 = 1d * 7 = 1w)
         private String resolveTo;
 
         public Boolean isEnabled() {

src/main/java/io/kamax/mxisd/config/MxisdConfig.java

@@ -83,6 +83,12 @@ public class MxisdConfig {
 
     }
 
+    public static MxisdConfig forDomain(String domain) {
+        MxisdConfig cfg = new MxisdConfig();
+        cfg.getMatrix().setDomain(domain);
+        return cfg;
+    }
+
     private AppServiceConfig appsvc = new AppServiceConfig();
     private AuthenticationConfig auth = new AuthenticationConfig();
     private DirectoryConfig directory = new DirectoryConfig();
@@ -309,6 +315,13 @@ public class MxisdConfig {
         this.wordpress = wordpress;
     }
 
+    public MxisdConfig inMemory() {
+        getKey().setPath(":memory:");
+        getStorage().getProvider().getSqlite().setDatabase(":memory:");
+
+        return this;
+    }
+
     public MxisdConfig build() {
         if (StringUtils.isBlank(getServer().getName())) {
             getServer().setName(getMatrix().getDomain());

src/main/java/io/kamax/mxisd/config/RegisterConfig.java

@@ -69,16 +69,16 @@ public class RegisterConfig {
         }
 
         private List<String> buildPatterns(List<String> domains) {
-            log.info("Building email policy");
+            log.debug("Building email policy");
             return domains.stream().map(d -> {
                 if (StringUtils.startsWith(d, "*")) {
-                    log.info("Found domain and subdomain policy");
+                    log.debug("Found domain and subdomain policy");
                     d = "(.*)" + d.substring(1);
                 } else if (StringUtils.startsWith(d, ".")) {
-                    log.info("Found subdomain-only policy");
+                    log.debug("Found subdomain-only policy");
                     d = "(.*)" + d;
                 } else {
-                    log.info("Found domain-only policy");
+                    log.debug("Found domain-only policy");
                 }
 
                 return "([^@]+)@" + d.replace(".", "\\.");
@@ -175,10 +175,10 @@ public class RegisterConfig {
     }
 
     public void build() {
-        log.info("--- Registration config ---");
+        log.debug("--- Registration config ---");
 
-        log.info("Before Build");
+        log.debug("Before Build");
-        log.info(GsonUtil.getPrettyForLog(this));
+        log.debug(GsonUtil.getPrettyForLog(this));
 
         new HashMap<>(getPolicy().getThreepid()).forEach((medium, policy) -> {
             if (ThreePidMedium.Email.is(medium)) {
@@ -194,8 +194,8 @@ public class RegisterConfig {
             getPolicy().getThreepid().put(medium, policy);
         });
 
-        log.info("After Build");
+        log.debug("After Build");
-        log.info(GsonUtil.getPrettyForLog(this));
+        log.debug(GsonUtil.getPrettyForLog(this));
     }
 
 }

src/main/java/io/kamax/mxisd/config/threepid/medium/GenericTemplateConfig.java

@@ -77,6 +77,7 @@ public class GenericTemplateConfig {
     private String invite;
     private Session session = new Session();
     private Map<String, String> generic = new HashMap<>();
+    private Map<String, String> placeholder = new HashMap<>();
 
     public String getInvite() {
         return invite;
@@ -98,4 +99,12 @@ public class GenericTemplateConfig {
         this.generic = generic;
     }
 
+    public Map<String, String> getPlaceholder() {
+        return placeholder;
+    }
+
+    public void setPlaceholder(Map<String, String> placeholder) {
+        this.placeholder = placeholder;
+    }
+
 }

src/main/java/io/kamax/mxisd/config/threepid/notification/NotificationConfig.java

@@ -20,7 +20,6 @@
 
 package io.kamax.mxisd.config.threepid.notification;
 
-import com.google.gson.JsonObject;
 import io.kamax.matrix.ThreePidMedium;
 import io.kamax.mxisd.threepid.notification.email.EmailRawNotificationHandler;
 import io.kamax.mxisd.threepid.notification.phone.PhoneNotificationHandler;
@@ -35,7 +34,7 @@ public class NotificationConfig {
     private transient final Logger log = LoggerFactory.getLogger(NotificationConfig.class);
 
     private Map<String, String> handler = new HashMap<>();
-    private Map<String, JsonObject> handlers = new HashMap<>();
+    private Map<String, Object> handlers = new HashMap<>();
 
     public NotificationConfig() {
         handler.put(ThreePidMedium.Email.getId(), EmailRawNotificationHandler.ID);
@@ -50,11 +49,11 @@ public class NotificationConfig {
         this.handler = handler;
     }
 
-    public Map<String, JsonObject> getHandlers() {
+    public Map<String, Object> getHandlers() {
         return handlers;
     }
 
-    public void setHandlers(Map<String, JsonObject> handlers) {
+    public void setHandlers(Map<String, Object> handlers) {
         this.handlers = handlers;
     }
 

src/main/java/io/kamax/mxisd/crypto/CryptoFactory.java

@@ -21,6 +21,7 @@
 package io.kamax.mxisd.crypto;
 
 import io.kamax.mxisd.config.KeyConfig;
+import io.kamax.mxisd.config.MxisdConfig;
 import io.kamax.mxisd.crypto.ed25519.Ed25519KeyManager;
 import io.kamax.mxisd.crypto.ed25519.Ed25519SignatureManager;
 import io.kamax.mxisd.storage.crypto.FileKeyStore;
@@ -54,8 +55,8 @@ public class CryptoFactory {
         return new Ed25519KeyManager(store);
     }
 
-    public static SignatureManager getSignatureManager(Ed25519KeyManager keyMgr) {
+    public static SignatureManager getSignatureManager(MxisdConfig cfg, Ed25519KeyManager keyMgr) {
-        return new Ed25519SignatureManager(keyMgr);
+        return new Ed25519SignatureManager(cfg, keyMgr);
     }
 
 }

src/main/java/io/kamax/mxisd/crypto/GenericKeyIdentifier.java

@@ -36,7 +36,7 @@ public class GenericKeyIdentifier implements KeyIdentifier {
 
     public GenericKeyIdentifier(KeyType type, String algo, String serial) {
         if (StringUtils.isAnyBlank(algo, serial)) {
-            throw new IllegalArgumentException("Aglorith and/or Serial cannot be blank");
+            throw new IllegalArgumentException("Algorithm and/or Serial cannot be blank");
         }
 
         this.type = Objects.requireNonNull(type);

src/main/java/io/kamax/mxisd/crypto/SignatureManager.java

@@ -20,12 +20,57 @@
 
 package io.kamax.mxisd.crypto;
 
+import com.google.gson.JsonElement;
 import com.google.gson.JsonObject;
+import io.kamax.matrix.event.EventKey;
+import io.kamax.matrix.json.MatrixJson;
 
 import java.nio.charset.StandardCharsets;
+import java.util.Objects;
 
 public interface SignatureManager {
 
+    /**
+     * Sign the message with the default domain and add the signature to the <code>signatures</code> key.
+     * <p>
+     * If the key does not exist yet, it is created. If the key exist, the produced signature will be merged with any
+     * existing ones.
+     *
+     * @param message The message to sign with the default domain and add the produced signature to
+     * @return The provided message with the new signature
+     * @throws IllegalArgumentException If the <code>signatures</code> key exists and its value is not a JSON object
+     */
+    JsonObject signMessageGson(JsonObject message) throws IllegalArgumentException;
+
+    /**
+     * Sign the message and add the signature to the <code>signatures</code> key.
+     * <p>
+     * If the key does not exist yet, it is created. If the key exist, the produced signature will be merged with any
+     * existing ones.
+     *
+     * @param domain  The domain under which the signature should be added
+     * @param message The message to sign and add the produced signature to
+     * @return The provided message with the new signature
+     * @throws IllegalArgumentException If the <code>signatures</code> key exists and its value is not a JSON object
+     */
+    default JsonObject signMessageGson(String domain, JsonObject message) throws IllegalArgumentException {
+        JsonElement signEl = message.remove(EventKey.Signatures.get());
+        JsonObject oldSigns = new JsonObject();
+        if (!Objects.isNull(signEl)) {
+            if (!signEl.isJsonObject()) {
+                throw new IllegalArgumentException("Message contains a signatures key that is not a JSON object value");
+            }
+
+            oldSigns = signEl.getAsJsonObject();
+        }
+
+        JsonObject newSigns = signMessageGson(domain, MatrixJson.encodeCanonical(message));
+        oldSigns.entrySet().forEach(entry -> newSigns.add(entry.getKey(), entry.getValue()));
+        message.add(EventKey.Signatures.get(), newSigns);
+
+        return message;
+    }
+
     /**
      * Sign the message and produce a <code>signatures</code> object that can directly be added to the object being signed.
      *

src/main/java/io/kamax/mxisd/crypto/ed25519/Ed25519SignatureManager.java

@@ -23,6 +23,8 @@ package io.kamax.mxisd.crypto.ed25519;
 import com.google.gson.JsonObject;
 import io.kamax.matrix.codec.MxBase64;
 import io.kamax.matrix.json.MatrixJson;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.config.ServerConfig;
 import io.kamax.mxisd.crypto.KeyIdentifier;
 import io.kamax.mxisd.crypto.Signature;
 import io.kamax.mxisd.crypto.SignatureManager;
@@ -35,12 +37,19 @@ import java.security.SignatureException;
 
 public class Ed25519SignatureManager implements SignatureManager {
 
+    private final ServerConfig cfg;
     private final Ed25519KeyManager keyMgr;
 
-    public Ed25519SignatureManager(Ed25519KeyManager keyMgr) {
+    public Ed25519SignatureManager(MxisdConfig cfg, Ed25519KeyManager keyMgr) {
+        this.cfg = cfg.getServer();
         this.keyMgr = keyMgr;
     }
 
+    @Override
+    public JsonObject signMessageGson(JsonObject message) throws IllegalArgumentException {
+        return signMessageGson(cfg.getName(), message);
+    }
+
     @Override
     public JsonObject signMessageGson(String domain, String message) {
         Signature sign = sign(message);

src/main/java/io/kamax/mxisd/exception/InternalServerError.java

@@ -33,8 +33,7 @@ public class InternalServerError extends HttpMatrixException {
         super(
                 HttpStatus.SC_INTERNAL_SERVER_ERROR,
                 "M_UNKNOWN",
-                "An internal server error occured. If this error persists, please contact support with reference #" +
+                "An internal server error occurred. Contact your administrator with reference Transaction #" + Instant.now().toEpochMilli()
-                        Instant.now().toEpochMilli()
         );
     }
 

src/main/java/io/kamax/mxisd/http/undertow/handler/BasicHttpHandler.java

@@ -31,6 +31,7 @@ import io.kamax.mxisd.proxy.Response;
 import io.kamax.mxisd.util.RestClientUtils;
 import io.undertow.server.HttpHandler;
 import io.undertow.server.HttpServerExchange;
+import io.undertow.server.handlers.form.FormData;
 import io.undertow.util.HttpString;
 import org.apache.commons.io.IOUtils;
 import org.apache.commons.lang3.StringUtils;
@@ -48,10 +49,7 @@ import java.net.InetSocketAddress;
 import java.net.URI;
 import java.net.URLDecoder;
 import java.nio.charset.StandardCharsets;
-import java.util.Deque;
+import java.util.*;
-import java.util.LinkedList;
-import java.util.Map;
-import java.util.Optional;
 
 public abstract class BasicHttpHandler implements HttpHandler {
 
@@ -122,6 +120,20 @@ public abstract class BasicHttpHandler implements HttpHandler {
         return GsonUtil.parseObj(getBodyUtf8(exchange));
     }
 
+    protected String getOrThrow(FormData data, String key) {
+        FormData.FormValue value = data.getFirst(key);
+        if (Objects.isNull(value)) {
+            throw new IllegalArgumentException("Form key " + key + " is missing");
+        }
+
+        String object = value.getValue();
+        if (Objects.isNull(object)) {
+            throw new IllegalArgumentException("Form key " + key + " does not have a value");
+        }
+
+        return object;
+    }
+
     protected void putHeader(HttpServerExchange ex, String name, String value) {
         ex.getResponseHeaders().put(HttpString.tryFromString(name), value);
     }

src/main/java/io/kamax/mxisd/http/undertow/handler/SaneHandler.java

@@ -97,7 +97,7 @@ public class SaneHandler extends BasicHttpHandler {
                 if (StringUtils.isNotBlank(e.getInternalReason())) {
                     log.error("Transaction #{} - {}", e.getReference(), e.getInternalReason());
                 } else {
-                    log.error("Transaction #{}", e);
+                    log.error("Transaction #{}", e.getReference(), e);
                 }
 
                 handleException(exchange, e);

src/main/java/io/kamax/mxisd/http/undertow/handler/auth/RestAuthHandler.java

@@ -36,7 +36,7 @@ public class RestAuthHandler extends BasicHttpHandler {
 
     public static final String Path = "/_matrix-internal/identity/v1/check_credentials";
 
-    private transient final Logger log = LoggerFactory.getLogger(RestAuthHandler.class);
+    private static final Logger log = LoggerFactory.getLogger(RestAuthHandler.class);
 
     private AuthManager mgr;
 
@@ -45,7 +45,7 @@ public class RestAuthHandler extends BasicHttpHandler {
     }
 
     @Override
-    public void handleRequest(HttpServerExchange exchange) throws Exception {
+    public void handleRequest(HttpServerExchange exchange) {
         JsonObject authData = parseJsonObject(exchange, "user");
         if (!authData.has("id") || !authData.has("password")) {
             throw new JsonMemberNotFoundException("Missing id or password keys");

src/main/java/io/kamax/mxisd/http/undertow/handler/directory/v1/UserDirectorySearchHandler.java

@@ -40,7 +40,7 @@ public class UserDirectorySearchHandler extends HomeserverProxyHandler {
     }
 
     @Override
-    public void handleRequest(HttpServerExchange exchange) throws Exception {
+    public void handleRequest(HttpServerExchange exchange) {
         String accessToken = getAccessToken(exchange);
         UserDirectorySearchRequest searchQuery = parseJsonTo(exchange, UserDirectorySearchRequest.class);
         URI target = URI.create(exchange.getRequestURL());

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/BulkLookupHandler.java

@@ -37,7 +37,7 @@ public class BulkLookupHandler extends LookupHandler {
 
     public static final String Path = IsAPIv1.Base + "/bulk_lookup";
 
-    private transient final Logger log = LoggerFactory.getLogger(SingleLookupHandler.class);
+    private static final Logger log = LoggerFactory.getLogger(SingleLookupHandler.class);
 
     private LookupStrategy strategy;
 

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/EphemeralKeyIsValidHandler.java

@@ -31,7 +31,7 @@ public class EphemeralKeyIsValidHandler extends KeyIsValidHandler {
 
     public static final String Path = IsAPIv1.Base + "/pubkey/ephemeral/isvalid";
 
-    private transient final Logger log = LoggerFactory.getLogger(EphemeralKeyIsValidHandler.class);
+    private static final Logger log = LoggerFactory.getLogger(EphemeralKeyIsValidHandler.class);
 
     private KeyManager mgr;
 

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/SessionTpidBindHandler.java

@@ -21,11 +21,15 @@
 package io.kamax.mxisd.http.undertow.handler.identity.v1;
 
 import com.google.gson.JsonObject;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.crypto.SignatureManager;
 import io.kamax.mxisd.exception.BadRequestException;
 import io.kamax.mxisd.http.IsAPIv1;
 import io.kamax.mxisd.http.io.identity.BindRequest;
+import io.kamax.mxisd.http.io.identity.SingeLookupReplyJson;
 import io.kamax.mxisd.http.undertow.handler.BasicHttpHandler;
 import io.kamax.mxisd.invitation.InvitationManager;
+import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.session.SessionManager;
 import io.undertow.server.HttpServerExchange;
 import io.undertow.util.QueryParameterUtils;
@@ -42,14 +46,16 @@ public class SessionTpidBindHandler extends BasicHttpHandler {
 
     public static final String Path = IsAPIv1.Base + "/3pid/bind";
 
-    private transient final Logger log = LoggerFactory.getLogger(SessionTpidBindHandler.class);
+    private static final Logger log = LoggerFactory.getLogger(SessionTpidBindHandler.class);
 
     private SessionManager mgr;
     private InvitationManager invMgr;
+    private SignatureManager signMgr;
 
-    public SessionTpidBindHandler(SessionManager mgr, InvitationManager invMgr) {
+    public SessionTpidBindHandler(SessionManager mgr, InvitationManager invMgr, SignatureManager signMgr) {
         this.mgr = mgr;
         this.invMgr = invMgr;
+        this.signMgr = signMgr;
     }
 
     @Override
@@ -74,8 +80,9 @@ public class SessionTpidBindHandler extends BasicHttpHandler {
         }
 
         try {
-            mgr.bind(bindReq.getSid(), bindReq.getSecret(), bindReq.getUserId());
+            SingleLookupReply lookup = mgr.bind(bindReq.getSid(), bindReq.getSecret(), bindReq.getUserId());
-            respond(exchange, new JsonObject());
+            JsonObject response = signMgr.signMessageGson(GsonUtil.makeObj(new SingeLookupReplyJson(lookup)));
+            respond(exchange, response);
         } catch (BadRequestException e) {
             log.info("requested session was not validated");
 

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/SessionValidateHandler.java

@@ -20,94 +20,36 @@
 
 package io.kamax.mxisd.http.undertow.handler.identity.v1;
 
-import io.kamax.mxisd.config.ServerConfig;
-import io.kamax.mxisd.config.ViewConfig;
 import io.kamax.mxisd.http.IsAPIv1;
-import io.kamax.mxisd.http.io.identity.SuccessStatusJson;
 import io.kamax.mxisd.http.undertow.handler.BasicHttpHandler;
 import io.kamax.mxisd.session.SessionManager;
 import io.kamax.mxisd.session.ValidationResult;
-import io.kamax.mxisd.util.FileUtil;
+import org.apache.commons.lang3.StringUtils;
-import io.undertow.server.HttpServerExchange;
-import io.undertow.util.HttpString;
-import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
-import java.io.IOException;
+public abstract class SessionValidateHandler extends BasicHttpHandler {
-import java.net.MalformedURLException;
-import java.net.URL;
-
-public class SessionValidateHandler extends BasicHttpHandler {
 
     public static final String Path = IsAPIv1.Base + "/validate/{medium}/submitToken";
 
     private transient final Logger log = LoggerFactory.getLogger(SessionValidateHandler.class);
 
     private SessionManager mgr;
-    private ServerConfig srvCfg;
-    private ViewConfig viewCfg;
 
-    public SessionValidateHandler(SessionManager mgr, ServerConfig srvCfg, ViewConfig viewCfg) {
+    public SessionValidateHandler(SessionManager mgr) {
         this.mgr = mgr;
-        this.srvCfg = srvCfg;
-        this.viewCfg = viewCfg;
     }
 
-    @Override
+    protected ValidationResult handleRequest(String sid, String secret, String token) {
-    public void handleRequest(HttpServerExchange exchange) {
+        if (StringUtils.isEmpty(sid)) {
-        String medium = getQueryParameter(exchange, "medium");
+            throw new IllegalArgumentException("sid is not set or is empty");
-        String sid = getQueryParameter(exchange, "sid");
-        String secret = getQueryParameter(exchange, "client_secret");
-        String token = getQueryParameter(exchange, "token");
-
-        boolean isHtmlRequest = false;
-        for (String v : exchange.getRequestHeaders().get("Accept")) {
-            if (StringUtils.startsWithIgnoreCase(v, "text/html")) {
-                isHtmlRequest = true;
-                break;
-            }
         }
 
-        if (isHtmlRequest) {
+        if (StringUtils.isEmpty(secret)) {
-            handleHtmlRequest(exchange, medium, sid, secret, token);
+            throw new IllegalArgumentException("client secret is not set or is empty");
-        } else {
-            handleJsonRequest(exchange, sid, secret, token);
-        }
         }
 
-    private void handleHtmlRequest(HttpServerExchange exchange, String medium, String sid, String secret, String token) {
+        return mgr.validate(sid, secret, token);
-        log.info("Validating session {} for medium {}", sid, medium);
-        ValidationResult r = mgr.validate(sid, secret, token);
-        log.info("Session {} was validated", sid);
-        if (r.getNextUrl().isPresent()) {
-            String url = r.getNextUrl().get();
-            try {
-                url = new URL(url).toString();
-            } catch (MalformedURLException e) {
-                log.info("Session next URL {} is not a valid one, will prepend public URL {}", url, srvCfg.getPublicUrl());
-                url = srvCfg.getPublicUrl() + r.getNextUrl().get();
-            }
-            log.info("Session {} validation: next URL is present, redirecting to {}", sid, url);
-            exchange.setStatusCode(302);
-            exchange.getResponseHeaders().add(HttpString.tryFromString("Location"), url);
-        } else {
-            try {
-                String data = FileUtil.load(viewCfg.getSession().getOnTokenSubmit().getSuccess());
-                writeBodyAsUtf8(exchange, data);
-            } catch (IOException e) {
-                throw new RuntimeException(e);
-            }
-        }
-    }
-
-    private void handleJsonRequest(HttpServerExchange exchange, String sid, String secret, String token) {
-        log.info("Requested: {}", exchange.getRequestURL());
-
-        mgr.validate(sid, secret, token);
-        log.info("Session {} was validated", sid);
-
-        respondJson(exchange, new SuccessStatusJson(true));
     }
 
 }

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/SessionValidationGetHandler.java

@@ -0,0 +1,82 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.http.undertow.handler.identity.v1;
+
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.config.ServerConfig;
+import io.kamax.mxisd.config.ViewConfig;
+import io.kamax.mxisd.session.SessionManager;
+import io.kamax.mxisd.session.ValidationResult;
+import io.kamax.mxisd.util.FileUtil;
+import io.undertow.server.HttpServerExchange;
+import io.undertow.util.HttpString;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+
+public class SessionValidationGetHandler extends SessionValidateHandler {
+
+    private transient final Logger log = LoggerFactory.getLogger(SessionValidationGetHandler.class);
+
+    private ServerConfig srvCfg;
+    private ViewConfig viewCfg;
+
+    public SessionValidationGetHandler(SessionManager mgr, MxisdConfig cfg) {
+        super(mgr);
+        this.srvCfg = cfg.getServer();
+        this.viewCfg = cfg.getView();
+    }
+
+    @Override
+    public void handleRequest(HttpServerExchange exchange) {
+        log.info("Handling GET request to validate session");
+
+        String sid = getQueryParameter(exchange, "sid");
+        String secret = getQueryParameter(exchange, "client_secret");
+        String token = getQueryParameter(exchange, "token");
+
+        ValidationResult r = handleRequest(sid, secret, token);
+        log.info("Session {} was validated", sid);
+        if (r.getNextUrl().isPresent()) {
+            String url = r.getNextUrl().get();
+            try {
+                url = new URL(url).toString();
+            } catch (MalformedURLException e) {
+                log.info("Session next URL {} is not a valid one, will prepend public URL {}", url, srvCfg.getPublicUrl());
+                url = srvCfg.getPublicUrl() + r.getNextUrl().get();
+            }
+            log.info("Session {} validation: next URL is present, redirecting to {}", sid, url);
+            exchange.setStatusCode(302);
+            exchange.getResponseHeaders().add(HttpString.tryFromString("Location"), url);
+        } else {
+            try {
+                String data = FileUtil.load(viewCfg.getSession().getOnTokenSubmit().getSuccess());
+                writeBodyAsUtf8(exchange, data);
+            } catch (IOException e) {
+                throw new RuntimeException(e);
+            }
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/SessionValidationPostHandler.java

@@ -0,0 +1,80 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.http.undertow.handler.identity.v1;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.http.io.identity.SuccessStatusJson;
+import io.kamax.mxisd.session.SessionManager;
+import io.undertow.server.HttpServerExchange;
+import io.undertow.server.handlers.form.FormData;
+import io.undertow.server.handlers.form.FormParserFactory;
+import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+
+public class SessionValidationPostHandler extends SessionValidateHandler {
+
+    private transient final Logger log = LoggerFactory.getLogger(SessionValidationPostHandler.class);
+
+    private FormParserFactory factory;
+
+    public SessionValidationPostHandler(SessionManager mgr) {
+        super(mgr);
+        factory = FormParserFactory.builder().build();
+    }
+
+
+    @Override
+    public void handleRequest(HttpServerExchange exchange) throws IOException {
+        log.info("Handling POST request to validate session");
+
+        String sid;
+        String secret;
+        String token;
+
+        String contentType = getContentType(exchange).orElseThrow(() -> new IllegalArgumentException("Content type header is not set"));
+        if (StringUtils.equals(contentType, "application/json")) { // FIXME use MIME parsing tools
+            log.info("Parsing as JSON data");
+
+            JsonObject body = parseJsonObject(exchange);
+            sid = GsonUtil.getStringOrThrow(body, "sid");
+            secret = GsonUtil.getStringOrThrow(body, "client_secret");
+            token = GsonUtil.getStringOrThrow(body, "token");
+        } else if (StringUtils.equals(contentType, "application/x-www-form-urlencoded")) { // FIXME use MIME parsing tools
+            log.info("Parsing as Form data");
+
+            FormData data = factory.createParser(exchange).parseBlocking();
+            sid = getOrThrow(data, "sid");
+            secret = getOrThrow(data, "client_secret");
+            token = getOrThrow(data, "token");
+        } else {
+            log.info("Unsupported Content type: {}", contentType);
+            throw new IllegalArgumentException("Unsupported Content type: " + contentType);
+        }
+
+        handleRequest(sid, secret, token);
+        respondJson(exchange, new SuccessStatusJson(true));
+    }
+
+}

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/SingleLookupHandler.java

@@ -21,9 +21,7 @@
 package io.kamax.mxisd.http.undertow.handler.identity.v1;
 
 import com.google.gson.JsonObject;
-import io.kamax.matrix.event.EventKey;
 import io.kamax.matrix.json.GsonUtil;
-import io.kamax.matrix.json.MatrixJson;
 import io.kamax.mxisd.config.MxisdConfig;
 import io.kamax.mxisd.config.ServerConfig;
 import io.kamax.mxisd.crypto.SignatureManager;
@@ -73,11 +71,8 @@ public class SingleLookupHandler extends LookupHandler {
             respondJson(exchange, "{}");
         } else {
             SingleLookupReply lookup = lookupOpt.get();
-
-            // FIXME signing should be done in the business model, not in the controller
             JsonObject obj = GsonUtil.makeObj(new SingeLookupReplyJson(lookup));
-            obj.add(EventKey.Signatures.get(), signMgr.signMessageGson(cfg.getName(), MatrixJson.encodeCanonical(obj)));
+            signMgr.signMessageGson(cfg.getName(), obj);
-
             respondJson(exchange, obj);
         }
     }

src/main/java/io/kamax/mxisd/invitation/InvitationManager.java

@@ -30,7 +30,6 @@ import io.kamax.mxisd.config.InvitationConfig;
 import io.kamax.mxisd.config.MxisdConfig;
 import io.kamax.mxisd.config.ServerConfig;
 import io.kamax.mxisd.crypto.*;
-import io.kamax.mxisd.dns.FederationDnsOverwrite;
 import io.kamax.mxisd.exception.BadRequestException;
 import io.kamax.mxisd.exception.ConfigurationException;
 import io.kamax.mxisd.exception.MappingAlreadyExistsException;
@@ -38,6 +37,7 @@ import io.kamax.mxisd.exception.ObjectNotFoundException;
 import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.ThreePidMapping;
 import io.kamax.mxisd.lookup.strategy.LookupStrategy;
+import io.kamax.mxisd.matrix.HomeserverFederationResolver;
 import io.kamax.mxisd.notification.NotificationManager;
 import io.kamax.mxisd.profile.ProfileManager;
 import io.kamax.mxisd.storage.IStorage;
@@ -57,13 +57,10 @@ import org.apache.http.impl.client.HttpClients;
 import org.apache.http.ssl.SSLContextBuilder;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-import org.xbill.DNS.*;
 
 import javax.net.ssl.HostnameVerifier;
 import javax.net.ssl.SSLContext;
 import java.io.IOException;
-import java.net.MalformedURLException;
-import java.net.URL;
 import java.nio.charset.StandardCharsets;
 import java.time.DateTimeException;
 import java.time.Instant;
@@ -85,7 +82,7 @@ public class InvitationManager {
     private LookupStrategy lookupMgr;
     private KeyManager keyMgr;
     private SignatureManager signMgr;
-    private FederationDnsOverwrite dns;
+    private HomeserverFederationResolver resolver;
     private NotificationManager notifMgr;
     private ProfileManager profileMgr;
 
@@ -100,7 +97,7 @@ public class InvitationManager {
             LookupStrategy lookupMgr,
             KeyManager keyMgr,
             SignatureManager signMgr,
-            FederationDnsOverwrite dns,
+            HomeserverFederationResolver resolver,
             NotificationManager notifMgr,
             ProfileManager profileMgr
     ) {
@@ -110,15 +107,15 @@ public class InvitationManager {
         this.lookupMgr = lookupMgr;
         this.keyMgr = keyMgr;
         this.signMgr = signMgr;
-        this.dns = dns;
+        this.resolver = resolver;
         this.notifMgr = notifMgr;
         this.profileMgr = profileMgr;
 
-        log.info("Loading saved invites");
+        log.debug("Loading saved invites");
         Collection<ThreePidInviteIO> ioList = storage.getInvites();
         ioList.forEach(io -> {
             io.getProperties().putIfAbsent(CreatedAtPropertyKey, defaultCreateTs);
-            log.info("Processing invite {}", GsonUtil.get().toJson(io));
+            log.debug("Processing invite {}", GsonUtil.get().toJson(io));
             ThreePidInvite invite = new ThreePidInvite(
                     MatrixID.asAcceptable(io.getSender()),
                     io.getMedium(),
@@ -130,6 +127,7 @@ public class InvitationManager {
             ThreePidInviteReply reply = new ThreePidInviteReply(io.getId(), invite, io.getToken(), "", Collections.emptyList());
             invitations.put(reply.getId(), reply);
         });
+        log.info("Loaded saved invites");
 
         // FIXME export such madness into matrix-java-sdk with a nice wrapper to talk to a homeserver
         try {
@@ -172,12 +170,6 @@ public class InvitationManager {
 
             // Enabled by default
             cfg.getInvite().getExpiration().setEnabled(true);
-
-            // We'll resolve to our computed User ID
-            cfg.getInvite().getExpiration().setResolveTo(mxId);
-
-            // One calendar week (60min/1h * 24 = 1d * 7 = 1w)
-            cfg.getInvite().getExpiration().setAfter(60 * 24 * 7);
         }
 
         if (cfg.getInvite().getExpiration().isEnabled()) {
@@ -213,56 +205,6 @@ public class InvitationManager {
         return reply.getInvite().getSender().getId() + ":" + reply.getInvite().getRoomId() + ":" + reply.getInvite().getMedium() + ":" + reply.getInvite().getAddress();
     }
 
-    private String getSrvRecordName(String domain) {
-        return "_matrix._tcp." + domain;
-    }
-
-    // TODO use caching mechanism
-    // TODO export in matrix-java-sdk
-    private String findHomeserverForDomain(String domain) {
-        Optional<String> entryOpt = dns.findHost(domain);
-        if (entryOpt.isPresent()) {
-            String entry = entryOpt.get();
-            log.info("Found DNS overwrite for {} to {}", domain, entry);
-            try {
-                return new URL(entry).toString();
-            } catch (MalformedURLException e) {
-                log.warn("Skipping homeserver Federation DNS overwrite for {} - not a valid URL: {}", domain, entry);
-            }
-        }
-
-        log.debug("Performing SRV lookup for {}", domain);
-        String lookupDns = getSrvRecordName(domain);
-        log.info("Lookup name: {}", lookupDns);
-
-        try {
-            List<SRVRecord> srvRecords = new ArrayList<>();
-            Record[] rawRecords = new Lookup(lookupDns, Type.SRV).run();
-            if (rawRecords != null && rawRecords.length > 0) {
-                for (Record record : rawRecords) {
-                    if (Type.SRV == record.getType()) {
-                        srvRecords.add((SRVRecord) record);
-                    } else {
-                        log.info("Got non-SRV record: {}", record.toString());
-                    }
-                }
-
-                srvRecords.sort(Comparator.comparingInt(SRVRecord::getPriority));
-                for (SRVRecord record : srvRecords) {
-                    log.info("Found SRV record: {}", record.toString());
-                    return "https://" + record.getTarget().toString(true) + ":" + record.getPort();
-                }
-            } else {
-                log.info("No SRV record for {}", lookupDns);
-            }
-        } catch (TextParseException e) {
-            log.warn("Unable to perform DNS SRV query for {}: {}", lookupDns, e.getMessage());
-        }
-
-        log.info("Performing basic lookup using domain name {}", domain);
-        return "https://" + domain + ":8448";
-    }
-
     private Optional<SingleLookupReply> lookup3pid(String medium, String address) {
         if (!cfg.getResolution().isRecursive()) {
             log.warn("/!\\ /!\\ --- RECURSIVE INVITE RESOLUTION HAS BEEN DISABLED --- /!\\ /!\\");
@@ -419,7 +361,7 @@ public class InvitationManager {
                     continue;
                 }
 
-                log.info("Invite {} has expired at TS {} - Expiring and resolving to {}", targetTs, targetMxid);
+                log.info("Invite {} has expired at TS {} - Expiring and resolving to {}", reply.getId(), targetTs, targetMxid);
                 publishMapping(reply, targetMxid);
             } catch (NumberFormatException | DateTimeException e) {
                 log.warn("Invite {} has an invalid creation TS, setting to default value of {}", reply.getId(), defaultCreateTs);
@@ -481,7 +423,7 @@ public class InvitationManager {
         String address = reply.getInvite().getAddress();
         String domain = reply.getInvite().getSender().getDomain();
         log.info("Discovering HS for domain {}", domain);
-        String hsUrlOpt = findHomeserverForDomain(domain);
+        String hsUrlOpt = resolver.resolve(domain).toString();
 
         // TODO this is needed as this will block if called during authentication cycle due to synapse implementation
         new Thread(() -> { // FIXME need to make this retry-able and within a general background working pool
@@ -570,6 +512,9 @@ public class InvitationManager {
                     publishMapping(reply, lookup.getMxid().getId());
                 } else {
                     log.info("No mapping for pending invite {}", getIdForLog(reply));
+                    if (lookupMgr.getLocalProviders().isEmpty()) {
+                        log.warn("No Identity store has been configured, this invite may never resolve");
+                    }
                 }
             } catch (Throwable t) {
                 log.error("Unable to process invite", t);

src/main/java/io/kamax/mxisd/lookup/SingleLookupReply.java

@@ -72,7 +72,7 @@ public class SingleLookupReply {
     }
 
     public SingleLookupReply(SingleLookupRequest request, _MatrixID mxid) {
-        this(request, mxid, Instant.now(), Instant.ofEpochMilli(0), Instant.ofEpochMilli(253402300799000L));
+        this(request, mxid, Instant.now(), Instant.now().minusSeconds(60), Instant.now().plusSeconds(5 * 60));
     }
 
     public SingleLookupReply(SingleLookupRequest request, _MatrixID mxid, Instant timestamp, Instant notBefore, Instant notAfter) {

src/main/java/io/kamax/mxisd/matrix/HomeserverFederationResolver.java

@@ -0,0 +1,216 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.matrix;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.matrix.json.InvalidJsonException;
+import io.kamax.mxisd.dns.FederationDnsOverwrite;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.xbill.DNS.*;
+
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.util.*;
+
+public class HomeserverFederationResolver {
+
+    private static final Logger log = LoggerFactory.getLogger(HomeserverFederationResolver.class);
+
+    private FederationDnsOverwrite dns;
+    private CloseableHttpClient client;
+
+    public HomeserverFederationResolver(FederationDnsOverwrite dns, CloseableHttpClient client) {
+        this.dns = dns;
+        this.client = client;
+    }
+
+    private String getDefaultScheme() {
+        return "https";
+    }
+
+    private int getDefaultPort() {
+        return 8448;
+    }
+
+    private String getDnsSrvPrefix() {
+        return "_matrix._tcp.";
+    }
+
+    private String buildSrvRecordName(String domain) {
+        return getDnsSrvPrefix() + domain;
+    }
+
+    private Optional<URL> resolveOverwrite(String domain) {
+        Optional<String> entryOpt = dns.findHost(domain);
+        if (!entryOpt.isPresent()) {
+            log.info("No DNS overwrite for {}", domain);
+            return Optional.empty();
+        }
+
+        try {
+            return Optional.of(new URL(entryOpt.get()));
+        } catch (MalformedURLException e) {
+            log.warn("Skipping homeserver Federation DNS overwrite for {} - not a valid URL: {}", domain, entryOpt.get());
+            return Optional.empty();
+        }
+    }
+
+    private Optional<URL> resolveLiteral(String domain) {
+        if (domain.contains("[") && domain.contains("]")) {
+            // This is an IPv6
+            if (domain.contains("]:")) {
+                // With a custom port, we return as is
+                return Optional.of(build(domain));
+            } else {
+                return Optional.of(build(domain + ":" + getDefaultPort()));
+            }
+        }
+
+        if (domain.contains(":")) {
+            // This is a domain or IPv4 with an explicit port, we return as is
+            return Optional.of(build(domain));
+        }
+
+        // At this point, we do not account for the provided string to be an IPv4 without a port. We will therefore
+        // perform well-known lookup and SRV record. While this is not needed, we don't expect the SRV to return anything
+        // and the well-known shouldn't either, but it might, leading to a wrong destination potentially.
+        //
+        // We accept this risk as mxisd is not meant to be used without DNS domain as per FAQ. We also provide resolution
+        // override facilities. Therefore, we accept to not handle this case until we get report of such unwanted behaviour
+        // that still fix mxisd use case and can't be resolved via override.
+
+        return Optional.empty();
+    }
+
+    private Optional<URL> resolveWellKnown(String domain) {
+        log.debug("Performing Well-known lookup for {}", domain);
+        HttpGet wnReq = new HttpGet("https://" + domain + "/.well-known/matrix/server");
+        try (CloseableHttpResponse wnRes = client.execute(wnReq)) {
+            int status = wnRes.getStatusLine().getStatusCode();
+            if (status == 200) {
+                try {
+                    JsonObject body = GsonUtil.parseObj(EntityUtils.toString(wnRes.getEntity()));
+                    String server = GsonUtil.getStringOrNull(body, "m.server");
+                    if (StringUtils.isNotBlank(server)) {
+                        log.debug("Found well-known entry: {}", server);
+                        return Optional.of(build(server));
+                    }
+                } catch (InvalidJsonException e) {
+                    log.info("Could not parse well-known resource: {}", e.getMessage());
+                }
+            } else {
+                log.info("Well-known did not return status code 200 but {}, ignoring", status);
+            }
+
+            return Optional.empty();
+        } catch (IOException e) {
+            throw new RuntimeException("Error while trying to lookup well-known for " + domain, e);
+        }
+    }
+
+    private Optional<URL> resolveDnsSrv(String domain) {
+        log.debug("Performing SRV lookup for {}", domain);
+        String lookupDns = buildSrvRecordName(domain);
+        log.debug("Lookup name: {}", lookupDns);
+
+        try {
+            List<SRVRecord> srvRecords = new ArrayList<>();
+            Record[] rawRecords = new Lookup(lookupDns, Type.SRV).run();
+            if (Objects.isNull(rawRecords) || rawRecords.length == 0) {
+                log.debug("No SRV record for {}", domain);
+                return Optional.empty();
+            }
+
+            for (Record record : rawRecords) {
+                if (Type.SRV == record.getType()) {
+                    srvRecords.add((SRVRecord) record);
+                } else {
+                    log.debug("Ignoring non-SRV record: {}", record.toString());
+                }
+            }
+
+            if (srvRecords.size() < 1) {
+                log.warn("DNS SRV records were found for {} but none is usable", lookupDns);
+                return Optional.empty();
+            }
+
+            srvRecords.sort(Comparator.comparingInt(SRVRecord::getPriority));
+            SRVRecord record = srvRecords.get(0);
+            return Optional.of(build(record.getTarget().toString(true) + ":" + record.getPort()));
+        } catch (TextParseException e) {
+            log.warn("Unable to perform DNS SRV query for {}: {}", lookupDns, e.getMessage());
+        }
+
+        return Optional.empty();
+    }
+
+    public URL build(String authority) {
+        try {
+            return new URL(getDefaultScheme() + "://" + authority);
+        } catch (MalformedURLException e) {
+            throw new RuntimeException("Could not build URL for " + authority, e);
+        }
+    }
+
+    public URL resolve(String domain) {
+        Optional<URL> s1 = resolveOverwrite(domain);
+        if (s1.isPresent()) {
+            URL dest = s1.get();
+            log.info("Resolution of {} via DNS overwrite to {}", domain, dest);
+            return dest;
+        }
+
+        Optional<URL> s2 = resolveLiteral(domain);
+        if (s2.isPresent()) {
+            URL dest = s2.get();
+            log.info("Resolution of {} as IP literal or IP/hostname with explicit port to {}", domain, dest);
+            return dest;
+        }
+
+        Optional<URL> s3 = resolveWellKnown(domain);
+        if (s3.isPresent()) {
+            URL dest = s3.get();
+            log.info("Resolution of {} via well-known to {}", domain, dest);
+            return dest;
+        }
+        // The domain needs to be resolved
+
+        Optional<URL> s4 = resolveDnsSrv(domain);
+        if (s4.isPresent()) {
+            URL dest = s4.get();
+            log.info("Resolution of {} via DNS SRV record to {}", domain, dest);
+            return dest;
+        }
+
+        URL dest = build(domain + ":" + getDefaultPort());
+        log.info("Resolution of {} to {}", domain, dest);
+        return dest;
+    }
+
+}

src/main/java/io/kamax/mxisd/matrix/IdentityServerUtils.java

@@ -79,7 +79,7 @@ public class IdentityServerUtils {
 
             JsonElement el = parser.parse(IOUtils.toString(res.getEntity().getContent(), StandardCharsets.UTF_8));
             if (!el.isJsonObject()) {
-                log.debug("IS {} did not send back an empty JSON object as per spec, not a valid IS");
+                log.debug("IS {} did not send back an empty JSON object as per spec, not a valid IS", remote);
                 return false;
             }
 
@@ -90,7 +90,7 @@ public class IdentityServerUtils {
         }
     }
 
-    public static String getSrvRecordName(String domain) {
+    private static String getSrvRecordName(String domain) {
         return "_matrix-identity._tcp." + domain;
     }
 

src/main/java/io/kamax/mxisd/session/SessionManager.java

@@ -32,15 +32,15 @@ import io.kamax.mxisd.exception.NotImplementedException;
 import io.kamax.mxisd.exception.SessionNotValidatedException;
 import io.kamax.mxisd.exception.SessionUnknownException;
 import io.kamax.mxisd.lookup.SingleLookupReply;
+import io.kamax.mxisd.lookup.SingleLookupRequest;
 import io.kamax.mxisd.lookup.ThreePidValidation;
 import io.kamax.mxisd.lookup.strategy.LookupStrategy;
 import io.kamax.mxisd.notification.NotificationManager;
 import io.kamax.mxisd.storage.IStorage;
 import io.kamax.mxisd.storage.dao.IThreePidSessionDao;
 import io.kamax.mxisd.threepid.session.ThreePidSession;
-import org.apache.commons.lang.RandomStringUtils;
+import org.apache.commons.lang3.RandomStringUtils;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang3.StringUtils;
-import org.apache.http.impl.client.CloseableHttpClient;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -58,23 +58,18 @@ public class SessionManager {
     private NotificationManager notifMgr;
     private LookupStrategy lookupMgr;
 
-    // FIXME export into central class, set version
-    private CloseableHttpClient client;
-
     public SessionManager(
             SessionConfig cfg,
             MatrixConfig mxCfg,
             IStorage storage,
             NotificationManager notifMgr,
-            LookupStrategy lookupMgr,
+            LookupStrategy lookupMgr
-            CloseableHttpClient client
     ) {
         this.cfg = cfg;
         this.mxCfg = mxCfg;
         this.storage = storage;
         this.notifMgr = notifMgr;
         this.lookupMgr = lookupMgr;
-        this.client = client;
     }
 
     private ThreePidSession getSession(String sid, String secret) {
@@ -128,7 +123,7 @@ public class SessionManager {
                 log.info("Generated new session {} to validate {} from server {}", sessionId, tpid, server);
 
                 storage.insertThreePidSession(session.getDao());
-                log.info("Stored session {}", sessionId, tpid, server);
+                log.info("Stored session {}", sessionId);
 
                 log.info("Session {} for {}: sending validation notification", sessionId, tpid);
                 notifMgr.sendForValidation(session);
@@ -139,12 +134,13 @@ public class SessionManager {
     }
 
     public ValidationResult validate(String sid, String secret, String token) {
+        log.info("Validating session {}", sid);
         ThreePidSession session = getSession(sid, secret);
-        log.info("Attempting validation for session {} from {}", session.getId(), session.getServer());
+        log.info("Session {} is from {}", session.getId(), session.getServer());
 
         session.validate(token);
         storage.updateThreePidSession(session.getDao());
-        log.info("Session {} has been validated locally", session.getId());
+        log.info("Session {} has been validated", session.getId());
 
         ValidationResult r = new ValidationResult(session);
         session.getNextLink().ifPresent(r::setNextUrl);
@@ -156,7 +152,7 @@ public class SessionManager {
         return new ThreePidValidation(session.getThreePid(), session.getValidationTime());
     }
 
-    public void bind(String sid, String secret, String mxidRaw) {
+    public SingleLookupReply bind(String sid, String secret, String mxidRaw) {
         // We make sure we have an acceptable User ID
         if (StringUtils.isEmpty(mxidRaw)) {
             throw new IllegalArgumentException("No Matrix User ID provided");
@@ -170,11 +166,16 @@ public class SessionManager {
 
         // Only accept binds if the domain matches our own
         if (!StringUtils.equalsIgnoreCase(mxCfg.getDomain(), mxid.getDomain())) {
-            throw new NotAllowedException("Only Matrix IDs from domain " + mxCfg + " can be bound");
+            throw new NotAllowedException("Only Matrix IDs from domain " + mxCfg.getDomain() + " can be bound");
         }
 
         log.info("Session {}: Binding of {}:{} to Matrix ID {} is accepted",
                 session.getId(), session.getThreePid().getMedium(), session.getThreePid().getAddress(), mxid.getId());
+
+        SingleLookupRequest request = new SingleLookupRequest();
+        request.setType(session.getThreePid().getMedium());
+        request.setThreePid(session.getThreePid().getAddress());
+        return new SingleLookupReply(request, mxid);
     }
 
     public void unbind(JsonObject reqData) {
@@ -195,6 +196,7 @@ public class SessionManager {
              */
 
             log.warn("A remote host attempted to unbind without proper authorization. Request was denied");
+            log.warn("See https://github.com/kamax-matrix/mxisd/wiki/mxisd-and-your-privacy for more info");
 
             if (!cfg.getPolicy().getUnbind().getFraudulent().getSendWarning()) {
                 log.info("Not sending notification to 3PID owner as per configuration");

src/main/java/io/kamax/mxisd/threepid/connector/email/BlackholeEmailConnector.java

@@ -0,0 +1,37 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.threepid.connector.email;
+
+public class BlackholeEmailConnector implements EmailConnector {
+
+    public static final String ID = "none";
+
+    @Override
+    public String getId() {
+        return ID;
+    }
+
+    @Override
+    public void send(String senderAddress, String senderName, String recipient, String content) {
+        //dev/null
+    }
+
+}

src/main/java/io/kamax/mxisd/threepid/connector/email/BuiltInEmailConnectorSupplier.java

@@ -33,6 +33,10 @@ public class BuiltInEmailConnectorSupplier implements EmailConnectorSupplier {
 
     @Override
     public Optional<EmailConnector> apply(EmailConfig cfg, Mxisd mxisd) {
+        if (StringUtils.equals(BlackholeEmailConnector.ID, cfg.getConnector())) {
+            return Optional.of(new BlackholeEmailConnector());
+        }
+
         if (StringUtils.equals(EmailSmtpConnector.ID, cfg.getConnector())) {
             EmailSmtpConfig smtpCfg = GsonUtil.get().fromJson(cfg.getConnectors().getOrDefault(EmailSmtpConnector.ID, new JsonObject()), EmailSmtpConfig.class);
             return Optional.of(new EmailSmtpConnector(smtpCfg));

src/main/java/io/kamax/mxisd/threepid/connector/email/EmailSmtpConnector.java

@@ -31,14 +31,17 @@ import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import javax.mail.Header;
 import javax.mail.Message;
 import javax.mail.MessagingException;
 import javax.mail.Session;
 import javax.mail.internet.InternetAddress;
 import javax.mail.internet.MimeMessage;
+import javax.mail.internet.MimeUtility;
 import java.io.UnsupportedEncodingException;
 import java.nio.charset.StandardCharsets;
 import java.util.Date;
+import java.util.Enumeration;
 import java.util.Properties;
 
 public class EmailSmtpConnector implements EmailConnector {
@@ -66,6 +69,10 @@ public class EmailSmtpConnector implements EmailConnector {
             sCfg.setProperty("mail.smtp.auth", "true");
         }
 
+        if (cfg.getTls() == 3) {
+            sCfg.setProperty("mail.smtp.ssl.enable", "true");
+        }
+
         session = Session.getInstance(sCfg);
     }
 
@@ -93,7 +100,16 @@ public class EmailSmtpConnector implements EmailConnector {
         try {
             InternetAddress sender = new InternetAddress(senderAddress, senderName);
             MimeMessage msg = new MimeMessage(session, IOUtils.toInputStream(content, StandardCharsets.UTF_8));
-            msg.setHeader("X-Mailer", Mxisd.Agent);
+
+            // We must encode our headers ourselves as we have no guarantee that they were in the provided data.
+            // This is required to support UTF-8 characters from user display names or room names in the subject header per example
+            Enumeration<Header> headers = msg.getAllHeaders();
+            while (headers.hasMoreElements()) {
+                Header header = headers.nextElement();
+                msg.setHeader(header.getName(), MimeUtility.encodeText(header.getValue()));
+            }
+
+            msg.setHeader("X-Mailer", MimeUtility.encodeText(Mxisd.Agent));
             msg.setSentDate(new Date());
             msg.setFrom(sender);
             msg.setRecipients(Message.RecipientType.TO, recipient);
@@ -101,8 +117,11 @@ public class EmailSmtpConnector implements EmailConnector {
 
             log.info("Sending invite to {} via SMTP using {}:{}", recipient, cfg.getHost(), cfg.getPort());
             SMTPTransport transport = (SMTPTransport) session.getTransport("smtp");
+
+            if (cfg.getTls() < 3) {
                 transport.setStartTLS(cfg.getTls() > 0);
                 transport.setRequireStartTLS(cfg.getTls() > 1);
+            }
 
             log.info("Connecting to {}:{}", cfg.getHost(), cfg.getPort());
             if (StringUtils.isAllEmpty(cfg.getLogin(), cfg.getPassword())) {

src/main/java/io/kamax/mxisd/threepid/connector/phone/BlackholePhoneConnector.java

@@ -24,14 +24,14 @@ public class BlackholePhoneConnector implements PhoneConnector {
 
     public static final String ID = "none";
 
-    @Override
-    public void send(String recipient, String content) {
-        //dev/null
-    }
-
     @Override
     public String getId() {
         return ID;
     }
 
+    @Override
+    public void send(String recipient, String content) {
+        //dev/null
+    }
+
 }

src/main/java/io/kamax/mxisd/threepid/connector/phone/BuiltInPhoneConnectorSupplier.java

@@ -33,15 +33,15 @@ public class BuiltInPhoneConnectorSupplier implements PhoneConnectorSupplier {
 
     @Override
     public Optional<PhoneConnector> apply(PhoneConfig cfg, Mxisd mxisd) {
+        if (StringUtils.equals(BlackholePhoneConnector.ID, cfg.getConnector())) {
+            return Optional.of(new BlackholePhoneConnector());
+        }
+
         if (StringUtils.equals(PhoneSmsTwilioConnector.ID, cfg.getConnector())) {
             PhoneTwilioConfig cCfg = GsonUtil.get().fromJson(cfg.getConnectors().getOrDefault(PhoneSmsTwilioConnector.ID, new JsonObject()), PhoneTwilioConfig.class);
             return Optional.of(new PhoneSmsTwilioConnector(cCfg));
         }
 
-        if (StringUtils.equals(BlackholePhoneConnector.ID, cfg.getConnector())) {
-            return Optional.of(new BlackholePhoneConnector());
-        }
-
         return Optional.empty();
     }
 

src/main/java/io/kamax/mxisd/threepid/connector/phone/PhoneSmsTwilioConnector.java

@@ -21,10 +21,12 @@
 package io.kamax.mxisd.threepid.connector.phone;
 
 import com.twilio.Twilio;
+import com.twilio.exception.ApiException;
 import com.twilio.rest.api.v2010.account.Message;
 import com.twilio.type.PhoneNumber;
 import io.kamax.mxisd.config.threepid.connector.PhoneTwilioConfig;
-import io.kamax.mxisd.exception.BadRequestException;
+import io.kamax.mxisd.exception.InternalServerError;
+import io.kamax.mxisd.exception.NotImplementedException;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -52,12 +54,17 @@ public class PhoneSmsTwilioConnector implements PhoneConnector {
     @Override
     public void send(String recipient, String content) {
         if (StringUtils.isBlank(cfg.getAccountSid()) || StringUtils.isBlank(cfg.getAuthToken()) || StringUtils.isBlank(cfg.getNumber())) {
-            throw new BadRequestException("Phone numbers cannot be validated at this time. Contact your administrator.");
+            log.error("Twilio connector in not fully configured and is missing mandatory configuration values.");
+            throw new NotImplementedException("Phone numbers cannot be validated at this time. Contact your administrator.");
         }
 
         recipient = "+" + recipient;
         log.info("Sending SMS notification from {} to {} with {} characters", cfg.getNumber(), recipient, content.length());
+        try {
             Message.creator(new PhoneNumber("+" + recipient), new PhoneNumber(cfg.getNumber()), content).create();
+        } catch (ApiException e) {
+            throw new InternalServerError(e);
+        }
     }
 
 }

src/main/java/io/kamax/mxisd/threepid/generator/GenericTemplateNotificationGenerator.java

@@ -68,6 +68,7 @@ public abstract class GenericTemplateNotificationGenerator extends PlaceholderNo
     @Override
     public String getForReply(IThreePidInviteReply invite) {
         log.info("Generating notification content for 3PID invite");
+        invite.getInvite().getProperties().putAll(cfg.getPlaceholder());
         return populateForReply(invite, getTemplateContent(cfg.getInvite()));
     }
 

src/main/java/io/kamax/mxisd/threepid/generator/PlaceholderNotificationGenerator.java

@@ -27,14 +27,17 @@ import io.kamax.mxisd.http.IsAPIv1;
 import io.kamax.mxisd.invitation.IMatrixIdInvite;
 import io.kamax.mxisd.invitation.IThreePidInviteReply;
 import io.kamax.mxisd.threepid.session.IThreePidSession;
-import org.apache.commons.lang.StringUtils;
+import io.kamax.mxisd.util.RestClientUtils;
 import org.apache.commons.lang.WordUtils;
+import org.apache.commons.lang3.StringUtils;
 
 import static io.kamax.mxisd.http.io.identity.StoreInviteRequest.Keys.RoomName;
 import static io.kamax.mxisd.http.io.identity.StoreInviteRequest.Keys.SenderDisplayName;
 
 public abstract class PlaceholderNotificationGenerator {
 
+    public static final String RegisterUrl = "REGISTER_URL";
+
     private MatrixConfig mxCfg;
     private ServerConfig srvCfg;
 
@@ -44,16 +47,26 @@ public abstract class PlaceholderNotificationGenerator {
     }
 
     protected String populateForCommon(ThreePid recipient, String input) {
+        if (StringUtils.isBlank(input)) {
+            return input;
+        }
+
         String domainPretty = WordUtils.capitalizeFully(mxCfg.getDomain());
 
         return input
                 .replace("%DOMAIN%", mxCfg.getDomain())
                 .replace("%DOMAIN_PRETTY%", domainPretty)
                 .replace("%RECIPIENT_MEDIUM%", recipient.getMedium())
-                .replace("%RECIPIENT_ADDRESS%", recipient.getAddress());
+                .replace("%RECIPIENT_MEDIUM_URL_ENCODED%", RestClientUtils.urlEncode(recipient.getMedium()))
+                .replace("%RECIPIENT_ADDRESS%", recipient.getAddress())
+                .replace("%RECIPIENT_ADDRESS_URL_ENCODED%", RestClientUtils.urlEncode(recipient.getAddress()));
     }
 
     protected String populateForInvite(IMatrixIdInvite invite, String input) {
+        if (StringUtils.isBlank(input)) {
+            return input;
+        }
+
         String senderName = invite.getProperties().getOrDefault(SenderDisplayName, "");
         String senderNameOrId = StringUtils.defaultIfBlank(senderName, invite.getSender().getId());
         String roomName = invite.getProperties().getOrDefault(RoomName, "");
@@ -70,25 +83,37 @@ public abstract class PlaceholderNotificationGenerator {
     }
 
     protected String populateForReply(IThreePidInviteReply invite, String input) {
+        if (StringUtils.isBlank(input)) {
+            return input;
+        }
+
         ThreePid tpid = new ThreePid(invite.getInvite().getMedium(), invite.getInvite().getAddress());
 
         String senderName = invite.getInvite().getProperties().getOrDefault(SenderDisplayName, "");
         String senderNameOrId = StringUtils.defaultIfBlank(senderName, invite.getInvite().getSender().getId());
         String roomName = invite.getInvite().getProperties().getOrDefault(RoomName, "");
         String roomNameOrId = StringUtils.defaultIfBlank(roomName, invite.getInvite().getRoomId());
+        String registerUrl = StringUtils.defaultIfBlank(invite.getInvite().getProperties().get(RegisterUrl), "https://" + mxCfg.getDomain());
 
         return populateForCommon(tpid, input)
+                .replace("%" + RegisterUrl + "%", registerUrl)
                 .replace("%SENDER_ID%", invite.getInvite().getSender().getId())
                 .replace("%SENDER_NAME%", senderName)
                 .replace("%SENDER_NAME_OR_ID%", senderNameOrId)
                 .replace("%INVITE_MEDIUM%", tpid.getMedium())
+                .replace("%INVITE_MEDIUM_URL_ENCODED%", RestClientUtils.urlEncode(tpid.getMedium()))
                 .replace("%INVITE_ADDRESS%", tpid.getAddress())
+                .replace("%INVITE_ADDRESS_URL_ENCODED%", RestClientUtils.urlEncode(tpid.getAddress()))
                 .replace("%ROOM_ID%", invite.getInvite().getRoomId())
                 .replace("%ROOM_NAME%", roomName)
                 .replace("%ROOM_NAME_OR_ID%", roomNameOrId);
     }
 
     protected String populateForValidation(IThreePidSession session, String input) {
+        if (StringUtils.isBlank(input)) {
+            return input;
+        }
+
         String validationLink = srvCfg.getPublicUrl() + IsAPIv1.getValidate(
                 session.getThreePid().getMedium(),
                 session.getId(),
@@ -102,10 +127,6 @@ public abstract class PlaceholderNotificationGenerator {
                 .replace("%NEXT_URL%", validationLink);
     }
 
-    protected String populateForRemoteValidation(IThreePidSession session, String input) {
-        return populateForValidation(session, input);
-    }
-
     protected String populateForFraudulentUndind(ThreePid tpid, String input) {
         return populateForCommon(tpid, input);
     }

src/main/java/io/kamax/mxisd/threepid/notification/BuiltInNotificationHandlerSupplier.java

@@ -20,7 +20,7 @@
 
 package io.kamax.mxisd.threepid.notification;
 
-import com.google.gson.JsonObject;
+import com.google.gson.JsonSyntaxException;
 import io.kamax.matrix.ThreePidMedium;
 import io.kamax.matrix.json.GsonUtil;
 import io.kamax.mxisd.Mxisd;
@@ -65,13 +65,18 @@ public class BuiltInNotificationHandlerSupplier implements NotificationHandlerSu
         if (StringUtils.equals(EmailRawNotificationHandler.ID, handler)) {
             Object o = mxisd.getConfig().getThreepid().getMedium().get(ThreePidMedium.Email.getId());
             if (Objects.nonNull(o)) {
-                EmailConfig emailCfg = GsonUtil.get().fromJson(GsonUtil.makeObj(o), EmailConfig.class);
+                EmailConfig emailCfg;
+                try {
+                    emailCfg = GsonUtil.get().fromJson(GsonUtil.makeObj(o), EmailConfig.class);
+                } catch (JsonSyntaxException e) {
+                    throw new ConfigurationException("Invalid configuration for threepid email notification");
+                }
 
-                if (org.apache.commons.lang.StringUtils.isBlank(emailCfg.getGenerator())) {
+                if (StringUtils.isBlank(emailCfg.getGenerator())) {
                     throw new ConfigurationException("notification.email.generator");
                 }
 
-                if (org.apache.commons.lang.StringUtils.isBlank(emailCfg.getConnector())) {
+                if (StringUtils.isBlank(emailCfg.getConnector())) {
                     throw new ConfigurationException("notification.email.connector");
                 }
 
@@ -94,9 +99,15 @@ public class BuiltInNotificationHandlerSupplier implements NotificationHandlerSu
         }
 
         if (StringUtils.equals(EmailSendGridNotificationHandler.ID, handler)) {
-            JsonObject cfgJson = mxisd.getConfig().getNotification().getHandlers().get(EmailSendGridNotificationHandler.ID);
+            Object cfgJson = mxisd.getConfig().getNotification().getHandlers().get(EmailSendGridNotificationHandler.ID);
             if (Objects.nonNull(cfgJson)) {
-                EmailSendGridConfig cfg = GsonUtil.get().fromJson(cfgJson, EmailSendGridConfig.class);
+                EmailSendGridConfig cfg;
+                try {
+                    cfg = GsonUtil.get().fromJson(GsonUtil.get().toJson(cfgJson), EmailSendGridConfig.class);
+                } catch (JsonSyntaxException e) {
+                    throw new ConfigurationException("Invalid configuration for threepid email sendgrid handler");
+                }
+
                 NotificationHandlers.register(() -> new EmailSendGridNotificationHandler(mxisd.getConfig(), cfg));
             }
         }
@@ -107,7 +118,12 @@ public class BuiltInNotificationHandlerSupplier implements NotificationHandlerSu
         if (StringUtils.equals(PhoneNotificationHandler.ID, handler)) {
             Object o = mxisd.getConfig().getThreepid().getMedium().get(ThreePidMedium.PhoneNumber.getId());
             if (Objects.nonNull(o)) {
-                PhoneConfig cfg = GsonUtil.get().fromJson(GsonUtil.makeObj(o), PhoneConfig.class);
+                PhoneConfig cfg;
+                try {
+                    cfg = GsonUtil.get().fromJson(GsonUtil.makeObj(o), PhoneConfig.class);
+                } catch (JsonSyntaxException e) {
+                    throw new ConfigurationException("Invalid configuration for threepid msisdn notification");
+                }
 
                 List<PhoneGenerator> generators = StreamSupport
                         .stream(ServiceLoader.load(PhoneGeneratorSupplier.class).spliterator(), false)

src/main/java/io/kamax/mxisd/threepid/notification/email/EmailSendGridNotificationHandler.java

@@ -33,7 +33,7 @@ import io.kamax.mxisd.notification.NotificationHandler;
 import io.kamax.mxisd.threepid.generator.PlaceholderNotificationGenerator;
 import io.kamax.mxisd.threepid.session.IThreePidSession;
 import io.kamax.mxisd.util.FileUtil;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang3.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -86,6 +86,9 @@ public class EmailSendGridNotificationHandler extends PlaceholderNotificationGen
     @Override
     public void sendForInvite(IMatrixIdInvite invite) {
         EmailTemplate template = cfg.getTemplates().getGeneric().get("matrixId");
+        if (StringUtils.isAllBlank(template.getBody().getText(), template.getBody().getHtml())) {
+            throw new FeatureNotAvailable("No template has been configured for Matrix ID invite notifications");
+        }
 
         Email email = getEmail();
         email.setSubject(populateForInvite(invite, template.getSubject()));
@@ -98,6 +101,10 @@ public class EmailSendGridNotificationHandler extends PlaceholderNotificationGen
     @Override
     public void sendForReply(IThreePidInviteReply invite) {
         EmailTemplate template = cfg.getTemplates().getInvite();
+        if (StringUtils.isAllBlank(template.getBody().getText(), template.getBody().getHtml())) {
+            throw new FeatureNotAvailable("No template has been configured for 3PID invite notifications");
+        }
+
         Email email = getEmail();
         email.setSubject(populateForReply(invite, template.getSubject()));
         email.setText(populateForReply(invite, getFromFile(template.getBody().getText())));
@@ -109,6 +116,10 @@ public class EmailSendGridNotificationHandler extends PlaceholderNotificationGen
     @Override
     public void sendForValidation(IThreePidSession session) {
         EmailTemplate template = cfg.getTemplates().getSession().getValidation();
+        if (StringUtils.isAllBlank(template.getBody().getText(), template.getBody().getHtml())) {
+            throw new FeatureNotAvailable("No template has been configured for validation notifications");
+        }
+
         Email email = getEmail();
         email.setSubject(populateForValidation(session, template.getSubject()));
         email.setText(populateForValidation(session, getFromFile(template.getBody().getText())));
@@ -120,6 +131,10 @@ public class EmailSendGridNotificationHandler extends PlaceholderNotificationGen
     @Override
     public void sendForFraudulentUnbind(ThreePid tpid) {
         EmailTemplate template = cfg.getTemplates().getSession().getUnbind().getFraudulent();
+        if (StringUtils.isAllBlank(template.getBody().getText(), template.getBody().getHtml())) {
+            throw new FeatureNotAvailable("No template has been configured for fraudulent unbind notifications");
+        }
+
         Email email = getEmail();
         email.setSubject(populateForCommon(tpid, template.getSubject()));
         email.setText(populateForCommon(tpid, getFromFile(template.getBody().getText())));

src/main/java/io/kamax/mxisd/util/RestClientUtils.java

@@ -25,12 +25,22 @@ import org.apache.http.client.methods.HttpPost;
 import org.apache.http.entity.ContentType;
 import org.apache.http.entity.StringEntity;
 
+import java.io.UnsupportedEncodingException;
+import java.net.URLEncoder;
 import java.nio.charset.StandardCharsets;
 
 public class RestClientUtils {
 
     private static Gson gson = GsonUtil.build();
 
+    public static String urlEncode(String value) {
+        try {
+            return URLEncoder.encode(value, StandardCharsets.UTF_8.name());
+        } catch (UnsupportedEncodingException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
     public static HttpPost post(String url, String body) {
         StringEntity entity = new StringEntity(body, StandardCharsets.UTF_8);
         entity.setContentType(ContentType.APPLICATION_JSON.toString());

src/main/resources/simplelogger.properties

@@ -1,2 +1,4 @@
 org.slf4j.simpleLogger.logFile=System.out
 org.slf4j.simpleLogger.log.org.xnio=warn
+org.slf4j.simpleLogger.log.com.j256.ormlite.table.TableUtils=warn
+org.slf4j.simpleLogger.log.com.mchange.v2.log.MLog=warn

src/main/resources/threepids/email/invite-template.eml

@@ -9,19 +9,12 @@ Content-Disposition: inline
 
 Hi,
 
-%SENDER_NAME_OR_ID% has invited you into a room [%ROOM_NAME_OR_ID%] on
+%SENDER_NAME_OR_ID% has invited you to the Matrix room [%ROOM_NAME_OR_ID%].
-Matrix. To join the conversation, register an account on https://%DOMAIN%
+To join the conversation, register an account using %REGISTER_URL%
 
+You may be required to provide the same email used for this invite during registration.
 You can also register an account on a public server and get in touch with them.
 
-
-About Matrix:
-
-Matrix is an open standard for interoperable, decentralised, real-time communication
-over IP, supporting group chat, file transfer, voice and video calling, integrations to
-other apps, bridges to other communication systems and much more. It can be used to power
-Instant Messaging, VoIP/WebRTC signalling, Internet of Things communication.
-
 Thanks,
 
 %DOMAIN_PRETTY% Admins
@@ -39,26 +32,26 @@ Content-Disposition: inline
 <html lang="en">
     <head>
         <style type="text/css">
-body {
+            body {
                 margin: 0px;
-}
+            }
 
-pre, code {
+            pre, code {
                 word-break: break-word;
                 white-space: pre-wrap;
-}
+            }
 
-#page {
+            #page {
                 font-family: 'Open Sans', Helvetica, Arial, Sans-Serif;
                 font-color: #454545;
                 font-size: 12pt;
                 width: 100%%;
                 padding: 20px;
-}
+            }
 
-#inner {
+            #inner {
                 width: 640px;
-}
+            }
         </style>
     </head>
     <body>
@@ -66,24 +59,17 @@ pre, code {
             <tr>
                 <td> </td>
                 <td id="inner">
-<p>Hi,</p>
+                    <p>Hi,</p>
 
-<p>%SENDER_NAME_OR_ID% has invited you into a room [%ROOM_NAME_OR_ID%] on
+                    <p>%SENDER_NAME_OR_ID% has invited you to the Matrix room [%ROOM_NAME_OR_ID%].<br/>
-Matrix. To join the conversation, register an account on <a href="https://%DOMAIN%">%DOMAIN%</a>.</p>
+                    To join the conversation, register an account on <a href="%REGISTER_URL%">%DOMAIN%</a>.</p>
 
-<pYou can also register an account on a public server and get in touch with them.</p>
+                    <p>You may be required to provide the same email used for this invite during registration.<br/>
+                    You can also register an account on a public server and get in touch with them.</p>
 
-<br>
+                    <p>Thanks,</p>
-<p>About Matrix:</p>
 
-<p>Matrix is an open standard for interoperable, decentralised, real-time communication
+                    <p>%DOMAIN_PRETTY% Admins</p>
-   over IP, supporting group chat, file transfer, voice and video calling, integrations to
-   other apps, bridges to other communication systems and much more. It can be used to power
-   Instant Messaging, VoIP/WebRTC signalling, Internet of Things communication.</p>
-
-<p>Thanks,</p>
-
-<p>%DOMAIN_PRETTY% Admins</p>
                 </td>
                 <td> </td>
             </tr>

src/main/resources/threepids/email/mxid-template.eml

@@ -9,7 +9,7 @@ Content-Disposition: inline
 
 Hi,
 
-%SENDER_NAME_OR_ID% has invited you into a room [%ROOM_NAME_OR_ID%] on Matrix.
+%SENDER_NAME_OR_ID% has invited you to the Matrix room [%ROOM_NAME_OR_ID%].
 
 Thanks,
 
@@ -28,26 +28,26 @@ Content-Disposition: inline
 <html lang="en">
     <head>
         <style type="text/css">
-body {
+            body {
                 margin: 0px;
-}
+            }
 
-pre, code {
+            pre, code {
                 word-break: break-word;
                 white-space: pre-wrap;
-}
+            }
 
-#page {
+            #page {
                 font-family: 'Open Sans', Helvetica, Arial, Sans-Serif;
                 font-color: #454545;
                 font-size: 12pt;
                 width: 100%%;
                 padding: 20px;
-}
+            }
 
-#inner {
+            #inner {
                 width: 640px;
-}
+            }
         </style>
     </head>
     <body>
@@ -55,13 +55,13 @@ pre, code {
             <tr>
                 <td> </td>
                 <td id="inner">
-<p>Hi,</p>
+                    <p>Hi,</p>
 
-<p>%SENDER_NAME_OR_ID% has invited you into a room [%ROOM_NAME_OR_ID%] on Matrix.</p>
+                    <p>%SENDER_NAME_OR_ID% has invited you to the Matrix room [%ROOM_NAME_OR_ID%].</p>
 
-<p>Thanks,</p>
+                    <p>Thanks,</p>
 
-<p>%DOMAIN_PRETTY% Admins</p>
+                    <p>%DOMAIN_PRETTY% Admins</p>
                 </td>
                 <td> </td>
             </tr>

src/main/resources/threepids/email/unbind-fraudulent.eml

@@ -61,26 +61,26 @@ Content-Disposition: inline
 <html lang="en">
     <head>
         <style type="text/css">
-body {
+            body {
                 margin: 0px;
-}
+            }
 
-pre, code {
+            pre, code {
                 word-break: break-word;
                 white-space: pre-wrap;
-}
+            }
 
-#page {
+            #page {
                 font-family: 'Open Sans', Helvetica, Arial, Sans-Serif;
                 font-color: #454545;
                 font-size: 12pt;
                 width: 100%%;
                 padding: 20px;
-}
+            }
 
-#inner {
+            #inner {
                 width: 640px;
-}
+            }
         </style>
     </head>
     <body>
@@ -88,42 +88,42 @@ pre, code {
             <tr>
                 <td> </td>
                 <td id="inner">
-<p>Hi,</p>
+                    <p>Hi,</p>
 
-<p><b>THIS IS IMPORTANT, PLEASE READ CAREFULLY</b>.<br/>
+                    <p><b>THIS IS IMPORTANT, PLEASE READ CAREFULLY</b>.<br/>
-If you are the system administrator of the Matrix installation, read the second section.</p>
+                    If you are the system administrator of the Matrix installation, read the second section.</p>
 
-<p>This is a notification email that a possibly unauthorized entity has attempted to alter your
+                    <p>This is a notification email that a possibly unauthorized entity has attempted to alter your
                        3PIDs (email, phone numbers, etc.) settings. The request was denied and no change has been made.</p>
 
-<p>This is so you are aware of a possible failure in case you just tried to remove a 3PID from your account.</p>
+                    <p>This is so you are aware of a possible failure in case you just tried to remove a 3PID from your account.</p>
 
-<p>If you do not understand this email, please forward it to your System administrator.</p>
+                    <p>If you do not understand this email, please forward it to your System administrator.</p>
 
-<hr>
+                    <hr>
 
-<p>As the system administrator:</p>
+                    <p>As the system administrator:</p>
 
-<p>If you are using synapse as a Homeserver, this is a known issue related to <a href="https://github.com/matrix-org/matrix-doc/issues/1194">MSC1194</a>
+                    <p>If you are using synapse as a Homeserver, this is a known issue related to <a href="https://github.com/matrix-org/matrix-doc/issues/1194">MSC1194</a>
                        and abuse of separation of concerns. As a privacy-centric product and to protect your privacy, the request was actively
                        blocked. We have written a more detailed explanation on our <a href="https://github.com/kamax-matrix/mxisd/wiki/mxisd-and-your-privacy">Privacy wiki page</a>
                        (<a href="https://github.com/kamax-matrix/mxisd/wiki/mxisd-and-your-privacy#msc1194-synapse-and-impacts-on-your-privacy">Direct link to section</a>)
                        so you can fully grasp the impact for you and your users.</p>
 
-<p>We have open an issue on the synapse repos to reflect the related privacy concerns and GDPR violation(s) and would
+                    <p>We have open an issue on the synapse repos to reflect the related privacy concerns and GDPR violation(s) and would
                        appreciate if you could comment on it or simply adds a thumbs up so the concerns are finally dealt with by the synapse dev team.<br/>
                        Issue: <a href="https://github.com/matrix-org/synapse/issues/4540">https://github.com/matrix-org/synapse/issues/4540</a></p>
 
-<p>If you are using another Homeserver or this came following no action from your own users, then you have been the target
+                    <p>If you are using another Homeserver or this came following no action from your own users, then you have been the target
                        of an unbind attack from a rogue entity which was blocked. You may want to check your logs to see the exact source of
                        the attack and take relevant actions following your policy.</p>
 
-<p>If you would like to disable these notifications, please see the
+                    <p>If you would like to disable these notifications, please see the
-<a href="https://github.com/kamax-matrix/mxisd/blob/master/docs/threepids/session/session.md#configuration">3PID sessions configuration documentation.</a></p>
+                    <a href="https://github.com/kamax-matrix/mxisd/blob/master/docs/threepids/session/session.md#configuration">3PID sessions configuration documentation.</a></p>
 
-<p>Thanks,</p>
+                    <p>Thanks,</p>
 
-<p>%DOMAIN_PRETTY% Admins</p>
+                    <p>%DOMAIN_PRETTY% Admins</p>
                 </td>
                 <td> </td>
             </tr>

src/main/resources/threepids/email/validate-template.eml

@@ -33,30 +33,30 @@ Content-Disposition: inline
 <html lang="en">
     <head>
         <style type="text/css">
-body {
+            body {
                 margin: 0px;
-}
+            }
 
-pre, code {
+            pre, code {
                 word-break: break-word;
                 white-space: pre-wrap;
-}
+            }
 
-#page {
+            #page {
                 font-family: 'Open Sans', Helvetica, Arial, Sans-Serif;
                 font-color: #454545;
                 font-size: 12pt;
                 width: 100%%;
                 padding: 20px;
-}
+            }
 
-#inner {
+            #inner {
                 width: 640px;
-}
+            }
 
-.notif_link a, .footer a {
+            .notif_link a, .footer a {
                 color: #76CFA6 ! important;
-}
+            }
         </style>
     </head>
     <body>

src/test/java/io/kamax/mxisd/test/MxisdDefaultTest.java

@@ -33,11 +33,7 @@ public class MxisdDefaultTest {
 
     @Test
     public void defaultConfig() {
-        MxisdConfig cfg = new MxisdConfig();
+        MxisdConfig cfg = MxisdConfig.forDomain(domain).inMemory();
-        cfg.getMatrix().setDomain(domain);
-        cfg.getKey().setPath(":memory:");
-        cfg.getStorage().getProvider().getSqlite().setDatabase(":memory:");
-
         Mxisd m = new Mxisd(cfg);
         m.start();
 

src/test/java/io/kamax/mxisd/test/MxisdTest.java

@@ -41,10 +41,7 @@ public class MxisdTest {
 
     @Before
     public void before() {
-        MxisdConfig cfg = new MxisdConfig();
+        MxisdConfig cfg = MxisdConfig.forDomain("localhost").inMemory();
-        cfg.getMatrix().setDomain("localhost");
-        cfg.getKey().setPath(":memory:");
-        cfg.getStorage().getProvider().getSqlite().setDatabase(":memory:");
 
         MemoryThreePid mem3pid = new MemoryThreePid();
         mem3pid.setMedium("email");

src/test/java/io/kamax/mxisd/test/auth/AuthManagerTest.java

@@ -0,0 +1,77 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.test.auth;
+
+import io.kamax.mxisd.Mxisd;
+import io.kamax.mxisd.auth.AuthManager;
+import io.kamax.mxisd.auth.UserAuthResult;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.config.memory.MemoryIdentityConfig;
+import io.kamax.mxisd.config.memory.MemoryThreePid;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static org.junit.Assert.assertTrue;
+
+public class AuthManagerTest {
+
+    private static AuthManager mgr;
+
+    // FIXME we should be able to easily build the class ourselves
+    // FIXME use constants
+    @BeforeClass
+    public static void beforeClass() {
+        MxisdConfig cfg = new MxisdConfig();
+        cfg.getMatrix().setDomain("localhost");
+        cfg.getKey().setPath(":memory:");
+        cfg.getStorage().getProvider().getSqlite().setDatabase(":memory:");
+
+        MemoryThreePid mem3pid = new MemoryThreePid();
+        mem3pid.setMedium("email");
+        mem3pid.setAddress("john@localhost");
+        MemoryIdentityConfig validCfg = new MemoryIdentityConfig();
+        validCfg.setUsername("john");
+        validCfg.setPassword("doe");
+        validCfg.getThreepids().add(mem3pid);
+        MemoryIdentityConfig illegalUser = new MemoryIdentityConfig();
+        illegalUser.setUsername("JANE");
+        illegalUser.setPassword("doe");
+        cfg.getMemory().setEnabled(true);
+        cfg.getMemory().getIdentities().add(validCfg);
+        cfg.getMemory().getIdentities().add(illegalUser);
+
+        Mxisd m = new Mxisd(cfg);
+        m.start();
+        mgr = m.getAuth();
+    }
+
+    @Test
+    public void basic() {
+        UserAuthResult result = mgr.authenticate("@john:localhost", "doe");
+        assertTrue(result.isSuccess());
+
+        // For backward-compatibility as per instructed by the spec, we do not fail on an illegal username
+        // This makes sure we don't break it
+        result = mgr.authenticate("@JANE:localhost", "doe");
+        assertTrue(result.isSuccess());
+    }
+
+}

src/test/java/io/kamax/mxisd/test/crypto/SignatureManagerTest.java

@@ -21,8 +21,10 @@
 package io.kamax.mxisd.test.crypto;
 
 import com.google.gson.JsonObject;
+import io.kamax.matrix.event.EventKey;
 import io.kamax.matrix.json.GsonUtil;
 import io.kamax.matrix.json.MatrixJson;
+import io.kamax.mxisd.config.MxisdConfig;
 import io.kamax.mxisd.crypto.Signature;
 import io.kamax.mxisd.crypto.SignatureManager;
 import io.kamax.mxisd.crypto.ed25519.Ed25519Key;
@@ -36,10 +38,14 @@ import org.junit.Test;
 
 import static org.hamcrest.core.Is.is;
 import static org.hamcrest.core.IsEqual.equalTo;
+import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertThat;
 
 public class SignatureManagerTest {
 
+    private static final String lookupData = "{\n" + "  \"not_before\": 0,\n" + "  \"address\": \"mxisd-federation-test@kamax.io\",\n"
+            + "  \"medium\": \"email\",\n" + "  \"mxid\": \"@mxisd-lookup-test:kamax.io\",\n"
+            + "  \"not_after\": 253402300799000,\n" + "  \"ts\": 1523482030147\n" + "}";
     private static SignatureManager signMgr;
 
     private static SignatureManager build(String keySeed) {
@@ -47,7 +53,7 @@ public class SignatureManagerTest {
         KeyStore store = new MemoryKeyStore();
         store.add(key);
 
-        return new Ed25519SignatureManager(new Ed25519KeyManager(store));
+        return new Ed25519SignatureManager(MxisdConfig.forDomain("localhost").inMemory().build(), new Ed25519KeyManager(store));
     }
 
     @BeforeClass
@@ -98,12 +104,19 @@ public class SignatureManagerTest {
 
     @Test
     public void onIdentityLookup() {
-        String value = MatrixJson.encodeCanonical("{\n" + "  \"address\": \"mxisd-federation-test@kamax.io\",\n"
+        String value = MatrixJson.encodeCanonical(lookupData);
-                + "  \"medium\": \"email\",\n" + "  \"mxid\": \"@mxisd-lookup-test:kamax.io\",\n"
-                + "  \"not_after\": 253402300799000,\n" + "  \"not_before\": 0,\n" + "  \"ts\": 1523482030147\n" + "}");
-
         String sign = "ObKA4PNQh2g6c7Yo2QcTcuDgIwhknG7ZfqmNYzbhrbLBOqZomU22xX9raufN2Y3ke1FXsDqsGs7WBDodmzZJCg";
         testSign(value, sign);
     }
 
+    @Test
+    public void onIdentityLookupFull() {
+        JsonObject data = GsonUtil.parseObj(lookupData);
+        signMgr.signMessageGson("localhost", data);
+        JsonObject signatures = EventKey.Signatures.getObj(data);
+        JsonObject domainSign = GsonUtil.getObj(signatures, "localhost");
+        String sign = GsonUtil.getStringOrThrow(domainSign, "ed25519:0");
+        assertEquals(sign, "ObKA4PNQh2g6c7Yo2QcTcuDgIwhknG7ZfqmNYzbhrbLBOqZomU22xX9raufN2Y3ke1FXsDqsGs7WBDodmzZJCg");
+    }
+
 }

src/test/java/io/kamax/mxisd/test/matrix/HomeserverFederationResolverTest.java

@@ -0,0 +1,64 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.test.matrix;
+
+import io.kamax.mxisd.Mxisd;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.dns.FederationDnsOverwrite;
+import io.kamax.mxisd.matrix.HomeserverFederationResolver;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.net.URL;
+
+import static org.junit.Assert.assertEquals;
+
+public class HomeserverFederationResolverTest {
+
+    private static HomeserverFederationResolver resolver;
+
+    @BeforeClass
+    public static void beforeClass() {
+        CloseableHttpClient client = HttpClients.custom()
+                .setUserAgent(Mxisd.Agent)
+                .setMaxConnPerRoute(Integer.MAX_VALUE)
+                .setMaxConnTotal(Integer.MAX_VALUE)
+                .build();
+
+        FederationDnsOverwrite fedDns = new FederationDnsOverwrite(new MxisdConfig().getDns().getOverwrite());
+        resolver = new HomeserverFederationResolver(fedDns, client);
+    }
+
+    @Test
+    public void hostnameWithoutPort() {
+        URL url = resolver.resolve("example.org");
+        assertEquals("https://example.org:8448", url.toString());
+    }
+
+    @Test
+    public void hostnameWithPort() {
+        URL url = resolver.resolve("example.org:443");
+        assertEquals("https://example.org:443", url.toString());
+    }
+
+}

src/test/java/io/kamax/mxisd/test/notification/EmailNotificationTest.java

@@ -32,7 +32,10 @@ import io.kamax.mxisd.config.MxisdConfig;
 import io.kamax.mxisd.config.threepid.connector.EmailSmtpConfig;
 import io.kamax.mxisd.config.threepid.medium.EmailConfig;
 import io.kamax.mxisd.invitation.MatrixIdInvite;
+import io.kamax.mxisd.invitation.ThreePidInvite;
+import io.kamax.mxisd.invitation.ThreePidInviteReply;
 import io.kamax.mxisd.threepid.connector.email.EmailSmtpConnector;
+import io.kamax.mxisd.threepid.generator.PlaceholderNotificationGenerator;
 import io.kamax.mxisd.threepid.session.ThreePidSession;
 import org.apache.commons.lang.RandomStringUtils;
 import org.junit.After;
@@ -45,6 +48,7 @@ import javax.mail.internet.MimeBodyPart;
 import javax.mail.internet.MimeMessage;
 import javax.mail.internet.MimeMultipart;
 import java.io.IOException;
+import java.util.ArrayList;
 import java.util.Collections;
 
 import static junit.framework.TestCase.assertEquals;
@@ -56,7 +60,8 @@ public class EmailNotificationTest {
     private final String user = "mxisd";
     private final String notifiee = "john";
     private final String sender = user + "@" + domain;
-    private final String senderEmail = "\"Mxisd Server (Unit Test)\" <" + sender + ">";
+    private final String senderName = "\"Mxisd Server あ (Unit Test)\" <" + sender + ">";
+    private final String senderNameEncoded = "=?UTF-8?Q?=22Mxisd_Server_=E3=81=82_=28Unit_T?= =?UTF-8?Q?est=29=22_=3Cmxisd=40localhost=3E?= <mxisd@localhost>";
     private final String target = notifiee + "@" + domain;
 
     private Mxisd m;
@@ -72,7 +77,7 @@ public class EmailNotificationTest {
         EmailConfig eCfg = new EmailConfig();
         eCfg.setConnector(EmailSmtpConnector.ID);
         eCfg.getIdentity().setFrom(sender);
-        eCfg.getIdentity().setName("Mxisd Server (Unit Test)");
+        eCfg.getIdentity().setName(senderName);
         eCfg.getConnectors().put(EmailSmtpConnector.ID, GsonUtil.makeObj(smtpCfg));
 
         MxisdConfig cfg = new MxisdConfig();
@@ -114,10 +119,33 @@ public class EmailNotificationTest {
         assertEquals(1, gm.getReceivedMessages().length);
         MimeMessage msg = gm.getReceivedMessages()[0];
         assertEquals(1, msg.getFrom().length);
-        assertEquals(senderEmail, msg.getFrom()[0].toString());
+        assertEquals(senderNameEncoded, msg.getFrom()[0].toString());
         assertEquals(1, msg.getRecipients(Message.RecipientType.TO).length);
     }
 
+    @Test
+    public void forThreepidInvite() throws MessagingException, IOException {
+        String registerUrl = "https://" + RandomStringUtils.randomAlphanumeric(20) + ".example.org/register";
+        gm.setUser(user, user);
+
+        _MatrixID sender = MatrixID.asAcceptable(user, domain);
+        ThreePidInvite inv = new ThreePidInvite(sender, ThreePidMedium.Email.getId(), target, "!rid:" + domain);
+        inv.getProperties().put(PlaceholderNotificationGenerator.RegisterUrl, registerUrl);
+        m.getNotif().sendForReply(new ThreePidInviteReply("a", inv, "b", "c", new ArrayList<>()));
+
+        assertEquals(1, gm.getReceivedMessages().length);
+        MimeMessage msg = gm.getReceivedMessages()[0];
+        assertEquals(1, msg.getFrom().length);
+        assertEquals(senderNameEncoded, msg.getFrom()[0].toString());
+        assertEquals(1, msg.getRecipients(Message.RecipientType.TO).length);
+
+        // We just check on the text/plain one. HTML is multipart and it's difficult so we skip
+        MimeMultipart content = (MimeMultipart) msg.getContent();
+        MimeBodyPart mbp = (MimeBodyPart) content.getBodyPart(0);
+        String mbpContent = mbp.getContent().toString();
+        assertTrue(mbpContent.contains(registerUrl));
+    }
+
     @Test
     public void forValidation() throws MessagingException, IOException {
         gm.setUser(user, user);
@@ -138,7 +166,7 @@ public class EmailNotificationTest {
         assertEquals(1, gm.getReceivedMessages().length);
         MimeMessage msg = gm.getReceivedMessages()[0];
         assertEquals(1, msg.getFrom().length);
-        assertEquals(senderEmail, msg.getFrom()[0].toString());
+        assertEquals(senderNameEncoded, msg.getFrom()[0].toString());
         assertEquals(1, msg.getRecipients(Message.RecipientType.TO).length);
 
         // We just check on the text/plain one. HTML is multipart and it's difficult so we skip

src/test/java/io/kamax/mxisd/test/util/RestClientUtilsTest.java

@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2019 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.test.util;
+
+import io.kamax.mxisd.util.RestClientUtils;
+import org.junit.Test;
+
+import static org.junit.Assert.assertEquals;
+
+public class RestClientUtilsTest {
+
+    @Test
+    public void urlEncode() {
+        String encoded = RestClientUtils.urlEncode("john+doe@example.org");
+        assertEquals("john%2Bdoe%40example.org", encoded);
+    }
+
+}